Question: Assessment of a brute-forcing utility for retrieving the RAT from a harmful document file.

This paper introduces an innovative approach to detecting malicious JavaScript-infused PDF documents through static analysis of the extracted JavaScript code. Despite Adobe's efforts to enhance PDF viewer security, the software's code base continues to harbor vulnerabilities, making it a target for sophisticated malware attacks. The prevalence of rapidly mutating PDF malware in the wild underscores the necessity for advanced protective measures that go beyond traditional signature-based detection methods. Our proposed technique, focusing on static analysis, offers a significant advantage over previous dynamic analysis methods. It boasts a tenfold reduction in runtime overhead and eliminates the need for specialized instrumentation. To validate its efficacy, we tested our method on a vast dataset sourced from VirusTotal's malware upload portal. The results affirm that our technique is not only effective against known and unknown malware strains but also well-suited for large-scale, high-throughput processing scenarios, demonstrating its potential for wide-scale deployment in combating PDF-based threats.

As the Internet has changed the way people communicate with each other in everyday life, the number of home Wi-Fi access routers has grown up significantly over the past few years. However, the security of routers used in every household is still in the low level. The common vulnerabilities in routers can be easily exploited by an attacker in order to obtain user’s sensitive information or even compromise the devices to be a part of the botnet network. Therefore, we developed one-stop service firmware analysis tool that can perform both static and dynamic analysis for the router firmware called “Firmaster”. textbfThe program is operated under graphical user interface (GUI) of Qt creator running on the Ubuntu Linux machine. textbfVulnerabilities of firmware analyzed by Firmaster program are based on OWASP’s Top 10 IoT Vulnerabilities 2014. Firmaster contains seven main functions: password cracking, SSL scanning, web static analysis, firmware update analysis, web dynamic analysis, port scanning and the summary report.

A intrusion detection evaluation test bed was developed which generated normal traffic similar to that on a government site containing 100’s of users on 1000’s of hosts. More than 300 instances of 38 different automated attacks were launched against victim UNIX hosts in seven weeks of training data and two weeks of test data. Six research groups participated in a blind evaluation and results were analyzed for probe, denialof-service (DoS), remote-to-local (R2L), and user to root (U2R) attacks. The best systems detected old attacks included in the training data, at moderate detection rates ranging from 63% to 93% at a false alarm rate of 10 false alarms per day. Detection rates were much worse for new and novel R2L and DoS attacks included only in the test data. The best systems failed to detect roughly half these new attacks which included damaging access to root-level privileges by remote users. These results suggest that further research should focus on developing techniques to find new attacks instead of extending existing rule-based

Alessandro Cimatti 1, Edmund Clarke2, Enrico Giunchiglia3, Fausto Giunchiglia 4, Marco Pistore1, Marco Roveri 1, Roberto Sebastiani 4, and Armando Tacchella 3 1 ITC-IRST, Via Sommarive 18, 38050 Trento, Italy fcimatti,pistore,roverig@irst.itc.it 2 Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh (PA), USA emc@cs.cmu.edu 3 DIST – Università di Genova, Viale Causa 13, 16145 Genova, Italy fenrico,tacg@mrg.dist.unige.it 4 Università di Trento, Via Sommarive 14, 38050 Trento, Italy ffausto,rsebag@science.unitn.it

We present PANDA, an open-source tool that has been purpose-built to support whole system reverse engineering. It is built upon the QEMU whole system emulator, and so analyses have access to all code executing in the guest and all data. PANDA adds the ability to record and replay executions, enabling iterative, deep, whole system analyses. Further, the replay log files are compact and shareable, allowing for repeatable experiments. A nine billion instruction boot of FreeBSD, e.g., is represented by only a few hundred MB. PANDA leverages QEMU's support of thirteen different CPU architectures to make analyses of those diverse instruction sets possible within the LLVM IR. In this way, PANDA can have a single dynamic taint analysis, for example, that precisely supports many CPUs. PANDA analyses are written in a simple plugin architecture which includes a mechanism to share functionality between plugins, increasing analysis code re-use and simplifying complex analysis development. We demonstrate PANDA's effectiveness via a number of use cases, including enabling an old but legitimately purchased game to run despite a lost CD key, in-depth diagnosis of an Internet Explorer crash, and uncovering the censorship activities and mechanisms of an IM client.

Timing attacks are usually used to attack weak computing devices such as smartcards. We show that timing attacks apply to general software systems. Specifically, we devise a timing attack against OpenSSL. Our experiments show that we can extract private keys from an OpenSSL-based web server running on a machine in the local network. Our results demonstrate that timing attacks against network servers are practical and therefore security systems should defend against them.

A drive-by download attack occurs when a user visits a webpage which attempts to automatically download malware without the user's consent. Attackers sometimes use a malware distribution network (MDN) to manage a large number of malicious webpages, exploits, and malware executables. In this paper, we provide a new method to determine these MDNs from the secondary URLs and redirect chains recorded by a high-interaction client honeypot. In addition, we propose a novel drive-by download detection method. Instead of depending on the malicious content used by previous methods, our algorithm first identifies and then leverages the URLs of the MDN's central servers, where a central server is a common server shared by a large percentage of the drive-by download attacks in the same MDN. A set of regular expression-based signatures are then generated based on the URLs of each central server. This method allows additional malicious webpages to be identified which launched but failed to execute a successful drive-by download attack. The new drive-by detection system named ARROW has been implemented, and we provide a large-scale evaluation on the output of a production drive-by detection system. The experimental results demonstrate the effectiveness of our method, where the detection coverage has been boosted by 96% with an extremely low false positive rate.

Malware is constantly adapting in order to avoid detection. Model-based malware detectors, such as SVM and neural networks, are vulnerable to so-called adversarial examples which are modest changes to detectable malware that allows the resulting malware to evade detection. Continuous-valued methods that are robust to adversarial examples of images have been developed using saddle-point optimization formulations. We are inspired by them to develop similar methods for the discrete, e.g. binary, domain which characterizes the features of malware. A specific extra challenge of malware is that the adversarial examples must be generated in a way that preserves their malicious functionality. We introduce methods capable of generating functionally preserved adversarial malware examples in the binary domain. Using the saddle-point formulation, we incorporate the adversarial examples into the training of models that are robust to them. We evaluate the effectiveness of the methods and others in the literature on a set of Portable Execution (PE) files. Comparison prompts our introduction of an online measure computed during training to assess general expectation of robustness.

Security breaches are largely caused by the vulnerable software. Since individuals and organizations mostly depend on softwares, it is important to produce in secured manner. The first step towards producing secured software is through gathering security requirements. This paper describes Software Security Requirements Gathering Instrument (SSRGI) that helps gather security requirements from the various stakeholders. This will guide the developers to gather security requirements along with the functional requirements and further incorporate security during other phases of software development. We subsequently present case studies that describe the integration of the SSRGI instrument with Software Requirements Specification (SRS) document as specified in standard IEEE 830-1998. Proposed SSRGI will support the software developers in gathering security requirements in detail during requirements gathering phase.

Besides a large set of malware categories such as worms and Trojan horses, Advanced Persistent Threat (APT) is another more sophisticated attack entity emerging in the cyber threats environment. In this paper we propose a model of the APT detection problem as well as a methodology to implement it on a generic organization network. From our knowledge, the proposed method is the first to address the problem of modeling an APT and to provide a possible detection framework.

Malware analysts still need to manually inspect malware samples that are considered suspicious by heuristic rules. They dissect software pieces and look for malware evidence in the code. The increasing number of malicious applications targeting Android devices raises the demand for analyzing them to find where the malcode is triggered when user interacts with them. In this paper a framework to monitor and visualize Android applications’ anomalous function calls is described. Our approach includes platformindependent application instrumentation, introducing hooks in order to trace restricted API functions used at runtime of the application. These function calls are collected at a central server where the application behavior filtering and a visualization take place. This can help Android malware analysts in visually inspecting what the application under study does, easily identifying such malicious functions.


Question: Assessment of a brute-forcing utility for retrieving the Remote Access Trojan from a harmful document file.

This paper delves into an advanced evaluation of a specialized tool designed for the extraction of Remote Access Trojans (RATs) from malicious document files through a brute forcing technique. Unlike conventional methods that may rely on signature-based detection or heuristic analysis, this tool employs a brute forcing approach to dissect and analyze the structure of document files. The evaluation focuses on the tool's efficiency, reliability, and its ability to accurately identify and extract RATs embedded within seemingly benign documents. The study also examines the tool's performance under various conditions, including different file formats and levels of obfuscation. Results indicate the tool's high success rate in RAT extraction, showcasing its potential as a valuable addition to the arsenal of cybersecurity professionals in combating targeted attacks that utilize malicious documents as a vector.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An investigation into the application of Direct Search Methods (DSM) for addressing the economic dispatch problem (EDP) in power systems, particularly when dealing with the valve-point effect. The valve-point effect refers to the non-smooth cost function characteristics of thermal power plants, which can significantly complicate the optimization process. This paper proposes a novel DSM tailored to overcome these complexities by directly searching for the optimal operating points without the need for gradient information. The method is tested on a variety of power system configurations, demonstrating its effectiveness in finding solutions that minimize total generation costs while accounting for operational constraints and valve-point effects. Results are compared with traditional optimization techniques, highlighting the superiority of the proposed DSM in terms of solution quality and computational efficiency.

**Similar Documents**: This research explores the utilization of Direct Search Methods (DSM) as an innovative solution to the economic dispatch problem (EDP) in electrical power systems, particularly when faced with the valve-point effect. The valve-point effect introduces irregularities into the cost function of thermal power plants, posing significant challenges to standard optimization algorithms. The proposed DSM is specifically designed to navigate these complexities by conducting a direct search for the most cost-effective operating points, bypassing the need for derivative information. The method's efficacy is evaluated across a range of power system scenarios, showcasing its ability to efficiently minimize total generation costs while adhering to operational constraints and accounting for valve-point effects. Comparative analysis against conventional optimization techniques underscores the proposed DSM's superior performance in both solution quality and computational speed.

In this paper we explore the problem of creating vulnerability signatures. A vulnerability signature matches all exploits of a given vulnerability, even polymorphic or metamorphic variants. Our work departs from previous approaches by focusing on the semantics of the program and vulnerability exercised by a sample exploit instead of the semantics or syntax of the exploit itself. We show the semantics of a vulnerability define a language which contains all and only those inputs that exploit the vulnerability. A vulnerability signature is a representation (e.g., a regular expression) of the vulnerability language. Unlike exploit-based signatures whose error rate can only be empirically measured for known test cases, the quality of a vulnerability signature can be formally quantified for all possible inputs. We provide a formal definition of a vulnerability signature and investigate the computational complexity of creating and matching vulnerability signatures. We also systematically explore the design space of vulnerability signatures. We identify three central issues in vulnerability-signature creation: how a vulnerability signature represents the set of inputs that may exercise a vulnerability, the vulnerability coverage (i.e., number of vulnerable program paths) that is subject to our analysis during signature creation, and how a vulnerability signature is then created for a given representation and coverage. We propose new data-flow analysis and novel adoption of existing techniques such as constraint solving for automatically generating vulnerability signatures. We have built a prototype system to test our techniques. Our experiments show that we can automatically generate a vulnerability signature using a single exploit which is of much higher quality than previous exploit-based signatures. In addition, our techniques have several other security applications, and thus may be of independent interest

Surveying a suite of algorithms that offer a solution to managing large document archives.

Kernel rootkits have posed serious security threats due to their stealthy manner. To hide their presence and activities, many rootkits hijack control flows by modifying control data or hooks in the kernel space. A critical step towards eliminating rootkits is to protect such hooks from being hijacked. However, it remains a challenge because there exist a large number of widely-scattered kernel hooks and many of them could be dynamically allocated from kernel heap and co-located together with other kernel data. In addition, there is a lack of flexible commodity hardware support, leading to the socalled protection granularity gap -- kernel hook protection requires byte-level granularity but commodity hardware only provides page level protection.
 To address the above challenges, in this paper, we present HookSafe, a hypervisor-based lightweight system that can protect thousands of kernel hooks in a guest OS from being hijacked. One key observation behind our approach is that a kernel hook, once initialized, may be frequently "read"-accessed, but rarely write"-accessed. As such, we can relocate those kernel hooks to a dedicated page-aligned memory space and then regulate accesses to them with hardware-based page-level protection. We have developed a prototype of HookSafe and used it to protect more than 5,900 kernel hooks in a Linux guest. Our experiments with nine real-world rootkits show that HookSafe can effectively defeat their attempts to hijack kernel hooks. We also show that HookSafe achieves such a large-scale protection with a small overhead (e.g., around 6% slowdown in performance benchmarks).

To address the growing concerns about the security of embedded systems, it is important to perform accurate analysis of firmware binaries, even when the source code or the hardware documentation are not available. However, research in this field is hindered by the lack of dedicated tools. For example, dynamic analysis is one of the main foundations of security analysis, e.g., through dynamic taint tracing or symbolic execution. Unlike static analysis, dynamic analysis relies on the ability to execute software in a controlled environment, often an instrumented emulator. However, emulating firmwares of embedded devices requires accurate models of all hardware components used by the system under analysis. Unfortunately, the lack of documentation and the large variety of hardware on the market make this approach infeasible in practice. In this paper we present Avatar, a framework that enables complex dynamic analysis of embedded devices by orchestrating the execution of an emulator together with the real hardware. We first introduce the basic mechanism to forward I/O accesses from the emulator to the embedded device, and then describe several techniques to improve the system’s performance by dynamically optimizing the distribution of code and data between the two environments. Finally, we evaluate our tool by applying it to three different security scenarios, including reverse engineering, vulnerability discovery and hardcoded backdoor detection. To show the flexibility of Avatar, we perform this analysis on three completely different devices: a GSM feature phone, a hard disk bootloader, and a wireless sensor node.

With more and more online services developed into web applications, security problems based on web applications become more serious now. Most intrusion detection systems are based on every single request to find the cyber-attack instead of users' behaviors, and these systems can only protect web application from known vulnerability rather than some zero-day attacks. In order to detect newly developed attacks, we analyze web logs from web servers and define users' behaviors to divide them into normal and malicious ones. The result shows that by using the feature of web resources to define users' behaviors, a higher accuracy rate and lower false alarm rate of intrusion detection can be obtained.

With the rapid technological advancements of sensors, Wireless Sensor Networks (WSNs) have become a popular technology for the Internet of Things (IoT). We investigated the security of WSNs in an environmental monitoring application with the goal to demonstrate the overall security. We implemented a Secure Temperature Monitoring System (STMS), which served as our WSN application. Our results revealed a security flaw found in the bootstrap loader (BSL) password used to protect MSP430 micro-controller units (MCUs). We demonstrated how the BSL password could be brute forced in a matter of days. Furthermore, we illustrate how an attacker can reverse engineer WSN applications to obtain critical security information such as encryption keys. We contribute a solution to patch the weak BSL password security flaw and improve the security of MSP430 MCU chips. The Secure-BSL patch we contribute allows the randomization of the BSL password. Our solution increases the brute force time to decades. The impractical brute force time enhances the security of the MSP430 and prevents future reverse engineering tactics. Our research serves as proof that the security of WSNs and the overall IoT technology is broken if we cannot protect these everyday objects at the physical layer.

Botnets have recently been identified as one of the most important threats to the security of the Internet. Traditionally, botnets organize themselves in an hierarchical manner with a central command and control location. This location can be statically defined in the bot, or it can be dynamically defined based on a directory server. Presently, the centralized characteristic of botnets is us eful to security professionals because it offers a central point of failure for the botnet. In the near future, we believe attackers will move to more resilient architectures. In particular, one class of botnet structure that has entered initial stages of development is peer-to-peer based architectures. In this paper, we present an overview of peer-to-peer botnets. We also present a case study of a Kademlia-based Trojan.Peacomm bot.

Practical software security metrics and measurements are essential to the development of secure software [18]. In this paper, we propose to use a software system’s attack surface measurement as an indicator of the system’s security; the larger the attack surface, the more insecure the system. We formalize the notion of a system’s attack surface using an I/O automata model of the system [15] and define a quantitative measure of the attack surface in terms of three kinds of resources used in attacks on the system: methods, channels, and data. We demonstrate the feasibility of our approach by measuring the attack surfaces of two open source FTP daemons and two IMAP servers. Software developers can use our attack surface measurement method in the software development process and software consumers can use the method in their decision making process. This research was sponsored by the US Army Research Office (ARO) under contract no. DAAD190210389, SAP Labs, LLC under award no. 1010751, and the Software Engineering Institute. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the U.S. government or any other entity. Report Documentation Page Form Approved

Increasingly advances in file carving, memory analysis and network forensics requires the ability to identify the underlying type of a file given only a file fragment. Work to date on this problem has relied on identification of specific byte sequences in file headers and footers, and the use of statistical analysis and machine learning algorithms taken from the middle of the file. We argue that these approaches are fundamentally flawed because they fail to consider the inherent internal structure in widely used file types such as PDF, DOC, and ZIP. We support our argument with a bottom-up examination of some popular formats and an analysis of TK PDF files. Based on our analysis, we argue that specialized methods targeted to each specific file type will be necessary to make progress in this area.

Application-level protocol specifications are useful for many security applications, including intrusion prevention and detection that performs deep packet inspection and traffic normalization, and penetration testing that generates network inputs to an application to uncover potential vulnerabilities. However, current practice in deriving protocol specifications is mostly manual. In this paper, we present Discoverer, a tool for automatically reverse engineering the protocol message formats of an application from its network trace. A key property of Discoverer is that it operates in a protocol-independent fashion by inferring protocol idioms commonly seen in message formats of many application-level protocols. We evaluated the efficacy of Discoverer over one text protocol (HTTP) and two binary protocols (RPC and CIFS/SMB) by comparing our inferred formats with true formats obtained from Ethereal [5]. For all three protocols, more than 90% of our inferred formats correspond to exactly one true format; one true format is reflected in five inferred formats on average; our inferred formats cover over 95% of messages, which belong to 30-40% of true formats observed in the trace.


Question: Assessment of a brute-forcing utility for retrieving the Remote Access Trojan from a tainted document file.

This paper introduces VMwatcher, a novel "out-of-the-box" malware detection approach designed to overcome the limitations of traditional host-based anti-malware systems, which are susceptible to stealthy malware techniques due to their internal operation within the very systems they protect. The core challenge faced by "out-of-the-box" solutions, which place malware detection mechanisms outside the protected virtual machine (VM), is the semantic gap—the loss of direct semantic understanding of the host environment. To address this, VMwatcher employs a technique called guest view casting, which systematically reconstructs the internal semantic views of a VM from an external perspective in a non-intrusive way. This technique involves mapping semantic definitions of guest operating system (OS) data structures and functions onto the virtual machine monitor (VMM)-level VM states, enabling the reconstruction of the semantic view. By bridging the semantic gap, VMwatcher showcases two distinctive malware detection capabilities: (1) view comparison-based malware detection, exemplified through rootkit detection, and (2) the external deployment of host-based anti-malware software, which results in enhanced detection accuracy and tamper-resistance. We have developed a proof-of-concept prototype of VMwatcher for both Linux and Windows platforms, and our experimental outcomes using real-world malware, such as sophisticated kernel-level rootkits, attest to its practicality and efficacy in detecting stealthy malware threats.

With the rapid growth of computer and network systems in recent years, there has also been a corresponding increase in cyber-crime. Cybercrime takes many forms and has garnered much attention in the media, making information security a more urgent and important priority. In order to fight cyber-crime, criminal evidence must be gathered from these computer-based systems. This is quite different from the collection of conventional criminal evidence and can confuse investigators attempting to deal with the forensics of cyber-crime, highlighting the importance of computer forensics. In this paper, we offer solutions to guard against cyber-crime through the implementation of software toolkits for computerbased systems. In this way, those who engage in criminal acts in cyber-space can be more easily apprehended. © 2006 Elsevier B.V. All rights reserved.

A wide range of malicious activities rely on the domain name service (DNS) to manage their large, distributed networks of infected machines. As a consequence, the monitoring and analysis of DNS queries has recently been proposed as one of the most promising techniques to detect and blacklist domains involved in malicious activities (e.g., phishing, spam, botnets command-and-control, etc.). EXPOSURE is a system we designed to detect such domains in real time, by applying 15 unique features grouped in four categories.
 We conducted a controlled experiment with a large, real-world dataset consisting of billions of DNS requests. The extremely positive results obtained in the tests convinced us to implement our techniques and deploy it as a free, online service. In this article, we present the Exposure system and describe the results and lessons learned from 17 months of its operation. Over this amount of time, the service detected over 100K malicious domains. The statistics about the time of usage, number of queries, and target IP addresses of each domain are also published on a daily basis on the service Web page.

This tutorial paper considers the issues of low-level software security from a language-based perspective, with the help of concrete examples. Four examples of low-level software attacks are covered in full detail; these examples are representative of the major types of attacks on C and C++ software that is compiled into machine code. Six examples of practical defenses against those attacks are also covered in detail; these defenses are selected because of their effectiveness, wide applicability, and low enforcement overhead.

Timing attacks are usually used to attack weak computing devices such as smartcards. We show that timing attacks apply to general software systems. Specifically, we devise a timing attack against OpenSSL. Our experiments show that we can extract private keys from an OpenSSL-based web server running on a machine in the local network. Our results demonstrate that timing attacks against network servers are practical and therefore security systems should defend against them.

This study analyzes 0-day vulnerabilities in the broader context of cybercrime and economic markets. The work is based on the interviews of several leading experts and on a field research of the authors. In particular, cybercrime is considered when involving traditional criminal activities or when military operations are involved. A description of different 0-day vulnerability markets - White, Black and Government markets - is provided, as well as the characteristics of malware factories and their major customers are discussed.

To be useful, software must respond to events in a predictable manner. The results can then be used as a window to the interior workings of the code, revealing some of the mechanisms of operations, which may be used to find ways to make it fail in a dangerous way. To some, the window is as clear as a six inch thick pane of lead, but to those with a high level of understanding it can be clear, or at the very least serve as a keyhole. This is an allusion to the old detective stories where someone looks through the keyhole to see what is behind the door. For these reasons, no software that interacts with humans can ever be considered completely secure, and human error in the development of the software can leave the equivalent of keyholes throughout the code.

Digital content is for copying: quotation, revision, plagiarism, and file sharing all create copies. Document fingerprinting is concerned with accurately identifying copying, including small partial copies, within large sets of documents.We introduce the class of local document fingerprinting algorithms, which seems to capture an essential property of any finger-printing technique guaranteed to detect copies. We prove a novel lower bound on the performance of any local algorithm. We also develop winnowing, an efficient local fingerprinting algorithm, and show that winnowing's performance is within 33% of the lower bound. Finally, we also give experimental results on Web data, and report experience with MOSS, a widely-used plagiarism detection service.

Many large companies today face the risk of data breaches via malicious software, compromising their business. These types of attacks are usually executed using hacker assets. Researching hacker assets within underground communities can help identify the tools which may be used in a cyberattack, provide knowledge on how to implement and use such assets and assist in organizing tools in a manner conducive to ethical reuse and education. This study aims to understand the functions and characteristics of assets in hacker forums by applying classification and topic modeling techniques. This research contributes to hacker literature by gaining a deeper understanding of hacker assets in well-known forums and organizing them in a fashion conducive to educational reuse. Additionally, companies can apply our framework to forums of their choosing to extract their assets and appropriate functions.

With only the binary executable of a program, it is useful to discover the program’s data structures and infer their syntactic and semantic definitions. Such knowledge is highly valuable in a variety of security and forensic applications. Although there exist efforts in program data structure inference, the existing solutions are not suitable for our targeted application scenarios. In this paper, we propose a reverse engineering technique to automatically reveal program data structures from binaries. Our technique, called REWARDS, is based on dynamic analysis. More specifically, each memory location accessed by the program is tagged with a timestamped type attribute. Following the program’s runtime data flow, this attribute is propagated to other memory locations and registers that share the same type. During the propagation, a variable’s type gets resolved if it is involved in a type-revealing execution point or “type sink”. More importantly, besides the forward type propagation, REWARDS involves a backward type resolution procedure where the types of some previously accessed variables get recursively resolved starting from a type sink. This procedure is constrained by the timestamps of relevant memory locations to disambiguate variables reusing the same memory location. In addition, REWARDS is able to reconstruct in-memory data structure layout based on the type information derived. We demonstrate that REWARDS provides unique benefits to two applications: memory image forensics and binary fuzzing for vulnerability discovery.

JavaScript is a browser scripting language that allows developers to create sophisticated client-side interfaces for web applications. However, JavaScript code is also used to carry out attacks against the user's browser and its extensions. These attacks usually result in the download of additional malware that takes complete control of the victim's platform, and are, therefore, called "drive-by downloads." Unfortunately, the dynamic nature of the JavaScript language and its tight integration with the browser make it difficult to detect and block malicious JavaScript code.
 This paper presents a novel approach to the detection and analysis of malicious JavaScript code. Our approach combines anomaly detection with emulation to automatically identify malicious JavaScript code and to support its analysis. We developed a system that uses a number of features and machine-learning techniques to establish the characteristics of normal JavaScript code. Then, during detection, the system is able to identify anomalous JavaScript code by emulating its behavior and comparing it to the established profiles. In addition to identifying malicious code, the system is able to support the analysis of obfuscated code and to generate detection signatures for signature-based systems. The system has been made publicly available and has been used by thousands of analysts.


Question: Uniqueness and coherence in synthesized conversations.

Introducing the SRILM Toolkit: A Comprehensive Resource for Language Modeling and Beyond

SRILM, a suite of C++ libraries, programs, and scripts, has been developed to facilitate the production and experimentation of statistical language models for speech recognition and other applications. Accessible for non-commercial use, SRILM is a versatile tool that supports the creation and assessment of various language model types rooted in N-gram statistics. Beyond language modeling, SRILM also encompasses related tasks such as statistical tagging and the management of N-best lists and word lattices. This paper offers an overview of the toolkit's capabilities, delving into its design and implementation. Key aspects highlighted include the toolkit's suitability for rapid prototyping, the reusability of its components, and the ease with which tools can be combined to address complex linguistic challenges.

This special issue highlights research articles that apply machine learning to robots and other systems that interact with users through more than one modality, such as speech, gestures, and vision. For example, a robot may coordinate its speech with its actions, taking into account (audio-)visual feedback during their execution. Machine learning provides interactive systems with opportunities to improve performance not only of individual components but also of the system as a whole. However, machine learning methods that encompass multiple modalities of an interactive system are still relatively hard to find. The articles in this special issue represent examples that contribute to filling this gap.

Perception and expression of emotion are key factors to the success of dialogue systems or conversational agents. However, this problem has not been studied in large-scale conversation generation so far. In this paper, we propose Emotional Chatting Machine (ECM) that can generate appropriate responses not only in content (relevant and grammatical) but also in emotion (emotionally consistent). To the best of our knowledge, this is the first work that addresses the emotion factor in large-scale conversation generation. ECM addresses the factor using three new mechanisms that respectively (1) models the high-level abstraction of emotion expressions by embedding emotion categories, (2) captures the change of implicit internal emotion states, and (3) uses explicit emotion expressions with an external emotion vocabulary. Experiments show that the proposed model can generate responses appropriate not only in content but also in emotion.

The CHiME challenge series aims to advance robust automatic speech recognition (ASR) technology by promoting research at the interface of speech and language processing, signal processing, and machine learning. This paper introduces the 5th CHiME Challenge, which considers the task of distant multimicrophone conversational ASR in real home environments. Speech material was elicited using a dinner party scenario with efforts taken to capture data that is representative of natural conversational speech and recorded by 6 Kinect microphone arrays and 4 binaural microphone pairs. The challenge features a single-array track and a multiple-array track and, for each track, distinct rankings will be produced for systems focusing on robustness with respect to distant-microphone capture vs. systems attempting to address all aspects of the task including conversational language modeling. We discuss the rationale for the challenge and provide a detailed description of the data collection procedure, the task, and the baseline systems for array synchronization, speech enhancement, and conventional and end-to-end ASR.

We describe RavenClaw, a new dialog management framework developed as a successor to the Agenda [1] architecture used in the CMU Communicator. RavenClaw introduces a clear separation between task and discourse behavior specification, and allows rapid development of dialog management components for spoken dialog systems operating in complex, goal-oriented domains. The system development effort is focused entirely on the specification of the dialog task, while a rich set of domain-independent conversational behaviors are transparently generated by the dialog engine. To date, RavenClaw has been applied to five different domains allowing us to draw some preliminary conclusions as to the generality of the approach. We briefly describe our experience in developing these systems.

Language generation from purely semantic representations is a challenging task. This paper addresses generating English from the Abstract Meaning Representation (AMR), consisting of re-entrant graphs whose nodes are concepts and edges are relations. The new method is trained statistically from AMRannotated English and consists of two major steps: (i) generating an appropriate spanning tree for the AMR, and (ii) applying tree-tostring transducers to generate English. The method relies on discriminative learning and an argument realization model to overcome data sparsity. Initial tests on held-out data show good promise despite the complexity of the task. The system is available open-source as part of JAMR at: http://github.com/jflanigan/jamr

The use of dialogue systems as a medium for human-machine interaction is an increasingly prevalent paradigm. A growing number of dialogue systems use conversation strategies that are learned from large datasets. There are well documented instances where interactions with these system have resulted in biased or even offensive conversations due to the data-driven training process. Here, we highlight potential ethical issues that arise in dialogue systems research, including: implicit biases in data-driven systems, the rise of adversarial examples, potential sources of privacy violations, safety concerns, special considerations for reinforcement learning systems, and reproducibility concerns. We also suggest areas stemming from these issues that deserve further investigation. Through this initial survey, we hope to spur research leading to robust, safe, and ethically sound dialogue systems.

We apply a decision tree based approach to pronoun resolution in spoken dialogue. Our system deals with pronouns with NPand non-NP-antecedents. We present a set of features designed for pronoun resolution in spoken dialogue and determine the most promising features. We evaluate the system on twenty Switchboard dialogues and show that it compares well to Byron’s (2002) manually tuned system.

Just as syntax differentiates coherent sentences from scrambled word strings, sequential images must also use a cognitive system to distinguish coherent narratives from random strings of images. We conducted experiments analogous to two classic psycholinguistic studies to examine structure and semantics in sequential images. We compared Normal comic strips with both structure and meaning to sequences with Semantics Only, Structure Only, or randomly Scrambled panels. Experiment 1 used a target-monitoring paradigm, and found that RTs were slowest to target panels in Scrambled sequences, intermediate in Structural Only and Semantic Only sequences, and fastest in Normal sequences. Experiment 2 measured ERPs to the same strips. The largest N400 appeared in Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in Normal sequences. Together, these findings suggest that sequential image comprehension is guided by an interaction between a structure and meaning, broadly analogous to syntax and semantics in language.

This article describes My Science Tutor (MyST), an intelligent tutoring system designed to improve science learning by students in 3rd, 4th, and 5th grades (7 to 11 years old) through conversational dialogs with a virtual science tutor. In our study, individual students engage in spoken dialogs with the virtual tutor Marni during 15 to 20 minute sessions following classroom science investigations to discuss and extend concepts embedded in the investigations. The spoken dialogs in MyST are designed to scaffold learning by presenting open-ended questions accompanied by illustrations or animations related to the classroom investigations and the science concepts being learned. The focus of the interactions is to elicit self-expression from students. To this end, Marni applies some of the principles of Questioning the Author, a proven approach to classroom conversations, to challenge students to think about and integrate new concepts with prior knowledge to construct enriched mental models that can be used to explain and predict scientific phenomena. In this article, we describe how spoken dialogs using Automatic Speech Recognition (ASR) and natural language processing were developed to stimulate students' thinking, reasoning and self explanations. We describe the MyST system architecture and Wizard of Oz procedure that was used to collect data from tutorial sessions with elementary school students. Using data collected with the procedure, we present evaluations of the ASR and semantic parsing components. A formal evaluation of learning gains resulting from system use is currently being conducted. This paper presents survey results of teachers' and children's impressions of MyST.

A challenging problem for spoken dialog systems is the design of utterance generation modules that are fast, flexible and general, yet produce high quality output in particular domains. A promising approach is trainable generation, which uses general-purpose linguistic knowledge automatically adapted to the application domain. This paper presents a trainable sentence planner for the MATCH dialog system. We show that trainable sentence planning can produce output comparable to that of MATCH’s template-based generator even for quite complex information presentations.


Question: Uniqueness and coherence in synthesized conversations.

This paper outlines an enhanced chart realization algorithm for Combinatory Categorial Grammar (CCG) that enables efficient processing of complex coordination phenomena, such as argument cluster coordination and gapping. The proposed algorithm introduces three innovative techniques to optimize chart realization: (i) utilizing rules to segment the input logical form into manageable sub-problems for independent resolution prior to integration; (ii) applying n-gram scores to prune chart edges, prioritizing those with higher linguistic probability over others with identical categories; and (iii) implementing a best-first anytime algorithm that leverages n-gram scores for agenda sorting, ensuring the most promising edges are processed first. The algorithm has been successfully integrated into the OpenCCG open source CCG parser, and preliminary testing reveals its capability to achieve processing speeds suitable for deployment in natural language dialogue systems, highlighting its potential for practical application.

Natural language generation (NLG) is a critical component of spoken dialogue and it has a significant impact both on usability and perceived quality. Most NLG systems in common use employ rules and heuristics and tend to generate rigid and stylised responses without the natural variation of human language. They are also not easily scaled to systems covering multiple domains and languages. This paper presents a statistical language generator based on a semantically controlled Long Short-term Memory (LSTM) structure. The LSTM generator can learn from unaligned data by jointly optimising sentence planning and surface realisation using a simple cross entropy training criterion, and language variation can be easily achieved by sampling from output candidates. With fewer heuristics, an objective evaluation in two differing test domains showed the proposed method improved performance compared to previous methods. Human judges scored the LSTM system higher on informativeness and naturalness and overall preferred it to the other systems.

Current dialogue systems generally operate in a pipelined, modular fashion on one complete utterance at a time. Evidence from human language understanding shows that human understanding operates incrementally and makes use of multiple sources of information during the parsing process, including traditionally “later” components such as pragmatics. In this paper we describe a spoken dialogue system that understands language incrementally, provides visual feedback on possible referents during the course of the user’s utterance, and allows for overlapping speech and actions. We further present findings from an empirical study showing that the resulting dialogue system is faster overall than its nonincremental counterpart. Furthermore, the incremental system is preferred to its nonincremental counterpart – beyond what is accounted for by factors such as speed and accuracy. These results indicate that successful incremental understanding systems will improve both performance and usability.

been developed during the last 20 years have proven to be quite successful, particularly in the domains of mathematics, science, and technology. They produce significant learning gains beyond classroom environments. They are capable of engaging most students’ attention and interest for hours. We have been working on a new generation of intelligent tutoring systems that hold mixedinitiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AUTOTUTOR is a conversational agent, with a talking head, that helps college students learn about computer literacy. ANDES, ATLAS, AND WHY2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations.

We describe our Alexa prize system (called ‘Alana’) which consists of an ensemble of bots, combining rule-based and machine learning systems, and using a contextual ranking mechanism to choose system responses. This paper reports on the version of the system developed and evaluated in the semi-finals of the competition (i.e. up to 15 August 2017), but not on subsequent enhancements. The ranker for this system was trained on real user feedback received during the competition, where we address the problem of how to train on the noisy and sparse feedback obtained during the competition. In order to avoid initial problems of inappropriate and boring utterances coming from big datasets such as Reddit and Twitter, we later focussed on ‘clean’ data sources such as news and facts. We report on experiments with different ranking functions and versions of our NewsBot. We find that a multiturn news strategy is beneficial, and that a ranker trained on the ratings feedback from users is also effective. Our system continuously improved using the data gathered over the course over the competition (1 July – 15 August) . Our final user score (averaged user rating over the whole semi-finals period) was 3.12, and we achieved 3.3 for the averaged user rating over the last week of the semi-finals (8-15 August 2017). We were also able to achieve long dialogues (average 10.7 turns) during the competition period. In subsequent weeks, after the end of the semi-final competition, we have achieved our highest scores of 3.52 (daily average, 18th October), 3.45 (weekly average on 23 and 24 October), and average dialogue lengths of 14.6 turns (1 October), and median dialogue length of 2.25 minutes (average for 7 days on 10th October).

We report on the construction of a real-time computer system capable of distinguishing speech signals from music signals over a wide range of digital audio input. We have examined 13 features intended to measure conceptually distinct properties of speech and/or music signals, and combined them in several multidimensional classification frameworks. We provide extensive data on system performance and the cross-validated training/test setup used to evaluate the system. For the datasets currently in use, the best classifier classifies with 5.8% error on a frame-by-frame basis, and 1.4% error when integrating long (2.4 second) segments of sound.

In recent years, there has been renewed interest in the NLP community in genuine language understanding and dialogue. Thus the long-standing issue of how the semantic content of language should be represented is reentering the communal discussion. This paper provides a brief “opinionated survey” of broadcoverage semantic representation (SR). It suggests multiple desiderata for such representations, and then outlines more than a dozen approaches to SR–some longstanding, and some more recent, providing quick characterizations, pros, cons, and some comments on imple-

We propose a method of constructing concurrent programs in which the synchronization skeleton of the program ~s automatically synthesized from a high-level (branching time) Temporal Logic specification. The synchronization skeleton is an abstraction of the actual program where detail irrelevant to synchronization is suppressed. For example, in the synchronization skeleton for a solution to the critical section problem each process's critical section may be viewed as a single node since the internal structure of the critical section is unimportant. Most solutions to synchronization problems in the literature are in fact given as synchronization skeletons. Because synchronization skeletons are in general finite state, the propositional version of Temporal Logic can be used to specify their properties. Our synthesis method exploits the (bounded) finite model property for an appropriate propositional Temporal Logic which asserts that if a formula of the logic is satisfiable, it is satisfiable in a finite model (of size bounded by a function of the length of the formula). Decision procedures have been devised which, given a formula of Temporal Logic, f, will decide whether f is satisfiable or unsatisfiable. If f is satisfiable, a finite model of f is constructed, in our application, unsatisfiability of f means that the specification is inconsistent (and must be reformulated). If the formula f is satisfiable, then the specification it expresses is consistent. A model for f with a finite number of states is constructed by the decision procedure. The synchronization skeleton of a program meeting the specification can be read from this model. The finite model property ensures that any program whose synchronization properties can be expressed in propositional Temporal Logic can be realized by a system of concurrently running processes, each of which is a finite state machine. initially, the synchronization skeletons we synthesize wi]] be for concurrent programs running in a shared-memory environment and for monitors. However, we believe that it is also possible to extend these techniques to synthesize distributed programs. One such application would be the automatic synthesis of network communication protocols from propositional Temporal Logic specifications. Previous efforts toward parallel program synthesis can be found in the work of [LA78] and iRK80]. [LA78] uses a specification language that is essentially predicate

UI on the Fly is a system that dynamically presents coordinated multimodal content through natural language and a small-screen graphical user interface. It adapts to the user’s preferences and situation. Multimodal Functional Unification Grammar (MUG) is a unification-based formalism that uses rules to generate content that is coordinated across several communication modes. Faithful variants are scored with a heuristic function.

This paper describes a speaker-adaptive HMM-based speech synthesis system. The new system, called ldquoHTS-2007,rdquo employs speaker adaptation (CSMAPLR+MAP), feature-space adaptive training, mixed-gender modeling, and full-covariance modeling using CSMAPLR transforms, in addition to several other techniques that have proved effective in our previous systems. Subjective evaluation results show that the new system generates significantly better quality synthetic speech than speaker-dependent approaches with realistic amounts of speech data, and that it bears comparison with speaker-dependent approaches even when large amounts of speech data are available. In addition, a comparison study with several speech synthesis techniques shows the new system is very robust: It is able to build voices from less-than-ideal speech data and synthesize good-quality speech even for out-of-domain sentences.

Spoken Language Understanding (SLU) and more specifically, semantic parsing is an indispensable task in each speech-enabled application. In this survey, we review the current research on SLU and semantic parsing with emphasis on machine learning techniques used for these tasks. Observing the current trends in semantic parsing, we conclude our discussion by suggesting some of the most promising future research trends.


Question: Uniqueness and coherence in produced conversations.

Reflecting on Two Decades of Statistical Language Modeling: Charting the Future Course
Over the past two decades, statistical language modeling has evolved significantly, with its primary goal being the accurate prediction of linguistic patterns to enhance speech recognition and other language-related technologies. From the inception of the groundbreaking model in 1980, numerous efforts have been dedicated to refining and advancing this field. This paper offers a comprehensive review of these advancements, highlighting promising avenues for future exploration. We advocate for a Bayesian framework as a means to harmoniously integrate theoretical linguistic insights with empirical data, thereby enriching the current landscape of statistical language modeling.

**Question**: A Novel Approach for the Detection of Breast Cancer Using Deep Learning Techniques

**Documents**: A novel framework for image classification based on deep learning
In this paper, a novel framework for image classification based on deep learning is proposed. The proposed framework consists of two main stages: feature extraction and classification. In the feature extraction stage, a deep convolutional neural network (CNN) is used to extract the features of the input images. The CNN consists of multiple convolutional layers followed by pooling layers. The convolutional layers learn the features of the input images, while the pooling layers reduce the dimensionality of the feature maps. In the classification stage, a support vector machine (SVM) is used to classify the extracted features. The SVM is trained on a set of labeled images, and the extracted features are used as the input to the SVM. The proposed framework is evaluated on several benchmark datasets, and the results show that it outperforms state-of-the-art image classification methods.

**Similar Documents**: A Deep Learning-Based Framework for Enhanced Image Classification
This paper presents a novel framework for image classification that harnesses the power of deep learning techniques. The proposed framework is comprised of two primary components: feature extraction and classification. In the feature extraction phase, a deep convolutional neural network (CNN) is employed to meticulously extract the essential features from input images. This CNN architecture includes a series of convolutional layers, which are adept at identifying image features, coupled with pooling layers that effectively reduce the dimensionality of feature maps. Subsequently, in the classification phase, a support vector machine (SVM) is utilized to categorize the features extracted from the images. The SVM is meticulously trained using a dataset of labeled images, with the extracted features serving as the input for the classification process. The efficacy of the proposed framework is demonstrated through rigorous evaluation on multiple benchmark datasets, where it consistently outperforms existing state-of-the-art image

Conversational machine comprehension requires a deep understanding of the conversation history. To enable traditional, single-turn models to encode the history comprehensively, we introduce FLOW, a mechanism that can incorporate intermediate representations generated during the process of answering previous questions, through an alternating parallel processing structure. Compared to shallow approaches that concatenate previous questions/answers as input, FLOW integrates the latent semantics of the conversation history more deeply. Our model, FLOWQA, shows superior performance on two recently proposed conversational challenges (+7.2% F1 on CoQA and +4.0% on QuAC). The effectiveness of FLOW also shows in other tasks. By reducing sequential instruction understanding to conversational machine comprehension, FLOWQA outperforms the best models on all three domains in SCONE, with +1.8% to +4.4% improvement in accuracy.

Spoken dialog systems typically rely on recognition confidence scores to guard against potential misundersta ndings. While confidence scores can provide an initial assessment for the reliability of the information obtained fro m the user, ideally systems should leverage information that is available in subsequent user responses to update and improve the accuracy of their beliefs. We present a machine-learning based solution for this problem. We use a compresse d representation of beliefs that tracks up to k hypotheses for each concept at any given time. We train a generalized l inear model to perform the updates. Experimental results show that the proposed approach significantly outperform s heuristic rules used for this task in current systems. Fu rthermore, a user study with a mixed-initiative spoken dialog sy stem shows that the approach leads to significant gains in task success and in the efficiency of the interaction, a cross a wide range of recognition error-rates.

The aim of our research is to enable spontaneous and efficient spatial reference to objects in human-robot interaction situations. This paper presents the iterative, empirically based design of a robotic system that uses a computational model for identifying objects on the basis of a range of spatial reference systems. The efficiency of the system is evaluated by two successive empirical studies involving uninformed users. The linguistic analysis points to the striking variability in speakers’ spontaneous strategies and preferences, and it motivates a number of modifications of the computational model.

Although coherence is an important aspect of any text generation system, it has received little attention in the context of machine translation (MT) so far. We hypothesize that the quality of document-level translation can be improved if MT models take into account the semantic relations among sentences during translation. We integrate the graph-based coherence model proposed by Mesgar and Strube (2016) with Docent1 (Hardmeier et al., 2012; Hardmeier, 2014) a document-level machine translation system. The application of this graph-based coherence modeling approach is novel in the context of machine translation. We evaluate the coherence model and its effects on the quality of the machine translation. The result of our experiments shows that our coherence model slightly improves the quality of translation in terms of the average Meteor score.

Statistical methods have long been the dominant approach in speech recognition and probabilistic modelling in ASR is now a mature technology. The use of statistical methods in other areas of spoken dialogue is however more recent and rather less mature. This paper reviews spoken dialogue systems from a statistical modelling perspective. The complete system is first presented as a partially observable Markov decision process. The various sub-components are then exposed by introducing appropriate intermediate variables. Samples of existing work are reviewed within this framework, including dialogue control and optimisation, semantic interpretation, goal detection, natural language generation and synthesis.


Question: Uniqueness and coherence in produced conversations.

The CMU-Cambridge Toolkit: An Enhanced Platform for Statistical Language Modeling

The CMU Statistical Language Modeling toolkit, a pivotal tool for the development and evaluation of bigram and trigram language models, was released to streamline the language modeling process. Currently, this toolkit is utilized by over a hundred academic, governmental, and industrial research labs in more than 50 countries worldwide. This paper introduces a refined version of the toolkit, detailing the conventional language modeling techniques implemented within it. We highlight the enhanced efficiency and expanded functionality of the new toolkit, setting it apart from its predecessors. The toolkit's advanced features and performance improvements make it a superior choice for language modeling tasks. To illustrate its practical application, we provide a case study demonstrating how the toolkit can be employed to construct and test a basic language model, showcasing its ease of use and effectiveness.

The problem of organizing information for multidocument summarization so that the generated summary is coherent has received relatively little attention. While sentence ordering for single document summarization can be determined from the ordering of sentences in the input article, this is not the case for multidocument summarization where summary sentences may be drawn from different input articles. In this paper, we propose a methodology for studying the properties of ordering information in the news genre and describe experiments done on a corpus of multiple acceptable orderings we developed for the task. Based on these experiments, we implemented a strategy for ordering information that combines constraints from chronological order of events and topical relatedness. Evaluation of our augmented algorithm shows a significant improvement of the ordering over two baseline strategies.

Just as syntax differentiates coherent sentences from scrambled word strings, sequential images must also use a cognitive system to distinguish coherent narratives from random strings of images. We conducted experiments analogous to two classic psycholinguistic studies to examine structure and semantics in sequential images. We compared Normal comic strips with both structure and meaning to sequences with Semantics Only, Structure Only, or randomly Scrambled panels. Experiment 1 used a target-monitoring paradigm, and found that RTs were slowest to target panels in Scrambled sequences, intermediate in Structural Only and Semantic Only sequences, and fastest in Normal sequences. Experiment 2 measured ERPs to the same strips. The largest N400 appeared in Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in Normal sequences. Together, these findings suggest that sequential image comprehension is guided by an interaction between a structure and meaning, broadly analogous to syntax and semantics in language.

Paste the appropriate copyright statement here. ACM now supports three different copyright statements: • ACM copyright: ACM holds the copyright on the work. This is the historical approach. • License: The author(s) retain copyright, but ACM receives an exclusive publication license. • Open Access: The author(s) wish to pay for the work to be open access. The additional fee must be paid to ACM. This text field is large enough to hold the appropriate release statement assuming it is single spaced in a sans-serif 7 point font. Every submission will be assigned their own unique DOI string to be included here. Abstract Twitch.tv, a streaming platform known for video game content, has grown tremendously since its inception in 2011. We examine communication practices in Twitch chats for the popular game Hearthstone, comparing massive chats with at least 10,000 concurrent viewers and small chats with fewer than 2,000 concurrent viewers. Due to the large scale and fast pace of massive chats, communication patterns no longer follow models developed in previous studies of computer-mediated communication. Rather than what other studies have described as communication breakdowns and information overload, participants in massive chats communicate in what we call “crowdspeak.”

GUS is the first o f a series o f experimental computer systems that we intend to construct as part o f a program of research on language understanding. In large measure, these systems will fill the role o f periodic progress reports, summarizing what we have learned, assessing the mutual coherence o f the various lines o f investigation we have been following, and saggestin# where more emphasis is needed in future work. GUS (Genial Understander System) is intended to engage a sympathetic and highly cooperative human in an English dialog, directed towards a specific goal within a very restricted domain o f discourse. As a starting point, G US was restricted to the role o f a travel agent in a conversation with a client who wants to make a simple return trip to a single city in California. There is good reason for restricting the domain o f discourse for a computer system which is to engage in an English dialog. Specializing the subject matter that the system can talk about permiis it to achieve some measure o f realism without encompassing all the possibilities o f human knowledge or o f the English language. It also provides the user with specific motivation for participating in the conversation, thus narrowing the range o f expectations that GUS must have about the user's purposes. A system restricted in this way will be more able to guide the conversation within the boundaries o f its competence. 1. Motivation and Design Issues Within its limitations, ous is able to conduct a more-or-less realistic dialog. But the outward behavior of this first system is not what makes it interesting or significant. There are, after all, much more convenient ways to plan a trip and, unlike some other artificial intelligence programs, (;us does not offer services or furnish information that are otherwise difficult or impossible to obtain. The system is i nteresting because of the phenomena of natural dialog that it attempts to model tThis work was done by the language understander project at the Xerox Palo Alto Research center. Additional affiliations: D. A. Norman, University of California, San Diego; H. Thompso6, University of California, Berkeley; and T. Winograd, Stanford University. Artificial Intelligence 8 0977), 155-173 Copyright © 1977 by North-Holland Publishing Company 156 D . G . BOBI~OW ET AL. and because of the principles of program organization around which it was de, Signed. Among the hallmarks of natural dialogs are unexpected and seemingly unpredictable sequences of events. We describe some of the forms that these can take below. "We then go on to discuss the modular design which makes the system re!atively insensitive t o the vagaries of ordinary conversation. 1.1. Problems of natural dialog The simple dialog shown in Fig. 1 illustrates some of the language-understanding problems we attacked. (The parenthesized numbers are for reference in the text). The problems illustrated in this figure, and described in the paragraphs below, include allowing both the client and the system to take the initiative, understanding indirect answers to questions, resolving anaphora, understanding fragments of sentences offered as answers to questions, and interpreting the discourse in the light of known conversational patterns. 1.1.1. Mixed initiative A typical contribution to a dialog, in addition to its more obvious functions, conveys an expectation about how the other participant will respond. This is clearest in the ease of a question, but it is true of all dialog. If one of the participants has very particular expectations and states them strongly whenever he speaks, and ff the other always responds in such a way as to meet the expectations conveyed, then the initiative remains with the first participant throughout. The success of interactive computer systems can often be traced to the skill with which their designers were able to assure them such a dominating position in the interaction. In natural conversations between humans, however, each participant usually assumes the initiative from time to time. Either clear expectations are not stated or simply not honored. GUS attempts to retain the initiative, but not to the extent of jeopardizing the natural flow of the conversation. To this extent it is a mixed-initiative system (see Carbonell [5, 6]). This is exemplified in the dialogue at (1) where the client volunteers more information than GUS requested. In addition to his destination, the client gives the date on which he wants to travel. Line (3) illustrates a ease where the client takes control of the conversation. GUS had found a potentially acceptable flight and asked for the client's approval. Instead of either giving or denying it, the client replied with a question of his own. 1.1.2. Indirect answers It is by no means always clear what constitutes an answer to a question. Frequently the purported answer is at best only a basis on which to infer the information requested. For example, when ous asks "Whatt ime do you want to leave?" it is seeking information to constrain the selection of a flight, client's res onse t o • • P " . this question, a t (2), does constrain the flight selection, b u t only indirectly. In Artificial Intelligence8 (1977), 155--17a : -

Whispered speech is a natural mode of speech in which voicing is absent – its acoustics differ significantly from normally spoken speech or so-called neutral speech, such that it is challenging to use only neutral speech to build speech processing and automatic recognition systems that can deal effectively with whisper. At the same time, humans can naturally produce and perceive whispered speech without explicit training. Tonal languages such as Mandarin present an interesting dilemma – tone is primarily encoded by pitch tracks which are absent during whispered speech, but humans can still tell tones apart. How humans manage to process whispered speech well without explicit training on it, whereas machine algorithms fail, is presently an unresolved question which could prove fruitful with study. This, however, is hindered by the lack of suitable, systematically collected corpora. We present iWhisper-Mandarin, a 25-hour parallel corpus of neutral and whispered Mandarin, designed to support research in linguistics and speech technology. We demonstrate and verify that earlier techniques applied to whispered speech from non-tonal languages also work with Mandarin, and present some preliminary studies on voice activity detection and whispered Mandarin speech recognition.

We propose detecting and responding to humor in spoken dialogue by extracting language and audio cues and subsequently feeding these features into a combined recurrent neural network (RNN) and logistic regression model. In this paper, we parse Switchboard phone conversations to build a corpus of punchlines and unfunny lines where punchlines precede laughter tokens in Switchboard transcripts. We create a combined RNN and logistic regression model that uses both acoustic and language cues to predict whether a conversational agent should respond to an utterance with laughter. Our model achieves an F1-score of 63.2 and accuracy of 73.9. This model outperforms our logistic language model (F1-score 56.6) and RNN acoustic model (59.4) as well as the final RNN model of D. Bertero, 2016 (52.9). Using our final model, we create a “laughbot” that audibly responds to a user with laughter when their utterance is classified as a punchline. A conversational agent outfitted with a humorrecognition system such as the one we present in this paper would be valuable as these agents gain utility in everyday life. Keywords—Chatbots; spoken natural language processing; deep learning; machine learning


Question: Uniqueness and coherence in produced dialogues.

In the realm of creating immersive and artistically compelling simulated environments, the Oz project at Carnegie Mellon University is pioneering a novel approach. This endeavor aims to construct captivating worlds populated by intelligent agents capable of exhibiting a wide range of behaviors. To achieve this, the project has developed an innovative agent architecture known as Tok, which is designed to support reactivity, goal-driven actions, emotional responses, and social interactions. In this paper, we outline the requirements and objectives of the Oz project, provide an overview of the Tok architecture, and detail the design of a specific agent that demonstrates the desired social behavior.

The Oz project is centered around four core components: a simulated physical environment, a set of automated agents, a user interface for human interaction, and a planner to orchestrate the user's experience. To ensure an engaging and believable experience for the user, the agents must possess a comprehensive set of capabilities. These include the ability to engage in goal-directed reactive behavior, express and manage emotional states, understand and enact social norms, and communicate through natural language. While each of these capacities can be tailored to the project's requirements, they must be integrated in a way that supports the creation of complex, dynamic, and believable agents.

In contrast to the complexity of real-world scenarios, Oz worlds can be simplified to serve artistic purposes. However, this simplification does not compromise the need for agents to handle uncertainties, respond promptly to stimuli, and operate in an environment populated by multiple agents. These challenges are similar to those faced by social robots in real-world settings, suggesting that the lessons learned from building broad agents in Oz could be applicable to the development of social robots.

The creation of broad agents, capable of integrating a wide array of capabilities, is an underexplored area of research. While there has been significant work on developing reactive systems, natural language processing, and emotion models, the integration of these diverse functionalities remains a challenge. The Oz project's focus on broad agent design and integration offers a unique perspective on this complex task. This paper provides an in-depth look at the structure of a specific agent designed to exhibit goal-directed behavior, emotional responses, and social interactions, highlighting the integration strategies employed in the Tok architecture. Further exploration of the integration challenges and solutions can be found in related literature.

This work not only contributes to the advancement of simulated worlds but also holds potential implications for the development of more sophisticated social robots and interactive systems. By pushing the boundaries of what is possible in artificial intelligence, the Oz project is paving the way for a new era of immersive and interactive experiences

Existing research on response generation for chatbot focuses on First Response Generation which aims to teach the chatbot to say the first response (e.g. a sentence) appropriate to the conversation context (e.g. the user’s query). In this paper, we introduce a new task Second Response Generation, termed as Improv chat, which aims to teach the chatbot to say the second response after saying the first response with respect the conversation context, so as to lighten the burden on the user to keep the conversation going. Specifically, we propose a general learning based framework and develop a retrieval based system which can generate the second responses with the users’ query and the chatbot’s first response as input. We present the approach to building the conversation corpus for Improv chat from public forums and social networks, as well as the neural networks based models for response matching and ranking. We include the preliminary experiments and results in this paper. This work could be further advanced with better deep matching models for retrieval base systems or generative models for generation based systems as well as extensive evaluations in real-life applications.

We conducted a laboratory-based observational study where pairs of people performed search tasks communicating verbally. Examination of the discourse allowed commonly used interactions to be identified for Spoken Conversational Search (SCS). We compared the interactions to existing models of search behaviour. We find that SCS is more complex and interactive than traditional search. This work enhances our understanding of different search behaviours and proposes research opportunities for an audio-only search system. Future work will focus on creating models of search behaviour for SCS and evaluating these against actual SCS systems.

Evaluating the output of NLG systems is notoriously difficult, and performin g assessments of text quality even more so. A range of automated and subject-based approaches to the evaluation of text quality h ave been taken, including comparison with a putative gold standard text, analysis of specific linguistic features of the output, expert review and task-based evaluation. In this paper we present the results of a variety of such approaches in the context of a case study application. We discuss the problems encountered in the implementation of each approach in the context of the literature, and propo se that a test based on the Turing test for machine intelligence offers a way forward in the evaluation of the subjective notion of text qua lity.

Writing is a skill of marking coherent words on a paper and composing a text. There are several criteria of good writing, two of them are cohesion and coherence. Research of cohesion and coherence writing has been conducted by using Centering Theory and Entity Transition Value method. However, the result can still be improved. Therefore, in this research, we tried to use Thematic Development approach which focused on the use of Theme and Rheme as a method to analyze coherence level of a paragraph, combined with CogNIAC rules and Centering Theory to analyze its cohesion. Besides improving the result of previous methods, this research aims to help users in evaluating and assessing their writing text. To achieve these objectives, the proposed method is compared to previous works as well as human judge. Based on the experiment, the proposed method yields average result of 91% which is nearly equivalent to human judge which is 92%. Thematic Development also yields better result compared to Centering Theory which is 29% and Entity Transition Value which is 0%, given the same data set of beginner and intermediate simple writing.

Understanding the ways in which information achieves widespread public awareness is a research question of significant interest. We consider whether, and how, the way in which the information is phrased — the choice of words and sentence structure — can affect this process. To this end, we develop an analysis framework and build a corpus of movie quotes, annotated with memorability information, in which we are able to control for both the speaker and the setting of the quotes. We find that there are significant differences between memorable and non-memorable quotes in several key dimensions, even after controlling for situational and contextual factors. One is lexical distinctiveness: in aggregate, memorable quotes use less common word choices, but at the same time are built upon a scaffolding of common syntactic patterns. Another is that memorable quotes tend to be more general in ways that make them easy to apply in new contexts — that is, more portable. We also show how the concept of “memorable language” can be extended across domains. To appear at ACL 2012; final version 1 Hello. My name is Inigo Montoya. Understanding what items will be retained in the public consciousness, and why, is a question of fundamental interest in many domains, including marketing, politics, entertainment, and social media; as we all know, many items barely register, whereas others catch on and take hold in many people’s minds. An active line of recent computational work has employed a variety of perspectives on this question. Building on a foundation in the sociology of diffusion [27, 31], researchers have explored the ways in which network structure affects the way information spreads, with domains of interest including blogs [1, 11], email [37], on-line commerce [22], and social media [2, 28, 33, 38]. There has also been recent research addressing temporal aspects of how different media sources convey information [23, 30, 39] and ways in which people react differently to information on different topics [28, 36]. Beyond all these factors, however, one’s everyday experience with these domains suggests that the way in which a piece of information is expressed — the choice of words, the way it is phrased — might also have a fundamental effect on the extent to which it takes hold in people’s minds. Concepts that attain wide reach are often carried in messages such as political slogans, marketing phrases, or aphorisms whose language seems intuitively to be memorable, “catchy,” or otherwise compelling. Our first challenge in exploring this hypothesis is to develop a notion of “successful” language that is precise enough to allow for quantitative evaluation. We also face the challenge of devising an evaluation setting that separates the phrasing of a message from the conditions in which it was delivered — highlycited quotes tend to have been delivered under compelling circumstances or fit an existing cultural, political, or social narrative, and potentially what appeals to us about the quote is really just its invocation of these extra-linguistic contexts. Is the form of the language adding an effect beyond or indepenar X iv :1 20 3. 63 60 v2 [ cs .C L ] 3 0 A pr 2 01 2 dent of these (obviously very crucial) factors? To investigate the question, one needs a way of controlling — as much as possible — for the role that the surrounding context of the language plays. The present work (i): Evaluating language-based memorability Defining what makes an utterance memorable is subtle, and scholars in several domains have written about this question. There is a rough consensus that an appropriate definition involves elements of both recognition — people should be able to retain the quote and recognize it when they hear it invoked — and production — people should be motivated to refer to it in relevant situations [15]. One suggested reason for why some memes succeed is their ability to provoke emotions [16]. Alternatively, memorable quotes can be good for expressing the feelings, mood, or situation of an individual, a group, or a culture (the zeitgeist): “Certain quotes exquisitely capture the mood or feeling we wish to communicate to someone. We hear them ... and store them away for future use” [10]. None of these observations, however, serve as definitions, and indeed, we believe it desirable to not pre-commit to an abstract definition, but rather to adopt an operational formulation based on external human judgments. In designing our study, we focus on a domain in which (i) there is rich use of language, some of which has achieved deep cultural penetration; (ii) there already exist a large number of external human judgments — perhaps implicit, but in a form we can extract; and (iii) we can control for the setting in which the text was used. Specifically, we use the complete scripts of roughly 1000 movies, representing diverse genres, eras, and levels of popularity, and consider which lines are the most “memorable”. To acquire memorability labels, for each sentence in each script, we determine whether it has been listed as a “memorable quote” by users of the widely-known IMDb (the Internet Movie Database), and also estimate the number of times it appears on the Web. Both of these serve as memorability metrics for our purposes. When we evaluate properties of memorable quotes, we compare them with quotes that are not assessed as memorable, but were spoken by the same character, at approximately the same point in the same movie. This enables us to control in a fairly fine-grained way for the confounding effects of context discussed above: we can observe differences that persist even after taking into account both the speaker and the setting. In a pilot validation study, we find that human subjects are effective at recognizing the more IMDbmemorable of two quotes, even for movies they have not seen. This motivates a search for features intrinsic to the text of quotes that signal memorability. In fact, comments provided by the human subjects as part of the task suggested two basic forms that such textual signals could take: subjects felt that (i) memorable quotes often involve a distinctive turn of phrase; and (ii) memorable quotes tend to invoke general themes that aren’t tied to the specific setting they came from, and hence can be more easily invoked for future (out of context) uses. We test both of these principles in our analysis of the data. The present work (ii): What distinguishes memorable quotes Under the controlled-comparison setting sketched above, we find that memorable quotes exhibit significant differences from nonmemorable quotes in several fundamental respects, and these differences in the data reinforce the two main principles from the human pilot study. First, we show a concrete sense in which memorable quotes are indeed distinctive: with respect to lexical language models trained on the newswire portions of the Brown corpus [21], memorable quotes have significantly lower likelihood than their nonmemorable counterparts. Interestingly, this distinctiveness takes place at the level of words, but not at the level of other syntactic features: the part-ofspeech composition of memorable quotes is in fact more likely with respect to newswire. Thus, we can think of memorable quotes as consisting, in an aggregate sense, of unusual word choices built on a scaffolding of common part-of-speech patterns. We also identify a number of ways in which memorable quotes convey greater generality. In their patterns of verb tenses, personal pronouns, and determiners, memorable quotes are structured so as to be more “free-standing,” containing fewer markers that indicate references to nearby text. Memorable quotes differ in other interesting aspects as well, such as sound distributions. Our analysis of memorable movie quotes suggests a framework by which the memorability of text in a range of different domains could be investigated. We provide evidence that such cross-domain properties may hold, guided by one of our motivating applications in marketing. In particular, we analyze a corpus of advertising slogans, and we show that these slogans have significantly greater likelihood at both the word level and the part-of-speech level with respect to a language model trained on memorable movie quotes, compared to a corresponding language model trained on non-memorable movie quotes. This suggests that some of the principles underlying memorable text have the potential to apply across different areas. Roadmap §2 lays the empirical foundations of our work: the design and creation of our movie-quotes dataset, which we make publicly available (§2.1), a pilot study with human subjects validating IMDbbased memorability labels (§2.2), and further study of incorporating search-engine counts (§2.3). §3 details our analysis and prediction experiments, using both movie-quotes data and, as an exploration of cross-domain applicability, slogans data. §4 surveys related work across a variety of fields. §5 briefly summarizes and indicates some future directions. 2 I’m ready for my close-up.

been developed during the last 20 years have proven to be quite successful, particularly in the domains of mathematics, science, and technology. They produce significant learning gains beyond classroom environments. They are capable of engaging most students’ attention and interest for hours. We have been working on a new generation of intelligent tutoring systems that hold mixedinitiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AUTOTUTOR is a conversational agent, with a talking head, that helps college students learn about computer literacy. ANDES, ATLAS, AND WHY2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations.

In this paper, we extend the state-of-the-art NLIDB system and present a dialog interface to relational databases. Dialog interface enables users to automatically exploit the semantic context of the conversation while asking natural language queries over RDBMS, thereby making it simpler to express complex questions in a natural, piece-wise manner. We propose novel ontology-driven techniques for addressing each of the dialog-specific challenges such as co-reference resolution, ellipsis resolution, and query disambiguation, and use them in determining the overall intent of the user query. We demonstrate the applicability and usefulness of dialog interface over two different domains viz. finance and healthcare.