Question: The Theory of Planned Behavior and Parental Engagement: A Conceptual Model for Reducing Academic Disparities.

This article aims to clarify the distinction between moderator and mediator variables in social psychological research, emphasizing their unique roles and the implications of conflating the two. Conceptually, moderators and mediators are distinct in their function within a causal framework. Moderators alter the strength or direction of the relationship between variables, whereas mediators explain the underlying mechanism through which an independent variable influences a dependent variable. Strategically, recognizing these differences is crucial for theory development and hypothesis testing, ensuring that the research design accurately reflects the theoretical model. The authors explore the implications of this distinction across various domains, including stress and coping, attitudes, and personality traits, demonstrating how the proper identification of moderators and mediators can enhance our understanding of complex phenomena. They also provide a guide to the statistical methods that can be employed to effectively analyze data involving both moderators and mediators, underscoring the importance of integrating these concepts into a comprehensive causal system. This clarification not only serves an educational purpose but also offers practical guidance for researchers in applying the moderator-mediator distinction to a broad range of psychological phenomena, thereby enriching the field's theoretical and empirical contributions.

The question at hand pertains to the Theory of Planned Behavior and parental involvement as a framework for addressing achievement gaps. The similar document discusses the importance of distinguishing between moderator and mediator variables in social psychological research, a concept that can be applied to understanding the factors influencing educational outcomes. By clearly defining the roles of moderators and mediators, researchers can more accurately identify the mechanisms through which parental involvement and other variables impact student achievement, thereby contributing to the development of effective interventions to close the achievement gaps. The detailed exploration of statistical methods for analyzing moderator and mediator effects provides a practical tool for educational researchers seeking to apply the Theory of Planned Behavior and other social psychological theories to the study of academic achievement.

In a 3-wave longitudinal study (with assessments 2 years apart) involving 186 early adolescents (M ages of approximately 9.3, 11.4, and 13.4), the hypothesis that parental warmth/positive expressivity predicts children's effortful control (EC) (a temperamental characteristic contributing to emotion regulation) 2 years later, which in turn predicts low levels of externalizing problems another 2 years later, was examined. The hypothesis that children's EC predicts parenting over time was also examined. Parents were observed interacting with their children; parents and teachers reported children's EC and externalizing problems; and children's persistence was assessed behaviorally. Children's EC mediated the relation between positive parenting and low levels of externalizing problems (whereas there was no evidence that children's EC predicted parenting).

Learning predictors for student retention is very difficult. After reviewing the literature, it is evident that there is considerable room for improvement in the current state of the art. As shown in this paper, improvements are possible if we (a) explore a wide range of learning methods; (b) take care when selecting attributes; (c) assess the efficacy of the learned theory not just by its median performance, but also by the variance in that performance; (d) study the delta of student factors between those who stay and those who are retained. Using these techniques, for the goal of predicting if students will remain for the first three years of an undergraduate degree, the following factors were found to be informative: family background and family’s social-economic status, high school GPA

Using data collected over a 6-year period on a sample of 1,039 European American children, 550 African American children, and 401 Hispanic children from the children of the National Longitudinal Survey of Youth, this study assessed whether maternal emotional support of the child moderates the relation between spanking and behavior problems. Children were 4–5 years of age in the first of 4 waves of data used (1988, 1990, 1992, 1994). At each wave, mothers reported their use of spanking and rated their children’s behavior problems. Maternal emotional support of the child was based on interviewer observations conducted as part of the Home Observation for Measurement of the Environment. For each of the 3 racial-ethnic groups, spanking predicted an increase in the level of problem behavior over time, controlling for income-needs ratio and maternal emotional support. Maternal emotional support moderated the link between spanking and problem behavior. Spanking was associated with an increase in behavior problems over time in the context of low levels of emotional support, but not in the context of high levels of emotional support. This pattern held for all 3 racial-ethnic groups.

Insights from appraisal theories of emotion are used to integrate elements of theories on collective action. Three experiments with disadvantaged groups systematically manipulated procedural fairness (Study 1), emotional social support (Study 2), and instrumental social support (Study 3) to examine their effects on collective action tendencies through group-based anger and group efficacy. Results of structural equation modeling showed that procedural fairness and emotional social support affected the group-based anger pathway (reflecting emotion-focused coping), whereas instrumental social support affected the group efficacy pathway (reflecting problem-focused coping), constituting 2 distinct pathways to collective action tendencies. Analyses of the means suggest that collective action tendencies become stronger the more fellow group members "put their money where their mouth is." The authors discuss how their dual pathway model integrates and extends elements of current approaches to collective action.

Recent policy initiatives in England have focused on promoting ‘interactive’ teaching in schools, with a clear expectation that this will lead to improvements in learning. This expectation is based on the perceived success of such approaches in other parts of the world.At the same time, there has been a large investment in Information and Communication Technology (ICT) resources, and particularly in interactive whiteboard technology. This paper explores the idea of interactive teaching in relation to the interactive technology which might be used to support it. It explains the development of a framework for the detailed analysis of teaching and learning in activity settings which is designed to represent the features and relationships involved in interactivity. When applied to a case study of interactive teaching during a lesson involving a variety of technology-based activities, the framework reveals a confusion of purpose in students’ use of an ICT resource that limits the potential for learning when students are working independently. Discussion of relationships between technical and pedagogical interactivity points a way forward concerning greater focus on learning goals during activity in order to enable learners to be more autonomous in exploiting ICT’s affordances, and the conclusion identifies the variables and issues which need to be considered in future research which will illuminate this path.

To explore possible reasons for low self-identification rates among undergraduates with learning disabilities (LD), we asked students (38 with LD, 100 without LD) attending two large, public, research-intensive universities to respond to a questionnaire designed to assess stereotypes about individuals with LD and conceptions of ability. Responses were coded into six categories of stereotypes about LD (low intelligence, compensation possible, process deficit, nonspecific insurmountable condition, working the system, and other), and into three categories of conceptions of intelligence (entity, incremental, neither). Consistent with past findings, the most frequent metastereotype reported by individuals in both groups related to generally low ability. In addition, students with LD were more likely to espouse views of intelligence as a fixed trait. As a whole, the study's findings have implications for our understanding of factors that influence self-identification and self-advocacy at the postsecondary level.

Article history: Received 24 May 2015 Received in revised form 2 October 2016 Accepted 18 November 2016 In our paper we develop and test the argument that intent to i-vote (to use on-line voting systems) drives intent to vote, while intent to i-vote is influenced by four key attitudes: performance expectation, perception on ease of use, trust in the internet and trust in the government.We show that these findings contradict thosewhich exclusively identified economical, legal, and cultural drivers to enhance democratic participation in the Central and Eastern European region. Rooted cardinally in the Technology Acceptance Model (TAM) six hypotheses were set, and then tested with partial least square (PLS) structural equation modelling. In the context of young, educated and internet-ready Hungarian voters the testing of the hypotheses has shown high level of on-line voting intent and that perception of on-line voting would enhance voting desire amongst young Hungarian internet users. Also, our findings show that performance expectation, perception on ease of use and trust in the internet are positively associated with i-voting intent. © 2016 Elsevier Inc. All rights reserved.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org. This article concerns theories about why and how information technology affects organizational life. Good theory guides research, which, when applied, increases the likelihood that information technology will be employed with desirable consequences for users, organizations, and other interested parties. But what is a good theory? Theories are often evaluated in terms of their content-the specific concepts used and the human values served. This article examines theories in terms of their structures-theorists' assumptions about the nature and direction of causal influence. Three dimensions of causal structure are considered-causal agency, logical structure, and level of analysis. Causal agency refers to beliefs about the nature of causality: whether external forces cause change, whether people act purposefully to accomplish intended objectives, or whether changes emerge unpredictably from the interaction of people and events. Logical structure refers to the temporal aspect of theory-static versus dynamic-and to the logical relationships between the "causes" and the outcomes. Level of analysis refers to the entities about which the theory poses concepts and relationships-individuals, groups, organizations, and society. While there are many possible structures for good theory about the role of information technology in organizational change, only a few of these structures can be seen in current theorizing. Increased awareness of the options, open discussion of their advantages and disadvantages , and explicit characterization of future theoretical statements in terms of the dimensions and categories discussed here should, we believe, promote the development of better theory.

Interest in online education is surging, as dramatized by the success of Khan Academy and recent Stanford online courses, but the technology for online education is in its infancy. Crowdsourcing mechanisms will likely be essential in order to reach the full potential of this medium. This paper sketches some of the challenges and directions we hope HCOMP researchers will ad-

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.. American Educational Research Association is collaborating with JSTOR to digitize, preserve and extend access to Educational Evaluation and Policy Analysis. This article examines the effects of professional development on teachers' instruction. Using a purposefully selected sample of about 207 teachers in 30 schools, in 10 districts infive states, we examine features of teachers' professional development and its effects on changing teaching practice in mathematics and science from 1996-1999. We found that professional developmentfocused on specific instructional practices increases teachers' use of those practices in the classroom. Furthermore, we found that specificfeatures, such as active learning opportunities, increase the effect of the professional development on teacher's instruction. What are the characteristics of professional development that affect teaching practice? This study adds to the knowledge base on effective professional development. The success of standards-based reform depends on teachers' ability to foster both basic knowledge and advanced thinking and problem solving among their students (Loucks-Desimone et al. Education 1999a), and surveys of teachers about their preservice preparation and in-service professional development experiences (e.g., Carey & Frechtling, 1997). In addition, there is a considerable amount of literature describing "best practices" in professional development, drawing on expert experiences (e.g., Loucks-Horsley et al., 1998). A professional consensus is emerging about particular characteristics of "high quality" professional development. These characteristics include a focus on content and how students learn content; in-depth, active learning opportunities; links to high standards, opportunities for teachers to engage in leadership roles; extended duration ; and the collective participation of groups of teachers from the same school, grade, or department. Although lists of characteristics such as these commonly appear in the literature on effective professional development, there is little direct evidence on the extent to which these characteristics are related to better teaching and increased student achievement. Some studies conducted over the past decade suggest that professional development experiences that share all or most of these characteristics can have a substantial, positive influence on teachers' classroom practice and student achieve-A few recent studies have begun to examine the relative importance of specific characteristics of professional development. Several studies have found that the intensity and …


Question: A dialogue management strategy for specialized domain question answering systems: does this meet your query needs?

This document discusses Microsoft Research Redmond's inaugural participation in the Text REtrieval Conference (TREC) question answering track. They adopted a data-intensive approach to web question answering, adjusting their system specifically for TREC QA. Two submissions were made for the primary QA track, designated as AskMSR and AskMSR2. The report highlights their exploration of data-driven methods for enhancing question answering capabilities on the web. A separate report within the same volume covers Microsoft Research Cambridge's entries for the filtering and Web tracks. The focus on data-intensive techniques for question answering reflects a strategic direction aimed at improving the performance and accuracy of automated responses in restricted domain question answering systems, aligning with the broader goals of advancing dialogue management in such systems.

and engineering J. Chu-Carroll J. Fan N. Schlaefer W. Zadrozny A key requirement for high-performing question-answering (QA) systems is access to high-quality reference corpora from which answers to questions can be hypothesized and evaluated. However, the topic of source acquisition and engineering has received very little attention so far. This is because most existing systems were developed under organized evaluation efforts that included reference corpora as part of the task specification. The task of answering Jeopardy!i questions, on the other hand, does not come with such a well-circumscribed set of relevant resources. Therefore, it became part of the IBM Watsoni effort to develop a set of well-defined procedures to acquire high-quality resources that can effectively support a high-performing QA system. To this end, we developed three procedures, i.e., source acquisition, source transformation, and source expansion. Source acquisition is an iterative development process of acquiring new collections to cover salient topics deemed to be gaps in existing resources based on principled error analysis. Source transformation refers to the process in which information is extracted from existing sources, either as a whole or in part, and is represented in a form that the system can most easily use. Finally, source expansion attempts to increase the coverage in the content of each known topic by adding new information as well as lexical and syntactic variations of existing information extracted from external large collections. In this paper, we discuss the methodology that we developed for IBM Watson for performing acquisition, transformation, and expansion of textual resources. We demonstrate the effectiveness of each technique through its impact on candidate recall and on end-to-end QA performance.

The precision of the answer is now essential for a question answering system, because of the large amount of free texts on the Internet. Attempting to achieve a high precision, we propose a question answering system supported by case grammar theory and based on VerbNet frames. It extracts the syntactic, thematic and semantic information from the question to filter out unmatched sentences in semantic level and to extract answer chunk (a phrase or a word that can answer the question) from the answer sentence. VerbNet is applied in our system to detect the verb frames in question and candidate sentences, so that the syntactic and thematic information as well as semantic information can be therefore obtained. Our question answering system works well especially for answering factoid questions. The experiments show that our approach is able to filter out semantically unmatched sentences effectively and therefore rank the correct answer (s) higher in the result list.

Answering questions correctly from standardized eighth-grade science tests is itself a test of machine intelligence.

This paper proposes a novel approach for building a Natural Language Interface to a Relational Database (NLI-RDB) using Conversational Agent (CA), Information Extraction (IE) and Object Relational Mapping (ORM) framework. The CA will help in disambiguating the user’s queries and guiding the user interaction. IE will play an important role in named entities extraction in order to map Natural Language queries into database queries. The ORM framework i.e. the Hibernate framework resolves the impedance mismatch between the Object Oriented Paradigms (OOP) and Relational Databases (RDBs) i.e. OOP concepts differ from RDB concepts, thus it reduces the complexity in generating SQL statements. Also, by utilizing ORM framework, the RDBs entities are mapped into real world objects, which bring the RDBs a step closer to the user. In addition, the ORM framework simplify the interaction between OOP and RDBs. The developed NLI-RDB system allows the user to interact with objects directly in natural language and through navigation, rather than by using SQL statements. This direct interaction tends to be easier and more acceptable for humans whom are nor technically orientated and have no SQL knowledge. The NLI-RDB system also offers friendly and interactive user interface in order to refine the query generated automatically. The NLI-RDB system has been evaluated by a group of participants through a combination of qualitative and quantitative measures. The experimental results show good performance of the prototype and excellent user’s satisfaction.

This paper describes a question answering system that is designed to capitalize on the tremendous amount of data that is now available online. Most question answering systems use a wide variety of linguistic resources. We focus instead on the redundancy available in large corpora as an important resource. We use this redundancy to simplify the query rewrites that we need to use, and to support answer mining from returned snippets. Our system performs quite well given the simplicity of the techniques being utilized. Experimental results show that question answering accuracy can be greatly improved by analyzing more and more matching passages. Simple passage ranking and n-gram extraction techniques work well in our system making it efficient to use with many backend retrieval engines.

The Carnegie Mellon Communicator system helps users create complex travel itineraries through a conversational interface. Itineraries consist of (multi-leg) flights, hotel and car reservations and are built from actual travel information for North America, obtained from the Web. The system manages dialog using a schema-based approach. Schemas correspond to major units of task information (such as a flight leg) and define conversational topics, or foci of interaction, meaningful to the user.

Teaching machines to read natural language documents remains an elusive challenge. Machine reading systems can be tested on their ability to answer questions posed on the contents of documents that they have seen, but until now large scale training and test datasets have been missing for this type of evaluation. In this work we define a new methodology that resolves this bottleneck and provides large scale supervised reading comprehension data. This allows us to develop a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure.

Community question answering services (CQAS) (e.g., Yahoo! Answers) provides a platform where people post questions and answer questions posed by others. Previous works analyzed the answer quality (AQ) based on answer-related features, but neglect the question-related features on AQ. Previous work analyzed how asker- and question-related features affect the question quality (QQ) regarding the amount of attention from users, the number of answers and the question solving latency, but neglect the correlation between QQ and AQ (measured by the rating of the best answer), which is critical to quality of service (QoS). We handle this problem from two aspects. First, we additionally use QQ in measuring AQ, and analyze the correlation between a comprehensive list of features (including answer-related features) and QQ. Second, we propose the first method that estimates the probability for a given question to obtain high AQ. Our analysis on the Yahoo! Answers trace confirmed that the list of our identified features exert influence on AQ, which determines QQ. For the correlation analysis, the previous classification algorithms cannot consider the mutual interactions between multiple ($>$ 2) classes of features. We then propose a novel Coupled Semi-Supervised Mutual Reinforcement-based Label Propagation (CSMRLP) algorithm for this purpose. Our extensive experiments show that CSMRLP outperforms the Mutual Reinforcement-based Label Propagation (MRLP) and five other traditional classification algorithms in the accuracy of AQ classification, and the effectiveness of our proposed method in AQ prediction. Finally, we provide suggestions on how to create a question that will receive high AQ, which can be exploited to improve the QoS of CQAS.

The study of artificial intelligence can be simplified into one goal: trying to mimic/enhance human senses. This paper attempts to combine computer vision and natural language processing to create a question answer system. This system takes a question and an image as input and outputs a response to the answer based on how the RCNN understands the question asked. The system correlates the question with the image by leveraging attention and memory mechanisms. Mentor: Arun Chaganty

In a spoken dialog system, determining which action a machine should take in a given situation is a difficult problem because automatic speech recognition is unreliable and hence the state of the conversation can never be known with certainty. Much of the research in spoken dialog systems centres on mitigating this uncertainty and recent work has focussed on three largely disparate techniques: parallel dialog state hypotheses, local use of confidence scores, and automated planning. While in isolation each of these approaches can improve action selection, taken together they currently lack a unified statistical framework that admits global optimization. In this paper we cast a spoken dialog system as a partially observable Markov decision process (POMDP). We show how this formulation unifies and extends existing techniques to form a single principled framework. A number of illustrations are used to show qualitatively the potential benefits of POMDPs compared to existing techniques, and empirical results from dialog simulations are presented which demonstrate significant quantitative gains. Finally, some of the key challenges to advancing this method – in particular scalability – are briefly outlined. 2006 Elsevier Ltd. All rights reserved.


Question: A dialogue management strategy for specialized domain question answering systems: Does this meet your needs?

This paper presents an innovative technique for constructing convolution kernels on sets composed of discrete structures, such as strings, trees, and graphs. The method is designed to be iterative, enabling the creation of a kernel on an infinite set from kernels involving the set's generators. This approach generalizes the family of radial basis kernels and can also be utilized to define kernels in the form of joint Gibbs probability distributions. The kernels can be derived from various sources, including hidden Markov random fields, generalized regular expressions, pair-HMMs, or ANOVA decompositions. The application of this method raises intriguing questions concerning the theory of infinitely divisible positive definite functions. To support the validity of the method, the paper delves into the fundamental theories of infinitely divisible positive definite functions and reproducing kernel Hilbert spaces, providing a comprehensive review of these concepts. The exploration of these theoretical foundations not only validates the method but also opens up new avenues for research in dialogue management for restricted domain question answering systems, potentially enhancing their effectiveness and adaptability.

This research focuses on natural, robot-led, human-robot interaction that enables a robot to discover what drinks a barman can prepare through continuous application of speech recognition, understanding and generation. Speech was recognised using Google Cloud’s speech to text API, understood by matching either the object or main verb of a sentence against a list of key words and, finally, generated using templates with variable parts. The difficulty lies in the large quantity of key words, as they are based on the properties of the ordered drinks. The results show that having the aforementioned interaction works well to some extent, i.e. the naturalness of the interaction was ranked 5.5 on average. Furthermore, the obtained precision when identifying the unavailable drinks was 0.625 and the obtained recall was 1, resulting in an F1 measure of 0.769.

This article describes My Science Tutor (MyST), an intelligent tutoring system designed to improve science learning by students in 3rd, 4th, and 5th grades (7 to 11 years old) through conversational dialogs with a virtual science tutor. In our study, individual students engage in spoken dialogs with the virtual tutor Marni during 15 to 20 minute sessions following classroom science investigations to discuss and extend concepts embedded in the investigations. The spoken dialogs in MyST are designed to scaffold learning by presenting open-ended questions accompanied by illustrations or animations related to the classroom investigations and the science concepts being learned. The focus of the interactions is to elicit self-expression from students. To this end, Marni applies some of the principles of Questioning the Author, a proven approach to classroom conversations, to challenge students to think about and integrate new concepts with prior knowledge to construct enriched mental models that can be used to explain and predict scientific phenomena. In this article, we describe how spoken dialogs using Automatic Speech Recognition (ASR) and natural language processing were developed to stimulate students' thinking, reasoning and self explanations. We describe the MyST system architecture and Wizard of Oz procedure that was used to collect data from tutorial sessions with elementary school students. Using data collected with the procedure, we present evaluations of the ASR and semantic parsing components. A formal evaluation of learning gains resulting from system use is currently being conducted. This paper presents survey results of teachers' and children's impressions of MyST.

Different software systems for automatic question answering have been developed in recent years. Some systems perform well on specific domains, but may not be appropriate for other domains. As the complexity and scaling of such information systems become ever greater, it is much more challenging to effectively and efficiently determine which toolkits, algorithms, knowledge bases or other resources should be integrated into a system so that one can achieve a desired or optimal level of performance on a given task. In this working notepaper, we present a generic framework that can be used for any machinereading task and automatically find the best configuration of algorithmic components as well as values of their corresponding parameters. Although we have designed the framework for all QA4MRE-2013 tasks (i.e. Main task, Biomedical about Alzheimer’s and Entrance Exam), our analysis will mostly focus on Biomedical about Alzheimer’s task. We introduce the Configuration Space Exploration (CSE) framework, an extension to the Unstructured Information Management Architecture (UIMA) which provides a general distributed solution for building and exploring possible configurations for any intelligent information system. For the Biomedial about Alzheimer’s task, CSE was used to generate more than 1000 different configurations from existing components; we selected the 3 best runs for submission. We achieved an average c@1 of 0.27; our highest score was 0.60 for reading-test-1, and our lowest was 0.0 for reading-test3. We further enhanced the system by introducing point-wise mutual information (PMI) scoring for answer ranking, which produced an average c@1 of 0.4025, with a highest score of 0.77 for reading test-1 and a lowest score of 0.2 for reading test-2.

We present Aardvark, a social search engine. With Aardvark, users ask a question, either by instant message, email, web input, text message, or voice. Aardvark then routes the question to the person in the user's extended social network most likely to be able to answer that question. As compared to a traditional web search engine, where the challenge lies in finding the right document to satisfy a user's information need, the challenge in a social search engine like Aardvark lies in finding the right person to satisfy a user's information need. Further, while trust in a traditional search engine is based on authority, in a social search engine like Aardvark, trust is based on intimacy. We describe how these considerations inform the architecture, algorithms, and user interface of Aardvark, and how they are reflected in the behavior of Aardvark users.

Massive Online Open Courses (MOOCs) which were introduced in 2008 has since drawn attention around the world for both its advantages as well as criticism on its drawbacks. One of the issues in MOOCs which is the lack of interactivity with the instructor has brought conversational bot into the picture to fill in this gap. In this study, a prototype of MOOCs conversational bot, MOOC-bot is being developed and integrated into MOOCs website to respond to the learner’s enquiries using text or speech input. MOOC-bot is using the popular Artificial Intelligence Markup Language (AIML) to develop its knowledge base, leverage from AIML’s capability to deliver appropriate responses and can be quickly adapted to new knowledge domains. The system architecture of MOOC-bot consists of knowledge base along with AIML interpreter, chat interface, MOOCs website and Web Speech API to provide speech recognition and speech synthesis capability. The initial MOOC-bot prototype has the general knowledge from the past Loebner Prize winner ALICE, course’s frequent asked questions, and a course’s content offered by Universiti Teknikal Malaysia Melaka (UTeM). The evaluation of MOOC-bot based on the past competition questions from Chatterbox Challenge (CBC) and Loebner Prize has shown that it was able to provide correct answers most of the time during the test and demonstrated the capability to prolong the conversation. The advantages of MOOCbot such as able to provide 24-hour service that can serve different time zones, able to have knowledge in multiple domains, and can be shared by multiple sites simultaneously have outweighed its existing limitations.

We present a game-based interface for acquiring common sense knowledge. In addition to being interactive and entertaining, our interface guides the knowledge acquisition process to learn about the most salient characteristics of a particular concept. We use statistical classification methods to discover the most informative characteristics in the Open Mind Common Sense knowledge base, and use these characteristics to play a game of 20 Questions with the user. Our interface also allows users to enter knowledge more quickly than a more traditional knowledge-acquisition interface. An evaluation showed that users enjoyed the game and that it increased the speed of knowledge acquisition.

This special issue highlights research articles that apply machine learning to robots and other systems that interact with users through more than one modality, such as speech, gestures, and vision. For example, a robot may coordinate its speech with its actions, taking into account (audio-)visual feedback during their execution. Machine learning provides interactive systems with opportunities to improve performance not only of individual components but also of the system as a whole. However, machine learning methods that encompass multiple modalities of an interactive system are still relatively hard to find. The articles in this special issue represent examples that contribute to filling this gap.

Search-oriented conversational systems rely on information needs expressed in natural language (NL). We focus here on the understanding of NL expressions for building keywordbased queries. We propose a reinforcementlearning-driven translation model framework able to 1) learn the translation from NL expressions to queries in a supervised way, and, 2) to overcome the lack of large-scale dataset by framing the translation model as a word selection approach and injecting relevance feedback as a reward in the learning process. Experiments are carried out on two TREC datasets. We outline the effectiveness of our approach.

The Tactical Language and Culture Training System (TLCTS) helps people quickly acquire communicative skills in foreign languages and cultures. More than 20,00 0 learners worldwide have used TLCTS courses. TLCTS utili zes artificial intelligence technologies in multiple wa ys: during the authoring process, and at run time to process l arner speech, interpret learner actions, control the resp on e of non-player characters, and evaluate and assess lear ner performance and proficiency. This paper describes the architecture of TLCTS and the artificial intelligence techn ologies that it employs, and presents results from multiple evaluation studies that demonstrate the benefits of learn ing foreign language and culture using this approach.

This article presents a deep analysis of the influence of context information on dialogue act recognition. We performed experiments on the annotated subsets of three different corpora: the widely explored Switchboard Dialog Act Corpus, as well as the unexplored LEGO and Cambridge Restaurant corpora. In contrast with previous work, especially in what concerns the Switchboard corpus, we used an event-based classification approach, using Support Vector Machines (SVMs), instead of the more common sequential approaches, such as Hidden Markov Models (HMMs). We opted for such an approach so that we could control the amount of provided context information and, thus, explore its range of influence. Our base features consist of n-grams, punctuation, and whwords. Context information is obtained from previous utterances and provided in three ways – n-grams, n-grams tagged with relative position, and dialogue act classifications. A comparative study was conducted to evaluate the performance of the three approaches. From it, we were able to assess the importance of context information on dialogue act recognition, as well as its range of influence for each of the three selected representations. In addition to the conclusions originated by the analysis, this work also produced results that advance the state-of-the-art, especially considering previous work on the Switchboard corpus. Furthermore, since, to our knowledge, the Email addresses: eugenio.ribeiro@l2f.inesc-id.pt (Eugénio Ribeiro), ricardo.ribeiro@inesc-id.pt (Ricardo Ribeiro), david.matos@inesc-id.pt (David Martins de Matos) Preprint submitted to Computer Speech and Language June 3, 2015 ar X iv :1 50 6. 00 83 9v 1 [ cs .C L ] 2 J un 2 01 5 remaining datasets had not been previously explored for this task, our experiments can be used as baselines for future work on those corpora.


Question: DOES THIS HELP YOU? Moving towards dialogue management in restricted domain question answering systems.

This document delves into the principles underlying Statistical Learning Theory, a critical framework for understanding the mechanisms and effectiveness of learning algorithms in machine learning. Unlike more empirical approaches, Statistical Learning Theory offers a theoretical foundation for evaluating and optimizing the performance of learning models. It explores the concepts of risk minimization, model complexity, and the trade-off between bias and variance. The theory is pivotal in addressing questions related to the generalization ability of a model, i.e., how well a model can make predictions on unseen data. This is achieved through the introduction of key concepts such as the Vapnik-Chervonenkis (VC) dimension, which quantifies the capacity of a model to learn from data. Moreover, the document discusses the role of the structural risk minimization (SRM) principle, which aims to find a balance between model complexity and training error to prevent overfitting. The SRM principle is essential in selecting the most appropriate model from a given hypothesis space. By providing a rigorous mathematical framework, Statistical Learning Theory not only aids in the design and analysis of learning algorithms but also in understanding the limitations and potential of machine learning models in various applications, including dialogue management systems for restricted domain question answering. This theoretical grounding is crucial for advancing the capabilities of such systems, ensuring they can efficiently and accurately respond to user queries within their designated domains.

Critiquing-based recommender systems elicit users’ feedback, called critiques, which they made on the recommended items. This conversational style of interaction is in contract to the standard model where users receive recommendations in a single interaction. Through the use of the critiquing feedback, the recommender systems are able to more accurately learn the users’ profiles, and therefore suggest better recommendations in the subsequent rounds. Critiquing-based recommenders have been widely studied in knowledge-, content-, and preference-based recommenders and are beginning to be tried in several online websites, such as MovieLens. This article examines the motivation and development of the subject area, and offers a detailed survey of the state of the art concerning the design of critiquing interfaces and development of algorithms for critiquing generation. With the help of categorization analysis, the survey reveals three principal branches of critiquing based recommender systems, using respectively natural language based, system-suggested, and user-initiated critiques. Representative example systems will be presented and analyzed for each branch, and their respective pros and cons will be discussed. Subsequently, a hybrid framework is developed to unify the advantages of different methods and overcome their respective limitations. Empirical findings from user studies are further presented, indicating how hybrid critiquing supports could effectively enable end-users to achieve more confident decisions. Finally, the article will point out several future trends to boost the advance of critiquing-based recommenders.

This paper describes Movie-DiC a Movie Dialogue Corpus recently collected for research and development purposes. The collected dataset comprises 132,229 dialogues containing a total of 764,146 turns that have been extracted from 753 movies. Details on how the data collection has been created and how it is structured are provided along with its main statistics and characteristics.

RDF question/answering (Q/A) allows users to ask questions in natural languages over a knowledge base represented by RDF. To answer a national language question, the existing work takes a two-stage approach: question understanding and query evaluation. Their focus is on question understanding to deal with the disambiguation of the natural language phrases. The most common technique is the joint disambiguation, which has the exponential search space. In this paper, we propose a systematic framework to answer natural language questions over RDF repository (RDF Q/A) from a graph data-driven perspective. We propose a semantic query graph to model the query intention in the natural language question in a structural way, based on which, RDF Q/A is reduced to subgraph matching problem. More importantly, we resolve the ambiguity of natural language questions at the time when matches of query are found. The cost of disambiguation is saved if there are no matching found. We compare our method with some state-of-the-art RDF Q/A systems in the benchmark dataset. Extensive experiments confirm that our method not only improves the precision but also speeds up query performance greatly.

In a spoken dialog system, determining which action a machine should take in a given situation is a difficult problem because automatic speech recognition is unreliable and hence the state of the conversation can never be known with certainty. Much of the research in spoken dialog systems centres on mitigating this uncertainty and recent work has focussed on three largely disparate techniques: parallel dialog state hypotheses, local use of confidence scores, and automated planning. While in isolation each of these approaches can improve action selection, taken together they currently lack a unified statistical framework that admits global optimization. In this paper we cast a spoken dialog system as a partially observable Markov decision process (POMDP). We show how this formulation unifies and extends existing techniques to form a single principled framework. A number of illustrations are used to show qualitatively the potential benefits of POMDPs compared to existing techniques, and empirical results from dialog simulations are presented which demonstrate significant quantitative gains. Finally, some of the key challenges to advancing this method – in particular scalability – are briefly outlined. 2006 Elsevier Ltd. All rights reserved.

The Carnegie Mellon Communicator system helps users create complex travel itineraries through a conversational interface. Itineraries consist of (multi-leg) flights, hotel and car reservations and are built from actual travel information for North America, obtained from the Web. The system manages dialog using a schema-based approach. Schemas correspond to major units of task information (such as a flight leg) and define conversational topics, or foci of interaction, meaningful to the user.

This paper summarizes our ongoing project on developing an architecture for a robot that can acquire new words and their meanings while engaging in multidomain dialogues. These two functions are crucial in making conversational service robots work in real tasks in the real world. Household robots and office robots need to be able to work in multiple task domains and they also need to engage in dialogues in multiple domains corresponding to those task domains. Lexical acquisition is necessary because speech understanding cannot be done without enough knowledge on words that are possibly spoken in the task domain. Our architecture is based on a multi-expert model in which multiple domain experts are employed and one of them is selected based on the user utterance and the situation to engage in the control of the dialogue and physical behaviors. We incorporate experts that have an ability to acquire new lexical entries and their meanings grounded on the physical world through spoken interactions. By appropriately selecting those experts, lexical acquisition in multi-domain dialogues becomes possible. An example robotic system based on this architecture that can acquire object names and location names demonstrates the viability of the architecture.

Natural language generation (NLG) is a critical component of spoken dialogue and it has a significant impact both on usability and perceived quality. Most NLG systems in common use employ rules and heuristics and tend to generate rigid and stylised responses without the natural variation of human language. They are also not easily scaled to systems covering multiple domains and languages. This paper presents a statistical language generator based on a semantically controlled Long Short-term Memory (LSTM) structure. The LSTM generator can learn from unaligned data by jointly optimising sentence planning and surface realisation using a simple cross entropy training criterion, and language variation can be easily achieved by sampling from output candidates. With fewer heuristics, an objective evaluation in two differing test domains showed the proposed method improved performance compared to previous methods. Human judges scored the LSTM system higher on informativeness and naturalness and overall preferred it to the other systems.

The human-generated question-answer pairs in the Web social communities are of great value for the research of automatic question-answering technique. Due to the large amount of noise information involved in such corpora, it is still a problem to detect the answers even though the questions are exactly located. Quantifying the semantic relevance between questions and their candidate answers is essential to answer detection in social media corpora. Since both the questions and their answers usually contain a small number of sentences, the relevance modeling methods have to overcome the problem of word feature sparsity. In this article, the deep learning principle is introduced to address the semantic relevance modeling task. Two deep belief networks with different architectures are proposed by us to model the semantic relevance for the question-answer pairs. According to the investigation of the textual similarity between the community-driven question-answering (cQA) dataset and the forum dataset, a learning strategy is adopted to promote our models’ performance on the social community corpora without hand-annotating work. The experimental results show that our method outperforms the traditional approaches on both the cQA and the forum corpora.

This paper investigates the problem of bootstrapping a statistical dialogue manager without access to training data and proposes a new probabilistic agenda-based method for simulating user behaviour. In experiments with a statistical POMDP dialogue system, the simulator was realistic enough to successfully test the prototype system and train a dialogue policy. An extensive study with human subjects showed that the learned policy was highly competitive, with task completion rates above 90%. 1 Background and Introduction 1.1 Bootstrapping Statistical Dialogue Managers One of the key advantages of a statistical approach to Dialogue Manager (DM) design is the ability to formalise design criteria as objective reward functions and to learn an optimal dialogue policy from real dialogue data. In cases where a system is designed from scratch, however, it is often the case that no suitable in-domain data is available for training the DM. Collecting dialogue data without a working prototype is problematic, leaving the developer with a classic chicken-and-egg problem. Wizard-of-Oz (WoZ) experiments can be carried out to record dialogues, but they are often time-consuming and the recorded data may show characteristics of humanhuman conversation rather than typical human-computer dialogue. Alternatively, human-computer dialogues can be recorded with a handcrafted DM prototype but neither of these two methods enables the system designer to test the implementation of the statistical DM and the learning algorithm. Moreover, the size of the recorded corpus (typically 10 dialogues) usually falls short of the requirements for training a statistical DM (typically 10 dialogues). 1.2 User Simulation-Based Training In recent years, a number of research groups have investigated the use of a two-stage simulation-based setup. A statistical user model is first trained on a limited amount of dialogue data and the model is then used to simulate dialogues with the interactively learning DM (see Schatzmann et al. (2006) for a literature review). The simulation-based approach assumes the presence of a small corpus of suitably annotated in-domain dialogues or out-of-domain dialogues with a matching dialogue format (Lemon et al., 2006). In cases when no such data is available, handcrafted values can be assigned to the model parameters given that the model is sufficiently simple (Levin et al., 2000; Pietquin and Dutoit, 2005) but the performance of dialogue policies learned this way has not been evaluated using real users.

This paper discusses the use of our information extraction (IE) system, Textract, in the questionanswering (QA) track of the recently held TREC-8 tests. One of our major objectives is to examine how IE can help IR (Information Retrieval) in applications like QA. Our study shows: (i) IE can provide solid support for QA; (ii ) low-level IE like Named Entity tagging is often a necessary component in handling most types of questions; (iii ) a robust natural language shallow parser provides a structural basis for handling questions; (iv) high-level domain independent IE, i.e. extraction of multiple relationships and general events, is expected to bring about a breakthrough in QA.


Question: Moving towards dialogue management in restricted domain question answering systems: Does this meet your query needs?

This paper introduces a probabilistic framework for dialog systems aimed at optimizing dialog strategies through a combination of supervised and reinforcement learning. We assert that the challenge of dialog design can be mathematically framed as an optimization task, where the objective function encapsulates various dialog aspects pertinent to the specific application domain. We demonstrate that any dialog system can be abstracted as a sequential decision-making process, characterized by its state space, action set, and strategy. By introducing assumptions regarding state transition probabilities and cost assignments, a dialog system can be modeled as a Markov decision process (MDP), a stochastic model. Within the MDP paradigm, a plethora of data-driven techniques, rooted in reinforcement learning, exist for identifying the optimal strategy—that which maximizes the chosen criterion. To harness the training data effectively, we advocate for a hybrid approach merging supervised and reinforcement learning. Supervised learning is employed to construct a user model by estimating the MDP parameters that elucidate the user's behavior. Subsequently, a reinforcement learning algorithm is utilized to ascertain the optimal strategy as the system engages with the simulated user. This methodology is evaluated on the task of an air travel information system (ATIS). The empirical outcomes presented herein affirm the feasibility of identifying a straightforward criterion, a state space representation, and a simulated user parameterization to autonomously learn a sophisticated dialog behavior, comparable to that crafted by experts through heuristic methods.

Learning from sparse and delayed reward is a central issue in reinforcement learning. In this paper, to tackle reward sparseness problem of task oriented dialogue management, we propose a curriculum based approach on the number of slots of user goals. This curriculum makes it possible to learn dialogue management for sets of user goals with large number of slots. We also propose a dialogue policy based on progressive neural networks whose modules with parameters are appended with previous parameters fixed as the curriculum proceeds, and this policy improves performances over the one with single set of parameters.

It has been proven that automatic conversational agents can be built up using the Endto-End Neural Response Generation (NRG) framework, and such a data-driven methodology requires a large number of dialog pairs for model training and reasonable evaluation metrics for testing. This paper proposes a Large Scale Domain-Specific Conversational Corpus (LSDSCC) composed of high-quality queryresponse pairs extracted from the domainspecific online forum, with thorough preprocessing and cleansing procedures. Also, a testing set, including multiple diverse responses annotated for each query, is constructed, and on this basis, the metrics for measuring the diversity of generated results are further presented. We evaluate the performances of neural dialog models with the widely applied diversity boosting strategies on the proposed dataset. The experimental results have shown that our proposed corpus can be taken as a new benchmark dataset for the NRG task, and the presented metrics are promising to guide the optimization of NRG models by quantifying the diversity of the generated responses reasonably.

This research focuses on natural, robot-led, human-robot interaction that enables a robot to discover what drinks a barman can prepare through continuous application of speech recognition, understanding and generation. Speech was recognised using Google Cloud’s speech to text API, understood by matching either the object or main verb of a sentence against a list of key words and, finally, generated using templates with variable parts. The difficulty lies in the large quantity of key words, as they are based on the properties of the ordered drinks. The results show that having the aforementioned interaction works well to some extent, i.e. the naturalness of the interaction was ranked 5.5 on average. Furthermore, the obtained precision when identifying the unavailable drinks was 0.625 and the obtained recall was 1, resulting in an F1 measure of 0.769.

Conversational modeling is an important task in natural language understanding and machine intelligence. Although previous approaches exist, they are often restricted to specific domains (e.g., booking an airline ticket) and require handcrafted rules. In this paper, we present a simple approach for this task which uses the recently proposed sequence to sequence framework. Our model converses by predicting the next sentence given the previous sentence or sentences in a conversation. The strength of our model is that it can be trained end-to-end and thus requires much fewer hand-crafted rules. We find that this straightforward model can generate simple conversations given a large conversational training dataset. Our preliminary results suggest that, despite optimizing the wrong objective function, the model is able to converse well. It is able extract knowledge from both a domain specific dataset, and from a large, noisy, and general domain dataset of movie subtitles. On a domainspecific IT helpdesk dataset, the model can find a solution to a technical problem via conversations. On a noisy open-domain movie transcript dataset, the model can perform simple forms of common sense reasoning. As expected, we also find that the lack of consistency is a common failure mode of our model.

Spoken Language Understanding (SLU) and more specifically, semantic parsing is an indispensable task in each speech-enabled application. In this survey, we review the current research on SLU and semantic parsing with emphasis on machine learning techniques used for these tasks. Observing the current trends in semantic parsing, we conclude our discussion by suggesting some of the most promising future research trends.

Conversational machine comprehension requires a deep understanding of the conversation history. To enable traditional, single-turn models to encode the history comprehensively, we introduce FLOW, a mechanism that can incorporate intermediate representations generated during the process of answering previous questions, through an alternating parallel processing structure. Compared to shallow approaches that concatenate previous questions/answers as input, FLOW integrates the latent semantics of the conversation history more deeply. Our model, FLOWQA, shows superior performance on two recently proposed conversational challenges (+7.2% F1 on CoQA and +4.0% on QuAC). The effectiveness of FLOW also shows in other tasks. By reducing sequential instruction understanding to conversational machine comprehension, FLOWQA outperforms the best models on all three domains in SCONE, with +1.8% to +4.4% improvement in accuracy.

Reiter and Dale’s book is about natural language generation (NLG), and describes methods for building systems of this kind. It offers a comprehensive overview of the field of NLG, from a perspective that has rarely been described elsewhere. It is very valuable for people with little or even no knowledge about NLG particulars who intend to investigate applications where NLG technology might play a certain role, as well as for people who want to obtain an application-oriented overview of the field. In this sense, all communities intended to be served by the book—students, academics in related areas, and software developers—are well addressed. In addition, some sections provide material that is probably new even to researchers with considerable experience in the field, such as the discussions on corpus analysis and domain modeling. In the introduction, the field of NLG is briefly characterized from researchand application-oriented perspectives and is illustrated by screen shots produced by several systems. Then conditions for effective uses of this technology are elaborated and contrasted with conditions where other techniques are more appropriate, and methods for determining the intended functionality of a system to be built are discussed. The main sections of the book are devoted to the prototypical architecture of applicationoriented NLG systems and their major processing phases: document planning, microplanning, and surface realization. Each of these three phases is illustrated by a number of detailed examples, demonstrating the successive refinements of utterance specifications in the course of processing. In the final section, the embedding of natural language processing technology is discussed, featuring typography, combined uses with graphics and hypertext, and integration with speech. The methods are illustrated by a large number of examples—the book contains more than 120 figures on its 248 pages. At the end of each section, a number of useful references for further reading are related to the section topics. In the appendix, a table summarizing the 35 systems referred to in the book is given. In general, the presentations in the book are beneficial for their high degree of detail in documenting representations. Running examples are illustrated by intermediate representations at various stages of processing. They demonstrate very well the increasing degrees of precision in which utterances are specified and the associated commitments about possible text portions that express these specifications. Particularly valuable are the comparisons of alternative representations, ranging from skeletal propositions over lexicalized case frames to canned text, including a comparison of the consequences for the distribution of work over the system components involved and discussions of the associated pros and cons. These chapters provide a number of valuable hints, given the currently low degree of standardization in the field. Further important aspects addressed, which have been widely ignored in the literature so far,

We propose Neural Responding Machine (NRM), a neural networ k-based response generator for Short-Text Conversation. NRM takes the general encoder-de coder framework: it formalizes the generation of response as a decoding process based on the lat ent representation of the input text, while both encoding and decoding are realized with recurren t n ural networks (RNN). The NRM is trained with a large amount of one-round conversation dat a collected from a microblogging service. Empirical study shows that NRM can generate gramma tically correct and content-wise appropriate responses to over 75% of the input text, outperf orming state-of-the-arts in the same setting, including retrieval-based and SMT-based models.

We describe an integrated approach for statistical modeling of discourse structure for natural conversational speech. Our model is based on 42`dialog acts' which were hand-labeled in 1155 conversations from the Switchboard corpus of spontaneous human-to-human telephone speech. We developed several models and algorithms to automatically detect dialog acts from transcribed or automatically recognized words and from prosodic properties of the speech signal, and by using a statistical discourse grammar. All of these components were probabilistic in nature and estimated from data, employing a variety of techniques (hidden Markov models, N-gram language models, maximum entropy estimation, decision tree classiiers, and neural networks). In preliminary studies, we achieved a dialog act labeling accuracy of 65% based on recognized words and prosody, and an accuracy of 72% based on word transcripts. Since humans achieve 84% on this task (with chance performance at 35%) we nd these results encouraging.

This article describes My Science Tutor (MyST), an intelligent tutoring system designed to improve science learning by students in 3rd, 4th, and 5th grades (7 to 11 years old) through conversational dialogs with a virtual science tutor. In our study, individual students engage in spoken dialogs with the virtual tutor Marni during 15 to 20 minute sessions following classroom science investigations to discuss and extend concepts embedded in the investigations. The spoken dialogs in MyST are designed to scaffold learning by presenting open-ended questions accompanied by illustrations or animations related to the classroom investigations and the science concepts being learned. The focus of the interactions is to elicit self-expression from students. To this end, Marni applies some of the principles of Questioning the Author, a proven approach to classroom conversations, to challenge students to think about and integrate new concepts with prior knowledge to construct enriched mental models that can be used to explain and predict scientific phenomena. In this article, we describe how spoken dialogs using Automatic Speech Recognition (ASR) and natural language processing were developed to stimulate students' thinking, reasoning and self explanations. We describe the MyST system architecture and Wizard of Oz procedure that was used to collect data from tutorial sessions with elementary school students. Using data collected with the procedure, we present evaluations of the ASR and semantic parsing components. A formal evaluation of learning gains resulting from system use is currently being conducted. This paper presents survey results of teachers' and children's impressions of MyST.


Question: A conversational approach for managing dialogue in specialized domain question answering systems.

This paper explores the exploitation of redundancy in the realm of question answering, specifically targeting brief factual inquiries such as "When was the Battle of Hastings?" or "Who wrote The Wind in the Willows?". Given that virtually any answer to these questions can be located on the Internet, our challenge lies in sifting through vast data volumes to pinpoint accurate responses. We tackle this by applying a sophisticated passage retrieval technique to identify potential answers and leveraging answer redundancy to ensure their validity. The crux of our method hinges on the premise that the web's expansive content repeatedly provides answers to most factual questions, embedded within diverse contexts. To execute this, we formulate a query from each question to sift through a significant corpus of web data, selecting concise passages that may contain the sought-after answer. These passages are then scrutinized to pinpoint candidate answers. Notably, the frequency of these candidates within the passages serves as a "voting" mechanism to identify the most probable answer. Our approach is rigorously evaluated using questions from the TREC-9 question-answering dataset. Additionally, we showcase its versatility by extending it to answer multiple-choice trivia questions, akin to those posed in trivia quizzes and television game shows, further substantiating its efficacy.

One of the most crucial problem on training personalized response generation models for conversational robots is the lack of large scale personal conversation data. To address the problem, we propose a two-phase approach, namely initialization then adaptation, to first pre-train an optimized RNN encoder-decoder model (LTS model) in a large scale conversational data for general response generation and then fine-tune the model in a small scale personal conversation data to generate personalized responses. For evaluation, we propose a novel human aided method, which can be seen as a quasi-Turing test, to evaluate the performance of the personalized response generation models. Experimental results show that the proposed personalized response generation model outperforms the state-of-the-art approaches to language model personalization and persona-based neural conversation generation on the automatic evaluation, offline human judgment and the quasi-Turing test.

This paper proposes a new dataset, Frames, composed of 1369 human-human dialogues with an average of 15 turns per dialogue. This corpus contains goal-oriented dialogues between users who are given some constraints to book a trip and assistants who search a database to find appropriate trips. The users exhibit complex decision-making behaviour which involve comparing trips, exploring different options, and selecting among the trips that were discussed during the dialogue. To drive research on dialogue systems towards handling such behaviour, we have annotated and released the dataset and we propose in this paper a task called frame tracking. This task consists of keeping track of different semantic frames throughout each dialogue. We propose a rule-based baseline and analyse the frame tracking task through this baseline.

Question answering (QA) systems are crucial when searching for exact answers for natural language questions in the biomedical domain. Answers to many of such questions can be extracted from the 26 millions biomedical publications currently included in MEDLINE when relying on appropriate natural language processing (NLP) tools. In this work we describe our participation in the task 4b of the BioASQ challenge using two QA systems that we developed for biomedicine. Preliminary results show that our systems achieved first and second positions in the snippet retrieval sub-task and for the generation of ideal answers.

Recently, the focus of many novel search applications shifted from short keyword queries to verbose natural language queries. Examples include question answering systems and dialogue systems, voice search on mobile devices and entity search engines like Facebook's Graph Search or Google's Knowledge Graph. However the performance of textbook information retrieval techniques for such verbose queries is not as good as that for their shorter counterparts. Thus, effective handling of verbose queries has become a critical factor for adoption of information retrieval techniques in this new breed of search applications. Over the past decade, the information retrieval community has deeply explored the problem of transforming natural language verbose queries using operations like reduction, weighting, expansion, reformulation and segmentation into more effective structural representations. However, thus far, there was not a coherent and organized tutorial on this topic. In this tutorial, we aim to put together various research pieces of the puzzle, provide a comprehensive and structured overview of various proposed methods, and also list various application scenarios where effective verbose query processing can make a significant difference.

Computer systems that can directly and accurately answer peoples' questions over a broad domain of human knowledge have been envisioned by scientists and writers since the advent of computers themselves. Open domain question answering holds tremendous promise for facilitating informed decision making over vast volumes of natural language content. Applications in business intelligence, healthcare, customer support, enterprise knowledge management, social computing, science and government would all benefit from deep language processing. The DeepQA project (www.ibm.com/deepqa) is aimed at illustrating how the advancement and integration of Natural Language Processing (NLP), Information Retrieval (IR), Machine Learning (ML), massively parallel computation and Knowledge Representation and Reasoning (KR&R) can greatly advance open-domain automatic Question Answering. An exciting proof-point in this challenge is to develop a computer system that can successfully compete against top human players at the Jeopardy! quiz show (www.jeopardy.com). Attaining champion-level performance Jeopardy! requires a computer to rapidly answer rich open-domain questions, and to predict its own performance on any given category/question. The system must deliver high degrees of precision and confidence over a very broad range of knowledge and natural language content and with a 3-second response time. To do this DeepQA generates, evidences and evaluates many competing hypotheses. A key to success is automatically learning and combining accurate confidences across an array of complex algorithms and over different dimensions of evidence. Accurate confidences are needed to know when to "buzz in" against your competitors and how much to bet. Critical for winning at Jeopardy!, High precision and accurate confidence computations are just as critical for providing real value in business settings where helping users focus on the right content sooner and with greater confidence can make all the difference. The need for speed and high precision demands a massively parallel compute platform capable of generating, evaluating and combing 1000's of hypotheses and their associated evidence. In this talk I will introduce the audience to the Jeopardy! Challenge and describe our technical approach and our progress on this grand-challenge problem.

Domain dependence of NLP systems is one of the major obstacles to their application in large-scale text analysis, also restricting the applicability of FrameNet semantic role labeling (SRL) systems. Yet, current FrameNet SRL systems are still only evaluated on a single in-domain test set. For the first time, we study the domain dependence of FrameNet SRL on a wide range of benchmark sets. We create a novel test set for FrameNet SRL based on user-generated web text and find that the major bottleneck for out-of-domain FrameNet SRL is the frame identification step. To address this problem, we develop a simple, yet efficient system based on distributed word representations. Our system closely approaches the state-of-the-art in-domain while outperforming the best available frame identification system out-of-domain. We publish our system and test data for research purposes.1

v This work focuses on incorporating sentiment information into task-oriented dialogue systems. v Current end-to-end approaches only consider user semantic inputs in learning and under-utilize other user information. v But the ultimate evaluator of dialog systems is the end-users and their sentiment is a direct reflection of user satisfaction and should be taken into consideration. v Therefore, we propose to include user sentiment obtained through multimodal information (acoustic, dialogic and textual), in the end-to-end learning framework to make systems more user-adaptive and effective. v We incorporated user sentiment information in both supervised and reinforcement learning settings. v In both settings, adding sentiment information reduced the dialog length and improved the task success rate on a bus information search task. Multimodal Sentiment Detector v We manually annotated 50 dialogs with 517 conversation turns to train this sentiment detector. The annotated set is open to public. v Prediction made by the detector will be used in the supervised learning and reinforcement learning. v Three sets of features: 1) Acoustic features; 2) Dialogic features; 3) Textual features. v Dialogic features include: 1) Interruption; 2) Button usage; 3) Repetitions; 4) Start over. These four categories of dialog features are chosen based on the previous literature and the observed statistics in the dataset.

We present the components of the ColumbiaGWU contribution to the 2016 TAC KBP BeSt Evaluation.

In this paper we explore deep learning models with memory component or attention mechanism for question answering task. We combine and compare three models, Neural Machine Translation [1], Neural Turing Machine [5], and Memory Networks [15] for a simulated QA data set [14]. This paper is the first one that uses Neural Machine Translation and Neural Turing Machines for solving QA tasks. Our results suggest that the combination of attention and memory have potential to solve certain QA problem.

In this work, we present a novel method for classifying comments in online discussions into a set of coarse discourse acts towards the goal of better understanding discussions at scale. To facilitate this study, we devise a categorization of coarse discourse acts designed to encompass general online discussion and allow for easy annotation by crowd workers. We collect and release a corpus of over 9,000 threads comprising over 100,000 comments manually annotated via paid crowdsourcing with discourse acts and randomly sampled from the site Reddit. Using our corpus, we demonstrate how the analysis of discourse acts can characterize different types of discussions, including discourse sequences such as Q&A pairs and chains of disagreement, as well as different communities. Finally, we conduct experiments to predict discourse acts using our corpus, finding that structured prediction models such as conditional random fields can achieve an F1 score of 75%. We also demonstrate how the broadening of discourse acts from simply question and answer to a richer set of categories can improve the recall performance of Q&A extraction.


Question: BabelNet: Constructing an extensive multilingual semantic network.

Introducing YAGO: A Comprehensive Ontology Derived from Wikipedia and WordNet
This paper introduces YAGO, a large-scale ontology that boasts high coverage and precision. YAGO has been meticulously constructed through an automated process that leverages the rich information available in Wikipedia and WordNet. The ontology encompasses a diverse array of entities and their relationships, currently featuring over 1.7 million entities and in excess of 15 million facts. YAGO's contents include not only a comprehensive taxonomic hierarchy delineated by Is-A relationships, but also a wide range of semantic connections between entities. These facts are meticulously extracted from Wikipedia's category system and infoboxes, and are augmented with taxonomic data from WordNet. To ensure YAGO's integrity, rigorous information extraction techniques and consistency checks have been employed, resulting in an impressive precision level of 95%, as validated through comprehensive evaluation. The ontology's design is grounded in a logically coherent model that ensures decidable consistency. Moreover, YAGO's architecture supports the natural representation of n-ary relations, all while maintaining compatibility with the Resource Description Framework Schema (RDFS). This makes YAGO a powerful resource for complex query formulations and data analysis.

This paper introduces NEMWEL, a system that performs Never-Ending MultiWord Expressions Learning. Instead of using a static corpus and classifier, NEMWEL applies supervised learning on automatically crawled news texts. Moreover, it uses its own results to periodically retrain the classifier, bootstrapping on its own results. In addition to a detailed description of the system’s architecture and its modules, we report the results of a manual evaluation. It shows that NEMWEL is capable of learning new expressions over time with improved precision.

This paper presents the results of a set of methods to cluster WordNet word senses. The methods rely on different information sources: confusion matrixes from Senseval-2 Word Sense Disambiguation systems, translation similarities, hand-tagged examples of the target word senses and examples obtained automatically from the web for the target word senses. The clustering results have been evaluated using the coarsegrained word senses provided for the lexical sample in Senseval-2. We have used Cluto, a general clustering environment, in order to test different clustering algorithms. The best results are obtained for the automatically obtained examples, yielding purity values up to 84% on average over 20 nouns.

Many research studies adopt manually selected patterns for semantic relation extraction. However, manually identifying and discovering patterns is time consuming and it is difficult to discover all potential candidates. Instead, we propose an automatic pattern construction approach to extract verb synonyms and antonyms from English newspapers. Instead of relying on a single pattern, we combine results indicated by multiple patterns to maximize the recall.

Words to express relations in natural language (NL) statements may be different from those to represen t properties in knowledge bases (KB). The vocabulary g p becomes barriers for knowledge base construction and retrieval. With the demo system called NL2KB in this paper, users can browse which properties in KB side may be mapped to for a given relati onal pattern in NL side. Besides, they can retrieve the sets of relational patterns in NL side for a given property in KB side. We describe how the mapping is established in detail. Although the mined patterns are used for Chinese knowledge base applications, t he methodology can be extended to other languages.

Entity linking (EL) is the task of disambiguating mentions in text by associating them with entries in a predefined database of mentions (persons, organizations, etc). Most previous EL research has focused mainly on one language, English, with less attention being paid to other languages, such as Spanish or Chinese. In this paper, we introduce LIEL, a Language Independent Entity Linking system, which provides an EL framework which, once trained on one language, works remarkably well on a number of different languages without change. LIEL makes a joint global prediction over the entire document, employing a discriminative reranking framework with many domain and language-independent feature functions. Experiments on numerous benchmark datasets, show that the proposed system, once trained on one language, English, outperforms several state-of-the-art systems in English (by 4 points) and the trained model also works very well on Spanish (14 points better than a competitor system), demonstrating the viability of the approach.

This paper describes our system submission to the CogALex-2016 Shared Task on Corpus-Based Identification of Semantic Relations. Our system won first place for Task-1 and second place for Task-2. The evaluation results of our system on the test set is 88.1% (79.0% for TRUE only) f-measure for Task-1 on detecting semantic similarity, and 76.0% (42.3% when excluding RANDOM) for Task-2 on identifying fine-grained semantic relations. In our experiments, we try word analogy, linear regression, and multi-task Convolutional Neural Networks (CNNs) with word embeddings from publicly available word vectors. We found that linear regression performs better in the binary classification (Task-1), while CNNs have better performance in the multi-class semantic classification (Task-2). We assume that word analogy is more suited for deterministic answers rather than handling the ambiguity of one-to-many and many-to-many relationships. We also show that classifier performance could benefit from balancing the distribution of labels in the training data.

Word Sense Disambiguation (WSD) is a key enabling-technology. Supervised WSD techniques are the best performing in public evaluations, but need large amounts of hand-tagging data. Existing hand-annotated corpora like SemCor (Miller et al., 1993), which is annotated with WordNet senses (Fellbaum, 1998) allow for a small improvement over the simple most frequent sense heuristic, as attested in the all-words track of the last Senseval competition (Snyder and Palmer, 2004). In theory, larger amounts of training data (SemCor has approx. 500M words) would improve the performance of supervised WSD, but no current project exists to provide such an expensive resource. Another problem of the supervised approach is that the inventory and distribution of senses changes dramatically from one domain to the other, requiring additional hand-tagging of corpora (Martı́nez and Agirre, 2000; Koeling et al., 2005). Supervised WSD is based on the “fixed-list of senses” paradigm, where the senses for a target word are a closed list coming from a dictionary or lexicon. Lexicographers and semanticists have long warned about the problems of such an approach, where senses are listed separately as discrete entities, and have argued in favor of more complex representations, where, for instance, senses are dense regions in a continuum (Cruse, 2000). Unsupervised Word Sense Induction and Discrimination (WSID, also known as corpus-based unsupervised systems) has followed this line of thinking, and tries to induce word senses directly from the corpus. Typical WSID systems involve clustering techniques, which group together similar examples. Given a set of induced clusters (which represent word uses or senses1), each new occurrence of the target word will be compared to the clusters and the most similar cluster will be selected as its sense. One of the problems of unsupervised systems is that of managing to do a fair evaluation. Most of current unsupervised systems are evaluated in-house, with a brief comparison to a re-implementation of a former system, leading to a proliferation of unsupervised systems with little ground to compare among them. The goal of this task is to allow for comparison across sense-induction and discrimination systems, and also to compare these systems to other supervised and knowledge-based systems. The paper is organizes as follows. Section 2 presents the evaluation framework used in this task. Section 3 presents the systems that participated in the task, and the official results. Finally, Section 4 draws the conclusions.

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while ‘multi-sense’ methods have been proposed and tested on artificial wordsimilarity tasks, we don’t know if they improve real natural language understanding tasks. In this paper we introduce a multisense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language un-

We present a parser for Abstract Meaning Representation (AMR). We treat Englishto-AMR conversion within the framework of string-to-tree, syntax-based machine translation (SBMT). To make this work, we transform the AMR structure into a form suitable for the mechanics of SBMT and useful for modeling. We introduce an AMR-specific language model and add data and features drawn from semantic resources. Our resulting AMR parser significantly improves upon state-of-the-art results.

Neural net advances improve computers' language ability in many fields.


Question: BabelNet: Constructing an extensive multilingual semantic network.

This article introduces a strategy to significantly enhance WordNet, a lexical database for the English language, by integrating extensive semantic relations derived from Wikipedia, an encyclopedic resource. The objective is to address the challenge of the knowledge acquisition bottleneck, a major hindrance to achieving high-performance Word Sense Disambiguation (WSD). The methodology proposed automates the augmentation of WordNet with a wealth of semantic connections. The results demonstrate that when equipped with a substantial and high-quality corpus of semantic relations, straightforward algorithms that require minimal knowledge can match the performance of sophisticated supervised WSD systems in a broad, coarse-grained context. Moreover, these algorithms surpass supervised systems when tested on domain-specific datasets of gold-standard quality. This breakthrough highlights the potential of leveraging large-scale, high-quality semantic resources to improve WSD algorithms without the need for extensive manual annotation or training data.

Distributional representations of words have been recently used in supervised settings for recognizing lexical inference relations between word pairs, such as hypernymy and entailment. We investigate a collection of these state-of-the-art methods, and show that they do not actually learn a relation between two words. Instead, they learn an independent property of a single word in the pair: whether that word is a “prototypical hypernym”.

• Neural network language model and distributed representation for words (Vector representation) • Capture syntactic and remantic regularities in language • Outperform state-of-the-art

This paper introduces Web1T5-Easy, a simple indexing solution that allows interactive searches of the Web 1T 5-gram database and a derived database of quasi-collocations. The latter is validated against co-occurrence data from the BNC and ukWaC on the automatic identification of non-compositional VPC.

Neural net advances improve computers' language ability in many fields.

In recent years, there has been renewed interest in the NLP community in genuine language understanding and dialogue. Thus the long-standing issue of how the semantic content of language should be represented is reentering the communal discussion. This paper provides a brief “opinionated survey” of broadcoverage semantic representation (SR). It suggests multiple desiderata for such representations, and then outlines more than a dozen approaches to SR–some longstanding, and some more recent, providing quick characterizations, pros, cons, and some comments on imple-

Guided by multiple heuristics, a unified taxonomy of entities and categories is distilled from the Wikipedia category network. A comprehensive evaluation, based on the analysis of upward generalization paths, demonstrates that the taxonomy supports generalizations which are more than twice as accurate as the state of the art. The taxonomy is available at http://headstaxonomy.com.

Extracting semantic relationships between entities is challenging. This paper investigates the incorporation of diverse lexical, syntactic and semantic knowledge in feature-based relation extraction using SVM. Our study illustrates that the base phrase chunking information is very effective for relation extraction and contributes to most of the performance improvement from syntactic aspect while additional information from full parsing gives limited further enhancement. This suggests that most of useful information in full parse trees for relation extraction is shallow and can be captured by chunking. We also demonstrate how semantic information such as WordNet and Name List, can be used in feature-based relation extraction to further improve the performance. Evaluation on the ACE corpus shows that effective incorporation of diverse features enables our system outperform previously best-reported systems on the 24 ACE relation subtypes and significantly outperforms tree kernel-based systems by over 20 in F-measure on the 5 ACE relation types.

Prior research into learning translations from source and target language monolingual texts has treated the task as an unsupervised learning problem. Although many techniques take advantage of a seed bilingual lexicon, this work is the first to use that data for supervised learning to combine a diverse set of signals derived from a pair of monolingual corpora into a single discriminative model. Even in a low resource machine translation setting, where induced translations have the potential to improve performance substantially, it is reasonable to assume access to some amount of data to perform this kind of optimization. Our work shows that only a few hundred translation pairs are needed to achieve strong performance on the bilingual lexicon induction task, and our approach yields an average relative gain in accuracy of nearly 50% over an unsupervised baseline. Large gains in accuracy hold for all 22 languages (low and high resource) that we investigate.

The DBpedia community project extracts structured, multilingual knowledge from Wikipedia and makes it freely available on the Web using Semantic Web and Linked Data technologies. The project extracts knowledge from 111 different language editions of Wikipedia. The largest DBpedia knowledge base which is extracted from the English edition of Wikipedia consists of over 400 million facts that describe 3.7 million things. The DBpedia knowledge bases that are extracted from the other 110 Wikipedia editions together consist of 1.46 billion facts and describe 10 million additional things. The DBpedia project maps Wikipedia infoboxes from 27 different language editions to a single shared ontology consisting of 320 classes and 1,650 properties. The mappings are created via a world-wide crowd-sourcing effort and enable knowledge from the different Wikipedia editions to be combined. The project publishes releases of all DBpedia knowledge bases for download and provides SPARQL query access to 14 out of the 111 language editions via a global network of local DBpedia chapters. In addition to the regular releases, the project maintains a live knowledge base which is updated whenever a page in Wikipedia changes. DBpedia sets 27 million RDF links pointing into over 30 external data sources and thus enables data from these sources to be used together with DBpedia data. Several hundred data sets on the Web publish RDF links pointing to DBpedia themselves and make DBpedia one of the central interlinking hubs in the Linked Open Data (LOD) cloud. In this system report, we give an overview of the DBpedia community project, including its architecture, technical implementation, maintenance, internationalisation, usage statistics and applications.

A shared bilingual word embedding space (SBWES) is an indispensable resource in a variety of cross-language NLP and IR tasks. A common approach to the SBWES induction is to learn a mapping function between monolingual semantic spaces, where the mapping critically relies on a seed word lexicon used in the learning process. In this work, we analyze the importance and properties of seed lexicons for the SBWES induction across different dimensions (i.e., lexicon source, lexicon size, translation method, translation pair reliability). On the basis of our analysis, we propose a simple but effective hybrid bilingual word embedding (BWE) model. This model (HYBWE) learns the mapping between two monolingual embedding spaces using only highly reliable symmetric translation pairs from a seed document-level embedding space. We perform bilingual lexicon learning (BLL) with 3 language pairs and show that by carefully selecting reliable translation pairs our new HYBWE model outperforms benchmarking BWE learning models, all of which use more expensive bilingual signals. Effectively, we demonstrate that a SBWES may be induced by leveraging only a very weak bilingual signal (document alignments) along with monolingual data.