Question: A completely automated system for creating crossword puzzles.

This article introduces a Maximum Entropy (MaxEnt) model designed for Part-Of-Speech (POS) tagging, achieving high accuracy in tagging unseen text based on a corpus annotated with POS tags. The MaxEnt model is characterized by its ability to concurrently leverage multiple contextual features for predicting the correct POS tag, setting it apart from traditional models. The paper also highlights the incorporation of specialized features aimed at addressing challenging tagging scenarios. It delves into the corpus consistency issues encountered during the implementation of these features and offers a training strategy to effectively manage these issues, ensuring that the model's performance remains robust and reliable. This comprehensive approach to POS tagging not only enhances the accuracy of tag prediction but also underscores the significance of feature selection and corpus consistency in the development of advanced linguistic models.

We introduce an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge (WSC), a common sense reasoning task that requires diverse, complex forms of inference and knowledge. Our method uses a knowledge hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine, then extracts and classifies knowledge from the returned results and weighs them to make a resolution. Our approach improves F1 performance on the full WSC by 0.21 over the previous best and represents the first system to exceed 0.5 F1. We further demonstrate that the approach is competitive on the Choice of Plausible Alternatives (COPA) task, which suggests that it is generally applicable.

Compilers and computer-aided design tools are essential for fine-grained control of nanoscale quantum-mechanical systems. A proposed four-phase design flow assists with computations by transforming a quantum algorithm from a high-level language program into precisely scheduled physical actions.

Internet users are keen on creating different kinds of morphs to avoid censorship, express strong sentiment or humor. For example, in Chinese social media, users often use the entity morph “方便面 (Instant Noodles)” to refer to “周永康 (Zhou Yongkang)” because it shares one character “康 (Kang)” with the well-known brand of instant noodles “康师傅 (Master Kang)”. We developed a wide variety of novel approaches to automatically encode proper and interesting morphs, which can effectively pass decoding tests 1.

Little has been done in the study of these intriguing questions, and I do not wish to give the impression that any extensive set of ideas exists that could be called a "theory." What is quite surprising, as far as the histories of science and philosophy are concerned, is that the major impetus for the fantastic growth of interest in brain processes, both psychological and physiological, has come from a device, a machine, the digital computer. In dealing with a human being and a human society, we enjoy the luxury of being irrational, illogical, inconsistent, and incomplete, and yet of coping. In operating a computer, we must meet the rigorous requirements for detailed instructions and absolute precision. If we understood the ability of the human mind to make effective decisions when confronted by complexity, uncertainty, and irrationality then we could use computers a million times more effectively than we do. Recognition of this fact has been a motivation for the spurt of research in the field of neurophysiology. The more we study the information processing aspects of the mind, the more perplexed and impressed we become. It will be a very long time before we understand these processes sufficiently to reproduce them. In any case, the mathematician sees hundreds and thousands of formidable new problems in dozens of blossoming areas, puzzles galore, and challenges to his heart's content. He may never resolve some of these, but he will never be bored. What more can he ask?

Dialogue systems are often regarded as being tedious and inflexible. We believe that one reason is rigid and inadaptable system utterances. A good dialogue system should automatically choose a formulation that reflects the user’s expectations. However, current dialogue system development environments only allow the definition of questions with unchangeable formulations. In this paper we present a new approach to the generation of system questions by only defining basic concepts. This is the basis for realising adaptive, user-tailored, and human-like system questions in dialogue systems.

As machines have become more intelligent, there has been a renewed interest in methods for measuring their intelligence. A common approach is to propose tasks for which a human excels, but one which machines find difficult. However, an ideal task should also be easy to evaluate and not be easily gameable. We begin with a case study exploring the recently popular task of image captioning and its limitations as a task for measuring machine intelligence. An alternative and more promising task is Visual Question Answering that tests a machine’s ability to reason about language and vision. We describe a dataset unprecedented in size created for the task that contains over 760,000 human generated questions about images. Using around 10 million human generated answers, machines may be easily evaluated.

Pavlov is a Programming By Demonstration (PBD) system that allows animated interfaces to be created without programming. Using a drawing editor and a clock, designers specify the behavior of a target interface by demonstrating stimuli (end-user actions or time) and the (time-stamped) graphical transformations that should be executed in response. This stimulus-response model allows interaction and animation to be defined in a uniform manner, and it allows for the demonstration of interactive animation, i.e., game-like behaviors in which the end-user (player) controls the speed and direction of object movement.

Full terms and conditions of use: http://pubsonline.informs.org/page/terms-and-conditions This article may be used only for the purposes of research, teaching, and/or private study. Commercial use or systematic downloading (by robots or other automatic processes) is prohibited without explicit Publisher approval, unless otherwise noted. For more information, contact permissions@informs.org. The Publisher does not warrant or guarantee the article’s accuracy, completeness, merchantability, fitness for a particular purpose, or non-infringement. Descriptions of, or references to, products or publications, or inclusion of an advertisement in this article, neither constitutes nor implies a guarantee, endorsement, or support of claims made of that product, publication, or service. © 1990 INFORMS

This paper illustrates an artiicial developmental system that is a computationally ee-cient technique for the automatic generation of complex Artiicial Neural Networks (ANN). Artiicial developmental system can develop a graph grammar into a modular ANN made of a combination of more simple subnetworks. A genetic algorithm is used to evolve coded grammars that generates ANNs for controlling a six-legged robot locomotion. A mechanism for the automatic deenition of sub-neural networks is incorporated. Using this mechanism, the genetic algorithm can automatically decompose a problem into subproblems, generate a subANN for solving the subproblem, and instantiate copies of this subANN to build a higher level ANN that solves the problem. We report some simulation results showing that the same problem cannot be solved if the mechanism for automatic deenition of sub-networks is suppressed. We support our argumentation with pictures describing the steps of development, how ANN structures are evolved, and how the ANNs compute.matic deention of sub-neural networks.

The Painting Fool is software that we hope will one day be taken seriously as a creative artist in its own right. This aim is being pursued as an Artificial Intelligence (AI) project, with the hope that the technical difficulties overcome along the way will lead to new and improved generic AI techniques. It is also being pursued as a sociological project, where the effect of software which might be deemed as creative is tested in the art world and the wider public. In this chapter, we summarise our progress so far in The Painting Fool project. To do this, we first compare and contrast The Painting Fool with software of a similar nature arising from AI and graphics projects. We follow this with a discussion of the guiding principles from Computational Creativity research that we adhere to in building the software. We then describe five projects with The Painting Fool where our aim has been to produce increasingly interesting and culturally valuable pieces of art. We end by discussing the issues raised in building an automated painter, and describe further work and future prospects for the project. By studying both the technical difficulties and sociological issues involved in engineering software for creative purposes, we hope to help usher in a new era where computers routinely act as our creative collaborators, as well as independent and creative artists, musicians, writers, designers, engineers and scientists, and contribute in meaningful and interesting ways to human culture.


Question: A completely automated system for creating crossword puzzles.

Introducing CrossWiz: A Comprehensive Crossword Puzzle Generator

CrossWiz is a sophisticated, fully automatic crossword puzzle generator that leverages advanced algorithms to create engaging and challenging crosswords. Unlike conventional methods that may require manual intervention or adjustment, CrossWiz operates entirely autonomously, ensuring a seamless and efficient puzzle generation process. By employing a combination of lexical databases, pattern recognition, and optimization techniques, CrossWiz is capable of generating crosswords that cater to a wide range of difficulty levels, from beginner to expert.

The core functionality of CrossWiz is centered around an optimized grid-filling algorithm. This algorithm takes into account the constraints of the grid size, word lengths, and the need for interlocking entries to produce high-quality puzzles. The system intelligently selects words from a vast lexical database, prioritizing those that offer the best fit and interlock with other entries. Additionally, CrossWiz incorporates a scoring system that evaluates the quality of potential word placements, ensuring that the final puzzle is both balanced and entertaining.

To further enhance the user experience, CrossWiz includes features for customizing puzzle themes, difficulty levels, and even the inclusion of specific words or phrases. This level of customization allows for the creation of crosswords that are tailored to specific interests or events, such as themed puzzles for holidays, birthdays, or educational purposes.

In terms of evaluation, CrossWiz has been tested extensively against existing crossword generation tools and has consistently outperformed them in terms of puzzle quality, variety, and generation speed. The system's robustness is attributed to its ability to adapt to various linguistic patterns and constraints, making it a versatile tool for both recreational and educational use.

Overall, CrossWiz represents a significant advancement in the field of automatic crossword generation, offering a comprehensive solution that meets the needs of crossword enthusiasts and educators alike. Whether for leisurely pursuits or as an educational tool, CrossWiz ensures that each puzzle generated is a unique and stimulating challenge.

We have developed an efficient way to determine the syntactic similarity of files and have applied it to every document on the World Wide Web. Using this mechanism, we built a clustering of all the documents that are syntactically similar. Possible applications include a “Lost and Found” service, filtering the results of Web searches, updating widely distributed web-pages, and identifying violations of intellectual property rights.

We report on research in multi-document summarization and on evaluation of summarization in the framework of cross-lingual information retrieval. This work was carried out during a summer workshop on Language Engineering held at Johns Hopkins University by a team of nine researchers from seven universities. The goals of the research were as follows: (1) to develop a toolkit for evaluation of single-document and multi-document summarizers, (2) to develop a modular multi-document summarizer, called MEAD, that works in both English and Chinese, and (3) to perform a meta-evaluation of four automatic summarizers, including MEAD, using several types of evaluation measures: some currently used by summarization researchers and a couple of novel techniques. Central to the experiments in this workshop was the cross-lingual experimental setup based on a large-scale Chinese and English parallel corpus. An extensive set of human judgments were specifically prepared by the Linguistic Data Consortium for our research. These human judgments include a) which documents are relevant to a certain query and b) which sentences in the relevant documents are most relevant to the query and which therefore constitute a good summary of the cluster. These judgments were used to construct variable-length multiand single document summaries as model summaries. Since one of the novel evaluation metrics that we used, Relevance Correlation, is based on the premise that good summaries preserve query relevance both within a language and across languages, we made use of a cross-lingual Information Retrieval (IR) engine. We evaluated the quality of the automatic summaries using co-selection and content-based evaluation, two established techniques. A relatively new metric, relative utility, was also extensively tested. Part of the new scientific contribution is the measurement of relevance correlation, which we introduced and systematically examined in this workshop. Relevance correlation measures the quality of summaries in comparison to the entire documents as a function of how much document relevance drops if summaries are indexed instead of documents. Our results show that this measure is sensible, in that it correlates with more established evaluation measures. Another contribution is the cross-lingual setup which allows us to automatically translate English queries into Chinese, perform Chinese IR with or without summarization. This allows us to calculate relevance correlation for English and for Chinese in parallel (i.e., for the same queries) and to take direct cross-lingual comparisons of evaluations. Additionally, an alternative way of constructing Chinese model summaries from English ones was implemented which relies on the sentence alignment of English and Chinese documents. The results of our large-scale meta-evaluation are numerous, but some of the highlights are the following: (1) All evaluation measures rank human summaries first, which is an appropriate and expected property of such measures, (2) Both relevance correlation and the content-based measures place leading sentence extracts ahead of the more sophisticated summarizers, (3) Relative utility ranks our system, MEAD, as the best summarizer for shorter summaries, although for longer summaries, lead-based summaries outperform MEAD, (4) Co-selection measurements show overall low agreement amongst humans (above chance), whereas relative utility reports higher numbers on the same data (but does not normalize for chance). The deliverable resources and software include: (1) a turn-key extractive multi-document summarizer, MEAD, which allows users to add their own features based on single sentences or pairs of sentences, (2) a large corpus of summaries produced by several automatic methods, including baseline and random summaries, (3) a collection of manual summaries produced by the Linguistic Data Consortium (LDC), (4) a battery of evaluation routines, (5) a collection of IR queries in English and Chinese and the corresponding relevance judgments from the Hong Kong news collection, (6) SMART relevance outputs for both full documents and summaries, (7) XML tools for processing of documents and summaries. JHU 2001 Summer workshop final report Evaluation of Text Summarization

been developed during the last 20 years have proven to be quite successful, particularly in the domains of mathematics, science, and technology. They produce significant learning gains beyond classroom environments. They are capable of engaging most students’ attention and interest for hours. We have been working on a new generation of intelligent tutoring systems that hold mixedinitiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AUTOTUTOR is a conversational agent, with a talking head, that helps college students learn about computer literacy. ANDES, ATLAS, AND WHY2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations.

Cross-device interactions involve input and output on multiple computing devices. Implementing and reasoning about interactions that cover multiple devices with a diversity of form factors and capabilities can be complex. To assist developers in programming cross-device interactions, we created DemoScript, a technique that automatically analyzes a cross-device interaction program while it is being written. DemoScript visually illustrates the step-by-step execution of a selected portion or the entire program with a novel, automatically generated cross-device storyboard visualization. In addition to helping developers understand the behavior of the program, DemoScript also allows developers to revise their program by interactively manipulating the cross-device storyboard. We evaluated DemoScript with 8 professional programmers and found that DemoScript significantly improved development efficiency by helping developers interpret and manage cross-device interaction; it also encourages testing to think through the script in a development process.

STRAND (Resnik, 1998) is a languageindependent system for automatic discovery of text in parallel translation on the World Wide Web. This paper extends the preliminary STRAND results by adding automatic language identification, scaling up by orders of magnitude, and formally evaluating performance. The most recent end-product is an automatically acquired parallel corpus comprising 2491 English-French document pairs, approximately 1.5 million words per language. 1 I n t r o d u c t i o n Text in parallel translation is a valuable resource in natural language processing. Statistical methods in machine translation (e.g. (Brown et al., 1990)) typically rely on large quantities of bilingual text aligned at the document or sentence level, and a number of approaches in the burgeoning field of crosslanguage information retrieval exploit parallel corpora either in place of or in addition to mappings between languages based on information from bilingual dictionaries (Davis and Dunning, 1995; Landauer and Littman, 1990; Hull and Oard, 1997; Oard, 1997). Despite the utility of such data, however, sources of bilingual text are subject to such limitations as licensing restrictions, usage fees, restricted domains or genres, and dated text (such as 1980's Canadian politics); or such sources simply may not exist for * This work was supported by Department of Defense contract MDA90496C1250, DARPA/ITO Contract N66001-97-C-8540, and a research grant from Sun Microsystems Laboratories. The author gratefully acknowledges the comments of the anonymous reviewers, helpful discussions with Dan Melamed and Doug Oard, and the assistance of Jeff Allen in the French-English experimental evaluation. language pairs of interest. Although the majority of Web content is in English, it also shows great promise as a source of multilingual content. Using figures from the Babel survey of multilinguality on the Web (h tZp : / /www. i s o c . o r g / ) , it is possible to estimate that as of June, 1997, there were on the order of 63000 primarily non-English Web servers, ranging over 14 languages. Moreover, a followup investigation of the non-English servers suggests that nearly a third contain some useful cross-language data, such as parallel English on the page or links to parallel English pages the follow-up also found pages in five languages not identified by the Babel study (Catalan, Chinese, Hungarian, Icelandic, and Arabic; Michael Littman, personal communication). Given the continued explosive increase in the size of the Web, the trend toward business organizations that cross national boundaries, and high levels of competition for consumers in a global marketplace, it seems impossible not to view multilingual content on the Web as an expanding resource. Moreover, it is a dynamic resource, changing in content as the world changes. For example, Diekema et al., in a presentation at the 1998 TREC-7 conference (Voorhees and Harman, 1998), observed that the performance of their cross-language information retrieval was hurt by lexical gaps such as Bosnia/Bosniethis illustrates a highly topical missing pair in their static lexical resource (which was based on WordNet 1.5). And Gey et al., also at TREC-7, observed that in doing cross-language retrieval using commercial machine translation systems, gaps in the lexicon (their example was acupuncture/Akupunktur) could make the difference between precision of 0.08 and precision of 0.83 on individual queries. ttesnik (1998) presented an algorithm called

Although commonly used in both commercial and experimental information retrieval systems, thesauri have not demonstrated consistent beneets for retrieval performance, and it is diicult to construct a thesaurus automatically for large text databases. In this paper, an approach, called PhraseFinder, is proposed to construct collection-dependent association thesauri automatically using large full-text document collections. The association thesaurus can be accessed through natural language queries in INQUERY, an information retrieval system based on the probabilistic inference network. Experiments are conducted in IN-QUERY to evaluate diierent types of association thesauri, and thesauri constructed for a variety of collections.

Abstractive single document summarization is considered as a challenging problem in the field of artificial intelligence and natural language processing. Meanwhile and specifically in the last two years, several deep learning summarization approaches were proposed that once again attracted the attention of researchers to this field.
 It is a well-known issue that deep learning approaches do not work well with small amounts of data. With some exceptions, this is, unfortunately, the case for most of the datasets available for the summarization task. Besides this problem, it should be considered that phonetic, morphological, semantic and syntactic features of the language are constantly changing over the time and unfortunately most of the summarization corpora are constructed from old resources. Another problem is the language of the corpora. Not only in the summarization field, but also in other fields of natural language processing, most of the corpora are only available in English. In addition to the above problems, license terms, and fees of the corpora are obstacles that prevent many academics and specifically non-academics from accessing these data.
 This work describes an open source framework to create an extendable multilingual corpus for abstractive single document summarization that addresses the above-mentioned problems. We describe a tool consisted of a scalable crawler and a centralized key-value store database to construct a corpus of an arbitrary size using a news aggregator service.

This paper advocates a new ML-based programming framework, called Programming by Feedback (PF), which involves a sequence of interactions between the active computer and the user. The latter only provides preference judgments on pairs of solutions supplied by the active computer. The active computer involves two components: the learning component estimates the user’s utility function and accounts for the user’s (possibly limited) competence; the optimization component explores the search space and returns the most appropriate candidate solution. A proof of principle of the approach is proposed, showing that PF requires a handful of interactions in order to solve some discrete and continuous benchmark problems.

Our goal is to automatically answer brief factual questions of the form ``When was the Battle of Hastings?'' or ``Who wrote The Wind in the Willows?''. Since the answer to nearly any such question can now be found somewhere on the Web, the problem reduces to finding potential answers in large volumes of data and validating their accuracy. We apply a method for arbitrary passage retrieval to the first half of the problem and demonstrate that answer redundancy can be used to address the second half. The success of our approach depends on the idea that the volume of available Web data is large enough to supply the answer to most factual questions multiple times and in multiple contexts. A query is generated from a question and this query is used to select short passages that may contain the answer from a large collection of Web data. These passages are analyzed to identify candidate answers. The frequency of these candidates within the passages is used to ``vote'' for the most likely answer. The approach is experimentally tested on questions taken from the TREC-9 question-answering test collection. As an additional demonstration, the approach is extended to answer multiple choice trivia questions of the form typically asked in trivia quizzes and television game shows.

We address the challenge of automatically generating questions from reading materials for educational practice and assessment. Our approach is to overgenerate questions, then rank them. We use manually written rules to perform a sequence of general purpose syntactic transformations (e.g., subject-auxiliary inversion) to turn declarative sentences into questions. These questions are then ranked by a logistic regression model trained on a small, tailored dataset consisting of labeled output from our system. Experimental results show that ranking nearly doubles the percentage of questions rated as acceptable by annotators, from 27% of all questions to 52% of the top ranked 20% of questions.


Question: A completely automated system for creating crossword puzzles.

This paper presents an innovative method utilizing Support Vector Machines (SVMs) for the precise identification of English base phrases, commonly referred to as chunks. SVMs are renowned for their exceptional generalization capabilities, particularly when dealing with input data characterized by high-dimensional feature spaces. An additional advantage of SVMs is their ability to train models with minimal computational burden, thanks to the Kernel principle, which ensures efficiency regardless of the dimensionality of the data. Our approach involves the strategic use of eight SVM-based systems, each trained on unique chunk representations, which are then combined through weighted voting. This ensemble technique has proven to outperform previous methods, as demonstrated by our experimental findings, showcasing superior accuracy in chunk identification.

**Question**: A Hybrid Genetic Algorithm for the Vehicle Routing Problem with Time Windows

**Documents**: A Hybrid Genetic Algorithm for the Vehicle Routing Problem with Time Windows
The vehicle routing problem with time windows (VRPTW) is a complex combinatorial optimization problem in the field of transportation. This paper presents a hybrid genetic algorithm (HGA) for the VRPTW. The algorithm combines the strengths of a genetic algorithm with those of local search methods, specifically 2-opt and Lin-Kernighan heuristics, to improve solution quality. The HGA uses a modified genetic crossover operator to generate offspring that respect the time windows and capacity constraints of the vehicles. Additionally, a new mutation operator is introduced to diversify the search and prevent premature convergence. The HGA is tested on a set of benchmark instances from the literature, and the results are compared to those obtained by a standard genetic algorithm and other metaheuristics. The HGA outperforms the other methods in terms of solution quality and computational time.

**Similar Documents**: This paper introduces a novel Hybrid Genetic Algorithm (HGA) tailored to address the complexities of the Vehicle Routing Problem with Time Windows (VRPTW), a significant challenge in transportation logistics. The proposed HGA integrates the robustness of a genetic algorithm with the refinement of local search strategies, namely the 2-opt and Lin-Kernighan heuristics, to enhance the quality of solutions. A key feature of the HGA is its custom crossover operator, designed to produce offspring that adhere to both time window and vehicle capacity constraints. To further enrich the search landscape and circumvent early stagnation, a distinctive mutation operator has been devised. The effectiveness of the HGA is substantiated through comprehensive testing on established benchmark instances, with outcomes demonstrating superior solution quality and efficiency compared to conventional genetic algorithms and alternative metahe

We present a system to automatically generate natural language descriptions from images. This system consists of two parts. The first part, content planning, smooths the output of computer vision-based detection and recognition algorithms with statistics mined from large pools of visually descriptive text to determine the best content words to use to describe an image. The second step, surface realization, chooses words to construct natural language sentences based on the predicted content and general statistics from natural language. We present multiple approaches for the surface realization step and evaluate each using automatic measures of similarity to human generated reference descriptions. We also collect forced choice human evaluations between descriptions from the proposed generation system and descriptions from competing approaches. The proposed system is very effective at producing relevant sentences for images. It also generates descriptions that are notably more true to the specific image content than previous work.

WordNet is perhaps the most important and widely used lexical resource for natural language processing systems up to now. WordNet: An Electronic Lexical Database, edited by Christiane Fellbaum, discusses the design of WordNet from both theoretical and historical perspectives, provides an up-to-date description of the lexical database, and presents a set of applications of WordNet. The book contains a foreword by George Miller, an introduction by Christiane Fellbaum, seven chapters from the Cognitive Sciences Laboratory of Princeton University, where WordNet was produced, and nine chapters contributed by scientists from elsewhere. Miller's foreword offers a fascinating account of the history of WordNet. He discusses the presuppositions of such a lexical database, how the top-level noun categories were determined, and the sources of the words in WordNet. He also writes about the evolution of WordNet from its original incarnation as a dictionary browser to a broad-coverage lexicon, and the involvement of different people during its various stages of development over a decade. It makes very interesting reading for casual and serious users of WordNet and anyone who is grateful for the existence of WordNet. The book is organized in three parts. Part I is about WordNet itself and consists of four chapters: "Nouns in WordNet" by George Miller, "Modifiers in WordNet" by Katherine Miller, "A semantic network of English verbs" by Christiane Fellbaum, and "Design and implementation of the WordNet lexical database and search software" by Randee Tengi. These chapters are essentially updated versions of four papers from Miller (1990). Compared with the earlier papers, the chapters in this book focus more on the underlying assumptions and rationales behind the design decisions. The description of the information contained in WordNet, however, is not as detailed as in Miller (1990). The main new additions in these chapters include an explanation of sense grouping in George Miller's chapter, a section about adverbs in Katherine Miller's chapter, observations about autohyponymy (one sense of a word being a hyponym of another sense of the same word) and autoantonymy (one sense of a word being an antonym of another sense of the same word) in Fellbaum's chapter, and Tengi's description of the Grinder, a program that converts the files the lexicographers work with to searchable lexical databases. The three papers in Part II are characterized as "extensions, enhancements and

We address the challenge of automatically generating questions from reading materials for educational practice and assessment. Our approach is to overgenerate questions, then rank them. We use manually written rules to perform a sequence of general purpose syntactic transformations (e.g., subject-auxiliary inversion) to turn declarative sentences into questions. These questions are then ranked by a logistic regression model trained on a small, tailored dataset consisting of labeled output from our system. Experimental results show that ranking nearly doubles the percentage of questions rated as acceptable by annotators, from 27% of all questions to 52% of the top ranked 20% of questions.

In this paper we describe Paul, a robotic installation that produces observational sketches of people. The sketches produced have been considered of interest by fine art professionals in recent art fairs and exhibitions, as well as by the public at large. We identify factors that may account for the perceived qualities of the produced sketches. A technical overview of the system is also presented.

The typical CAPTCHA GUI consists of two parts: a character image with noise, and an and J. Saul, "Asirra: a CAPTCHA that exploits interest-aligned manual image categorization," in 14th ACM conference on Computer and Communications. CAPTCHAs (short for "Completely Automated Public Turing test to tell Asirra: A CAPTCHA that Exploits Interest-Aligned Manual Image Categorization. (30)J. Elson, J. R. Douceur, J. Howell, and J. Saul, “Asirra: A CAPTCHA that exploits interest-aligned manual image categorization,” in Proc. ACM CCS, 2007, pp.

For decades, researchers have debated the pros and cons of adaptive user interfaces with enthusiastic AI practitioners often confronting skeptical HCI experts (Shneiderman & Maes, 1997). This paper summarizes the SUPPLE project’s six years of work analyzing the characteristics of successful adaptive interfaces and developing efficient algorithms for their automatic generation (Gajos & Weld, 2004; Gajos et al., 2005; Gajos & Weld, 2005; Gajos et al., 2006; Gajos, Wobbrock, & Weld, 2007; Gajos et al., 2008; Gajos, Wobbrock, & Weld, 2008). We conclude that personalized user interfaces, which are adapted to a person’s devices, tasks, preferences and abilities, can improve user satisfaction and performance. Further, we demonstrate that automatic generation of these interfaces is computationally feasible. We developed two systems which autonomously construct a broad range of personalized adaptive interfaces: 1) SUPPLE/ARNAULD uses decision-theoretic optimization to automatically generate user interfaces adapted to target device, usage patterns, and a learned model of user preferences, and 2) SUPPLE++ first performs a one-time assessment of a person’s motor abilities and then automatically generates user interfaces adapted to that user’s abilities. Our experiments show that these automatically generated interfaces significantly improve speed, accuracy and satisfaction of users with motor impairments compared to manufacturers’ defaults. We also provide the first characterization of the design space of adaptive interfaces and demonstrate how such interfaces can significantly improve the quality and efficiency of daily interactions for typical users.

ANIMATE, HUMAN, etc. and encode type restrictions on nouns and adjectives and on the arguments of verbs. Subject codes use another set of primitives to classify senses of words by subject (ECONOMICS, ENGINEERING, etc.). Guthrie et al. (1991) demonstrate a typical use of this information: in addition to using the Lesk-based method of counting overlaps between definitions and contexts, they impose a correspondence of subject codes in an iterative process. No quantitative evaluation of this method is available, but Cowie et al. (1992) improve the method using simulated annealing and report results of 47% for sense distinctions and 72% for homographs. The use of LDOCE box codes, however, is problematic: the codes are not systematic (see, for example, Fontenelle, 1990); in later work, Braden-Harder (1993) showed that simply matching box or subject codes is not sufficient for disambiguation. For example, in I tipped the driver, the codes for several senses of the words in the sentence satisfy the necessary constraints (e.g. tip-money + human object or tip-tilt + movable solid object). In many ways, the supple7 Note that the assumptions underlying this method are very similar to Quillian’s: Thus one may think of a full concept analogically as consisting of all the information one would have if he looked up what will be called the “patriarch” word in a dictionary, then looked up every word in each of its definitions, then looked up every word found in each of these, and so on, continually branching outward[...] (Quillian, 1968, p. 238). However, Quillian’s network also keeps track of semantic relationships among the words encountered along the path between two words, which are encoded in his semantic network; the neural network avoids the overhead of creating the semantic network but loses this relational information. 13 mentary information in the LDOCE, and in particular the subject codes, are similar to those in a thesaurus, which, however, are more systematically structured. Inconsistencies in dictionaries, noted earlier, are not the only and perhaps not the major source of their limitations for WSD. While dictionaries provide detailed information at the lexical level, they lack pragmatic information that enters into sense determination (see, e.g., Hobbs, 1987). For example, the link between ash and tobacco, cigarette or tray in a network such as Quillian’s is very indirect, whereas in the Brown Corpus, the word ash co-occurs frequently with one of these words. It is therefore not surprising that corpora have become a primary source of information for WSD; this development is outlined below in section 2.3. 1.3.2 Thesauri. Thesauri provide information about relationships among words, most notably synonymy. Roget's International Thesaurus, which was put into machine-tractable form in the 1950's8 and has been used in a variety of applications including machine translation (Masterman, 1957), information retrieval (Sparck Jones, 1964, 1986), and content analysis (Sedelow and Sedelow, 1969; see also Sedelow and Sedelow, 1986, 1992), also supplies an explicit concept hierarchy consisting of up to eight increasingly refined levels. Typically, each occurrence of the same word under different categories of the thesaurus represent different senses of that word; i.e., the categories correspond roughly to word senses (Yarowsky, 1992). A set of words in the same category are semantically related. The earliest known use of Roget’s for WSD is the work of Masterman (1957), described above in section 2.1. Several years later, Patrick (1985) used Roget’s to discriminate among verb senses, by examining semantic clusters formed by “e-chains” derived from the thesaurus (Bryan, 1973, 1974; see also Sedelow and Sedelow, 1986). He uses “word-strong neighborhoods,” comprising word groups in low-level semicolon groups, which are the most closely related semantically in the thesaurus, and words connected to the group via chains. He is able to discriminate the correct sense of verbs such as inspire (to raise the spirits vs. to inhale, breathe in, sniff, etc.), question (to doubt vs. to ask a question) with “high reliability.” Bryan's earlier work had already demonstrated that homographs can be distinguished by applying a metric based on relationships defined by his chains (Bryan, 1973, 1974). Similar work is described in Sedelow and Mooney (1988). Yarowsky (1992) derives classes of words by starting with words in common categories in Roget's (4th ed.). A 100-word context of each word in the category is extracted from a corpus (the 1991 electronic text of Grolier's Ency8 The work of Masterman (1957) and Sparck Jones (1964) relied on a version of Roget’s that was hand-punched onto cards in the 1950’s; the Sedelow’s (1969) work relied on a machine readable version of the 3rd Edition. Roget’s is now widely available via anonymous ftp from various sites.

An automata theory course can be taught in an interactive, hands-on manner using a computer. At Duke we have been using the software tool JFLAP to provide interaction and feedback in CPS 140, our automata theory course. JFLAP is a tool for designing and running nondeterministic versions of finite automata, pushdown automata, and Turing machines. Recently, we have enhanced JFLAP to allow one to study the proofs of several theorems that focus on conversions of languages, from one form to another, such as converting an NFA to a DFA and then to a minimum state DFA. In addition, our enhancements combined with other tools allow one to interactively study LL and LR parsing methods.

Automatically solving algebra word problems has raised considerable interest recently. Existing state-of-the-art approaches mainly rely on learning from human annotated equations. In this paper, we demonstrate that it is possible to efficiently mine algebra problems and their numerical solutions with little to no manual effort. To leverage the mined dataset, we propose a novel structured-output learning algorithm that aims to learn from both explicit (e.g., equations) and implicit (e.g., solutions) supervision signals jointly. Enabled by this new algorithm, our model gains 4.6% absolute improvement in accuracy on the ALG514 benchmark compared to the one without using implicit supervision. The final model also outperforms the current state-of-the-art approach by 3%.

This paper presents a computational study of part of the lexical-acquisition task faced by children, namely the acquisition of word-to-meaning mappings. It first approximates this task as a formal mathematical problem. It then presents an implemented algorithm for solving this problem, illustrating its operation on a small example. This algorithm offers one precise interpretation of the intuitive notions of cross-situational learning and the principle of contrast applied between words in an utterance. It robustly learns a homonymous lexicon despite noisy multi-word input, in the presence of referential uncertainty, with no prior knowledge that is specific to the language being learned. Computational simulations demonstrate the robustness of this algorithm and illustrate how algorithms based on cross-situational learning and the principle of contrast might be able to solve lexical-acquisition problems of the size faced by children, under weak, worst-case assumptions about the type and quantity of data available.


Question: A completely automated system for creating crossword puzzles.

WebPuzzleMaster: A Web-Driven Solution for Crossword Enthusiasts
In the realm of artificial intelligence, language games offer a captivating challenge, exemplified by the creation of WebPuzzleMaster, a system designed to solve crosswords by leveraging the vast resources of the internet as its knowledge base. This innovative approach marks a departure from traditional literature, as it is the first solver capable of tackling non-English crosswords, with potential for multilingual capabilities. Despite being in its preliminary phase, WebPuzzleMaster has already achieved remarkable results, demonstrating performance comparable to that of a novice human player: crosswords deemed "easy" by experts are solved within competitive time frames, achieving an 80% accuracy in word selection and over 90% in letter accuracy.
Introduction
Crosswords, a globally popular linguistic puzzle, have long intrigued researchers in artificial intelligence. The focus has primarily been on the NP-complete crossword generation problem, which can now be efficiently solved for reasonable sizes. However, the challenge of solving crosswords from clues, considered AI-complete, is significantly more complex due to the absence of a closed-world assumption and the necessity for human-level knowledge. With the advent of advanced technologies such as search engines, information retrieval, and machine learning, AI systems are now capable of processing real-life concepts with semantic understanding. WebPuzzleMaster, our software system, is built on the premise of utilizing the internet as its primary knowledge source, recognizing its extensive and continuously updating repository of human knowledge.
This paper offers an overview of the system's modules and the various subproblems encountered in the project. Unlike the Proverb system, which achieved human-like performance through specific expert modules and a substantial crossword database, WebPuzzleMaster emphasizes the generality of its knowledge and language independence. At its core, the system features a module for specialized web-based question answering, termed clue-answering. The second objective, selecting the optimal set of candidate answers for filling the crossword grid, is addressed as a Probabilistic-CSP.
Italian crosswords, while structurally akin to American versions, include unique complexities such as two-letter words and references to socio-cultural and political topics. This introduces an additional layer of challenge, requiring a broad, up-to-date, and ambiguity-resilient knowledge base. To assess WebPuzzleMaster's capabilities, a dataset of 685 Italian crosswords was compiled, with 60 puzzles randomly selected for testing. The system's preliminary version has already shown promise, achieving an average of 70%

Surveying a suite of algorithms that offer a solution to managing large document archives.

A major obstacle to finding program errors in a real system is knowing what correctness rules the system must obey. These rules are often undocumented or specified in an ad hoc manner. This paper demonstrates techniques that automatically extract such checking information from the source code itself, rather than the programmer, thereby avoiding the need for a priori knowledge of system rules. The cornerstone of our approach is inferring programmer “beliefs” that we then cross-check for contradictions. Beliefs are facts implied by code: a dereference of a pointer, p, implies a belief that p is non-null, a call to “unlock(l)” implies that l was locked, etc. For beliefs we know the programmer must hold, such as the pointer dereference above, we immediately flag contradictions as errors. For beliefs that the programmer may hold, we can assume these beliefs hold and use a statistical analysis to rank the resulting errors from most to least likely. For example, a call to “spin lock” followed once by a call to “spin unlock” implies that the programmer may have paired these calls by coincidence. If the pairing happens 999 out of 1000 times, though, then it is probably a valid belief and the sole deviation a probable error. The key feature of this approach is that it requires no a priori knowledge of truth: if two beliefs contradict, we know that one is an error without knowing what the correct belief is. Conceptually, our checkers extract beliefs by tailoring rule “templates” to a system – for example, finding all functions that fit the rule template “<a> must be paired with <b>.” We have developed six checkers that follow this conceptual framework. They find hundreds of bugs in real systems such as Linux and OpenBSD. From our experience, they give a dramatic reduction in the manual effort needed to check a large system. Compared to our previous work [9], these template checkers find ten to one hundred times more rule instances and derive properties we found impractical to specify manually.

The world wide web represents vast stores of information. However, the sheer amount of such information makes it practically impossible for any human user to be aware of much of it. Therefore, it would be very helpful to have a system that automatically discovers relevant, yet previously unknown information, and reports it to users in human-readable form. As the first attempt to accomplish such a goal, we proposed a new clustering algorithm and compared it with existing clustering algorithms. The proposed method is motivated by constructive and competitive learning from neural network research. In the construction phase, it tries to find the optimal number of clusters by adding a new cluster when the intrinsic difference between the instance presented and the existing clusters is detected. Each cluster then moves toward the optimal cluster center according to the learning rate by adjusting its weight vector. From the experimental results on the three different real world data sets, the proposed method shows an even trend of performance across the different domains, while the performance of our algorithm on text domains was better than that reported in previous research.

Manually querying search engines in order to accumulate a large bodyof factual information is a tedious, error-prone process of piecemealsearch. Search engines retrieve and rank potentially relevantdocuments for human perusal, but do not extract facts, assessconfidence, or fuse information from multiple documents. This paperintroduces KnowItAll, a system that aims to automate the tedious process ofextracting large collections of facts from the web in an autonomous,domain-independent, and scalable manner.The paper describes preliminary experiments in which an instance of KnowItAll, running for four days on a single machine, was able to automatically extract 54,753 facts. KnowItAll associates a probability with each fact enabling it to trade off precision and recall. The paper analyzes KnowItAll's architecture and reports on lessons learned for the design of large-scale information extraction systems.

Answering questions correctly from standardized eighth-grade science tests is itself a test of machine intelligence.

Given an image, how do we automatically assign keywords to it? In this paper, we propose a novel, graph-based approach (GCap) which outperforms previously reported methods for automatic image captioning. Moreover, it is fast and scales well, with its training and testing time linear to the data set size. We report auto-captioning experiments on the "standard" Corel image database of 680 MBytes, where GCap outperforms recent, successful auto-captioning methods by up to 10 percentage points in captioning accuracy (50% relative improvement).

We describe new applications of the theory of automata to natural language processing: the representation of very large scale dictionaries and the indexation of natural language texts. They are based on new algorithms that we introduce and describe in detail. In particular, we give pseudocodes for the de-terminization of string to string transducers, the deterministic union of p-subsequential string to string transducers, and the indexation by automata. We report several experiments illustrating the applications.

Paper pop-ups are interesting three-dimensional books that fascinate people of all ages. The design and construction of these pop-up books however are done manually and require a lot of time and effort. This has led to computer-assisted or automated tools for designing paper pop-ups. This paper proposes an approach for automatically converting a 3D model into a multi-style paper pop-up. Previous automated approaches have only focused on single-style pop-ups, where each is made of a single type of pop-up mechanisms. In our work, we combine multiple styles in a pop-up, which is more representative of actual artist’s creations. Our method abstracts a 3D model using suitable primitive shapes that both facilitate the formation of the considered pop-up mechanisms and closely approximate the input model. Each shape is then abstracted using a set of 2D patches that combine to form a valid pop-up. We define geometric conditions that ensure the validity of the combined pop-up structures. In addition, our method also employs an image-based approach for producing the patches to preserve the textures, finer details and important contours of the input model. Finally, our system produces a printable design layout and decides an assembly order for the construction instructions. The feasibility of our results is verified by constructing the actual paper pop-ups from the designs generated by our system.

Destination maps are navigational aids designed to show anyone within a region how to reach a location (the destination). Hand-designed destination maps include only the most important roads in the region and are non-uniformly scaled to ensure that all of the important roads from the highways to the residential streets are visible. We present the first automated system for creating such destination maps based on the design principles used by mapmakers. Our system includes novel algorithms for selecting the important roads based on mental representations of road networks, and for laying out the roads based on a non-linear optimization procedure. The final layouts are labeled and rendered in a variety of styles ranging from informal to more formal map styles. The system has been used to generate over 57,000 destination maps by thousands of users. We report feedback from both a formal and informal user study, as well as provide quantitative measures of success.

This paper presents a cross platform multilingual multimedia Indian Sign Language (ISL) dictionary building tool. ISL is a linguistically under-investigated language with no source of well documented electronic data. Research on ISL linguistics also gets hindered due to a lack of ISL knowledge and the unavailability of any educational tools. Our system can be used to associate signs corresponding to a given text. The current system also facilitates the phonological annotation of Indian signs in the form of HamNoSys structure. The generated HamNoSys string can be given as input to an avatar module to produce an animated sign representation.


Question: A statistical syntax-directed translation method utilizing an extended domain of locality.

Introducing SRILM: A Comprehensive Language Modeling Toolkit for Statistical Syntax-Directed Translation

SRILM, a suite of C++ libraries, programs, and auxiliary scripts, has been developed to facilitate the production and experimentation of statistical language models, specifically for speech recognition and related applications. This toolkit is offered free of charge for non-commercial use. SRILM's capabilities encompass the creation and assessment of various language model types grounded in N-gram statistics, in addition to ancillary functions such as statistical tagging and the manipulation of N-best lists and word lattices. This paper outlines the toolkit's comprehensive features, delving into its design philosophy and implementation, with a focus on the toolkit's rapid prototyping capabilities, reusability, and the seamless integration of its tools. SRILM's design emphasizes flexibility and efficiency, making it a powerful resource for researchers and developers working in the field of statistical syntax-directed translation and language modeling.

In most of the dependency parsing studies, dependency relations within a sentence are often presented as a tree structure. Whilst the tree structure is sufficient to represent the surface relations, deep dependencies which may result to multi-headed relations require more general dependency structures, namely Directed Acyclic Graphs (DAGs). This study proposes a new dependency DAG parsing approach which uses a dynamic oracle within a shift-reduce transitionbased parsing framework. Although there is still room for improvement on performance with more feature engineering, we already obtain competitive performances compared to static oracles as a result of our initial experiments conducted on the ITU-METU-Sabancı Turkish Treebank (IMST).

MAXIMUM ENTROPY MODELS FOR NATURAL LANGUAGE AMBIGUITY RESOLUTION Adwait Ratnaparkhi Supervisor Professor Mitch Marcus This thesis demonstrates that several important kinds of natural language ambiguities can be resolved to state of the art accuracies using a single statistical modeling technique based on the principle of maximum entropy We discuss the problems of sentence boundary detection part of speech tagging prepo sitional phrase attachment natural language parsing and text categorization under the maximum entropy framework In practice we have found that maximum entropy models o er the following advantages State of the art Accuracy The probability models for all of the tasks discussed per form at or near state of the art accuracies or outperform competing learning algo rithms when trained and tested under similar conditions Methods which outperform those presented here require much more supervision in the form of additional human involvement or additional supporting resources Knowledge Poor Features The facts used to model the data or features are linguis tically very simple or knowledge poor but yet succeed in approximating complex linguistic relationships Reusable Software Technology The mathematics of the maximum entropy framework

In this paper, we propose a probabilistic parsing model that defines a proper conditional probability distribution over nonprojective dependency trees for a given sentence, using neural representations as inputs. The neural network architecture is based on bi-directional LSTMCNNs, which automatically benefits from both wordand character-level representations, by using a combination of bidirectional LSTMs and CNNs. On top of the neural network, we introduce a probabilistic structured layer, defining a conditional log-linear model over nonprojective trees. By exploiting Kirchhoff’s Matrix-Tree Theorem (Tutte, 1984), the partition functions and marginals can be computed efficiently, leading to a straightforward end-to-end model training procedure via back-propagation. We evaluate our model on 17 different datasets, across 14 different languages. Our parser achieves state-of-the-art parsing performance on nine datasets.

Kleene's regular expressions, which can be used for describing sequential circuits, were defined using three operators (union, concatenation and iterate) on sets of sequences. Word descriptions of problems can be more easily put in the regular expression language if the language is enriched by the inclusion of other logical operations. However, il~ the problem of converting the regular expression description to a state diagram, the existing methods either cannot handle expressions with additional operators, or are made quite complicated by the presence of such operators. In this paper the notion of a derivative of a regular expression is introduced atld the properties of derivatives are discussed. This leads, in a very natural way, to the construction of a state diagram from a regular expression containing any number of logical operators.

We propose a new hierarchical Bayesian n-gram model of natural languages. Our model makes use of a generalization of the commonly used Dirichlet distributions called Pitman-Yor processes which produce power-law distributions more closely resembling those in natural languages. We show that an approximation to the hierarchical Pitman-Yor language model recovers the exact formulation of interpolated Kneser-Ney, one of the best smoothing methods forn-gram language models. Experiments verify that our model gives cross entropy results superior to interpolated Kneser-Ney and comparable to modified Kneser-Ney.

Most neural machine translation (NMT) models are based on the sequential encoder-decoder framework, which makes no use of syntactic information. In this paper, we improve this model by explicitly incorporating source-side syntactic trees. More specifically, we propose (1) a bidirectional tree encoder which learns both sequential and tree structured representations; (2) a tree-coverage model that lets the attention depend on the source-side syntax. Experiments on Chinese-English translation demonstrate that our proposed models outperform the sequential attentional model as well as a stronger baseline with a bottom-up tree encoder and word coverage.1

A PCFGparsingcandramaticallyreducethetime requiredto find theexactViterbi parseby conservatively estimatingoutsideViterbi probabilities.Wediscussvariousestimatesandgive efficientalgorithmsfor computingthem.On Penntreebanksentences, ourmostdetailedestimate reducesthetotalnumberof edgesprocessedto lessthan3%of thatrequiredby exhausti ve parsing,andevenasimplerestimatewhichcanbepre-computedin underaminutestill reduces thework by a factorof 5. ThealgorithmextendstheclassicA graphsearchprocedureto a certainhypergraphassociatedwith parsing.Unlike best-firstandfinite-beammethodsfor achieving thiskind of speed-up, theA parseris guaranteedto returnthemostlikely parse,not justanapproximation.Thealgorithmis alsocorrectfor awide rangeof parsercontrolstrategies andmaintainsaworst-casecubictime bound. A Parsing: FastExact Viterbi ParseSelection

We explore efficient domain adaptation for the task of statistical machine translation based on extracting sentences from a large generaldomain parallel corpus that are most relevant to the target domain. These sentences may be selected with simple cross-entropy based methods, of which we present three. As these sentences are not themselves identical to the in-domain data, we call them pseudo in-domain subcorpora. These subcorpora – 1% the size of the original – can then used to train small domain-adapted Statistical Machine Translation (SMT) systems which outperform systems trained on the entire corpus. Performance is further improved when we use these domain-adapted models in combination with a true in-domain model. The results show that more training data is not always better, and that best results are attained via proper domain-relevant data selection, as well as combining inand general-domain systems during decoding.

Discriminative reranking is one method for constructing high-performance statistical parsers (Collins, 2000). A discriminative reranker requires a source of candidate parses for each sentence. This paper describes a simple yet novel method for constructing sets of 50-best parses based on a coarse-to-fine generative parser (Charniak, 2000). This method generates 50-best lists that are of substantially higher quality than previously obtainable. We used these parses as the input to a MaxEnt reranker (Johnson et al., 1999; Riezler et al., 2002) that selects the best parse from the set of parses for each sentence, obtaining an f-score of 91.0% on sentences of length 100 or less.

Text normalization is an important first step towards enabling many Natural Language Processing (NLP) tasks over informal text. While many of these tasks, such as parsing, perform the best over fully grammatically correct text, most existing text normalization approaches narrowly define the task in the word-to-word sense; that is, the task is seen as that of mapping all out-of-vocabulary non-standard words to their in-vocabulary standard forms. In this paper, we take a parser-centric view of normalization that aims to convert raw informal text into grammatically correct text. To understand the real effect of normalization on the parser, we tie normalization performance directly to parser performance. Additionally, we design a customizable framework to address the often overlooked concept of domain adaptability, and illustrate that the system allows for transfer to new domains with a minimal amount of data and effort. Our experimental study over datasets from three domains demonstrates that our approach outperforms not only the state-of-the-art wordto-word normalization techniques, but also manual word-to-word annotations.


Question: A statistical syntax-directed translation method with an extended domain of locality.

This paper outlines the alignment template approach, a phrase-based statistical machine translation method that accommodates versatile many-to-many word relationships. By considering the context of words within the translation model, this approach can explicitly learn and adapt to local variations in word order between source and target languages. The model is articulated through a log-linear framework, offering a more flexible alternative to the conventional source-channel model, thereby facilitating easier extensions. The learning process for phrasal translations, the feature functions applied, and the search algorithm are detailed. Evaluation of the method is conducted across three distinct translation tasks. For the German-English speech translation task of Verbmobil, the study dissects the impact of various system components. On the French-English Hansards translation task, the alignment template system outperforms a single-word-based model, demonstrating statistically significant improvements. In the 2002 National Institute of Standards and Technology (NIST) Chinese-English machine translation evaluation, the system achieves statistically significant higher NIST scores compared to both research and commercial translation systems, showcasing its efficacy in diverse translation scenarios.

A connection between universal codes and the problems of prediction and statistical estimation is established. A‘known lower bound for the mean length of universal codes is sharpened and generalized, and optimum universal codes constructed. The bound is defined to give the information in strings relative to the considered class of processes. The earlier derived minimum description length criterion for estimation of parameters, including their number, is given a fundamental information, theoretic justification by showing that its estimators achieve the information in the strings. It is also shown that one cannot do prediction in Manuscript received July 13,1983; revised January 16,1984. This work was presented in part at the IEEE International Symposium on Information Theory, St. Jovite, Canada, September 26-30, 1983. This work was done while the author was Visiting Professor at the Department of System Science, University of California, Los Angeles, while on leave from the IBM Research Laboratory, San Jose, CA 95193. Gaussian autoregressive moving average (ARMA) processes below a bound, which is determined by the information in the data.

Spelling error is broadly classified in two categories namely non word error and real word error. In this paper a localized real word error detection and correction method is proposed where the scores of bigrams generated by immediate left and right neighbour of the candidate word and the trigram of these three words are combined. A single character position error model is assumed so that if a word W is erroneous then the correct word belongs to the set of real words S generated by single character edit operation on W. The above combined score is calculated also on all members of S. These words are ranked in the decreasing order of the score. By observing the rank and using a rule based approach, the error decision and correction candidates are simultaneously selected. The approach gives comparable accuracy with other existing approaches but is computationally attractive. Since only left and right neighbor are involved, multiple errors in a sentence can also be detected ( if the error occurs in every alternate words ).

Presents parameter estimation methods common with discrete probability distributions, which is of particular interest in text modeling. Starting with maximum likelihood, a posteriori and Bayesian estimation, central concepts like conjugate distributions and Bayesian networks are reviewed. As an application, the model of latent Dirichlet allocation (LDA) is explained in detail with a full derivation of an approximate inference algorithm based on Gibbs sampling, including a discussion of Dirichlet hyperparameter estimation. History: version 1: May 2005, version 2.4: August 2008.

The $64,000 question in computational linguistics these days is: “What should I read to learn about statistical natural language processing?” I have been asked this question over and over, and each time I have given basically the same reply: there is no text that addresses this topic directly, and the best one can do is find a good probability-theory textbook and a good information-theory textbook, and supplement those texts with an assortment of conference papers and journal articles. Understanding the disappointment this answer provoked, I was delighted to hear that someone had finally written a bookdirectly addressing this topic. However, after reading Eugene Charniak’s Statistical Language Learning, I have very mixed feelings about the impact this bookmight have on the ever-growing field of statistical NLP. The book begins with a very brief description of the classic artificial intelligence approach toNLP (chapter 1), including morphology, syntax, semantics, and pragmatics. It presents a few definitions from probability theory and information theory (chapter 2), then proceeds to introduce hidden Markov models (chapters 3–4) and probabilistic context-free grammars (chapters 5–6). The book concludes with a few chapters discussing advanced topics in statistical language learning, such as grammar induction (chapter 7), syntactic disambiguation (chapter 8), word clustering (chapter 9), andword sense disambiguation (chapter 10). To its credit, the book serves as an interesting popular discussion of statistical modeling in NLP. It is well-written and entertaining, and very accessible to the reader with a limited mathematical background. It presents a good selection of statistical NLP topics to introduce the reader to the field. And the descriptions of the forward-backward algorithm for hidden Markov models and the inside-outside algorithm for probabilistic context-free grammars are intuitive and easy to follow. However, as a resource for someone interested in entering this area of research, this book falls far short of its author’s goals. These goals are clearly stated in the preface:

A key question facing the parsing community is how to compare parsers which use different grammar formalisms and produce different output. Evaluating a parser on the same resource used to create it can lead to non-comparable accuracy scores and an over-optimistic view of parser performance. In this paper we evaluate a CCG parser on DepBank, and demonstrate the difficulties in converting the parser output into DepBank grammatical relations. In addition we present a method for measuring the effectiveness of the conversion, which provides an upper bound on parsing accuracy. The CCG parser obtains an F-score of 81.9% on labelled dependencies, against an upper bound of 84.8%. We compare the CCG parser against the RASP parser, outperforming RASPby over 5% overall and on the majority of dependency types.

This paper evaluates the translation quality of machine translation systems for 8 language pairs: translating French, German, Spanish, and Czech to English and back. We carried out an extensive human evaluation which allowed us not only to rank the different MT systems, but also to perform higher-level analysis of the evaluation process. We measured timing and intraand inter-annotator agreement for three types of subjective evaluation. We measured the correlation of automatic evaluation metrics with human judgments. This meta-evaluation reveals surprising facts about the most commonly used methodologies.

In this paper we present an overview of recent developments in discourse theory and parsing under the Linguistic Discourse Model (LDM) framework, a semantic theory of discourse structure. We give a novel approach to the problem of discourse segmentation based on discourse semantics and sketch a limited but robust approach to symbolic discourse parsing based on syntactic, semantic and lexical rules. To demonstrate the utility of the system in a real application, we briefly describe the architecture of the PALSUMM system, a symbolic summarization system being developed at FX Palo Alto Laboratory that uses discourse structures constructed using the theory outlined to summarize written English prose texts. 1

This paper will focus on the semantic representation of verbs in computer systems and its impact on lexical selection problems in machine translation (MT). Two groups of English and Chinese verbs are examined to show that lexical selection must be based on interpretation of the sentence as well as selection restrictions placed on the verb arguments. A novel representation scheme is suggested, and is compared to representations with selection restrictions used in transfer-based MT. We see our approach as closely aligned with knowledge-based MT approaches (KBMT), and as a separate component that could be incorporated into existing systems. Examples and experimental results will show that, using this scheme, inexact matches can achieve correct lexical selection.

We introduce a neural semantic parser which is interpretable and scalable. Our model converts natural language utterances to intermediate, domain-general natural language representations in the form of predicate-argument structures, which are induced with a transition system and subsequently mapped to target domains. The semantic parser is trained end-to-end using annotated logical forms or their denotations. We achieve the state of the art on SPADES and GRAPHQUESTIONS and obtain competitive results on GEOQUERY and WEBQUESTIONS. The induced predicate-argument structures shed light on the types of representations useful for semantic parsing and how these are different from linguistically motivated ones.1

We have developed Preksha: a Hindi text visualizer. Preksha generates a 3D scene by understanding a given Hindi text. We propose the Earley's parsing algorithm using Tree adjoining grammar (TAG) for Hindi language understanding. A brief overview of the TAG formalism and the Earley's parsing algorithm is presented. Preksha's Hindi language parsing is explainedwith the Lisp notation. The maincontribution of this paper is to apply TAG parsing for morphologically-richfree-order language Hindi for automatic text visualization (ATV).


Question: A statistical syntax-directed translation method using an extended domain of locality.

This paper presents an in-depth exploration of five statistical models designed to capture the intricacies of the translation process, alongside algorithms for estimating their parameters. Given a corpus of sentence pairs known to be translations of each other, these models enable the computation of probabilities associated with various word-by-word alignments between the sentences. A key contribution is the development of an algorithm aimed at identifying the most probable alignment, despite its suboptimal nature. This alignment effectively captures the lexical correspondences across the bilingual sentences. Our extensive French-English dataset, derived from the Canadian Parliamentary proceedings, has been instrumental in our research. Although our focus has been on these two languages, the universal linguistic simplicity of our algorithms suggests their potential effectiveness across a variety of language pairs. We argue that word-by-word alignments are a fundamental characteristic of any sizable bilingual corpus, a hypothesis supported by the minimal linguistic assumptions embedded in our methodologies.

Existing entailment datasets mainly pose problems which can be answered without attention to grammar or word order. Learning syntax requires comparing examples where different grammar and word order change the desired classification. We introduce several datasets based on synthetic transformations of natural entailment examples in SNLI or FEVER, to teach aspects of grammar and word order. We show that without retraining, popular entailment models are unaware that these syntactic differences change meaning. With retraining, some but not all popular entailment models can learn to compare the syntax properly.

We have developed Preksha: a Hindi text visualizer. Preksha generates a 3D scene by understanding a given Hindi text. We propose the Earley's parsing algorithm using Tree adjoining grammar (TAG) for Hindi language understanding. A brief overview of the TAG formalism and the Earley's parsing algorithm is presented. Preksha's Hindi language parsing is explainedwith the Lisp notation. The maincontribution of this paper is to apply TAG parsing for morphologically-richfree-order language Hindi for automatic text visualization (ATV).

It is difficult to develop style-preserving source-to-source transformation engines for C and C++. The main reason is not the complexity of those languages, but the use of the C pre-processor (cpp), especially ifdefs and macros. This has for example hindered the development of refactoring tools for C and C++. In this paper we propose to combine multiple techniques and heuristics to parse C/C++ source files as-is, while still having only a few modifications to the original grammars of C and C++. We rely on the fact that in most C and C++ software, programmers follow a limited number of conventions on the use of cpp which makes it possible to disambiguate different situations by just looking at the context, names, or indentation of cpp constructs. We have implemented a parser, Yacfe, based on these techniques and evaluated it on 16 large open source projects. Yacfe can on average parse 96% of those projects correctly. As a side effect, we also found mistakes in code that was not compiled because it was protected by particular ifdefs, but that was still analyzed by Yacfe. Using Yacfe on new projects may require adapting some of our techniques. We found that as conventions and idioms are shared by many projects, the adaptation time is on average less than 2 hours for a new project.

Statistical machine translation (SMT) performance suffers when models are trained on only small amounts of parallel data. The learned models typically have both low accuracy (incorrect translations and feature scores) and low coverage (high out-of-vocabulary rates). In this work, we use an additional data resource, comparable corpora, to improve both. Beginning with a small bitext and corresponding phrase-based SMT model, we improve coverage by using bilingual lexicon induction techniques to learn new translations from comparable corpora. Then, we supplement the model’s feature space with translation scores estimated over comparable corpora in order to improve accuracy. We observe improvements between 0.5 and 1.7 BLEU translating Tamil, Telugu, Bengali, Malayalam, Hindi, and Urdu into English.

Grammatical error correction (GEC) is the task of automatically correcting grammatical errors in written text. Earlier attempts to grammatical error correction involve rule-based and classifier approaches which are limited to correcting only some particular type of errors in a sentence. As sentences may contain multiple errors of different types, a practical error correction system should be able to detect and correct all errors. In this report, we investigate GEC as a translation task from incorrect to correct English and explore some machine translation approaches for developing end-to-end GEC systems for all error types. We apply Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) approaches to GEC and show that they can correct multiple errors of different types in a sentence when compared to the earlier methods which focus on individual errors. We also discuss some of the weakness of machine translation approaches. Finally, we also experiment on a candidate re-ranking technique to re-rank the hypotheses generated by machine translation systems. With regression models, we try to predict a grammaticality score for each candidate hypothesis and re-rank them according to the score.

We describe a framework for inducing probabilistic grammars from corpora of positive samples. First, samples are incorporated by adding ad-hoc rules to a working grammar; subsequently, elements of the model (such as states or nonterminals) are merged to achieve generalization and a more compact representation. The choice of what to merge and when to stop is governed by the Bayesian posterior probability of the grammar given the data, which formalizes a trade-off between a close fit to the data and a default preference for simpler models (‘Occam’s Razor’). The general scheme is illustrated using three types of probabilistic grammars: Hidden Markov models, class-based n-grams, and stochastic context-free grammars.

Kleene's regular expressions, which can be used for describing sequential circuits, were defined using three operators (union, concatenation and iterate) on sets of sequences. Word descriptions of problems can be more easily put in the regular expression language if the language is enriched by the inclusion of other logical operations. However, il~ the problem of converting the regular expression description to a state diagram, the existing methods either cannot handle expressions with additional operators, or are made quite complicated by the presence of such operators. In this paper the notion of a derivative of a regular expression is introduced atld the properties of derivatives are discussed. This leads, in a very natural way, to the construction of a state diagram from a regular expression containing any number of logical operators.

Long and complicated sentences prove to be a stumbling block for current systems relying on NL input. These systems stand to gain from methods that syntactically simplify such sentences. To simplify a sentence, we need an idea of the structure of the sentence, to identify the components to be separated out. Obviously a parser could be used to obtain the complete structure of the sentence. However, full parsing is slow and prone to failure, especially on complex sentences. In this paper, we consider two alternatives to full parsing which could be used for simpli cation. The rst approach uses a Finite State Grammar (FSG) to produce noun and verb groups while the second uses a Supertagging model to produce dependency linkages. We discuss the impact of these two input representations on the simpli cation process. 1 Reasons for Text Simpli cation Long and complicated sentences prove to be a stumbling block for current systems which rely on natural language input. These systems stand to gain from methods that preprocess such sentences so as to make them simpler. Consider, for example, the following sentence: (1) The embattled Major government survived a crucial vote on coal pits closure as its last-minute concessions curbed the extent of Tory revolt over an issue that generated unusual heat in the House of Commons and brought the miners to London streets. Such sentences are not uncommon in newswire texts. Compare this with the multi-sentence version which has been manually simpli ed: (2) The embattled Major government survived a crucial vote on coal pits closure. Its last-minute concessions curbed the extent of On leave from the National Centre for Software Technology, Gulmohar Cross Road No. 9, Juhu, Bombay 400 049, India Tory revolt over the coal-mine issue. This issue generated unusual heat in the House of Commons. It also brought the miners to

We investigate the identification and, to some extent, the classification of collocational word groups that consist of an adjectival modifier (A), an accusative object noun (N), and a verb (V) by means of parsing a newspaper corpus with a lexicalized probabilistic grammar.1 Data triples are extracted from the resulting Viterbi parses and subsequently evaluated by means of a statistical association measure. The extraction results are then compared to predefined descriptive classes of ANV-triples. We also use a decision tree algorithm to classify part of the data obtained, on the basis of a small set of manually classified examples. Much of the candidate data is lexicographically relevant: the triples include idiomatic combinations (e.g. (sich) eine goldene Nase verdienen, ‘get rich’, lit. ‘earn oneself a golden nose’), combinations of N+V and N+A collocations (e.g. (eine) klare Absage erteilen, ‘refuse resolutely’, lit. ‘give a clear refusal’), next to cases where N+V or N+A collocations are found, in combination with other (not necessarily collocational) context partners. To extract such data from text corpora, a grammar is needed that captures verb+object relations: simple pattern matching on part-of-speech shapes is not sufficient. Statistical tools then allow to order the data in a way useful for subsequent manual selection by lexicographers. 1This work has been carried out in the context of the Transferbereich 32: Automatische Exzerption, a DFG-funded project aiming at the creation of support tools for the corpus-based updating of printed dictionaries in lexicography, carried out in cooperation with the publishers Langenscheidt KG and Duden BIFAB AG.

Existing neural semantic parsers mainly utilize a sequence encoder, i.e., a sequential LSTM, to extract word order features while neglecting other valuable syntactic information such as dependency graph or constituent trees. In this paper, we first propose to use the syntactic graph to represent three types of syntactic information, i.e., word order, dependency and constituency features. We further employ a graph-to-sequence model to encode the syntactic graph and decode a logical form. Experimental results on benchmark datasets show that our model is comparable to the state-ofthe-art on Jobs640, ATIS and Geo880. Experimental results on adversarial examples demonstrate the robustness of the model is also improved by encoding more syntactic information.