Question: An empirical investigation into unsupervised Chinese word segmentation techniques for SMT on large-scale corpora.

This document details the outcomes of the Second International Chinese Word Segmentation Bakeoff, an event conducted in the summer of 2005 to assess the latest advancements in the field of word segmentation. Twenty-three participating teams contributed a total of 130 result sets across two competition tracks, using four distinct corpora for evaluation. The findings revealed a notable enhancement in word segmentation technology since the last bakeoff, held two years prior. However, the challenge of out-of-vocabulary words remains a critical issue, underscoring its significance in the ongoing development of segmentation techniques. Despite the progress made, the out-of-vocabulary problem continues to demand attention, highlighting the need for further research and innovation in this area.

This study describes the construction of the TOCFL (Test Of Chinese as a Foreign Language) learner corpus, including the collection and grammatical error annotation of 2,837 essays written by Chinese language learners originating from a total of 46 different mother-tongue languages. We propose hierarchical tagging sets to manually annotate grammatical errors, resulting in 33,835 inappropriate usages. Our built corpus has been provided for the shared tasks on Chinese grammatical error diagnosis. These demonstrate the usability of our learner corpus annotation.

Since sentences in patent texts are long, they are difficult to translate by a machine. Although statistical machine translation is one of the major streams of the field, long patent sentences are difficult to translate not using syntactic analysis. We propose the combination of a rule based method and a statistical method. It is a rule based machine translation (RMT) with a statistical based post editor (SPE). The evaluation by the NIST score shows RMT+SPE is more accurate than RMT only. Manual checks, however, show the outputs of RMT+SPE often have strange expressions in the target language. So we propose a new evaluation measure NMG (normalized mean grams). Although NMG is based on n-gram, it counts the number of words in the longest word sequence matches between the test sentence and the target language reference corpus. We use two reference corpora. One is the reference translation only the other is a large scaled target language corpus. In the former case, RMT+SPE wins in the later case, RMT wins.

This paper investigates neural characterbased morphological tagging for languages with complex morphology and large tag sets. Character-based approaches are attractive as they can handle rarelyand unseen words gracefully. We evaluate on 14 languages and observe consistent gains over a state-of-the-art morphological tagger across all languages except for English and French, where we match the state-of-the-art. We compare two architectures for computing characterbased word vectors using recurrent (RNN) and convolutional (CNN) nets. We show that the CNN based approach performs slightly worse and less consistently than the RNN based approach. Small but systematic gains are observed when combining the two architectures by ensembling.

Generating an article automatically with computer program is a challenging task in artificial intelligence and natural language processing. In this paper, we target at essay generation, which takes as input a topic word in mind and generates an organized article under the theme of the topic. We follow the idea of text planning (Reiter and Dale, 1997) and develop an essay generation framework. The framework consists of three components, including topic understanding, sentence extraction and sentence reordering. For each component, we studied several statistical algorithms and empirically compared between them in terms of qualitative or quantitative analysis. Although we run experiments on Chinese corpus, the method is language independent and can be easily adapted to other language. We lay out the remaining challenges and suggest avenues for future research.

We explore efficient domain adaptation for the task of statistical machine translation based on extracting sentences from a large generaldomain parallel corpus that are most relevant to the target domain. These sentences may be selected with simple cross-entropy based methods, of which we present three. As these sentences are not themselves identical to the in-domain data, we call them pseudo in-domain subcorpora. These subcorpora – 1% the size of the original – can then used to train small domain-adapted Statistical Machine Translation (SMT) systems which outperform systems trained on the entire corpus. Performance is further improved when we use these domain-adapted models in combination with a true in-domain model. The results show that more training data is not always better, and that best results are attained via proper domain-relevant data selection, as well as combining inand general-domain systems during decoding.

We demonstrate that an unlexicalized PCFG can parse much more accurately than previously shown, by making use of simple, linguistically motivated state splits, which break down false independence assumptions latent in a vanilla treebank grammar. Indeed, its performance of 86.36% (LP/LR F 1) is better than that of earlylexicalizedPCFG models, and surprisingly close to the current state-of-theart. This result has potential uses beyond establishing a strong lower bound on the maximum possible accuracy of unlexicalized models: an unlexicalized PCFG is much more compact, easier to replicate, and easier to interpret than more complex lexical models, and the parsing algorithms are simpler, more widely understood, of lower asymptotic complexity, and easier to optimize. In the early 1990s, as probabilistic methods swept NLP, parsing work revived the investigation of probabilistic context-free grammars ( PCFGs) (Booth and Thomson, 1973; Baker, 1979). However, early results on the utility ofPCFGs for parse disambiguation and language modeling were somewhat disappointing. A conviction arose that lexicalizedPCFGs (where head words annotate phrasal nodes) were the key tool for high performancePCFG parsing. This approach was congruent with the great success of word n-gram models in speech recognition, and drew strength from a broader interest in lexicalized grammars, as well as demonstrations that lexical dependencies were a key tool for resolving ambiguities such asPPattachments (Ford et al., 1982; Hindle and Rooth, 1993). In the following decade, great success in terms of parse disambiguation and even language modeling was achieved by various lexicalized PCFG models (Magerman, 1995; Charniak, 1997; Collins, 1999; Charniak, 2000; Charniak, 2001). However, several results have brought into question how large a role lexicalization plays in such parsers. Johnson (1998) showed that the performance of anunlexicalizedPCFGover the Penn treebank could be improved enormously simply by annotating each node by its parent category. The Penn treebank coveringPCFGis a poor tool for parsing because the context-freedom assumptions it embodies are far too strong, and weakening them in this way makes the model much better. More recently, Gildea (2001) discusses how taking the bilexical probabilities out of a good current lexicalized PCFG parser hurts performance hardly at all: by at most 0.5% for test text from the same domain as the training data, and not at all for test text from a different domain. 1 But it is precisely these bilexical dependencies that backed the intuition that lexicalized PCFGs should be very successful, for example in Hindle and Rooth’s demonstration fromPPattachment. We take this as a reflection of the fundamental sparseness of the lexical dependency information available in the Penn Treebank. As a speech person would say, one million words of training data just isn’t enough. Even for topics central to the treebank’s Wall Street Journal text, such as stocks, many very plausible dependencies occur only once, for example stocks stabilized, while many others occur not at all, for example stocks skyrocketed .2 The best-performing lexicalized PCFGs have increasingly made use of subcategorization 3 of the 1There are minor differences, but all the current best-known lexicalized PCFGs employ bothmonolexicalstatistics, which describe the phrasal categories of arguments and adjuncts t hat appear around a head lexical item, and bilexicalstatistics, or dependencies, which describe the likelihood of a head word tak ing as a dependent a phrase headed by a certain other word. 2This observation motivates various classor similaritybased approaches to combating sparseness, and this remains a promising avenue of work, but success in this area has proven somewhat elusive, and, at any rate, current lexicalized PCFGs do simply use exact word matches if available, and interpola te with syntactic category-based estimates when they are not. 3In this paper we use the term subcategorizationin the original general sense of Chomsky (1965), for where a syntactic ca tcategories appearing in the Penn treebank. Charniak (2000) shows the value his parser gains from parentannotation of nodes, suggesting that this information is at least partly complementary to information derivable from lexicalization, and Collins (1999) uses a range of linguistically motivated and carefully hand-engineered subcategorizations to break down wrong context-freedom assumptions of the naive Penn treebank covering PCFG, such as differentiating “baseNPs” from noun phrases with phrasal modifiers, and distinguishing sentences with empty subjects from those where there is an overt subject NP. While he gives incomplete experimental results as to their efficacy, we can assume that these features were incorporated because of beneficial effects on parsing that were complementary to lexicalization. In this paper, we show that the parsing performance that can be achieved by an unlexicalized PCFG is far higher than has previously been demonstrated, and is, indeed, much higher than community wisdom has thought possible. We describe several simple, linguistically motivated annotations which do much to close the gap between a vanilla PCFG and state-of-the-art lexicalized models. Specifically, we construct anunlexicalizedPCFG which outperforms the lexicalized PCFGs of Magerman (1995) and Collins (1996) (though not more recent models, such as Charniak (1997) or Collins (1999)). One benefit of this result is a much-strengthened lower bound on the capacity of an unlexicalized PCFG. To the extent that no such strong baseline has been provided, the community has tended to greatly overestimate the beneficial effect of lexicalization in probabilistic parsing, rather than looking critically at where lexicalized probabilities are both neededto make the right decision and availablein the training data. Secondly, this result affirms the value of linguistic analysis for feature discovery. The result has other uses and advantages: an unlexicalized PCFGis easier to interpret, reason about, and improve than the more complex lexicalized models. The grammar representation is much more compact, no longer requiring large structures that store lexicalized probabilities. The parsing algorithms have lower asymptotic complexity4 and have much smaller grammar egory is divided into several subcategories, for example di viding verb phrases into finite and non-finite verb phrases, rath e than in the modern restricted usage where the term refers onl y to the syntactic argument frames of predicators. 4O(n3) vs. O(n5) for a naive implementation, or vs. O(n4) if using the clever approach of Eisner and Satta (1999). constants. An unlexicalizedPCFG parser is much simpler to build and optimize, including both standard code optimization techniques and the investigation of methods for search space pruning (Caraballo and Charniak, 1998; Charniak et al., 1998). It is not our goal to argue against the use of lexicalized probabilities in high-performance probabilistic parsing. It has been comprehensively demonstrated that lexical dependencies are useful in resolving major classes of sentence ambiguities, and a parser should make use of such information where possible. We focus here on using unlexicalized, tructural context because we feel that this information has been underexploited and underappreciated. We see this investigation as only one part of the foundation for state-of-the-art parsing which employsboth lexical and structural conditioning. 1 Experimental Setup To facilitate comparison with previous work, we trained our models on sections 2–21 of the WSJsection of the Penn treebank. We used the first 20 files (393 sentences) of section 22 as a development set (devset ). This set is small enough that there is noticeable variance in individual results, but it allowed rapid search for good features via continually reparsing the devset in a partially manual hill-climb. All of section 23 was used as a test set for the final model. For each model, input trees were annotated or transformed in some way, as in Johnson (1998). Given a set of transformed trees, we viewed the local trees as grammar rewrite rules in the standard way, and used (unsmoothed) maximum-likelihood estimates for rule probabilities. 5 To parse the grammar, we used a simple array-based Java implementation of a generalizedCKY parser, which, for our final best model, was able to exhaustively parse all sentences in section 23 in 1GB of memory, taking approximately 3 sec for average length sentences. 6 5The tagging probabilities were smoothed to accommodate unknown words. The quantityP(tag|word) was estimated as follows: words were split into one of several categories wordclass, based on capitalization, suffix, digit, and other character features. For each of these categories, we took th e maximum-likelihood estimate of P(tag|wordclass). This distribution was used as a prior against which observed tagging s, if any, were taken, givingP(tag|word) = [c(tag, word) + κ P(tag|wordclass)]/[c(word)+κ]. This was then inverted to give P(word|tag). The quality of this tagging model impacts all numbers; for example the raw treebank grammar’s devset F 1 is 72.62 with it and 72.09 without it. 6The parser is available for download as open source at: http://nlp.stanford.edu/downloads/lex-parser.shtml

In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.

Previous work demonstrated that Web counts can be used to approximate bigram counts, suggesting that Web-based frequencies should be useful for a wide variety of Natural Language Processing (NLP) tasks. However, only a limited number of tasks have so far been tested using Web-scale data sets. The present article overcomes this limitation by systematically investigating the performance of Web-based models for several NLP tasks, covering both syntax and semantics, both generation and analysis, and a wider range of n-grams and parts of speech than have been previously explored. For the majority of our tasks, we find that simple, unsupervised models perform better when n-gram counts are obtained from the Web rather than from a large corpus. In some cases, performance can be improved further by using backoff or interpolation techniques that combine Web counts and corpus counts. However, unsupervised Web-based models generally fail to outperform supervised state-of-the-art models trained on smaller corpora. We argue that Web-based models should therefore be used as a baseline for, rather than an alternative to, standard supervised models.

WordFreak is a natural language annotation tool that has been designed to be easy to extend to new domains and tasks. Specifically, a plug-in architecture has been developed which allows components to be added to WordFreak for customized visualization, annotation specification, and automatic annotation, without re-compilation. The APIs for these plug-ins provide mechanisms to allow automatic annotators or taggers to guide future annotation to supports active learning. At present WordFreak can be used to annotate a number of different types of annotation in English, Chinese, and Arabic including: constituent parse structure and dependent annotations, and ACE named-entity and coreference annotation. The Java source code for WordFreak is distributed under the Mozilla Public License 1.1 via SourceForge at: http://wordfreak.sourceforge.net. This site also provides screenshots, and a web deployable version of WordFreak.

We propose a new unsupervised learning technique for extracting information from large text collections. We model documents as if they were generated by a two-stage stochastic process. Each author is represented by a probability distribution over topics, and each topic is represented as a probability distribution over words for that topic. The words in a multi-author paper are assumed to be the result of a mixture of each authors' topic mixture. The topic-word and author-topic distributions are learned from data in an unsupervised manner using a Markov chain Monte Carlo algorithm. We apply the methodology to a large corpus of 160,000 abstracts and 85,000 authors from the well-known CiteSeer digital library, and learn a model with 300 topics. We discuss in detail the interpretation of the results discovered by the system including specific topic and author models, ranking of authors by topic and topics by author, significant trends in the computer science literature between 1990 and 2002, parsing of abstracts by topics and authors and detection of unusual papers by specific authors. An online query interface to the model is also discussed that allows interactive exploration of author-topic models for corpora such as CiteSeer.


Question: An empirical investigation of unsupervised Chinese word segmentation techniques for SMT on large-scale corpora.

This paper delves into the mathematical underpinnings of statistical machine translation, specifically focusing on parameter estimation. A sequence of five statistical models for the translation process is delineated, accompanied by algorithms for estimating the parameters of these models using pairs of sentences that are translations of each other. Central to these models is the notion of word-by-word alignment between sentence pairs. For any given pair of sentences, each model calculates the probability associated with each potential word-by-word alignment. An algorithm is provided to identify the most probable alignment, which, despite not being optimal, effectively captures the word-by-word relationships within the sentence pair. The authors have access to a vast dataset of French and English text from the Canadian Parliament proceedings, which has been the primary focus of their research. However, they believe that due to the minimal linguistic content in their algorithms, these methodologies would be equally effective for other language pairs. Similarly, they argue that word-by-word alignments are a fundamental characteristic of any substantial bilingual corpus, owing to the minimal linguistic bias in their algorithms.

Query spelling correction is an important component of modern search engines that can help users to express an information need more accurately and thus improve search quality. In this work we proposed and implemented an end-to-end speller correction system, namely CloudSpeller. The CloudSpeller system uses a Hidden Markov Model to effectively model major types of spelling errors in a unified framework, in which we integrate a large-scale lexicon constructed using Wikipedia, an error model trained from high confidence correction pairs, and the Microsoft Web N-gram service. Our system achieves excellent performance on two search query spelling correction datasets, reaching 0.960 and 0.937 F1 scores on the TREC dataset and the MSN dataset respectively.

Statistical language models estimate the distribution of various natural language phenomena for the purpose of speech recognition and other language technologies. Since the first significant model was proposed in 1980, many attempts have been made to improve the state of the art. We review them, point to a few promising directions, and argue for a Bayesian approach to integration of linguistic theories with data.

Today, parallel corpus-based systems dominate the transliteration landscape. But the resource-scarce languages do not enjoy the luxury of large parallel transliteration corpus. For these languages, rule-based transliteration is the only viable option. In this article, we show that by properly harnessing the monolingual resources in conjunction with manually created rule base, one can achieve reasonable transliteration performance. We achieve this performance by exploiting the power of Character Sequence Modeling (CSM), which requires only monolingual resources. We present the results of our rule-based system for Hindi to English, English to Hindi, and Persian to English transliteration tasks. We also perform extrinsic evaluation of transliteration systems in the context of Cross Lingual Information Retrieval. Another important contribution of our work is to explain the widely varying accuracy numbers reported in transliteration literature, in terms of the entropy of the language pairs and the datasets involved.

We contribute 5-gram counts and language models trained on the Common Crawl corpus, a collection over 9 billion web pages. This release improves upon the Google n-gram counts in two key ways: the inclusion of low-count entries and deduplication to reduce boilerplate. By preserving singletons, we were able to use Kneser-Ney smoothing to build large language models. This paper describes how the corpus was processed with emphasis on the problems that arise in working with data at this scale. Our unpruned Kneser-Ney English 5-gram language model, built on 975 billion deduplicated tokens, contains over 500 billion unique n-grams. We show gains of 0.5–1.4 BLEU by using large language models to translate into various languages.

Automatic evaluation metrics for Machine Translation (MT) systems, such as BLEU and the related NIST metric, are becoming increasingly important in MT. Yet, their behaviors are not fully understood. In this paper, we analyze some flaws in the BLEU/NIST metrics. With a better understanding of these problems, we can better interpret the reported BLEU/NIST scores. In addition, this paper reports a novel method of calculating the confidence intervals for BLEU/NIST scores using bootstrapping. With this method, we can determine whether two MT systems are significantly different from each other.

Automated discourse analysis tools based on Natural Language Processing (NLP) aiming at the diagnosis of languageimpairing dementias generally extract several textual metrics of narrative transcripts. However, the absence of sentence boundary segmentation in the transcripts prevents the direct application of NLP methods which rely on these marks to function properly, such as taggers and parsers. We present the first steps taken towards automatic neuropsychological evaluation based on narrative discourse analysis, presenting a new automatic sentence segmentation method for impaired speech. Our model uses recurrent convolutional neural networks with prosodic, Part of Speech (PoS) features, and word embeddings. It was evaluated intrinsically on impaired, spontaneous speech, as well as, normal, prepared speech, and presents better results for healthy elderly (CTL) (F1 = 0.74) and Mild Cognitive Impairment (MCI) patients (F1 = 0.70) than the Conditional Random Fields method (F1 = 0.55 and 0.53, respectively) used in the same context of our study. The results suggest that our model is robust for impaired speech and can be used in automated discourse analysis tools to differentiate narratives produced by MCI and CTL.

Personal Attributes Extraction in Chinese Text Task is designed to extract person specific attributes, for example, spouse/husband, children, education, title etc. from unstructured Chinese texts. According to the characteristics of the personal attributes extraction, we propose a new method based on deep learning to extract attributes from the documents in this paper. Experiments show that this method can improve the accuracy of personal attributes extraction effectively.

The amount of readily available on-line text has reached hundreds of billions of words and continues to grow. Yet for most core natural language tasks, algorithms continue to be optimized, tested and compared after training on corpora consisting of only one million words or less. In this paper, we evaluate the performance of different learning methods on a prototypical natural language disambiguation task, confusion set disambiguation, when trained on orders of magnitude more labeled data than has previously been used. We are fortunate that for this particular application, correctly labeled training data is free. Since this will often not be the case, we examine methods for effectively exploiting very large corpora when labeled data comes at a cost.

Word similarity computation has important applications in many areas, such as natural language processing, intelligence information retrieval, document clustering, automatic answering, word sense disambiguation, machine translation etc. This article makes an intensive study of word similarity computation based on HowNet and the word similarity is computed in three steps: (1) computes sememes similarity, (2) computes concepts similarity using the weight sum method of sememes similarity, (3) takes the maximum similarity of concepts as words similarity. This article mainly introduces numbering the sememes and presents a more accurate method of decomposing the concept description type. The experiment shows that the algorithm of word similarity presented in this article is simple and feasible, and the computed results are relatively consistent with awareness.

tra Statistical machine translation systems are usually trained on large amounts of bilingual text and monolingual text. In this paper, we propose a method to perform domain adaptation for statistical machine translation, where in-domain bilingual corpora do not exist. This method first uses out-of-domain corpora to train a baseline system and then uses in-domain translation dictionaries and in-domain monolingual corpora to improve the indomain performance. We propose an algorithm to combine these different resources in a unified framework. Experimental results indicate that our method achieves absolute improvements of 8.16 and 3.36 BLEU scores on Chinese to English translation and English to French translation respectively, as compared with the baselines using only out-ofdomain corpora.


Question: Acquiring knowledge about semantic resemblance.

Introducing Probabilistic Latent Semantic Indexing (PLSI): A Statistical Framework for Document Indexing

Probabilistic Latent Semantic Indexing (PLSI) represents a cutting-edge technique for automated document indexing, grounded in a statistical latent class model designed for factor analysis of count data. This method, distinguished by its ability to handle domain-specific synonymy and polysemous words, is trained on a corpus of text documents through an advanced adaptation of the Expectation Maximization algorithm. Unlike traditional Latent Semantic Indexing (LSI) methods that rely on Singular Value Decomposition, PLSI is underpinned by robust statistical principles, offering a comprehensive generative data model. Performance evaluations on diverse test collections have shown significant improvements over direct term matching techniques and conventional LSI, with the most notable gains achieved through the strategic integration of models with varying dimensionalities.

Several recent approaches use distributional similarity for making symbolic reasoning more flexible. While an important step in the right direction, the use of similarity has a number of inherent limitations. We argue that similarity-based reasoning should be complemented with commonsense reasoning patterns such as interpolation and a fortiori inference. We show how the required background knowledge for these inference patterns can be obtained from distributional models.

The met* method distinguishes selected examples of metonymy from metaphor and from literalness and anomaly in short English sentences. In the met* method, literalness is distinguished because it satisfies contextual constraints that the nonliteral others all violate. Metonymy is discriminated from metaphor and anomaly in a way that [1] supports Lakoff and Johnson's (1980) view that in metonymy one entity stands for another whereas in metaphor one entity is viewed as another, [2] permits chains of metonymies (Reddy 1979), and [3] allows metonymies to co-occur with instances of either literalness, metaphor, or anomaly. Metaphor is distinguished from anomaly because the former contains a relevant analogy, unlike the latter. The met* method is part of Collative Semantics, a semantics for natural language processing, and has been implemented in a computer program called meta5. Some examples of meta5's analysis of metaphor and metonymy are given. The met* method is compared with approaches from artificial intelligence, linguistics, philosophy, and psychology.

Fine-grained sense distinctions are one of the major obstacles to successful Word Sense Disambiguation. In this paper, we present a method for reducing the granularity of the WordNet sense inventory based on the mapping to a manually crafted dictionary encoding sense hierarchies, namely the Oxford Dictionary of English. We assess the quality of the mapping and the induced clustering, and evaluate the performance of coarse WSD systems in the Senseval-3 English all-words task.

The work reported here experimentally investigates a striking generalization about vocabulary acquisition: Noun learning is superior to verb learning in the earliest moments of child language development. The dominant explanation of this phenomenon in the literature invokes differing conceptual requirements for items in these lexical categories: Verbs are cognitively more complex than nouns and so their acquisition must await certain mental developments in the infant. In the present work, we investigate an alternative hypothesis; namely, that it is the information requirements of verb learning, not the conceptual requirements, that crucially determine the acquisition order. Efficient verb learning requires access to structural features of the exposure language and thus cannot take place until a scaffolding of noun knowledge enables the acquisition of clause-level syntax. More generally, we experimentally investigate the hypothesis that vocabulary acquisition takes place via an incremental constraint-satisfaction procedure that bootstraps itself into successively more sophisticated linguistic representations which, in turn, enable new kinds of vocabulary learning. If the experimental subjects were young children, it would be difficult to distinguish between this information-centered hypothesis and the conceptual change hypothesis. Therefore the experimental "learners" are adults. The items to be "acquired" in the experiments were the 24 most frequent nouns and 24 most frequent verbs from a sample of maternal speech to 18-24-month-old infants. The various experiments ask about the kinds of information that will support identification of these words as they occur in mother-to-child discourse. Both the proportion correctly identified and the type of word that is identifiable changes significantly as a function of information type. We discuss these results as consistent with the incremental construction of a highly lexicalized grammar by cognitively and pragmatically sophisticated human infants, but inconsistent with a procedure in which lexical acquisition is independent of and antecedent to syntax acquisition.

Language-endowed intelligent agents benefit from leveraging lexical knowledge falling at different points along a spectrum of compositionality. This means that robust computational lexicons should include not only the compositional expectations of argument-taking words, but also non-compositional collocations (idioms), semi-compositional collocations that might be difficult for an agent to interpret (e.g., standard metaphors), and even collocations that could be compositionally analyzed but are so frequently encountered that recording their meaning increases the efficiency of interpretation. In this paper we argue that yet another type of stringto-meaning mapping can also be useful to intelligent agents: remembered semantic analyses of actual text inputs. These can be viewed as super-specific multi-word expressions whose recorded interpretations mimic a person’s memories of knowledge previously learned from language input. These differ from typical annotated corpora in two ways. First, they provide a full, context-sensitive semantic interpretation rather than select features. Second, they are are formulated in the ontologically-grounded metalanguage used in a particular agent environment, meaning that the interpretations contribute to the dynamically evolving cognitive capabilites of agents configured in that environment.

This work uses parallel monolingual corpora for a detailed study of the task of sub-sentential paraphrase acquisition. We argue that the scarcity of this type of resource is compensated by the fact that it is the most suited type for studies on paraphrasing. We propose a large exploration of this task with experiments on two languages with five different acquisition techniques, selected for their complementarity, their combinations, as well as four monolingual corpus types of varying comparability. We report, under all conditions, a significant improvement over all techniques by validating candidate paraphrases using a maximum entropy classifier. An important result of our study is the identification of difficult-to-acquire paraphrase pairs, which are classified and quantified in a bilingual typology.

This paper presents a novel approach to the problem of paraphrase identification. Although paraphrases often make use of synonymous or near synonymous terms, many previous approaches have either ignored or made limited use of information about similarities between word meanings. We present an algorithm for paraphrase identification which makes extensive use of word similarity information derived from WordNet (Fellbaum, 1998). The approach is evaluated using the Microsoft Research Paraphrase Corpus (Dolan et al., 2004), a standard resource for this task, and found to outperform previously published methods.

Keywords: ontologies < controlled vocabularies < index languages < language types < (language), knowledge acquisition < knowledge engineering < artificial intelligence < computer applications < computer operations < (activities and operations)

Guided by multiple heuristics, a unified taxonomy of entities and categories is distilled from the Wikipedia category network. A comprehensive evaluation, based on the analysis of upward generalization paths, demonstrates that the taxonomy supports generalizations which are more than twice as accurate as the state of the art. The taxonomy is available at http://headstaxonomy.com.

The cognitive and neural systems that enable conceptual processing must support the ability to combine (and recombine) concepts to form an infinite number of ideas. Two candidate neural systems for conceptual combination-the left anterior temporal lobe (ATL) and the left angular gyrus (AG)-have been characterized as "semantic hubs" due to both functional and anatomical properties; however, these two regions likely support different aspects of composition. Here we consider two hypotheses for the role of AG in conceptual combination, both of which differ from a putative role for the ATL in "feature-based" combinatorics (i.e., meaning derived by combining concepts' features). Firstly, we examine whether AG is more sensitive to function-argument relations of the sort that arise when a predicate is combined with its arguments. Secondly, we examine the non-mutually exclusive possibility that AG represents information carried on a verb in particular, whether this be information about event composition or about thematic relations denoted uniquely by verbs. We identified voxels that respond differentially to two-word versus one-word stimuli, and we measured the similarity of the patterns in these voxels evoked by (1) pairs of two-word phrases that shared a noun that was an argument, thus sharing function-argument composition (e.g. eats meat and with meat), in comparison with two-word phrases that shared only a noun, not an argument (e.g., eats meat and tasty meat); and (2) stimulus pairs that shared only an event (operationalized here as sharing a verb; e.g. eats meat and eats quickly), in comparison to both of the above. We found that activity patterns in left AG tracked information relating to the presence of an event-denoting verb in a pair of two-word phrases. We also found that the neural similarity in AG voxel patterns between two phrases sharing a verb correlated with subjects' ratings of how similar the meanings of those two verb phrases were. These findings indicate that AG represents information specific to verbs, perhaps event structure or thematic relations mediated by verbs, as opposed to argument structure in general.


Question: Acquiring knowledge about semantic resemblance.

This paper explores and critically assesses an automated technique for grouping English words based on their syntactic distribution patterns. The methodology employs deterministic annealing to determine the most optimal set of clusters with the least distortion. As the annealing parameter ascends, the stability of existing clusters diminishes, leading to their division and the creation of a hierarchical, probabilistic clustering structure. These clusters serve as a foundation for developing class models that predict word co-occurrence. The effectiveness of these models is gauged by their performance on unseen test data, providing insights into the robustness of the clustering approach in capturing semantic similarity through distributional analysis.

Knowledge can often be represented using entities connected by relations. For example, the fact that tennis ball is round can be represented as “TennisBall HasShape Round”, where a “TennisBall” is one entity, “HasShape” is a relation and “Round” is another entity. A knowledge base is a way to store such structured information, a knowledge base stores triples of the “an entity-relation-an entity” form, and a real world knowledge base often has millions or billions of such triples. There are several well-known knowledge bases including FreeBase [1], WordNet [2], YAGO [3], etc. They are important in fields like reasoning and question answering; for instance if one asks “what is the shape of a tennis ball”, we can search the knowledge base for the triple as “TennisBall HasShape Round” and output “round” as the answer.

Learning to recognize of object classes is one of the most important functionalities of vision. It is estimated that humans are able to learn tens of thousands of visual categori es in their life. Given the photometric and geometric variabilities displayed by objects as well as the high degree of intra-clas s variabilities, we hypothesize that humans achieve such a fe at by using knowledge and information cumulated throughout the learning process. In recent years, a handful of pioneering p apers have applied various forms of knowledge transfer algorithms to the problem of learning object classes. We first review some o f these papers by loosely grouping them into three categories : transfer through prior parameters, transfer through shared features or parts, and transfer through contextual information. In the second half of the paper, we detail a recent algorithm proposed by the author. This incremental learning scheme us es information from object classes previously learned in the f orm of prior models to train a new object class model. Training images can be presented in an incremental way. We present experimental results tested with this model on a large numbe r of object categories.

Semantic entailment is the problem of determining if the meaning of a given sentence entails that of another. This is a fundamental problem in natural language understanding that provides a broad framework for studying language variability and has a large number of applications. We present a principled approach to this problem that builds on inducing rerepresentations of text snippets into a hierarchical knowledge representation along with a sound inferential mechanism that makes use of it to prove semantic entailment.

The work reported here experimentally investigates a striking generalization about vocabulary acquisition: Noun learning is superior to verb learning in the earliest moments of child language development. The dominant explanation of this phenomenon in the literature invokes differing conceptual requirements for items in these lexical categories: Verbs are cognitively more complex than nouns and so their acquisition must await certain mental developments in the infant. In the present work, we investigate an alternative hypothesis; namely, that it is the information requirements of verb learning, not the conceptual requirements, that crucially determine the acquisition order. Efficient verb learning requires access to structural features of the exposure language and thus cannot take place until a scaffolding of noun knowledge enables the acquisition of clause-level syntax. More generally, we experimentally investigate the hypothesis that vocabulary acquisition takes place via an incremental constraint-satisfaction procedure that bootstraps itself into successively more sophisticated linguistic representations which, in turn, enable new kinds of vocabulary learning. If the experimental subjects were young children, it would be difficult to distinguish between this information-centered hypothesis and the conceptual change hypothesis. Therefore the experimental "learners" are adults. The items to be "acquired" in the experiments were the 24 most frequent nouns and 24 most frequent verbs from a sample of maternal speech to 18-24-month-old infants. The various experiments ask about the kinds of information that will support identification of these words as they occur in mother-to-child discourse. Both the proportion correctly identified and the type of word that is identifiable changes significantly as a function of information type. We discuss these results as consistent with the incremental construction of a highly lexicalized grammar by cognitively and pragmatically sophisticated human infants, but inconsistent with a procedure in which lexical acquisition is independent of and antecedent to syntax acquisition.

A lexicon is a linguistic object and hence is not the same thing as an ontology, which is non-linguistic. Nonetheless, word senses are in many ways similar to ontological concepts and the relationships found between word senses resemble the relationships found between concepts. Although the arbitrary and semi-arbitrary distinctions made by natural languages limit the degree to which these similarities can be exploited, a lexicon can nonetheless serve in the development of an ontology, especially in a technical domain.

This paper describes an automatic indexing procedure that uses the “IS-A” relations contained within WordNet and the set of nouns contained in a text to select a sense for each plysemous noun in the text. The result of the indexing procedure is a vector in which some of the terms represent word senses instead of word stems. Retrieval experiments comparing the effectivenss of these sense-based vectors vs. stem-based vectors show the stem-based vectors to be superior overall, although the sense-based vectors do improve the performance of some queries. The overall degradation is due in large part to the difficulty of disambiguating senses in short query statements. An analysis of these results suggests two conclusions: the IS-A links define a generalization/specialization hierarchy that is not sufficient to reliably select the correct sense of a noun from the set of fine sense distinctions in WordNet; and missing correct matches because of incorrect sense resolution has a much more deleterious effect on retrieval performance than does making spurious matches.

The goal of giving a well-defined meaning to information is currently shared by endeavors such as the Semantic Web as well as by current trends within Knowledge Management. They all depend on the large-scale formalization of knowledge and on the availability of formal metadata about information resources. However, the question how to provide the necessary formal metadata in an effective and efficient way is still not solved to a satisfactory extent. Certainly, the most effective way to provide such metadata as well as formalized knowledge is to let humans encode them directly into the system, but this is neither efficient nor feasible. Furthermore, as current social studies show, individual knowledge is often less powerful than the collective knowledge of a certain community.As a potential way out of the knowledge acquisition bottleneck, we present a novel methodology that acquires collective knowledge from the World Wide Web using the GoogleTM API. In particular, we present PANKOW, a concrete instantiation of this methodology which is evaluated in two experiments: one with the aim of classifying novel instances with regard to an existing ontology and one with the aim of learning sub-/superconcept relations.

We present a family of neural-network– inspired models for computing continuous word representations, specifically designed to exploit both monolingual and multilingual text. This framework allows us to perform unsupervised training of embeddings that exhibit higher accuracy on syntactic and semantic compositionality, as well as multilingual semantic similarity, compared to previous models trained in an unsupervised fashion. We also show that such multilingual embeddings, optimized for semantic similarity, can improve the performance of statistical machine translation with respect to how it handles words not present in the parallel data.

In this paper we describe a fuzzy logic-based language processing method, which is applied to speech recognition. Our purpose is to create a system that can learn from a linguistic corpus the fuzzy semantic relations between the concepts represented by words and use such relations to process the word sequences generated by speech recognition systems. In particular, the system will be able to predict the words failed to be recognized by a speech recognition system. This will help to increase the accuracy of a speech recognition system. This will also serve as the first stage of deep semantic processing of speech recognition results by providing “semantic relatedness” between the recognized words. We report the fuzzy inference rule learning system, which we have developed and also report the experimental results based on the system. Key-words: Fuzzy logic, Natural Language Analysis, Speech Recognition, Corpus Linguistics.

This paper presents a framework for knowledge discovery and concept exploration. In order to enhance the concept exploration capability of knowledge-based systems and to alleviate the limitations of the manual browsing approach, we have developed two spreading activation-based algorithms for concept exploration in large, heterogeneous networks of concepts (e.g., multiple thesauri). One algorithm, which is based on the symbolic Al paradigm, performs a conventional branch-and-bound search on a semantic net representation to identify other highly relevant concepts (a serial, optimal search process). The second algorithm, which is based on the neural network approach, executes the Hopfield net parallel relaxation and convergence process to identify “convergent” concepts for some initial queries (a parallel, heuristic search process). Both algorithms can be adopted for automatic, multiple-thesauri consultation. We tested these two algorithms on a large text-based knowledge network of about 13,000 nodes (terms) and 80,000 directed links in the area of computing technologies. This knowledge network was created from two external thesauri and one automatically generated thesaurus. We conducted experiments to compare the behaviors and performances of the two algorithms with the hypertext-like browsing process. Our experiment revealed that manual browsing achieved higher-term recall but lower-term precision in comparison to the algorithmic systems. However, it was also a much more laborious and cognitively demanding process. In document retrieval, there were no statistically significant differences in document recall and precision between the algorithms and the manual browsing process. In light of the effort required by the manual browsing process, our proposed algorithmic approach presents a viable option for efficiently traversing largescale, multiple thesauri (knowledge network).


Question: Acquiring knowledge about semantic resemblance.

Introducing Probabilistic Latent Semantic Analysis (PLSA): An Innovative Statistical Method for Analyzing Two-Mode and Co-occurrence Data
Probabilistic Latent Semantic Analysis (PLSA) emerges as a groundbreaking statistical technique for dissecting two-mode and co-occurrence data, with broad applications in information retrieval, natural language processing, machine learning from textual content, and related fields. Unlike conventional Latent Semantic Analysis (LSA), which relies on linear algebra and performs a Singular Value Decomposition (SVD) of co-occurrence matrices, PLSA adopts a mixture decomposition rooted in a latent class model, offering a more principled and statistically robust approach. To prevent overfitting, a generalization of the maximum likelihood model fitting method, termed tempered Expectation-Maximization (EM), is introduced. This strategy consistently outperforms LSA across various experiments, showcasing PLSA's superiority in handling complex data structures and enhancing semantic analysis accuracy.

ABSTRACT There is overwhelming evidence that children's first words are primarily nouns even across languages. These data are interpreted as evidence of a "Natural Partitions Theory," one that holds that the concepts referred to by nouns are conceptually more basic than those referred to by verbs or prepositions. Analysis of data from cross-linguistic studies designed to eliminate Inonconceptual differences between words in a language that might account for the Larlier acquisition of nouns--such as the position of a word in the sentence, morphological transparency (the ease with which the root can be heard in various uses of the word), and patterns of language teaching strategies--leaves some version of the Natural Partitions hypothesis as the most reasonable view of early vocabulary acquisition. A cross-linguistic examination of a single sentence suggests that if objecthood is created by spatial relations among perceptual elements, then good concrete objects are particularly cohesive collections of percepts. These highly cohesive collections of perceptual information tend to be lexicalized as nouns by almost every language. Object concepts are given to language users by the world and can be learned one at a time, while predicate concepts form a system in each language., that, from a child's point of view, must be discovered. (HTE)

As the editors of this volume remind us, polysemy has been a vexing issue for the understanding of language since antiquity. For half a century, it has been a major bottleneck for natural language processing. It contributed to the failure of early machine translation research (remember Bar-Hillel’s famous pen and box example) and is still plaguing most natural language processing and information retrieval applications. A recent issue of this journal described the state of the art in automatic sense disambiguation (Ide and Véronis 1998), and Senseval system competitions have revealed the immense difficulty of the task (http://www.sle.sharp.co.uk/senseval2). However, no significant progress can be made on the computational aspects of polysemy without serious advances in theoretical issues. At the same time, theoretical work can be fostered by computational results and problems, and language-processing applications can provide a unique test bed for theories. It was therefore an excellent idea to gather both theoretical and applied contributions in the same book. Yael Ravin and Claudia Leacock are well-known names to those who work on the theoretical and computational aspects of word meaning. In this volume, they bring together a collection of essays from leading researchers in the field. As far as I can tell, these essays are not reprints or expanded versions of conference papers, as is often the case for edited works; instead, they seem to have been specially commissioned for the purposes of this book, which makes it even more exciting to examine. The book is composed of 11 chapters. It is not formally divided into parts, but chapters dealing more specifically with the computational aspects of polysemy are grouped together at the end (and constitute about one-third of the volume). Chapter 1 is an overview written by the volume editors. Yael Ravin and Claudia Leacock survey the main theories of meaning and their treatment of polysemy. These include the classical Aristotelian approach revived by Katz and Fodor (1963); Rosch’s (1977) prototypical approach, which has its roots in Wittgenstein’s Philosophical Investigations (1953); and the relational approach recently exemplified by WordNet (Fellbaum 1998), which (although the authors do not mention it) can be traced back to Peirce’s (1931–1958) and Selz’s (1913, 1922) graphs and which gained popularity with Quillian’s (1968) semantic networks. In the course of this overview, Ravin and Leacock put the individual chapters into perspective by relating them to the various theories.

In recent years, we have seen an increasing amount of interest in low-dimensional vector representations of words. Among other things, these facilitate computing word similarity and relatedness scores. The most well-known example of algorithms to produce representations of this sort are the word2vec approaches. In this paper, we investigate a new model to induce such vector spaces for medical concepts, based on a joint objective that exploits not only word co-occurrences but also manually labeled documents, as available from sources such as PubMed. Our extensive experimental analysis shows that our embeddings lead to significantly higher correlations with human similarity and relatedness assessments than previous work. Due to the simplicity and versatility of vector representations, these findings suggest that our resource can easily be used as a drop-in replacement to improve any systems relying on medical concept similarity measures.

Entity Disambiguation aims to link mentions of ambiguous entities to a knowledge base (e.g., Wikipedia). Modeling topical coherence is crucial for this task based on the assumption that information from the same semantic context tends to belong to the same topic. This paper presents a novel deep semantic relatedness model (DSRM) based on deep neural networks (DNN) and semantic knowledge graphs (KGs) to measure entity semantic relatedness for topical coherence modeling. The DSRM is directly trained on large-scale KGs and it maps heterogeneous types of knowledge of an entity from KGs to numerical feature vectors in a latent space such that the distance between two semantically-related entities is minimized. Compared with the state-ofthe-art relatedness approach proposed by (Milne and Witten, 2008a), the DSRM obtains 19.4% and 24.5% reductions in entity disambiguation errors on two publicly available datasets respectively.

We propose a novel graph-based approach for constructing concept hierarchy from a large text corpus. Our algorithm, GraBTax, incorporates both statistical co-occurrences and lexical similarity in optimizing the structure of the taxonomy. To automatically generate topic-dependent taxonomies from a large text corpus, GraBTax first extracts topical terms and their relationships from the corpus. The algorithm then constructs a weighted graph representing topics and their associations. A graph partitioning algorithm is then used to re-cursively partition the topic graph into a tax-onomy. For evaluation, we apply GraBTax to articles, primarily computer science, in the CiteSeerX digital library and search engine. The quality of the resulting concept hierarchy is assessed by both human judges and comparison with Wikipedia categories.

We describe a method for the automatic acquisition of the hyponymy lexical relation from unrestricted text. Two goals motivate the approach: (i) avoidance of the need for pre-encoded knowledge and (ii) applicability across a wide range of text. We identify a set of lexicosyntactic patterns that are easily recognizable, that occur frequently and across text genre boundaries, and that indisputably indicate the lexical relation of interest. We describe a method for discovering these patterns and suggest that other lexical relations will also be acquirable in this way. A subset of the acquisition algorithm is implemented and the results are used to augment and critique the structure of a large hand-built thesaurus. Extensions and applications to areas such as information retrieval are suggested.

This paper describes an automatic method for acquiring hyponymy relations from HTML documents on the WWW. Hyponymy relations can play a crucial role in various natural language processing systems. Most existing acquisition methods for hyponymy relations rely on particular linguistic patterns, such as “ NP such as NP”. Our method, however, does not use such linguistic patterns, and we expect that our procedure can be applied to a wide range of expressions for which existing methods cannot be used. Our acquisition algorithm uses clues such as itemization or listing in HTML documents and statistical measures such as document frequencies and verb-noun co-occurrences.

Learning commonsense knowledge from natural language text is nontrivial due to reporting bias: people rarely state the obvious, e.g., “My house is bigger than me.” However, while rarely stated explicitly, this trivial everyday knowledge does influence the way people talk about the world, which provides indirect clues to reason about the world. For example, a statement like, “Tyler entered his house” implies that his house is bigger than Tyler. In this paper, we present an approach to infer relative physical knowledge of actions and objects along five dimensions (e.g., size, weight, and strength) from unstructured natural language text. We frame knowledge acquisition as joint inference over two closely related problems: learning (1) relative physical knowledge of object pairs and (2) physical implications of actions when applied to those object pairs. Empirical results demonstrate that it is possible to extract knowledge of actions and objects from language and that joint inference over different types of knowledge improves performance.

Text document clustering plays an important role in providing intuitive navigation and browsing mechanisms by organizing large sets of documents into a small number of meaningful clusters. The bag of words representation used for these clustering methods is often unsatisfactory as it ignores relationships between important terms that do not cooccur literally. In order to deal with the problem, we integrate core ontologies as background knowledge into the process of clustering text documents. Our experimental evaluations compare clustering techniques based on precategorizations of texts from Reuters newsfeeds and on a smaller domain of an eLearning course about Java. In the experiments, improvements of results by background knowledge compared to a baseline without background knowledge can be shown in many interesting combinations.

Introduction Linguistic Resources Knowledge-based machine translation (KBMT) systems have achieved excellent results in constrained domains, but have not yet scaled up to newspaper text. The reason is that knowledge resources (lexicons, grammar rules, world models) must be painstakingly handcrafted from scratch. One of the hypotheses being tested in the PANGLOSS machine translation project is whether or not these resources can be semi-automatically acquired on a very large scale. This paper focuses on the construction of a large ontology (or knowledge base, or world model) for supporting KBMT. It contains representations for some 70,000 commonly encountered objects, processes, qualities, and relations. The ontology was constructed by merging various online dictionaries, semantic networks, and bilingual resources, through semi-automatic methods. Some of these methods (e.g., conceptual matching of semantic taxonomies) are broadly applicable to problems of importing/exporting knowledge from one KB to another. Other methods (e.g., bilingual matching) allow a knowledge engineer to build up an index to a KB in a second language, such as Spanish or Japanese. USC/Information Sciences Institute 4676 Admiralty Way Marina del Rey, CA 90292 knight,luk @isi.edu


Question: Acquiring knowledge about semantic resemblance.

In our TREC-3 participation, we employed Latent Semantic Indexing (LSI) for both document routing and ad-hoc retrieval tasks. For the routing challenges, we constructed an LSI space utilizing the training documents. Our analysis involved comparing profiles developed exclusively from topic words (without training data) against those derived from the average of relevant documents (ignoring topic words). As anticipated, profiles based on the centroid of relevant documents outperformed those relying solely on topic words by a notable 30%. This feedback technique proved to be highly effective, showcasing competitive routing performance when contrasted with other systems. Further enhancements in performance were achieved through various integrations of information from topic words and relevant documents, albeit with minor improvements. In the context of ad-hoc tasks, we contrasted LSI with keyword vector matching, which involves no dimension reduction. Even with the extensive TREC topic statements, LSI demonstrated slight advantages over keyword vector matching, highlighting its effectiveness in semantic similarity assessment.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design

**Similar Documents**: This paper introduces a novel evolutionary learning algorithm, termed Hybrid Genetic Algorithm-Particle Swarm Optimization (HGAPSO), aimed at automating the design of recurrent neural/fuzzy networks. HGAPSO integrates the strengths of genetic algorithm (GA) and particle swarm optimization (PSO) to create a robust optimization approach. In HGAPSO, new generations are not solely produced through traditional GA operations such as crossover and mutation; they also benefit from PSO principles. The elite strategy is central to HGAPSO, where the top-performing individuals are not directly replicated but are instead enhanced. These elites are treated as particles within a swarm, and PSO is employed to refine them, akin to a maturation process in nature. The enhanced elites constitute half of the new generation, while the rest are generated through GA operations on these refined elites. HGAPSO's application to recurrent neural/fuzzy network design is demonstrated through two case studies: a fully connected recurrent neural network for temporal sequence production and a Takagi-Sugeno-Kang-type recurrent fuzzy network for dynamic plant control. Comparative results against standard GA and PSO methods reveal the superior performance of HGAPSO in recurrent network design problems. This innovative approach could potentially be adapted to address complex optimization challenges, such as the economic dispatch problem with valve-point effect, by leveraging its robust optimization capabilities to find optimal solutions in a high

We present a game-based interface for acquiring common sense knowledge. In addition to being interactive and entertaining, our interface guides the knowledge acquisition process to learn about the most salient characteristics of a particular concept. We use statistical classification methods to discover the most informative characteristics in the Open Mind Common Sense knowledge base, and use these characteristics to play a game of 20 Questions with the user. Our interface also allows users to enter knowledge more quickly than a more traditional knowledge-acquisition interface. An evaluation showed that users enjoyed the game and that it increased the speed of knowledge acquisition.

Vectorial representations of words have grown remarkably popular in natural language processing and machine translation. The recent surge in deep learning-inspired methods for producing distributed representations has been widely noted even outside these fields. Existing representations are typically trained on large monolingual corpora using context-based prediction models. In this paper, we propose extending pre-existing word representations by exploiting Wiktionary. This process results in a substantial extension of the original word vector representations, yielding a large multilingual dictionary of word embeddings. We believe that this resource can enable numerous monolingual and cross-lingual applications, as evidenced in a series of monolingual and cross-lingual semantic experiments that we have conducted.

This paper will focus on the semantic representation of verbs in computer systems and its impact on lexical selection problems in machine translation (MT). Two groups of English and Chinese verbs are examined to show that lexical selection must be based on interpretation of the sentence as well as selection restrictions placed on the verb arguments. A novel representation scheme is suggested, and is compared to representations with selection restrictions used in transfer-based MT. We see our approach as closely aligned with knowledge-based MT approaches (KBMT), and as a separate component that could be incorporated into existing systems. Examples and experimental results will show that, using this scheme, inexact matches can achieve correct lexical selection.

Word Sense Disambiguation (WSD) is a key enabling-technology. Supervised WSD techniques are the best performing in public evaluations, but need large amounts of hand-tagging data. Existing hand-annotated corpora like SemCor (Miller et al., 1993), which is annotated with WordNet senses (Fellbaum, 1998) allow for a small improvement over the simple most frequent sense heuristic, as attested in the all-words track of the last Senseval competition (Snyder and Palmer, 2004). In theory, larger amounts of training data (SemCor has approx. 500M words) would improve the performance of supervised WSD, but no current project exists to provide such an expensive resource. Another problem of the supervised approach is that the inventory and distribution of senses changes dramatically from one domain to the other, requiring additional hand-tagging of corpora (Martı́nez and Agirre, 2000; Koeling et al., 2005). Supervised WSD is based on the “fixed-list of senses” paradigm, where the senses for a target word are a closed list coming from a dictionary or lexicon. Lexicographers and semanticists have long warned about the problems of such an approach, where senses are listed separately as discrete entities, and have argued in favor of more complex representations, where, for instance, senses are dense regions in a continuum (Cruse, 2000). Unsupervised Word Sense Induction and Discrimination (WSID, also known as corpus-based unsupervised systems) has followed this line of thinking, and tries to induce word senses directly from the corpus. Typical WSID systems involve clustering techniques, which group together similar examples. Given a set of induced clusters (which represent word uses or senses1), each new occurrence of the target word will be compared to the clusters and the most similar cluster will be selected as its sense. One of the problems of unsupervised systems is that of managing to do a fair evaluation. Most of current unsupervised systems are evaluated in-house, with a brief comparison to a re-implementation of a former system, leading to a proliferation of unsupervised systems with little ground to compare among them. The goal of this task is to allow for comparison across sense-induction and discrimination systems, and also to compare these systems to other supervised and knowledge-based systems. The paper is organizes as follows. Section 2 presents the evaluation framework used in this task. Section 3 presents the systems that participated in the task, and the official results. Finally, Section 4 draws the conclusions.

Whether computational algorithms such as latent semantic analysis (LSA) can both extract meaning from language and advance theories of human cognition has become a topic of debate in cognitive science, whereby accounts of symbolic cognition and embodied cognition are often contrasted. Albeit for different reasons, in both accounts the importance of statistical regularities in linguistic surface structure tends to be underestimated. The current article gives an overview of the symbolic and embodied cognition accounts and shows how meaning induction attributed to a specific statistical process or to activation of embodied representations should be attributed to language itself. Specifically, the performance of LSA can be attributed to the linguistic surface structure, more than special characteristics of the algorithm, and embodiment findings attributed to perceptual simulations can be explained by distributional linguistic information.

raising the issue of the implications of findings in neuropsychological subjects for category learning [Knowlton, B. (1999) What can neuropsychology tell us about category learning? Trends Cognit. Sci. 3, 123–124]1. Knowlton and Squire2,3 reported various dissociations between categorization and recognition in normal and amnesic subjects to support their view that multiple memory systems mediate these tasks. The multiple-system view is more complex than a single-system approach is to modeling categorization and recognition. On grounds of parsimony, a question of scientific interest is whether a single-system model can suffice to account for these data. In support of this possibility, we demonstrated that an extremely simple version of a single-system exemplar model accounted in quantitative detail for the bulk of the dissociation findings reported by Knowlton and Squire, as long as plausible assumptions were made regarding parameter differences in memory ability between groups4. At the same time, we acknowledged some remaining challenges for the exemplar model, and concluded our article by stating, ‘Whether or not a full account of Knowlton and Squire’s complete set of dissociation findings is available within the exemplar approach remains a subject for future research, but we feel that we have made an important start’ (Ref. 4, p. 255). We also outlined various directions in which the exemplar model could be extended to meet these challenges. In the first part of her commentary1, Knowlton reiterates these acknowledged limitations of the simple exemplar model and argues against the possible lines of extension that we suggested. Her first point pertains to the dissociation exhibited by the severe amnesic E.P., who despite showing zero ability to discriminate between old and new exemplars in a recognition task, displayed normal performance in a classification task5. We had pointed out that stimulus conditions were dramatically different across the particular recognition and classification tasks in question. In the test phase of the recognition task, presentations of the target item were rare and widely spaced between distractor items. By contrast, in the classification test, members of the target category were frequently presented. We noted that if E.P. had even a small residual N o s o f s k y a n d Z a k i – R e s p o n s e Update Comment

We propose to shift the goal of recognition from naming to describing. Doing so allows us not only to name familiar objects, but also: to report unusual aspects of a familiar object (“spotty dog”, not just “dog”); to say something about unfamiliar objects (“hairy and four-legged”, not just “unknown”); and to learn how to recognize new objects with few or no visual examples. Rather than focusing on identity assignment, we make inferring attributes the core problem of recognition. These attributes can be semantic (“spotty”) or discriminative (“dogs have it but sheep do not”). Learning attributes presents a major new challenge: generalization across object categories, not just across instances within a category. In this paper, we also introduce a novel feature selection method for learning attributes that generalize well across categories. We support our claims by thorough evaluation that provides insights into the limitations of the standard recognition paradigm of naming and demonstrates the new abilities provided by our attribute-based framework.

Similes are easily obtained from web-driven and casebased reasoning approaches. Still, generating thoughtful figurative descriptions with meaningful relation to narrative context and author style has not yet been fully explored. In this paper, the author prepares the foundation for a computational model which can achieve this level of aesthetic complexity. This paper also introduces and evaluates a possible architecture for generating and ranking figurative comparisons on par with humans: the

Distributional semantics creates vectorspace representations that capture many forms of semantic similarity, but their relation to semantic entailment has been less clear. We propose a vector-space model which provides a formal foundation for a distributional semantics of entailment. Using a mean-field approximation, we develop approximate inference procedures and entailment operators over vectors of probabilities of features being known (versus unknown). We use this framework to reinterpret an existing distributionalsemantic model (Word2Vec) as approximating an entailment-based model of the distributions of words in contexts, thereby predicting lexical entailment relations. In both unsupervised and semi-supervised experiments on hyponymy detection, we get substantial improvements over previous results.

The question of whether the neural encodings of objects are similar across different people is one of the key questions in cognitive neuroscience. This article examines the commonalities in the internal representation of objects, as measured with fMRI, across individuals in two complementary ways. First, we examine the commonalities in the internal representation of objects across people at the level of interobject distances, derived from whole brain fMRI data, and second, at the level of spatially localized anatomical brain regions that contain sufficient information for identification of object categories, without making the assumption that their voxel patterns are spatially matched in a common space. We examine the commonalities in internal representation of objects on 3T fMRI data collected while participants viewed line drawings depicting various tools and dwellings. This exploratory study revealed the extent to which the representation of individual concepts, and their mutual similarity, is shared across participants.


Question: Acquiring knowledge about semantic resemblance.

Venture with us into the realm of contemporary literature, where a fresh tome awaits your eager eyes. Indeed, this is a newly minted volume that has captured the intrigue of numerous avid readers—will you join their ranks? Undoubtedly, you should. Embracing this book will not burden you with undue hardship; instead, it will enrich your existence. Despite some individuals' apprehensions that reading is a strenuous endeavor, you must hold steadfast in your belief that you can conquer this narrative. The sensation of difficulty often arises from a dearth of inspiration regarding what to read next, or when the material at hand fails to captivate your interest sufficiently. However, you must remain resolute in your quest for knowledge, for the world of books is vast and filled with endless possibilities.

In the context of modern information retrieval, we are embarking on a journey through the latest advancements and methodologies in the field. This new book serves as a beacon for those seeking to navigate the complex landscape of data and knowledge. Whether you are a seasoned professional or a curious novice, this tome promises to deliver insights and strategies for optimizing your search for relevant information. It delves into the intricacies of semantic similarity, a critical component in understanding how information is interconnected and how it can be retrieved efficiently. By exploring the latest algorithms and techniques, this book aims to equip you with the tools necessary to excel in the ever-evolving field of information retrieval. So, let us embark on this intellectual adventure together, confident in the knowledge that every page turned will bring us closer to mastering the art of modern information retrieval.

Semantic entailment is the problem of determining if the meaning of a given sentence entails that of another. This is a fundamental problem in natural language understanding that provides a broad framework for studying language variability and has a large number of applications. We present a principled approach to this problem that builds on inducing rerepresentations of text snippets into a hierarchical knowledge representation along with a sound inferential mechanism that makes use of it to prove semantic entailment.

This paper reports our submissions to semantic textual similarity task, i.e., task 2 in Semantic Evaluation 2015. We built our systems using various traditional features, such as string-based, corpus-based and syntactic similarity metrics, as well as novel similarity measures based on distributed word representations, which were trained using deep learning paradigms. Since the training and test datasets consist of instances collected from various domains, three different strategies of the usage of training datasets were explored: (1) use all available training datasets and build a unified supervised model for all test datasets; (2) select the most similar training dataset and separately construct a individual model for each test set; (3) adopt multi-task learning framework to make full use of available training sets. Results on the test datasets show that using all datasets as training set achieves the best averaged performance and our best system ranks 15 out of 73.

We describe a neural network model that jointly learns distributed representations of texts and knowledge base (KB) entities. Given a text in the KB, we train our proposed model to predict entities that are relevant to the text. Our model is designed to be generic with the ability to address various NLP tasks with ease. We train the model using a large corpus of texts and their entity annotations extracted from Wikipedia. We evaluated the model on three important NLP tasks (i.e., sentence textual similarity, entity linking, and factoid question answering) involving both unsupervised and supervised settings. As a result, we achieved state-of-the-art results on all three of these tasks. Our code and trained models are publicly available for further academic research.

The Semantic Web has fuelled the appearance of numerous open-source knowledge bases. Knowledge bases enable new types of information search, going beyond classical query answering and into the realm of exploratory search, and providing answers to new types of user questions. One such question is how two entities are comparable, i.e., what are similarities and differences between the information known about the two entities. Entity comparison is an important task and a widely used functionality available in many information systems. Yet it is usually domain-specific and depends on a fixed set of aspects to compare. In this paper we propose a formal framework for domain-independent entity comparison that provides similarity and difference explanations for input entities. We model explanations as conjunctive queries, we discuss how multiple explanations for an entity pair can be ranked and we provide a polynomial-time algorithm for generating most specific similarity explanations.

This paper presents the results of a set of methods to cluster WordNet word senses. The methods rely on different information sources: confusion matrixes from Senseval-2 Word Sense Disambiguation systems, translation similarities, hand-tagged examples of the target word senses and examples obtained automatically from the web for the target word senses. The clustering results have been evaluated using the coarsegrained word senses provided for the lexical sample in Senseval-2. We have used Cluto, a general clustering environment, in order to test different clustering algorithms. The best results are obtained for the automatically obtained examples, yielding purity values up to 84% on average over 20 nouns.

Over the past years, distributed semantic representations have proved to be effective and flexible keepers of prior knowledge to be integrated into downstream applications. This survey focuses on the representation of meaning. We start from the theoretical background behind word vector space models and highlight one of their major limitations: the meaning conflation deficiency, which arises from representing a word with all its possible meanings as a single vector. Then, we explain how this deficiency can be addressed through a transition from the word level to the more fine-grained level of word senses (in its broader acceptation) as a method for modelling unambiguous lexical meaning. We present a comprehensive overview of the wide range of techniques in the two main branches of sense representation, i.e., unsupervised and knowledge-based. Finally, this survey covers the main evaluation procedures and applications for this type of representation, and provides an analysis of four of its important aspects: interpretability, sense granularity, adaptability to different domains and compositionality.

Vectorial representations of words have grown remarkably popular in natural language processing and machine translation. The recent surge in deep learning-inspired methods for producing distributed representations has been widely noted even outside these fields. Existing representations are typically trained on large monolingual corpora using context-based prediction models. In this paper, we propose extending pre-existing word representations by exploiting Wiktionary. This process results in a substantial extension of the original word vector representations, yielding a large multilingual dictionary of word embeddings. We believe that this resource can enable numerous monolingual and cross-lingual applications, as evidenced in a series of monolingual and cross-lingual semantic experiments that we have conducted.

Knowledge can often be represented using entities connected by relations. For example, the fact that tennis ball is round can be represented as “TennisBall HasShape Round”, where a “TennisBall” is one entity, “HasShape” is a relation and “Round” is another entity. A knowledge base is a way to store such structured information, a knowledge base stores triples of the “an entity-relation-an entity” form, and a real world knowledge base often has millions or billions of such triples. There are several well-known knowledge bases including FreeBase [1], WordNet [2], YAGO [3], etc. They are important in fields like reasoning and question answering; for instance if one asks “what is the shape of a tennis ball”, we can search the knowledge base for the triple as “TennisBall HasShape Round” and output “round” as the answer.

Automatic lexical acquisition has been an active area of research in computational linguistics for over two decades, but the automatic identification of new word-senses has received attention only very recently. Previous work on this topic has been limited by the availability of appropriate evaluation resources. In this paper we present the largest corpus-based dataset of diachronic sense differences to date, which we believe will encourage further work in this area. We then describe several extensions to a state-of-the-art topic modelling approach for identifying new word-senses. This adapted method shows superior performance on our dataset of two different corpus pairs to that of the original method for both: (a) types having taken on a novel sense over time; and (b) the token instances of such novel senses.

WordNet is perhaps the most important and widely used lexical resource for natural language processing systems up to now. WordNet: An Electronic Lexical Database, edited by Christiane Fellbaum, discusses the design of WordNet from both theoretical and historical perspectives, provides an up-to-date description of the lexical database, and presents a set of applications of WordNet. The book contains a foreword by George Miller, an introduction by Christiane Fellbaum, seven chapters from the Cognitive Sciences Laboratory of Princeton University, where WordNet was produced, and nine chapters contributed by scientists from elsewhere. Miller's foreword offers a fascinating account of the history of WordNet. He discusses the presuppositions of such a lexical database, how the top-level noun categories were determined, and the sources of the words in WordNet. He also writes about the evolution of WordNet from its original incarnation as a dictionary browser to a broad-coverage lexicon, and the involvement of different people during its various stages of development over a decade. It makes very interesting reading for casual and serious users of WordNet and anyone who is grateful for the existence of WordNet. The book is organized in three parts. Part I is about WordNet itself and consists of four chapters: "Nouns in WordNet" by George Miller, "Modifiers in WordNet" by Katherine Miller, "A semantic network of English verbs" by Christiane Fellbaum, and "Design and implementation of the WordNet lexical database and search software" by Randee Tengi. These chapters are essentially updated versions of four papers from Miller (1990). Compared with the earlier papers, the chapters in this book focus more on the underlying assumptions and rationales behind the design decisions. The description of the information contained in WordNet, however, is not as detailed as in Miller (1990). The main new additions in these chapters include an explanation of sense grouping in George Miller's chapter, a section about adverbs in Katherine Miller's chapter, observations about autohyponymy (one sense of a word being a hyponym of another sense of the same word) and autoantonymy (one sense of a word being an antonym of another sense of the same word) in Fellbaum's chapter, and Tengi's description of the Grinder, a program that converts the files the lexicographers work with to searchable lexical databases. The three papers in Part II are characterized as "extensions, enhancements and


Question: What activity results in this? Progress in predicting basic physical action-outcome scenarios.

In this study, we investigate the capability of learning the fundamental physical attributes of objects from a collection of untagged video clips. Inspired by the human capacity to comprehend basic physical principles at a tender age, we posit that this skill is crucial for the development of computational vision systems. Our research encompasses diverse physical situations, such as objects descending inclined planes, experiencing collisions, being connected to springs, and impacting various surfaces. The outcomes of these scenarios are influenced by numerous physical properties, including mass, density, and the coefficient of restitution, and our objective is to automatically infer these attributes. To this end, we have amassed a dataset of 17,408 video sequences, featuring 101 distinct objects of varying materials and appearances—shapes, colors, and sizes. This collection, coined as Physics 101, serves as a repository for exploring the physical properties centered around objects. We propose an unsupervised learning model designed to explicitly incorporate fundamental physical laws into its structure. Utilizing observations automatically extracted from videos, our model operates as a form of self-supervision. Experimental results affirm that our approach can successfully learn the physical properties of objects from video data. Moreover, we highlight the model's generative capabilities, enabling it to tackle additional tasks such as predicting the outcomes of physical scenarios.

Our goal is to automate the understanding of natural hand-object manipulation by developing computer visionbased techniques. Our hypothesis is that it is necessary to model the grasp types of hands and the attributes of manipulated objects in order to accurately recognize manipulation actions. Specifically, we focus on recognizing hand grasp types, object attributes and actions from a single image within an unified model. First, we explore the contextual relationship between grasp types and object attributes, and show how that context can be used to boost the recognition of both grasp types and object attributes. Second, we propose to model actions with grasp types and object attributes based on the hypothesis that grasp types and object attributes contain complementary information for characterizing different actions. Our proposed action model outperforms traditional appearance-based models which are not designed to take into account semantic constraints such as grasp types or object attributes. Experiment results on public egocentric activities datasets strongly support our hypothesis.

Humans can effortlessly recognize others' actions in the presence of complex transformations, such as changes in viewpoint. Several studies have located the regions in the brain involved in invariant action recognition; however, the underlying neural computations remain poorly understood. We use magnetoencephalography decoding and a data set of well-controlled, naturalistic videos of five actions (run, walk, jump, eat, drink) performed by different actors at different viewpoints to study the computational steps used to recognize actions across complex transformations. In particular, we ask when the brain discriminates between different actions, and when it does so in a manner that is invariant to changes in 3D viewpoint. We measure the latency difference between invariant and noninvariant action decoding when subjects view full videos as well as form-depleted and motion-depleted stimuli. We were unable to detect a difference in decoding latency or temporal profile between invariant and noninvariant action recognition in full videos. However, when either form or motion information is removed from the stimulus set, we observe a decrease and delay in invariant action decoding. Our results suggest that the brain recognizes actions and builds invariance to complex transformations at the same time and that both form and motion information are crucial for fast, invariant action recognition. NEW & NOTEWORTHY The human brain can quickly recognize actions despite transformations that change their visual appearance. We use neural timing data to uncover the computations underlying this ability. We find that within 200 ms action can be read out of magnetoencephalography data and that this representation is invariant to changes in viewpoint. We find form and motion are needed for this fast action decoding, suggesting that the brain quickly integrates complex spatiotemporal features to form invariant action representations.