Question: Analyzing subjectivity and sentiment in modern standard Arabic texts.

This study delves into the complexities of phrase-level sentiment analysis, particularly focusing on distinguishing between a word's prior polarity and its contextual polarity within a sentence. Traditionally, sentiment analysis algorithms rely on lexicons containing words labeled with their inherent polarity. However, the actual sentiment conveyed by a word in a specific context can often contradict its predefined polarity. For instance, words with positive connotations can be used in negative contexts, and vice versa. Additionally, words that are strongly polarized in isolation can become neutral when embedded in certain contexts, rendering them ineffective in sentiment expression. The objective of this research is to develop a method that can accurately differentiate between a word's prior and contextual polarity, emphasizing the identification of features crucial for this task. A significant component of this challenge involves recognizing when polar words are utilized in non-polar settings. To address this, the study evaluates various features that can help discern between neutral and polar occurrences, as well as features that distinguish between positive and negative contextual polarity. The assessment includes testing the performance of these features across different machine learning algorithms. The results indicate that, with one exception, combining all features yields the highest accuracy. The study also examines how the inclusion of neutral instances impacts the effectiveness of features in distinguishing between positive and negative polarities. It reveals that the presence of neutral instances significantly diminishes performance, suggesting that enhancing the system's capability to detect neutrality could be the key to improving overall accuracy across all polarity classes.

Nowadays, sentiment analysis is a popular research area in computer science. which aims to determine a person’s or a group’s mood, behaviour and opinion about any topic with using textual documents. With the proliferation of social micro-blogging sites, digital opinion text data is increased. Thus, many sentiment analysis researches are performed on these public data in different sociological fields, such as finance, economy and politics. In this paper, a novel sentiment analysis method is proposed on micro-blogging sites which uses new user metrics. Proposed method is used to measure financial community’s sentiment polarity on micro-blogging sites. In addition to that we analyze the correlation between the mood of financial community and the behavior of the Borsa Istanbul 100 index weekly by Pearson correlation coefficient method. Our test results show that this novel sentiment analysis method improves to the accuracy of linear relationship between the behavior of the stock market and the sentiment polarity of the financial community.

Sentiment analysis is a growing field of research, driven by both commercial applications and academic interest. In this paper, we explore multiclass classification of diary-like blog posts for the sentiment dimensions of valence and arousal, where the aim of the task is to predict the level of valence and arousal of a post on a ordinal five-level scale, from very negative/low to very positive/high, respectively. We show how to map discrete affective states into ordinal scales in these two dimensions, based on the psychological model of Russell's circumplex model of affect and label a previously available corpus with multidimensional, real-valued annotations. Experimental results using regression and one-versus-all approaches of support vector machine classifiers show that although the latter approach provides better exact ordinal class prediction accuracy, regression techniques tend to make smaller scale errors.

A Supervised Learning Approach for Imbalanced Text Classification of Biomedical Literature Triage

Sarcasm is a form of speech act in which the speakers convey their message in an implicit way. The inherently ambiguous nature of sarcasm sometimes makes it hard even for humans to decide whether an utterance is sarcastic or not. Recognition of sarcasm can benefit many sentiment analysis NLP applications, such as review summarization, dialogue systems and review ranking systems. In this paper we experiment with semisupervised sarcasm identification on two very different data sets: a collection of 5.9 million tweets collected from Twitter, and a collection of 66000 product reviews from Amazon. Using the Mechanical Turk we created a gold standard sample in which each sentence was tagged by 3 annotators, obtaining F-scores of 0.78 on the product reviews dataset and 0.83 on the Twitter dataset. We discuss the differences between the datasets and how the algorithm uses them (e.g., for the Amazon dataset the algorithm makes use of structured information). We also discuss the utility of Twitter #sarcasm hashtags for the task.

This paper introduces an approach to sentiment analysis which uses support vector machines (SVMs) to bring together diverse sources of potentially pertinent information, including several favorability measures for phrases and adjectives and, where available, knowledge of the topic of the text. Models using the features introduced are further combined with unigram models which have been shown to be effective in the past (Pang et al., 2002) and lemmatized versions of the unigram models. Experiments on movie review data from Epinions.com demonstrate that hybrid SVMs which combine unigram-style feature-based SVMs with those based on real-valued favorability measures obtain superior performance, producing the best results yet published using this data. Further experiments using a feature set enriched with topic information on a smaller dataset of music reviews handannotated for topic are also reported, the results of which suggest that incorporating topic information into such models may also yield improvement.

The given paper describes modern approach to the task of sentiment analysis of movie reviews by using deep learning recurrent neural networks and decision trees. These methods are based on statistical models, which are in a nutshell of machine learning algorithms. The fertile area of research is the application of Google's algorithm Word2Vec presented by Tomas Mikolov, Kai Chen, Greg Corrado and Jeffrey Dean in 2013. The main idea of Word2Vec is the representations of words with the help of vectors in such manner that semantic relationships between words preserved as basic linear algebra operations. The extra advantage of the mentioned algorithm above the alternatives is computational efficiency. This paper focuses on using Word2Vec model for text classification by their sentiment type.

Current approaches in aspect-based sentiment analysis ignore or neutralize unhandled issues emerging from the lexicon-based scoring (i.e., SentiWordNet), whereby lexical sentiment analysis only classifies text based on affect word presence and word count are limited to these surface features. This is coupled with considerably low detection rate among implicit concepts in the text. To address this issues, this paper proposed the use of ontology to i) enhance aspect extraction process by identifying features pertaining to implicit entities, and ii) eliminate lexicon-based sentiment scoring issues which, in turn, improve sentiment analysis and summarization accuracy. Concept-level sentiment analysis aims to go beyond word-level analysis by employing ontologies which act as a semantic knowledge base rather than the lexicon. The outcome is an Ontology-Based Product Sentiment Summarization (OBPSS) framework which outperformed other existing summarization systems in terms of aspect extraction and sentiment scoring. The improved performance is supported by the sentence-level linguistic rules applied by OBPSS in providing a more accurate sentiment analysis.

This paper provides a thorough evaluation of a set of six important Arabic OCR systems available in the market; namely: Abbyy FineReader, Leadtools, Readiris, Sakhr, Tesseract and NovoVerus. We test the OCR systems using a randomly selected images from the well known Arabic Printed Text Image database (250 images from the APTI database) and using a set of 8 images from an Arabic book. The APTI database contains 45.313.600 of both decomposable and non-decomposable word images. In the evaluation, we conduct two tests. The first test is based on usual metrics used in the literature. In the second test, we provide a novel measure for Arabic language, which can be used for other non-Latin languages.

Twitter has emerged as a major social media platform and generated great interest from sentiment analysis researchers. Despite this attention, state-of-the-art Twitter sentiment analysis approaches perform relatively poorly with reported classification accuracies often below 70%, adversely impacting applications of the derived sentiment information. In this research, we investigate the unique challenges presented by Twitter sentiment analysis and review the literature to determine how the devised approaches have addressed these challenges. To assess the state-of-the-art in Twitter sentiment analysis, we conduct a benchmark evaluation of 28 top academic and commercial systems in tweet sentiment classification across five distinctive data sets. We perform an error analysis to uncover the causes of commonly occurring classification errors. To further the evaluation, we apply select systems in an event detection case study. Finally, we summarize the key trends and takeaways from the review and benchmark evaluation and provide suggestions to guide the design of the next generation of approaches.

A FUZZY LOGIC-BASED TEXT CLASSIFICATION METHOD FOR SOCIAL MEDIA


Question: Analyzing subjectivity and sentiment in Modern Standard Arabic texts.

This case study delves into the manual tagging process for sentence-level categorization, specifically focusing on distinguishing between subjective and objective clauses within the Wall Street Journal. Four annotators were tasked with classifying these clauses based on detailed tagging guidelines. The level of consensus among the annotators was examined, and using this data, a final classification was assigned to each clause. To validate these classifications, the study explored correlations within the dataset between the subjective category and a fundamental semantic class proposed by Quirk et al. (1985), thereby offering empirical backing to the subjective-objective categorization. This research not only highlights the intricacies involved in manual tagging but also underscores the importance of such categorization in understanding language nuances in a specific context.

We present the first shared task on detecting the intensity of emotion felt by the speaker of a tweet. We create the first datasets of tweets annotated for anger, fear, joy, and sadness intensities using a technique called best–worst scaling (BWS). We show that the annotations lead to reliable fine-grained intensity scores (rankings of tweets by intensity). The data was partitioned into training, development, and test sets for the competition. Twenty-two teams participated in the shared task, with the best system obtaining a Pearson correlation of 0.747 with the gold intensity scores. We summarize the machine learning setups, resources, and tools used by the participating teams, with a focus on the techniques and resources that are particularly useful for the task. The emotion intensity dataset and the shared task are helping improve our understanding of how we convey more or less intense emotions through language.

Vancouver, October 2005. OpinionFinder: A system for subjectivity analysis Theresa Wilson‡, Paul Hoffmann‡, Swapna Somasundaran†, Jason Kessler†, Janyce Wiebe†‡, Yejin Choi§, Claire Cardie§, Ellen Riloff∗, Siddharth Patwardhan∗ ‡Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA 15260 †Department of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260 §Department of Computer Science, Cornell University, Ithaca, NY 14853 ∗School of Computing, University of Utah, Salt Lake City, UT 84112

Sentiment analysis (SA) has become one of the most active and progressively popular areas in information retrieval and text mining due to the expansion of the World Wide Web (WWW). SA deals with the computational treatment or the classification of user’s sentiments, opinions and emotions hidden within the text. Aspect extraction is the most vital and extensively explored phase of SA to carry out the classification of sentiments in precise manners. During the last decade, enormous number of research has focused on identifying and extracting aspects. Therefore, in this survey, a comprehensive overview has been attempted for different aspect extraction techniques and approaches. These techniques have been categorized in accordance with the adopted approach. Despite being a traditional survey, a comprehensive comparative analysis is conducted among different approaches of aspect extraction, which not only elaborates the performance of any technique but also guides the reader to compare the accuracy with other state-of-the-art and most recent approaches.

Sentiment Analysis deals with the analysis of emotions, opinions and facts in the sentences which are expressed by the people. It allows us to track attitudes and feelings of the people by analyzing blogs, comments, reviews and tweets about all the aspects. The development of Internet has strong influence in all types of industries like tourism, healthcare and any business. The availability of Internet has changed the way of accessing the information and sharing their experience among users. Social media provide this information and these comments are trusted by other users. This paper recognizes the use and impact of social media on healthcare industry by analyzing the users’ feelings expressed in the form of free text, thereby gives the quality indicators of services or features related with them. In this paper, a sentiment classifier model using improved Term Frequency Inverse Document Frequency (TFIDF) method and linear regression model has been proposed to classify online reviews, tweets or customer feedback for various features. The model involves the process of gathering online user reviews about hospitals and analyzes those reviews in terms of sentiments expressed. Information Extraction process filters irrelevant reviews, extracts sentimental words of features identified and quantifies the sentiment of features using sentiment dictionary. Emotionally expressed positive or negative words are assigned weights using the classification prescribed in the dictionary. The sentiment analysis on tweets/reviews is done for various features using Natural Language Processing (NLP) and Information Retrieval (IR) techniques. The proposed linear regression model using the senti-score predicts the star rating of the feature of service. The statistical results show that improved TF-IDF method gives better accuracy when compared with TF and TF-IDF methods, used for representing the text. The senti-score obtained as a result of text analysis (user feedback) on features gives not only the opinion summarization but also the comparative results on various features of different competitors. This information can be used by business to focus on the low scored features so as to improve their business and ensure a very high level of user satisfaction.

Sentiment Analysis (SA) and summarization has recently become the focus of many researchers, because analysis of online text is beneficial and demanded in many different applications. One such application is product-based sentiment summarization of multi-documents with the purpose of informing users about pros and cons of various products. This paper introduces a novel solution to target-oriented (i.e. aspect-based) sentiment summarization and SA of short informal texts with a main focus on Twitter posts known as "tweets". We compare different algorithms and methods for SA polarity detection and sentiment summarization. We show that our hybrid polarity detection system not only outperforms the unigram state-of-the-art baseline, but also could be an advantage over other methods when used as a part of a sentiment summarization system. Additionally, we illustrate that our SA and summarization system exhibits a high performance with various useful functionalities and features.

Textual information in the world can be broadly categorized into two main types: facts and opinions. Facts are objective expressions about entities, events and their properties. Opinions are usually subjective expressions that describe people’s sentiments, appraisals or feelings toward entities, events and their properties. The concept of opinion is very broad. In this chapter, we only focus on opinion expressions that convey people’s positive or negative sentiments. Much of the existing research on textual information processing has been focused on mining and retrieval of factual information, e.g., information retrieval, Web search, text classification, text clustering and many other text mining and natural language processing tasks. Little work had been done on the processing of opinions until only recently. Yet, opinions are so important that whenever we need to make a decision we want to hear others’ opinions. This is not only true for individuals but also true for organizations.

Political views frequently conflict in the coverage of contentious political issues, potentially causing serious social problems. We present a novel social annotation analysis approach for identification of news articles' political orientation. The approach focuses on the behavior of individual commenters. It uncovers commenters' sentiment patterns towards political news articles, and predicts the political orientation from the sentiments expressed in the comments. It takes advantage of commenters' participation as well as their knowledge and intelligence condensed in the sentiment of comments, thereby greatly reduces the high complexity of political view identification. We conduct extensive study on commenters' behaviors, and discover predictive commenters showing a high degree of regularity in their sentiment patterns. We develop and evaluate sentiment pattern-based methods for political view identification.

Existing target-specific sentiment recognition methods consider only a single target per tweet, and have been shown to miss nearly half of the actual targets mentioned. We present a corpus of UK election tweets, with an average of 3.09 entities per tweet and more than one type of sentiment in half of the tweets. This requires a method for multi-target specific sentiment recognition, which we develop by using the context around a target as well as syntactic dependencies involving the target. We present results of our method on both a benchmark corpus of single targets and the multi-target election corpus, showing state-of-the art performance in both corpora and outperforming previous approaches to multi-target sentiment task as well as deep learning models for singletarget sentiment.

Indian epics have not been analyzed computationally to the extent that Greek epics have. In this paper, we show how interesting insights can be derived from the ancient epic Mahabharata by applying a variety of analytical techniques based on a combination of natural language processing, sentiment/emotion analysis and social network analysis methods. One of our key findings is the pattern of significant changes in the overall sentiment of the epic story across its eighteen chapters and the corresponding characterization of the primary protagonists in terms of their sentiments, emotions, centrality and leadership attributes in the epic saga.

Across the globe, people are voicing their opinion in social media and various other online fora. Given such data, modern deep learning-based sentiment analysis methods excel at determining the sentiment polarity of what is being said about companies, products, etc. Unfortunately, such deep methods require significant training data, while for many languages, resources and training data are scarce. In this work, we present a cross-lingual propagation algorithm that yields sentiment embedding vectors for numerous languages. We then rely on a dual-channel convolutional neural architecture to incorporate them into the network. This allows us to achieve gains in deep sentiment analysis across a range of languages and domains.


Question: Analyzing subjectivity and sentiment in modern standard Arabic text.

This paper explores the task of Sentiment Analysis in Modern Standard Arabic (MSA), focusing on the detection and classification of subjective expressions and sentiments within text. Sentiment Analysis is a complex process, particularly in languages like MSA, which possess rich morphology and syntax. We propose a method that effectively identifies the sentiment of opinions in MSA texts. Our approach involves two main components: a word sentiment analysis module and a sentence-level sentiment aggregation mechanism. The word sentiment analysis component evaluates the polarity of individual words, while the sentence-level aggregation mechanism combines these individual word sentiments to determine the overall sentiment of a sentence. We experimented with a variety of models for classifying sentiment at both the word and sentence levels, achieving promising outcomes. The results demonstrate the efficacy of our method in accurately determining the sentiment of opinions in MSA, thereby contributing to the broader field of Sentiment Analysis in non-English languages.

Political views frequently conflict in the coverage of contentious political issues, potentially causing serious social problems. We present a novel social annotation analysis approach for identification of news articles' political orientation. The approach focuses on the behavior of individual commenters. It uncovers commenters' sentiment patterns towards political news articles, and predicts the political orientation from the sentiments expressed in the comments. It takes advantage of commenters' participation as well as their knowledge and intelligence condensed in the sentiment of comments, thereby greatly reduces the high complexity of political view identification. We conduct extensive study on commenters' behaviors, and discover predictive commenters showing a high degree of regularity in their sentiment patterns. We develop and evaluate sentiment pattern-based methods for political view identification.

The use of the web as a universal communication platform generates large volumes of data (Big data), which in many cases, need to be processed so that they can become useful knowledge in face of the sceptics who have doubts about the credibility of such information. The use of web data that comes from educational contexts needs to be addressed, since that large amount of unstructured information is not being valued, losing valuable information that can be used. To solve this problem, we propose the use of data mining techniques such as sentiment analysis to validate the information that comes from the educational platforms. The objective of this research is to propose a methodology that allows the user to apply sentiment analysis in a simple way, because although some researchers have done it, very few do with data in the educational context. The results obtained prove that the proposal can be used in similar cases.

With the advent of Web 2.0, people became more eager to express and share their opinions on web regarding day-to-day activities and global issues as well. Evolution of social media has also contributed immensely to these activities, thereby providing us a transparent platform to share views across the world. These electronic Word of Mouth (eWOM) statements expressed on the web are much prevalent in business and service industry to enable customer to share his/her point of view. In the last one and half decades, research communities, academia, public and service industries are working rigorously on sentiment analysis, also known as, opinion mining, to extract and analyze public mood and views. In this regard, this paper presents a rigorous survey on sentiment analysis, which portrays views presented by over one hundred articles published in the last decade regarding necessary tasks, approaches, and applications of sentiment analysis. Several sub-tasks need to be performed for sentiment analysis which in turn can be accomplished using various approaches and techniques. This survey covering published literature during 2002-2015, is organized on the basis of sub-tasks to be performed, machine learning and natural language processing techniques used and applications of sentiment analysis. The paper also presents open issues and along with a summary table of a hundred and sixty one articles.

The presented work aims at generating a systematically annotated corpus that can support the enhancement of sentiment analysis tasks in Telugu using wordlevel sentiment annotations. From OntoSenseNet, we extracted 11,000 adjectives, 253 adverbs, 8483 verbs and sentiment annotation is being done by language experts. We discuss the methodology followed for the polarity annotations and validate the developed resource. This work aims at developing a benchmark corpus, as an extension to SentiWordNet, and baseline accuracy for a model where lexeme annotations are applied for sentiment predictions. The fundamental aim of this paper is to validate and study the possibility of utilizing machine learning algorithms, word-level sentiment annotations in the task of automated sentiment identification. Furthermore, accuracy is improved by annotating the bi-grams extracted from the target corpus.

Online social media, such as Reddit, has become an important resource to share personal experiences and communicate with others. Among other personal information, some social media users communicate about mental health problems they are experiencing, with the intention of getting advice, support or empathy from other users. Here, we investigate the language of Reddit posts specific to mental health, to define linguistic characteristics that could be helpful for further applications. The latter include attempting to identify posts that need urgent attention due to their nature, e.g. when someone announces their intentions of ending their life by suicide or harming others. Our results show that there are a variety of linguistic features that are discriminative across mental health user communities and that can be further exploited in subsequent classification tasks. Furthermore, while negative sentiment is almost uniformly expressed across the entire data set, we demonstrate that there are also condition-specific vocabularies used in social media to communicate about particular disorders. Source code and related materials are available from: https: //github.com/gkotsis/ reddit-mental-health.

Aspect Based Sentiment Analysis (ABSA) is the task of mining and summarizing opinions from text about specific entities and their aspects. This article describes two datasets for the development and testing of ABSA systems for French which comprise user reviews annotated with relevant entities, aspects and polarity values. The first dataset contains 457 restaurant reviews (2365 sentences) for training and testing ABSA systems, while the second contains 162 museum reviews (655 sentences) dedicated to out-of-domain evaluation. Both datasets were built as part of SemEval-2016 Task 5 “Aspect-Based Sentiment Analysis” where seven different languages were represented, and are publicly available for research purposes. This article provides examples and statistics by annotation type, summarizes the annotation guidelines and discusses their cross-lingual applicability. It also explains how the data was used for evaluation in the SemEval ABSA task and briefly presents the results obtained for French.

Sentiment analysis is considered to be a category of machine learning and natural language processing. It is used to extricate, recognize, or portray opinions from different content structures, including news, audits and articles and categorizes them as positive, neutral and negative. It is difficult to predict election results from tweets in different Indian languages. We used Twitter Archiver tool to get tweets in Hindi language. We performed data (text) mining on 42,235 tweets collected over a period of a month that referenced five national political parties in India, during the campaigning period for general state elections in 2016. We made use of both supervised and unsupervised approaches. We utilized Dictionary Based, Naive Bayes and SVM algorithm to build our classifier and classified the test data as positive, negative and neutral. We identified the sentiment of Twitter users towards each of the considered Indian political parties. The results of the analysis for Naive Bayes was the BJP (Bhartiya Janta Party), for SVM it was the BJP (Bhartiya Janta Party) and for the Dictionary Approach it was the Indian Nathional Congress. SVM predicted a 78.4% chance that the BJP would win more elections in the general election due to the positive sentiment they received in tweets. As it turned out, BJP won 60 out of 126 constituencies in the 2016 general election, far more than any other political party as the next party (the Indian National Congress) only won 26 out of 126 constituencies.

The advent of the Social Web has enabled anyone with an Internet connection to easily create and share their ideas, opinions and content with millions of other people around the world. In pace with a global deluge of videos from billions of computers, smartphones, tablets, university projectors and security cameras, the amount of multimodal content on the Web has been growing exponentially, and with that comes the need for decoding such information into useful knowledge. In this paper, a multimodal affective data analysis framework is proposed to extract user opinion and emotions from video content. In particular, multiple kernel learning is used to combine visual, audio and textual modalities. The proposed framework outperforms the state-of-the-art model in multimodal sentiment analysis research with a margin of 10-13% and 3-5% accuracy on polarity detection and emotion recognition, respectively. The paper also proposes an extensive study on decision-level fusion.

With the problem of increased web resources and the huge amount of information available, the necessity of having automatic summarization systems appeared. Since summarization is needed the most in the process of searching for information on the web, where the user aims at a certain domain of interest according to his query, domain-based summaries would serve the best. Despite the existence of plenty of research work in the domain-based summarization in English, there is lack of them in Arabic due to the shortage of existing knowledge bases. In this paper an Ontology-based Summarization System for Arabic Documents, OSSAD, is introduced. Domain knowledge is extracted from an Arabic corpus and represented by topic related concepts/keywords and the lexical relations among them. The user’s query is first expanded by using the Arabic WordNet and then by adding the domain-specific knowledge base to the expansion. For summarization, decision tree algorithm (C4.5) is used, which was trained by a set of features extracted from the original documents. For the testing dataset, Essex Arabic Summaries Corpus (EASC) was used. Recall Oriented Understudy for Gisting Evaluation (ROUGE) was used to compare OSSAD summaries with the human summaries along with other automatic summarization systems, showing that the proposed approach demonstrated promising results.

The volume of discussion about a product in weblogs has recently been shown to correlate with the product’s financial performance. In this paper, we study whether applying sentiment analysis methods to weblog data results in better correlation than volume only, in the domain of movies. Our main finding is that positive sentiment is indeed a better predictor for movie success when applied to a limited context around references to the movie in weblogs, posted prior to its release. If my film makes one more person miserable, I’ve done my job.


Question: Analyzing subjectivity and sentiment in Modern Standard Arabic texts.

This paper reflects on our multi-year endeavor to construct a comprehensive, annotated corpus of Arabic text, known as the Penn Arabic Treebank. We delve into the following key aspects: (a) scrutinize significant linguistic characteristics of Arabic in relation to methodological decisions, (b) elucidate our rationale for adopting the annotation style of the Penn English Treebank—challenging Arabic-speaking annotators to work within an unfamiliar grammatical framework—over traditional Arabic grammatical styles, which would necessitate NLP researchers to grapple with a novel system; (c) demonstrate the critical role of human annotation and the complexities faced in automatic analysis, particularly concerning orthographic ambiguity management by both morphological analyzers and human annotators; (d) provide a detailed case study of our Arabic Treebank methodology, zeroing in on a specific linguistic construction through morphological analysis, tagging, and syntactic analysis, tracing its evolution through the entire annotation process; and (e) conclude with an overview of our accomplishments to date and outline the future tasks that lie ahead. Our experience underscores the intricacies involved in balancing linguistic accuracy and annotation efficiency in the creation of a large-scale annotated Arabic corpus.

We present a new approach for documentlevel sentiment inference, where the goal is to predict directed opinions (who feels positively or negatively towards whom) for all entities mentioned in a text. To encourage more complete and consistent predictions, we introduce an ILP that jointly models (1) sentenceand discourse-level sentiment cues, (2) factual evidence about entity factions, and (3) global constraints based on social science theories such as homophily, social balance, and reciprocity. Together, these cues allow for rich inference across groups of entities, including for example that CEOs and the companies they lead are likely to have similar sentiment towards others. We evaluate performance on new, densely labeled data that provides supervision for all pairs, complementing previous work that only labeled pairs mentioned in the same sentence. Experiments demonstrate that the global model outperforms sentence-level baselines, by providing more coherent predictions across sets of related entities.

Existing target-specific sentiment recognition methods consider only a single target per tweet, and have been shown to miss nearly half of the actual targets mentioned. We present a corpus of UK election tweets, with an average of 3.09 entities per tweet and more than one type of sentiment in half of the tweets. This requires a method for multi-target specific sentiment recognition, which we develop by using the context around a target as well as syntactic dependencies involving the target. We present results of our method on both a benchmark corpus of single targets and the multi-target election corpus, showing state-of-the art performance in both corpora and outperforming previous approaches to multi-target sentiment task as well as deep learning models for singletarget sentiment.

This paper describes an approach and our experiences from the development, deployment and usability testing of a Natural Language Processing (NLP) and Information Retrieval system that supports the moderation of user comments on a large newspaper website. We highlight some of the differences between industry-oriented and academic research settings and their influence on the decisions made in the data collection and annotation processes, selection of document representation and machine learning methods. We report on classification results, where the problems to solve and the data to work with come from a commercial enterprise. In this context typical for NLP research, we discuss relevant industrial aspects. We believe that the challenges faced as well as the solutions proposed for addressing them can provide insights to others working in a similar setting. Data and experiment code related to this paper are available for download at https://ofai.github.io/million-post-corpus.

The emergence of online social networks in the past few years has generated an enormous amount of information about potentially any subject. Valuable data containing users' opinions and thoughts are available on those repositories and several sentiment analysis techniques have been proposed that address the problem of understanding the opinion orientation of the users' postings. In this paper, we take a different perspective to the problem through a user centric approach, which uses a graph to model users (and their postings) and applies link mining techniques to infer opinions of users. Preliminary experiments on a Twitter corpus have shown promising results.

Variation in language is ubiquitous, and is particularly evident in newer forms of writing such as social media. Fortunately, variation is not random, but is usually linked to social factors. By exploiting linguistic homophily — the tendency of socially linked individuals to use language similarly — it is possible to build models that are more robust to variation. In this paper, we focus on social network communities, which make it possible to generalize sociolinguistic properties from authors in the training set to authors in the test sets, without requiring demographic author metadata. We detect communities via standard graph clustering algorithms, and then exploit these communities by learning community-specific projections of word embeddings. These projections capture shifts in word meaning in different social groups; by modeling them, we are able to improve the overall accuracy of Twitter sentiment analysis by a significant margin over competitive prior work.

Sentiment analysis is a growing field of research, driven by both commercial applications and academic interest. In this paper, we explore multiclass classification of diary-like blog posts for the sentiment dimensions of valence and arousal, where the aim of the task is to predict the level of valence and arousal of a post on a ordinal five-level scale, from very negative/low to very positive/high, respectively. We show how to map discrete affective states into ordinal scales in these two dimensions, based on the psychological model of Russell's circumplex model of affect and label a previously available corpus with multidimensional, real-valued annotations. Experimental results using regression and one-versus-all approaches of support vector machine classifiers show that although the latter approach provides better exact ordinal class prediction accuracy, regression techniques tend to make smaller scale errors.

Sentiment analysis is very perplexing and massive issue in the field of social data mining. Twitter is one of the mostly used social media where people discuss on various issues in a dense way. The tweets about a particular topic give peoples’ views, opinions, orientations, inclinations about that topic. In this work, we have used pre-labeled (with positive, negative and neutral opinion) tweets on particular topics for sentiment classification. Opinion score of each tweet is calculated using feature vectors. These opinion score is used to classify the tweets into positive, negative and neutral classes. Then using various machine learning classifiers the accuracy of predicted classification with respect to actual classification is being calculated and compared using supervised learning model. Along with building a sentiment classification model, analysis of tweets is being carried out by visualizing the wordcloud of tweets using R.

The machine learning approach is represented with two methods: the maximum entropy method and support vector machine. Text representation for the maximum entropy method includes the information about the proportion of positive and negative words and collocations, the quantity of interrogation and exclamation marks, emoticons, obscene language. For the support vector machine binary vectors with cosine normalization are built on texts.

In the recent years, microblogging services, as Twitter, have become a popular tool for expressing feelings, opinions, broadcasting news, and communicating with friends. Twitter users produced more than 340 million tweets per day which may be consider a rich source of user information. We take a supervised approach to the problem, but leverage existing hashtags in Twitter for building our training data. Finally, we tested the Spanish emotional corpus applying two different machine learning algorithms for emotion identification reaching about 65% accuracy.

This paper focuses on how to improve aspect-level opinion mining for online customer reviews. We first propose a novel generative topic model, the Joint Aspect/Sentiment (JAS) model, to jointly extract aspects and aspect-dependent sentiment lexicons from online customer reviews. An aspect-dependent sentiment lexicon refers to the aspect-specific opinion words along with their aspect-aware sentiment polarities with respect to a specific aspect. We then apply the extracted aspect-dependent sentiment lexicons to a series of aspect-level opinion mining tasks, including implicit aspect identification, aspect-based extractive opinion summarization, and aspect-level sentiment classification. Experimental results demonstrate the effectiveness of the JAS model in learning aspect- dependent sentiment lexicons and the practical values of the extracted lexicons when applied to these practical tasks.


Question: Evolutionary shifts in the connection between grammar and the lexicon during development.

Exploring the Evolution of Grammar from the Lexicon

This paper delves into the intricate process by which grammar emerges from the lexicon, focusing on the developmental shifts in the relationship between these two fundamental linguistic components. Traditionally, grammar and the lexicon have been viewed as distinct entities, each governed by its own set of rules and mechanisms. However, recent studies have highlighted the interdependence and co-evolutionary nature of grammar and the lexicon, particularly during the early stages of language acquisition.

In the context of language development, the lexicon serves as the foundation upon which grammatical structures are built. Initially, children acquire words and their meanings, gradually organizing them into more complex structures as they develop an understanding of grammatical rules. This evolution is not a linear process but rather a dynamic interplay where the lexicon and grammar influence and shape each other.

The lexicon provides the raw material for grammar, with words and their semantic properties influencing the formation of grammatical patterns. For instance, the acquisition of verb tenses is closely linked to the lexicon, as children must first understand the meaning of time-related words before they can correctly apply temporal markers. Similarly, the development of noun phrases is influenced by the lexicon, as the understanding of noun properties and their relationships to other words is crucial for the correct formation of phrases.

Conversely, grammar plays a crucial role in shaping the lexicon. The rules of syntax guide the organization of words into meaningful sentences, influencing the way words are stored and accessed in memory. Grammar also facilitates the acquisition of new words, as it provides a framework for predicting and understanding the function and meaning of unfamiliar lexical items.

As children's language skills mature, the relationship between grammar and the lexicon becomes more refined. The lexicon expands, incorporating new words and meanings, while grammatical structures become more complex and nuanced. This development is marked by a shift from reliance on the lexicon for grammatical organization to a more sophisticated interplay where grammar and the lexicon co-evolve in a mutually reinforcing manner.

Understanding the emergence of grammar from the lexicon is crucial for theories of language acquisition and has implications for language teaching and learning. It highlights the importance of a balanced approach to language instruction, one that recognizes the interdependence of grammar and vocabulary and promotes their integrated development. By acknowledging the dynamic relationship between grammar and the lexicon, educators can create more effective strategies for fostering language skills in learners of all ages.

Grammatical error correction (GEC) is the task of automatically correcting grammatical errors in written text. Previous research has mainly focussed on individual error types and current commercial proofreading tools only target limited error types. As sentences produced by learners may contain multiple errors of different types, a practical error correction system should be able to detect and correct all errors. In this thesis, we investigate GEC for learners of English as a Second Language (ESL). Specifically, we treat GEC as a translation task from incorrect into correct English, explore new models for developing end-to-end GEC systems for all error types, study system performance for each error type, and examine model generali-sation to different corpora. First, we apply Statistical Machine Translation (SMT) to GEC and prove that it can form the basis of a competitive all-errors GEC system. We implement an SMT-based GEC system which contributes to our winning system submitted to a shared task in 2014. Next, we propose a ranking model to re-rank correction candidates generated by an SMT-based GEC system. This model introduces new linguistic information and we show that it improves correction quality. Finally, we present the first study using Neural Machine Translation (NMT) for GEC. We demonstrate that NMT can be successfully applied to GEC and help capture new errors missed by an SMT-based GEC system. While we focus on GEC for English, our methods presented in this thesis can be easily applied to any language. Acknowledgements First and foremost, I owe a huge debt of gratitude to my supervisor, Ted Briscoe, who has patiently guided me through my PhD and always been very helpful, understanding and supportive. I cannot thank him enough for providing me with opportunities that helped me grow as a researcher and a critical thinker. I am immensely grateful to my examiners, Paula Buttery and Stephen Pulman, for their thorough reading of my thesis, their valuable comments and an enjoyable viva. My appreciation extends to my fellow members of the Natural Language and Information Processing research group, with whom I have always enjoyed discussing our work and other random things. My gratitude goes to Stephen Clark and Ann Copestake for giving me early feedback on my work as well as Christopher Bryant for generously reading my thesis draft. I would especially like to thank Mariano Felice for being not just a great colleague but also a dear friend. A special mention has …

We demonstrate that a state-of-the-art parser can be built using only a lexical tagging model and a deterministic grammar, with no explicit model of bi-lexical dependencies. Instead, all dependencies are implicitly encoded in an LSTM supertagger that assigns CCG lexical categories. The parser significantly outperforms all previously published CCG results, supports efficient and optimal A∗ decoding, and benefits substantially from semisupervised tri-training. We give a detailed analysis, demonstrating that the parser can recover long-range dependencies with high accuracy and that the semi-supervised learning enables significant accuracy gains. By running the LSTM on a GPU, we are able to parse over 2600 sentences per second while improving state-of-the-art accuracy by 1.1 F1 in domain and up to 4.5 F1 out of domain.

We introduce (1) a novel stochastic inversion transduction grammar formalism for bilingual language modeling of sentence-pairs, and (2) the concept of bilingual parsing with a variety of parallel corpus analysis applications. Aside from the bilingual orientation, three major features distinguish the formalism from the finite-state transducers more traditionally found in computational linguistics: it skips directly to a context-free rather than finite-state base, it permits a minimal extra degree of ordering flexibility, and its probabilistic formulation admits an efficient maximum-likelihood bilingual parsing algorithm. A convenient normal form is shown to exist. Analysis of the formalism's expressiveness suggests that it is particularly well suited to modeling ordering shifts between languages, balancing needed flexibility against complexity constraints. We discuss a number of examples of how stochastic inversion transduction grammars bring bilingual constraints to bear upon problematic corpus analysis tasks such as segmentation, bracketing, phrasal alignment, and parsing.

The authors examine the implications of research on Chinese for theories of reading and propose the lexical constituency model as a general framework for word reading across writing systems. Word identities are defined by 3 interlinked constituents (orthographic, phonological, and semantic). The implemented model simulates the time course of graphic, phonological, and semantic priming effects, including immediate graphic facilitation followed by graphic inhibition with simultaneous phonological facilitation, a pattern unique to the Chinese writing system. Pseudocharacter primes produced only facilitation, supporting the model's assumption that lexical thresholds determine phonological and semantic, but not graphic, effects. More generally, both universal reading processes and writing system constraints exist. Although phonology is universal, its activation process depends on how the writing system structures graphic units.

We present two methods for unsupervised segmentation of words into morphemelike units. The model utilized is especially suited for languages with a rich morphology, such as Finnish. The first method is based on the Minimum Description Length (MDL) principle and works online. In the second method, Maximum Likelihood (ML) optimization is used. The quality of the segmentations is measured using an evaluation method that compares the segmentations produced to an existing morphological analysis. Experiments on both Finnish and English corpora show that the presented methods perform well compared to a current stateof-the-art system.

In generaltext-to-speechsystems, it is notpossibleto guaranteethat a lexiconwill containall wordsfoundin a text, thereforesomesystemfor predictingpronunciationfrom theword itself is necessary . Herewe presenta generalframework for building letter to sound (LTS) rulesfrom a word list in a language.The techniquecanbe fully automatic,thougha small amountof handseedingcangive betterresults.We have appliedthis techniqueto English(UK and US), Frenchand German. The generatedmodelsachieve, 75%, 58%, 93% and89%, respecti vely, wordscorrectfor held out data from theword lists. To testour modelson moretypical datawe alsoanalyzedgeneral text, to find which wordsdo not appearin our lexicon. Theseunknown wordswereusedasamorerealistictestcorpusfor ourmodels. We also discussthe distribution and type of suchunknown words.

Two sets of items can share the same underlying conceptual structure, while appearing unrelated at a surface level. Humans excel at recognizing and using alignments between such underlying structures in many domains of cognition, most notably in analogical reasoning. Here we show that structural alignment reveals how different people's neural representations of word meaning are preserved across different languages, such that patterns of brain activation can be used to translate words from one language to another. Groups of Chinese and English speakers underwent fMRI scanning while reading words in their respective native languages. Simply by aligning structures representing the two groups' neural semantic spaces, we successfully infer all seven Chinese–English word translations. Beyond language translation, conceptual structural alignment underlies many aspects of high-level cognition, and this work opens the door to deriving many such alignments directly from neural representational content.

The efficacy of syntactic features for topic-independent authorship attribution is evaluated, taking a feature set of frequencies of words and punctuation marks as baseline. The features are ‘deep’ in the sense that they are derived by parsing the subject texts, in contrast to ‘shallow’ syntactic features for which a part-of-speech analysis is enough. The experiments are conducted on a corpus of novels written around the year 1900 by 20 different authors, and cover two tasks. In the first task, text samples are taken from books by one author, and the goal is to pair samples from the same book. In the second task, text samples are taken from several authors, but only one sample from each book, and the goal is to pair samples from the same author. In the first task, the baseline feature set outperformed the syntax-based feature set, but for the second task, the outcome was the opposite. This suggests that, compared to lexical features such as vocabulary and punctuation, syntactic features are more robust to changes in topic.

RNN language models have achieved state-ofthe-art results on various tasks, but what exactly they are representing about syntax is as yet unclear. Here we investigate whether RNN language models learn humanlike word order preferences in syntactic alternations. We collect language model surprisal scores for controlled sentence stimuli exhibiting major syntactic alternations in English: heavy NP shift, particle shift, the dative alternation, and the genitive alternation. We show that RNN language models reproduce human preferences in these alternations based on NP length, animacy, and definiteness. We collect human acceptability ratings for our stimuli, in the first acceptability judgment experiment directly manipulating the predictors of syntactic alternations. We show that the RNNs’ performance is similar to the human acceptability ratings and is not matched by an n-gram baseline model. Our results show that RNNs learn the abstract features of weight, animacy, and definiteness which underlie soft constraints on syntactic alternations. The best-performing models for many natural language processing tasks in recent years have been recurrent neural networks (RNNs) (Elman, 1990; Sutskever et al., 2014; Goldberg, 2017), but the black-box nature of these models makes it hard to know exactly what generalizations they have learned about their linguistic input: Have they learned generalizations stated over hierarchical structures, or only dependencies among relatively local groups of words (Linzen et al., 2016; Gulordava et al., 2018; Futrell et al., 2018)? Do they represent structures analogous to syntactic dependency trees (Williams et al., 2018), and can they represent complex relationships such as filler–gap dependencies (Chowdhury and Zamparelli, 2018; Wilcox et al., 2018)? In order to make progress with RNNs, it is crucial to determine what RNNs actually learn given currently standard practices; then we can design network architectures, objective functions, and training practices to build on strengths and alleviate weaknesses (Linzen, 2018). In this work, we investigate whether RNNs trained on a language modeling objective learn certain syntactic preferences exhibited by humans, especially those involving word order. We draw on a rich literature from quantitative linguistics that has investigated these preferences in corpora and experiments (e.g., McDonald et al., 1993; Stallings et al., 1998; Bresnan et al., 2007; Rosenbach, 2008). Word order preferences are a key aspect of human linguistic knowledge. In many cases, they can be captured using local co-occurrence statistics: for example, the preference for subject–verb– object word order in English can often be captured directly in short word strings, as in the dramatic preference for I ate apples over I apples ate. However, some word order preferences are more abstract and can only be stated in terms of higherorder linguistic units and abstract features. For example, humans exhibit a general preference for word orders in which words linked in syntactic dependencies are close to each other: such sentences are produced more frequently and comprehended more easily (Hawkins, 1994; Futrell et al., 2015; Temperley and Gildea, 2018). We are interested in whether RNNs learn abstract word order preferences as a way of probing their syntactic knowledge. If RNNs exhibit these preferences for appropriately controlled stimuli, then on some level they have learned the abstractions required to state them. Knowing whether RNNs show human-like word order preferences also bears on their suitability as language generation systems. White and Rajkumar (2012) have shown that language gener-

The authors outline a cognitive and computational account of causal learning in children. They propose that children use specialized cognitive systems that allow them to recover an accurate "causal map" of the world: an abstract, coherent, learned representation of the causal relations among events. This kind of knowledge can be perspicuously understood in terms of the formalism of directed graphical causal models, or Bayes nets. Children's causal learning and inference may involve computations similar to those for learning causal Bayes nets and for predicting with them. Experimental results suggest that 2- to 4-year-old children construct new causal maps and that their learning is consistent with the Bayes net formalism.


Question: Evolutionary shifts in the connection between grammar and the lexicon during development.

This study explores a novel strategy for rapid word learning in uncertain environments by leveraging cross-situational statistics. Given the vast array of potential word-to-object pairings in natural learning settings, prior theories have emphasized constraints like linguistic, social, cognitive, and attentional factors at any given instant. However, this paper introduces a cross-situational learning method that hinges on calculating distributional statistics across words, referents, and crucially, the co-occurrences of words and referents over multiple moments. To test this hypothesis, adults were briefly exposed to a series of trials featuring several spoken words and a set of individual object images. No direct information about the word-image correspondences was provided within each trial. Despite this, learners were able to deduce the word-image mappings by discerning statistical patterns across trials. Various learning scenarios were designed to manipulate the level of within-trial reference ambiguity, the number of trials, and the duration of each trial. The robust learning outcomes across different conditions strongly suggest that learners are adept at computing cross-trial statistics with high accuracy, enabling them to quickly acquire word-referent associations even in highly complex and ambiguous learning environments.

Language is used to communicate ideas. Ideas are mental tools for coping with a complex and uncertain world. Thus human conceptual structures should be key to language meaning, and probability—the mathematics of uncertainty— should be indispensable for describing both language and thought. Indeed, probabilistic models are enormously useful in modeling human cognition (Tenenbaum et al., 2011) and aspects of natural language (Bod et al., 2003; Chater et al., 2006). With a few early exceptions (e.g. Adams, 1975; Cohen, 1999b), probabilistic tools have only recently been used in natural language semantics and pragmatics. In this chapter we synthesize several of these modeling advances, exploring a formal model of interpretation grounded, via lexical semantics and pragmatic inference, in conceptual structure. Flexible human cognition is derived in large part from our ability to imagine possibilities (or possible worlds). A rich set of concepts, intuitive theories, and other mental representations support imagining and reasoning about possible worlds—together we will call these the conceptual lexicon. We posit that this collection of concepts also forms the set of primitive elements available for lexical semantics: word meanings can be built from the pieces of conceptual structure. Larger semantic structures are then built from word meanings by composition, ultimately resulting in a sentence meaning which is a phrase in the “language of thought” provided by the conceptual lexicon. This expression is truth-functional in that it takes on a Boolean value for each imagined world, and it can thus be used as the basis for belief updating. However, the connection between cognition, semantics, and belief is not direct: because language must flexibly adapt to the context of communication, the connection between lexical representation and interpreted meaning is mediated by pragmatic inference.

We describe the ATILF-LLF system built for the MWE 2017 Shared Task on automatic identification of verbal multiword expressions. We participated in the closed track only, for all the 18 available languages. Our system is a robust greedy transition-based system, in which MWE are identified through a MERGE transition. The system was meant to accommodate the variety of linguistic resources provided for each language, in terms of accompanying morphological and syntactic information. Using per-MWE Fscore, the system was ranked first1 for all but two languages (Hungarian and Romanian).

Language learnabi l i ty has been inves t igated . This refers to the following s i tua t ion: A class of possible languages is specified, t oge the r wi th a method of present ing informat ion to the learner about an unknown language, which is to be chosen from the class. The quest ion is now asked, " I s the informat ion sufficient to determine which of the possible languages is the unknown language?" M a n y definitions of l ea rnab i l i ty are possible, bu t only the following is considered here: Time is quant ized and has a finite s t a r t ing time. At each t ime the learner receives a un i t of in format ion and is to make a guess as to the i den t i t y of the unknown language on the basis of the informat ion received so far. This process continues forever. The class of languages will be considered lea~nabIe with respect to the specified me thod of in format ion p resen ta t ion if there is an a lgor i thm t h a t the learner can use to make his guesses, the a lgor i thm hav ing the following p roper ty : Given any language of the class, there is some finite t ime af te r which the guesses will all be the same and they will be correct . In this pre l iminary invest igat ion, a language is t aken to be a set of s tr ings on some finite a lphabet . The a lphabe t is the same for all languages of tile class. Several var ia t ions of each of the following two basic methods of informat ion presenta t ion are inves t iga ted: A text for a language generates the s t r ings of the language in any order such t h a t every s t r ing of the language occurs a t least once. An informant for a language tells whe the r a s t r ing is in the language, and chooses the s t r ings in some order such t h a t every s t r ing occurs at least once. I t was found t h a t the class of context-sensi t ive languages is learnable from an informant, but that not even the class of regular languages is learnable from a text.

We present a model of visually-grounded language learning based on stacked gated recurrent neural networks which learns to predict visual features given an image description in the form of a sequence of phonemes. The learning task resembles that faced by human language learners who need to discover both structure and meaning from noisy and ambiguous data across modalities. We show that our model indeed learns to predict features of the visual context given phonetically transcribed image descriptions, and show that it represents linguistic information in a hierarchy of levels: lower layers in the stack are comparatively more sensitive to form, whereas higher layers are more sensitive to meaning.

Automatic lexical acquisition has been an active area of research in computational linguistics for over two decades, but the automatic identification of new word-senses has received attention only very recently. Previous work on this topic has been limited by the availability of appropriate evaluation resources. In this paper we present the largest corpus-based dataset of diachronic sense differences to date, which we believe will encourage further work in this area. We then describe several extensions to a state-of-the-art topic modelling approach for identifying new word-senses. This adapted method shows superior performance on our dataset of two different corpus pairs to that of the original method for both: (a) types having taken on a novel sense over time; and (b) the token instances of such novel senses.

This paper relates to the challenge of morphological tagging and lemmatization in morphologically rich languages by example of German and Latin. We focus on the question what a practitioner can expect when using state-of-the-art solutions out of the box. Moreover, we contrast these with old(er) methods and implementations for POS tagging. We examine to what degree recent efforts in tagger development pay out in improved accuracies — and at what cost, in terms of training and processing time. We also conduct in-domain vs. out-domain evaluation. Out-domain evaluations are particularly insightful because the distribution of the data which is being tagged by a user will typically differ from the distribution on which the tagger has been trained. Furthermore, two lemmatization techniques are evaluated. Finally, we compare pipeline tagging vs. a tagging approach that acknowledges dependencies between inflectional categories.

WordNet is perhaps the most important and widely used lexical resource for natural language processing systems up to now. WordNet: An Electronic Lexical Database, edited by Christiane Fellbaum, discusses the design of WordNet from both theoretical and historical perspectives, provides an up-to-date description of the lexical database, and presents a set of applications of WordNet. The book contains a foreword by George Miller, an introduction by Christiane Fellbaum, seven chapters from the Cognitive Sciences Laboratory of Princeton University, where WordNet was produced, and nine chapters contributed by scientists from elsewhere. Miller's foreword offers a fascinating account of the history of WordNet. He discusses the presuppositions of such a lexical database, how the top-level noun categories were determined, and the sources of the words in WordNet. He also writes about the evolution of WordNet from its original incarnation as a dictionary browser to a broad-coverage lexicon, and the involvement of different people during its various stages of development over a decade. It makes very interesting reading for casual and serious users of WordNet and anyone who is grateful for the existence of WordNet. The book is organized in three parts. Part I is about WordNet itself and consists of four chapters: "Nouns in WordNet" by George Miller, "Modifiers in WordNet" by Katherine Miller, "A semantic network of English verbs" by Christiane Fellbaum, and "Design and implementation of the WordNet lexical database and search software" by Randee Tengi. These chapters are essentially updated versions of four papers from Miller (1990). Compared with the earlier papers, the chapters in this book focus more on the underlying assumptions and rationales behind the design decisions. The description of the information contained in WordNet, however, is not as detailed as in Miller (1990). The main new additions in these chapters include an explanation of sense grouping in George Miller's chapter, a section about adverbs in Katherine Miller's chapter, observations about autohyponymy (one sense of a word being a hyponym of another sense of the same word) and autoantonymy (one sense of a word being an antonym of another sense of the same word) in Fellbaum's chapter, and Tengi's description of the Grinder, a program that converts the files the lexicographers work with to searchable lexical databases. The three papers in Part II are characterized as "extensions, enhancements and

We introduce two ways to detect entailment using distributional semantic representations of phrases. Our first experiment shows that the entailment relation between adjective-noun constructions and their head nouns (big cat |= cat), once represented as semantic vector pairs, generalizes to lexical entailment among nouns (dog |= animal). Our second experiment shows that a classifier fed semantic vector pairs can similarly generalize the entailment relation among quantifier phrases (many dogs|=some dogs) to entailment involving unseen quantifiers (all cats|=several cats). Moreover, nominal and quantifier phrase entailment appears to be cued by different distributional correlates, as predicted by the type-based view of entailment in formal semantics.

FrameNet is a three-year NSF-supported project in corpus-based computational lexicography, now in its second year (NSF IRI-9618838, "Tools for Lexicon Building"). The project's key features are (a) a commitment to corpus evidence for semantic and syntactic generalizations, and (b) the representation of the valences of its target words (mostly nouns, adjectives, and verbs) in which the semantic portion makes use of frame semantics. The resulting database will contain (a) descriptions of the semantic frames underlying the meanings of the words described, and (b) the valence representation (semantic and syntactic) of several thousand words and phrases, each accompanied by (c) a representative collection of annotated corpus attestations, which jointly exemplify the observed linkings between "frame elements" and their syntactic realizations (e.g. grammatical function, phrase type, and other syntactic traits). This report will present the project's goals and workflow, and information about the computational tools that have been adapted or created in-house for this work. 1 I n t r o d u c t i o n The Berkeley FrameNet project 1 is producing frame-semantic descriptions of several thousand English lexical items and backing up these descriptions with semantically annotated attestations from contemporary English corpora 2. 1The project is based at the International Computer Science Institute (1947 Center Street, Berkeley, CA). A fuller bibliography may be found in (Lowe et ai., 1997) 2Our main corpus is the British National Corpus. We have access to it through the courtesy of Oxford University Press; the POS-tagged and lemmatized version we use was prepared by the Institut flit Maschinelle Sprachverarbeitung of the University of Stuttgart). The These descriptions are based on hand-tagged semantic annotations of example sentences extracted from large text corpora and systematic analysis of the semantic patterns they exemplify by lexicographers and linguists. The primary emphasis of the project therefore is the encoding, by humans, of semantic knowledge in machine-readable form. The intuition of the lexicographers is guided by and constrained by the results of corpus-based research using highperformance software tools. The semantic domains to be covered are" HEALTH CARE, CHANCE, PERCEPTION, COMMUNICATION, TRANSACTION, TIME, SPACE, BODY (parts and functions of the body), MOTION, LIFE STAGES, SOCIAL CONTEXT, EMOTION and COGNITION. 1.1 Scope of t h e P r o j e c t The results of the project are (a) a lexical resource, called the FrameNet database 3, and (b) associated software tools. The database has three major components (described in more detail below: • Lexicon containing entries which are composed of: (a) some conventional dictionary-type data, mainly for the sake of human readers; (b) FORMULAS which capture the morphosyntactic ways in which elements of the semantic frame can be realized within the phrases or sentences built up around the word; (c) links to semantically ANNOTATED EXAMEuropean collaborators whose participation has made this possible are Sue Atkins, Oxford University Press, and Ulrich Held, IMS-Stuttgart. SThe database will ultimately contain at least 5,000 lexical entries together with a parallel annotated corpus, these in formats suitable for integration into applications which use other lexical resources such as WordNet and COMLEX. The final design of the database will be selected in consultation with colleagues at Princeton (WordNet), ICSI, and IMS, and with other members of the NLP community.

In this paper, we show that local features computed from the derivations of tree substitution grammars — such as the identify of particular fragments, and a count of large and small fragments — are useful in binary grammatical classification tasks. Such features outperform n-gram features and various model scores by a wide margin. Although they fall short of the performance of the hand-crafted feature set of Charniak and Johnson (2005) developed for parse tree reranking, they do so with an order of magnitude fewer features. Furthermore, since the TSGs employed are learned in a Bayesian setting, the use of their derivations can be viewed as the automatic discovery of tree patterns useful for classification. On the BLLIP dataset, we achieve an accuracy of 89.9% in discriminating between grammatical text and samples from an n-gram language model.


Question: Evolutionary alterations in the connection between grammar and the lexicon during development.

In the initial stages of language acquisition, infants face the formidable challenge of identifying individual words within a largely uninterrupted flow of speech. This critical learning task may be facilitated by leveraging distributional cues to pinpoint word boundaries. For instance, infants might compute the transitional probabilities between the sounds in their linguistic environment, utilizing the varying strengths of these probabilities to conjecture where words begin and end. This process could be further refined by the presence of language-specific prosodic signals that are often correlated with word boundaries. To empirically evaluate these propositions, we conducted an experiment where adults were briefly exposed to an artificial language. In this language, the only available cues for word segmentation were the transitional probabilities between syllables. Our findings revealed that the subjects were indeed capable of learning the words in this constructed language. Moreover, the introduction of specific prosodic cues significantly improved their performance. These outcomes provide compelling evidence that distributional cues could be a crucial factor in the early word segmentation abilities of language learners, potentially shaping the foundational relationship between grammar and the lexicon.

A central problem in grounded language acquisition is learning the correspondences between a rich world state and a stream of text which references that world state. To deal with the high degree of ambiguity present in this setting, we present a generative model that simultaneously segments the text into utterances and maps each utterance to a meaning representation grounded in the world state. We show that our model generalizes across three domains of increasing difficulty—Robocup sportscasting, weather forecasts (a new domain), and NFL recaps.

This article brief ly reviews four computational models of reading and the types of empirical findings that they are designed to explain: (1) theDual-Route model of word identification (Coltheart et al., 2001); (2) the SimpleRecurrent Network model of sentence processing (Elman, 1990); (3) the Construction–Integration model of discourse representation (Kintsch, 1988); and (4) the E-Z Reader model of eye-movement control in reading (Reichle et al., 2012). These particular models are reviewed because they provide comprehensive accounts of a large number of empirical findings in each of their respective domains, and because they have advanced our understanding of the cognitive processes involved in reading by motivating new empirical studies. Future models of reading will build upon the success of their predecessors by integrating models from two or more of the aforementioned domains, thereby providing a means to examine the compatibility of their theoretical assumptions and more comprehensive accounts of the cognitive processes involved in reading. The perceptual, cognitive, and motor processes that support skilled reading are numerous, extremely complex, and work together in a highly coordinated manner. For those reasons, any deep understanding of what happens in the mind during reading requires the use of computational models that describe those processes and how they jointly determine both the on-line behavior that is observed during reading (e.g., the patterns of eye movements that readers make) and the products of that behavior (e.g., the mental representations that readers generate to understand the text). As will be discussed in this article, such models provide theoretical frameworks for understanding the mental processes involved in reading, but also advance our understanding by generating novel predictions that can be tested with experiments. In the remainder of this article, I will describe four existing models that have already played important roles in furthering our understanding of the psychology of reading. These models were selected for inclusion in this article because they account for a wide range of empirical phenomena within a broad area of reading research, and because they have motivated a large number of new empirical studies. The models were also selected because they collectively span the range of topics that have traditionally been studied by reading researchers: (1) word identification; (2) sentence processing; (3) the representation of discourse; and (4) how these different processes interact with other cognitive processes (e.g., attention) to determine the patterns of eye movements that are observed during reading. Because one goal of this article is to introduce the models in a way that is accessible to a broad audience, I will describe the models using schematic diagrams rather than their equations; for a more formal description of the models, see either the original articles or Reichle’s review of the models. Word-Identification Models Word-identification models describe how the visual features corresponding to a word on a printed page are used to access that word’s pronunciation and meaning, as well as behavioral deficits that result from impairment of these processes due to developmental delay or brain © 2015 The Author Language and Linguistics Compass © 2015 John Wiley & Sons Ltd

We describe a series o,f five statistical models o,f the translation process and give algorithms,for estimating the parameters o,f these models given a set o,f pairs o,f sentences that are translations o,f one another. We define a concept o,f word-by-word alignment between such pairs o,f sentences. For any given pair of such sentences each o,f our models assigns a probability to each of the possible word-by-word alignments. We give an algorithm for seeking the most probable o,f these alignments. Although the algorithm is suboptimal, the alignment thus obtained accounts well for the word-by-word relationships in the pair o,f sentences. We have a great deal o,f data in French and English from the proceedings o,f the Canadian Parliament. Accordingly, we have restricted our work to these two languages; but we,feel that because our algorithms have minimal linguistic content they would work well on other pairs o,f languages. We also ,feel, again because of the minimal linguistic content o,f our algorithms, that it is reasonable to argue that word-by-word alignments are inherent in any sufficiently large bilingual corpus.

Grammatical error correction (GEC) is the task of automatically correcting grammatical errors in written text. Previous research has mainly focussed on individual error types and current commercial proofreading tools only target limited error types. As sentences produced by learners may contain multiple errors of different types, a practical error correction system should be able to detect and correct all errors. In this thesis, we investigate GEC for learners of English as a Second Language (ESL). Specifically, we treat GEC as a translation task from incorrect into correct English, explore new models for developing end-to-end GEC systems for all error types, study system performance for each error type, and examine model generali-sation to different corpora. First, we apply Statistical Machine Translation (SMT) to GEC and prove that it can form the basis of a competitive all-errors GEC system. We implement an SMT-based GEC system which contributes to our winning system submitted to a shared task in 2014. Next, we propose a ranking model to re-rank correction candidates generated by an SMT-based GEC system. This model introduces new linguistic information and we show that it improves correction quality. Finally, we present the first study using Neural Machine Translation (NMT) for GEC. We demonstrate that NMT can be successfully applied to GEC and help capture new errors missed by an SMT-based GEC system. While we focus on GEC for English, our methods presented in this thesis can be easily applied to any language. Acknowledgements First and foremost, I owe a huge debt of gratitude to my supervisor, Ted Briscoe, who has patiently guided me through my PhD and always been very helpful, understanding and supportive. I cannot thank him enough for providing me with opportunities that helped me grow as a researcher and a critical thinker. I am immensely grateful to my examiners, Paula Buttery and Stephen Pulman, for their thorough reading of my thesis, their valuable comments and an enjoyable viva. My appreciation extends to my fellow members of the Natural Language and Information Processing research group, with whom I have always enjoyed discussing our work and other random things. My gratitude goes to Stephen Clark and Ann Copestake for giving me early feedback on my work as well as Christopher Bryant for generously reading my thesis draft. I would especially like to thank Mariano Felice for being not just a great colleague but also a dear friend. A special mention has …

The words people use in their daily lives can reveal important aspects of their social and psychological worlds. With advances in computer technology, text analysis allows researchers to reliably and quickly assess features of what people say as well as subtleties in their linguistic styles. Following a brief review of several text analysis programs, we summarize some of the evidence that links natural word use to personality, social and situational fluctuations, and psychological interventions. Of particular interest are findings that point to the psychological value of studying particles-parts of speech that include pronouns, articles, prepositions, conjunctives, and auxiliary verbs. Particles, which serve as the glue that holds nouns and regular verbs together, can serve as markers of emotional state, social identity, and cognitive styles.

This work uses parallel monolingual corpora for a detailed study of the task of sub-sentential paraphrase acquisition. We argue that the scarcity of this type of resource is compensated by the fact that it is the most suited type for studies on paraphrasing. We propose a large exploration of this task with experiments on two languages with five different acquisition techniques, selected for their complementarity, their combinations, as well as four monolingual corpus types of varying comparability. We report, under all conditions, a significant improvement over all techniques by validating candidate paraphrases using a maximum entropy classifier. An important result of our study is the identification of difficult-to-acquire paraphrase pairs, which are classified and quantified in a bilingual typology.

This paper summarizes our ongoing project on developing an architecture for a robot that can acquire new words and their meanings while engaging in multidomain dialogues. These two functions are crucial in making conversational service robots work in real tasks in the real world. Household robots and office robots need to be able to work in multiple task domains and they also need to engage in dialogues in multiple domains corresponding to those task domains. Lexical acquisition is necessary because speech understanding cannot be done without enough knowledge on words that are possibly spoken in the task domain. Our architecture is based on a multi-expert model in which multiple domain experts are employed and one of them is selected based on the user utterance and the situation to engage in the control of the dialogue and physical behaviors. We incorporate experts that have an ability to acquire new lexical entries and their meanings grounded on the physical world through spoken interactions. By appropriately selecting those experts, lexical acquisition in multi-domain dialogues becomes possible. An example robotic system based on this architecture that can acquire object names and location names demonstrates the viability of the architecture.

Syntactic simplification is the process of reducing the grammatical complexity of a text, while retaining its information content and meaning. The aim of syntactic simplification is to make text easier to comprehend for human readers, or process by programs. In this thesis, I describe how syntactic simplification can be achieved using shallow robust analysis, a small set of hand-crafted simplification rules and a detailed analysis of the discourse-level aspects of syntactically rewriting text. I offer a treatment of relative clauses, apposition, coordination and subordination. I present novel techniques for relative clause and appositive attachment. I argue that these attachment decisions are not purely syntactic. My approaches rely on a shallow discourse model and on animacy information obtained from a lexical knowledge base. I also show how clause and appositive boundaries can be determined reliably using a decision procedure based on local context, represented by part-of-speech tags and noun chunks. I then formalise the interactions that take place between syntax and discourse during the simplification process. This is important because the usefulness of syntactic simplification in making a text accessible to a wider audience can be undermined if the rewritten text lacks cohesion. I describe how various generation issues like sentence ordering, cue-word selection, referring-expression generation, determiner choice and pronominal use can be resolved so as to preserve conjunctive and anaphoric cohesive-relations during syntactic simplification. In order to perform syntactic simplification, I have had to address various natural language processing problems, including clause and appositive identification and attachment, pronoun resolution and referring-expression generation. I evaluate my approaches to solving each problem individually, and also present a holistic evaluation of my syntactic simplification system.

Recently, many NLP tasks have benefited from distributed word representation. However, it remains unknown whether embedding models are really immune to the typological diversity of languages, despite the language-independent architecture. Here we investigate three representative models on a large set of language samples by mapping dense embedding to sparse linguistic property space. Experiment results reveal the language universal and specific properties encoded in various word representation. Additionally, strong evidence supports the utility of word form, especially for inflectional languages.

Iterated learning describes the process whereby an individual learns their behaviour by exposure to another individual's behaviour, who themselves learnt it in the same way. It can be seen as a key mechanism of cultural evolution. We review various methods for understanding how behaviour is shaped by the iterated learning process: computational agent-based simulations; mathematical modelling; and laboratory experiments in humans and non-human animals. We show how this framework has been used to explain the origins of structure in language, and argue that cultural evolution must be considered alongside biological evolution in explanations of language origins.


Question: Evolutionary shifts in the connection between grammar and the lexicon during development.

This study investigates the capacity of 8-month-old infants to learn from statistical cues in speech, highlighting the role of experience-dependent mechanisms in language acquisition. Traditionally, language learning theories have focused on the significance of experience-independent mechanisms. However, our findings suggest that infants can segment words from continuous speech using solely statistical information about the adjacency of speech sounds. Remarkably, this ability to segment words is achieved after only two minutes of exposure to speech, indicating that infants possess a potent mechanism for computing the statistical characteristics of their linguistic input. This research underscores the importance of statistical learning in early language development, challenging prevailing views on the primacy of experience-independent mechanisms.

This paper outlines current cognitive perspectives on second language acquisition (SLA). The Associative-Cognitive CREED holds that SLA is governed by the same principles of associative and cognitive learning that underpin the rest of human knowledge. The major principles of the framework are that SLA is Construction-based, Rational, Exemplar-driven, Emergent, and Dialectic. Language learning involves the acquisition of constructions that map linguistic form and function. Competence and performance both emerge from the dynamic system that is the frequency-tuned conspiracy of memorized exemplars of use of these constructions, with competence being the integrated sum of prior usage and performance being its dynamic contextualized activation. The system is rational in that it optimally reflects prior first language (L1) usage. The L1 tunes the ways in which learners attend to language. Learned-attention transfers to L2 and it is this L1 entrenchment that limits the endstate of usage-based SLA. But these limitations can be overcome by recruiting learner consciousness, putting them into a dialectic tension between the conflicting forces of their current stable states of interlanguage and the evidence of explicit form-focused feedback, either linguistic, pragmatic, or metalinguistic, that allows socially scaffolded development. The paper directs the reader to recent review articles in these key areas and weighs the implications of this framework.

We present a novel methodology involving mappings between different modes of semantic representations. We propose distributional semantic models as a mechanism for representing the kind of world knowledge inherent in the system of abstract symbols characteristic of a sophisticated community of language users. Then, motivated by insight from ecological psychology, we describe a model approximating affordances, by which we mean a language learner’s direct perception of opportunities for action in an environment. We present a preliminary experiment involving mapping between these two representational modalities, and propose that our methodology can become the basis for a cognitively inspired model of grounded language learning.

Statistical language models estimate the distribution of various natural language phenomena for the purpose of speech recognition and other language technologies. Since the first significant model was proposed in 1980, many attempts have been made to improve the state of the art. We review them, point to a few promising directions, and argue for a Bayesian approach to integration of linguistic theories with data.

Neural probabilistic language models (NPLMs) have been shown to be competitive with and occasionally superior to the widely-used n-gram language models. The main drawback of NPLMs is their extremely long training and testing times. Morin and Bengio have proposed a hierarchical language model built around a binary tree of words, which was two orders of magnitude faster than the nonhierarchical model it was based on. However, it performed considerably worse than its non-hierarchical counterpart in spite of using a word tree created using expert knowledge. We introduce a fast hierarchical language model along with a simple feature-based algorithm for automatic construction of word trees from the data. We then show that the resulting models can outperform non-hierarchical neural models as well as the best n-gram models.

The question of whether language affects our categorization of perceptual continua is of particular interest for the domain of color where constraints on categorization have been proposed both within the visual system and in the visual environment. Recent research (Roberson, Davies, & Davidoff, 2000; Roberson et al., in press) found substantial evidence of cognitive color differences between different language communities, but concerns remained as to how representative might be a tiny, extremely remote community. The present study replicates and extends previous findings using additional paradigms among a larger community in a different visual environment. Adult semi-nomadic tribesmen in Southern Africa carried out similarity judgments, short-term memory and long-term learning tasks. They showed different cognitive organization of color to both English and another language with the five color terms. Moreover, Categorical Perception effects were found to differ even between languages with broadly similar color categories. The results provide further evidence of the tight relationship between language and cognition.

We describe the ATILF-LLF system built for the MWE 2017 Shared Task on automatic identification of verbal multiword expressions. We participated in the closed track only, for all the 18 available languages. Our system is a robust greedy transition-based system, in which MWE are identified through a MERGE transition. The system was meant to accommodate the variety of linguistic resources provided for each language, in terms of accompanying morphological and syntactic information. Using per-MWE Fscore, the system was ranked first1 for all but two languages (Hungarian and Romanian).

We introduce two ways to detect entailment using distributional semantic representations of phrases. Our first experiment shows that the entailment relation between adjective-noun constructions and their head nouns (big cat |= cat), once represented as semantic vector pairs, generalizes to lexical entailment among nouns (dog |= animal). Our second experiment shows that a classifier fed semantic vector pairs can similarly generalize the entailment relation among quantifier phrases (many dogs|=some dogs) to entailment involving unseen quantifiers (all cats|=several cats). Moreover, nominal and quantifier phrase entailment appears to be cued by different distributional correlates, as predicted by the type-based view of entailment in formal semantics.

Automatic lexical acquisition has been an active area of research in computational linguistics for over two decades, but the automatic identification of new word-senses has received attention only very recently. Previous work on this topic has been limited by the availability of appropriate evaluation resources. In this paper we present the largest corpus-based dataset of diachronic sense differences to date, which we believe will encourage further work in this area. We then describe several extensions to a state-of-the-art topic modelling approach for identifying new word-senses. This adapted method shows superior performance on our dataset of two different corpus pairs to that of the original method for both: (a) types having taken on a novel sense over time; and (b) the token instances of such novel senses.

Recent studies in the literature have shown that syntax remains a significant barrier to novice computer science students in the field. While this syntax barrier is known to exist, whether and how it varies across programming languages has not been carefully investigated. For this article, we conducted four empirical studies on programming language syntax as part of a larger analysis into the, so called, programming language wars. We first present two surveys conducted with students on the intuitiveness of syntax, which we used to garner formative clues on what words and symbols might be easy for novices to understand. We followed up with two studies on the accuracy rates of novices using a total of six programming languages: Ruby, Java, Perl, Python, Randomo, and Quorum. Randomo was designed by randomly choosing some keywords from the ASCII table (a metaphorical placebo). To our surprise, we found that languages using a more traditional C-style syntax (both Perl and Java) did not afford accuracy rates significantly higher than a language with randomly generated keywords, but that languages which deviate (Quorum, Python, and Ruby) did. These results, including the specifics of syntax that are particularly problematic for novices, may help teachers of introductory programming courses in choosing appropriate first languages and in helping students to overcome the challenges they face with syntax.

Co-speech gestures embody a form of manual action that is tightly coupled to the language system. As such, the co-occurrence of speech and co-speech gestures is an excellent example of the interplay between language and action. There are, however, other ways in which language and action can be thought of as closely related. In this paper we will give an overview of studies in cognitive neuroscience that examine the neural underpinnings of links between language and action. Topics include neurocognitive studies of motor representations of speech sounds, action-related language, sign language and co-speech gestures. It will be concluded that there is strong evidence on the interaction between speech and gestures in the brain. This interaction however shares general properties with other domains in which there is interplay between language and action.