Question: Natural policy gradient techniques utilizing parameter-based exploration for control tasks.

This paper introduces a simulation-based optimization technique for enhancing the average reward in a Markov Reward Process (MRP) that is influenced by a set of parameters. The method is particularly applicable to Markov Decision Processes (MDPs) where optimization occurs within a parameterized set of policies. Unlike conventional methods, this algorithm relies on the simulation of a single sample path and can be executed in an online manner. A convergence result with a probability of one is provided, supported by contracts with Siemens AG, Munich, Germany, and Alcatel Bell, Belgium, as well as by contract DMI with the National Science Foundation.

The paper begins by addressing the challenges posed by large state spaces and the absence of an exact model, common issues in many practical situations. To overcome these difficulties, the paper explores two main ideas: the use of simulation for estimating quantities of interest without model-based computations, and the application of parametric representations to tackle the curse of dimensionality. Parametric representations and associated algorithms can be broadly categorized into three main areas:

1. Parametrized value functions: Instead of associating a value with each state, a parametric form is used, where r is a vector of tunable parameters (weights), and V is an approximation architecture. This could be a multilayer perceptron or other representations such as polynomials, linear combinations of feature vectors, or state aggregation.

2. Parametrized policies: An alternative approach considered in this paper is to bypass the tuning of a parametrized value function. Instead, a class of policies described by a parameter vector is considered. Simulation is utilized to estimate the gradient of the performance metric with respect to the parameters, and the policy is improved by updating the parameters in a gradient direction.

3. Actor-critic methods: This approach combines parametrizations of the policy (actor) and the value function (critic).

The focus of this paper is on methods based on policy parametrization and approximate gradient improvement. The paper starts with a formula for the gradient of the performance metric, which has been presented in different forms and for various contexts. It then suggests a method for estimating the terms in that formula, leading to a simulation-based method that updates the parameter vector at every regeneration time in an approximate gradient direction. Furthermore, it shows how to construct an online method that updates the parameter vector at each time step. The resulting method has conceptual similarities with those described in previous works but operates under different assumptions and with different convergence results.

The gradient estimator can also be derived or interpreted in terms of likelihood

Reinforcement learning algorithms are a powerful machine learning technique. However, much of the work on these algorithms has been developed with regard to discrete nite-state Markovian problems, which is too restrictive for many real-world environments. Therefore, it is desirable to extend these methods to high dimensional continuous state-spaces, which requires the use of function approximation to generalise the information learnt by the system. In this report, the use of back-propagation neural networks (Rumelhart, Hinton and Williams 1986) is considered in this context. We consider a number of di erent algorithms based around Q-Learning (Watkins 1989) combined with the Temporal Di erence algorithm (Sutton 1988), including a new algorithm (Modi ed Connectionist Q-Learning), and Q( ) (Peng and Williams 1994). In addition, we present algorithms for applying these updates on-line during trials, unlike backward replay used by Lin (1993) that requires waiting until the end of each trial before updating can occur. On-line updating is found to be more robust to the choice of training parameters than backward replay, and also enables the algorithms to be used in continuously operating systems where no end of trial conditions occur. We compare the performance of these algorithms on a realistic robot navigation problem, where a simulated mobile robot is trained to guide itself to a goal position in the presence of obstacles. The robot must rely on limited sensory feedback from its surroundings, and make decisions that can be generalised to arbitrary layouts of obstacles. These simulations show that on-line learning algorithms are less sensitive to the choice of training parameters than backward replay, and that the alternative update rules of MCQ-L and Q( ) are more robust than standard Q-learning updates. 1

Recent research has shown the benefit of framing problems of imitation learning as solutions to Markov Decision Problems. This approach reduces learning to the problem of recovering a utility function that makes the behavior induced by a near-optimal policy closely mimic demonstrated behavior. In this work, we develop a probabilistic approach based on the principle of maximum entropy. Our approach provides a well-defined, globally normalized distribution over decision sequences, while providing the same performance guarantees as existing methods. We develop our technique in the context of modeling realworld navigation and driving behaviors where collected data is inherently noisy and imperfect. Our probabilistic approach enables modeling of route preferences as well as a powerful new approach to inferring destinations and routes based on partial trajectories.

Can one parallelize complex exploration– exploitation tradeoffs? As an example, consider the problem of optimal highthroughput experimental design, where we wish to sequentially design batches of experiments in order to simultaneously learn a surrogate function mapping stimulus to response and identify the maximum of the function. We formalize the task as a multiarmed bandit problem, where the unknown payoff function is sampled from a Gaussian process (GP), and instead of a single arm, in each round we pull a batch of several arms in parallel. We develop GP-BUCB, a principled algorithm for choosing batches, based on the GP-UCB algorithm for sequential GP optimization. We prove a surprising result; as compared to the sequential approach, the cumulative regret of the parallel algorithm only increases by a constant factor independent of the batch size B. Our results provide rigorous theoretical support for exploiting parallelism in Bayesian global optimization. We demonstrate the effectiveness of our approach on two real-world applications.

The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. Training is recast as a control problem and this allows us to formulate necessary optimality conditions in continuous time using the Pontryagin’s maximum principle (PMP). A modification of the method of successive approximations is then used to solve the PMP, giving rise to an alternative training algorithm for deep learning. This approach has the advantage that rigorous error estimates and convergence results can be established. We also show that it may avoid some pitfalls of gradient-based methods, such as slow convergence on flat landscapes near saddle points. Furthermore, we demonstrate that it obtains favorable initial convergence rate periteration, provided Hamiltonian maximization can be efficiently carried out a step which is still in need of improvement. Overall, the approach opens up new avenues to attack problems associated with deep learning, such as trapping in slow manifolds and inapplicability of gradient-based methods for discrete trainable variables.

Motion and path-planning algorithms often use complex cost functions for both global navigation and local smoothing of trajectories. Obtaining good results typically requires carefully hand-engineering the trade-offs between different terms in the cost function. In practice, it is often much easier to demonstrate a few good trajectories. In this paper, we describe an efficient algorithm which - when given access to a few trajectory demonstrations - can automatically infer good trade-offs between the different costs. In our experiments, we apply our algorithm to the problem of navigating a robotic car in a parking lot.

Designing gaits and corresponding control policies is a key challenge in robot locomotion. Even with a viable controller parametrization, finding near-optimal parameters can be daunting. Typically, this kind of parameter optimization requires specific expert knowledge and extensive robot experiments. Automatic black-box gait optimization methods greatly reduce the need for human expertise and time-consuming design processes. Many different approaches for automatic gait optimization have been suggested to date. However, no extensive comparison among them has yet been performed. In this article, we thoroughly discuss multiple automatic optimization methods in the context of gait optimization. We extensively evaluate Bayesian optimization, a model-based approach to black-box optimization under uncertainty, on both simulated problems and real robots. This evaluation demonstrates that Bayesian optimization is particularly suited for robotic applications, where it is crucial to find a good set of gait parameters in a small number of experiments.

Most learning algorithms are not invariant to the scale of the function that is being approximated. We propose to adaptively normalize the targets used in learning. This is useful in value-based reinforcement learning, where the magnitude of appropriate value approximations can change over time when we update the policy of behavior. Our main motivation is prior work on learning to play Atari games, where the rewards were all clipped to a predetermined range. This clipping facilitates learning across many different games with a single learning algorithm, but a clipped reward function can result in qualitatively different behavior. Using the adaptive normalization we can remove this domain-specific heuristic without diminishing overall performance.

Reinforcement Learning (RL) algorithms allow artificial agents to improve their selection of actions so as to increase rewarding experiences in their environments. The learning can become intractably slow as the state space of the environment grows. This has motivated methods that learn internal representations of the agents state by a function approximator. Impressive results have been produced by using deep artificial neural networks to learn the state representations. However, deep reinforcement learning algorithms require solving a non-convex and nonlinear unconstrained optimization problem. Methods for solving the optimization problems in deep RL are restricted to the class of first-order algorithms, like stochastic gradient descent (SGD). The major drawback of the SGD methods is that they have the undesirable effect of not escaping saddle points. Furthermore, these methods require exhaustive trial and error to fine-tune many learning parameters. Using second derivative information can result in improved convergence properties, but computing the Hessian matrix for large-scale problems is not practical. QuasiNewton methods, like SGD, require only first-order gradient information, but they can result in superlinear convergence, which makes them attractive alternatives. The limitedmemory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) approach is one of the most popular quasi-Newton methods that construct positive definite Hessian approximations. In this paper, we introduce an efficient optimization method, based on the limited memory BFGS quasi-Newton method using line search strategy – as an alternative to SGD methods. Our method bridges the disparity between first order methods and second order methods by continuing to use gradient information to calculate a low-rank Hessian approximations. We provide empirical results on variety of the classic ATARI 2600 games. Our results show a robust convergence with preferred generalization characteristics, as well as fast training time and no need for the experience replaying mechanism.

The reinforcement learning community has made great strides in designing algorithms capable of exceeding human performance on specific tasks. These algorithms are mostly trained one task at the time, each new task requiring to train a brand new agent instance. This means the learning algorithm is general, but each solution is not; each agent can only solve the one task it was trained on. In this work, we study the problem of learning to master not one but multiple sequentialdecision tasks at once. A general issue in multi-task learning is that a balance must be found between the needs of multiple tasks competing for the limited resources of a single learning system. Many learning algorithms can get distracted by certain tasks in the set of tasks to solve. Such tasks appear more salient to the learning process, for instance because of the density or magnitude of the in-task rewards. This causes the algorithm to focus on those salient tasks at the expense of generality. We propose to automatically adapt the contribution of each task to the agent’s updates, so that all tasks have a similar impact on the learning dynamics. This resulted in state of the art performance on learning to play all games in a set of 57 diverse Atari games. Excitingly, our method learned a single trained policy with a single set of weights that exceeds median human performance. To our knowledge, this was the first time a single agent surpassed human-level performance on this multi-task domain. The same approach also demonstrated state of the art performance on a set of 30 tasks in the 3D reinforcement learning platform DeepMind Lab.

We consider learning in a Markov decision process where we are not explicitly given a reward function, but where instead we can observe an expert demonstrating the task that we want to learn to perform. This setting is useful in applications (such as the task of driving) where it may be difficult to write down an explicit reward function specifying exactly how different desiderata should be traded off. We think of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and give an algorithm for learning the task demonstrated by the expert. Our algorithm is based on using "inverse reinforcement learning" to try to recover the unknown reward function. We show that our algorithm terminates in a small number of iterations, and that even though we may never recover the expert's reward function, the policy output by the algorithm will attain performance close to that of the expert, where here performance is measured with respect to the expert's unknown reward function.


Question: Natural policy gradient techniques utilizing parameter-based exploration for control tasks.

This paper explores the application of the Expectation-Maximization (EM) procedure to reinforcement learning, specifically focusing on Hinton's Relative Payoff Procedure (RPP). Unlike traditional reinforcement learning algorithms that rely on stochastic gradient ascent, the RPP is a static method that can significantly alter parameter values while ensuring an increase in mean return. The theoretical underpinning of this claim is grounded in a correspondence between the RPP and a variant of the EM algorithm. Under certain conditions, it is proven that the RPP can enhance the mean return, despite potentially substantial modifications to the parameters. This connection between the RPP and EM not only enriches our understanding of reinforcement learning algorithms but also opens up new avenues for optimizing control tasks through natural policy gradient methods with parameter-based exploration. By leveraging the insights from EM, reinforcement learning algorithms can be refined to better navigate complex control problems, offering a promising direction for enhancing the efficiency and effectiveness of policy optimization in dynamic environments.

We study the process of multi-agent reinforcement learning in the context of load balancing in a distributed system, without use of either central coordination or explicit communication. We rst de ne a precise framework in which to study adaptive load balancing, important features of which are its stochastic nature and the purely local information available to individual agents. Given this framework, we show illuminating results on the interplay between basic adaptive behavior parameters and their e ect on system e ciency. We then investigate the properties of adaptive load balancing in heterogeneous populations, and address the issue of exploration vs. exploitation in that context. Finally, we show that naive use of communication may not improve, and might even harm system e ciency.

Stochastic gradient optimization is a class of widely used algorithms for training machine learning models. To optimize an objective, it uses the noisy gradient computed from the random data samples instead of the true gradient computed from the entire dataset. However, when the variance of the noisy gradient is large, the algorithm might spend much time bouncing around, leading to slower convergence and worse performance. In this paper, we develop a general approach of using control variate for variance reduction in stochastic gradient. Data statistics such as low-order moments (pre-computed or estimated online) is used to form the control variate. We demonstrate how to construct the control variate for two practical problems using stochastic gradient optimization. One is convex—the MAP estimation for logistic regression, and the other is non-convex—stochastic variational inference for latent Dirichlet allocation. On both problems, our approach shows faster convergence and better performance than the classical approach.

We discsus Hinton’s (1989) relative payoff procedure (RPP), a static reinforcement learning algorithm whose foundation is not stochastic gradient ascent. We show circumstances under which applying the RPP is guaranteed to increase the mean return, even though it can make large changes in the values of the parameters. The proof is based on a mapping between the RPP and a form of the expectation-maximisation procedure of Dempster, Laird & Rubin (1976).

We present a method for designing a robust control policy for an uncertain system subject to temporal logic specifications. The system is modeled as a finite Markov Decision Process (MDP) whose transition probabilities are not exactly known but are known to belong to a given uncertainty set. A robust control policy is generated for the MDP that maximizes the worst-case probability of satisfying the specification over all transition probabilities in this uncertainty set. To this end, we use a procedure from probabilistic model checking to combine the system model with an automaton representing the specification. This new MDP is then transformed into an equivalent form that satisfies assumptions for stochastic shortest path dynamic programming. A robust version of dynamic programming solves for a ε-suboptimal robust control policy with time complexity O(log1/ε) times that for the non-robust case.

Intelligent control of robotic arms has huge potential over the coming years, but as of now will often fail to adapt when presented with new and unfamiliar environments. Recent trends to solve this problem have seen a shift to endto-end solutions using deep reinforcement learning to learn policies from visual input, rather than relying on a handcrafted, modular pipeline. Building upon the recent success of deep Q-networks, we present an approach which uses threedimensional simulations to train a 7-DOF robotic arm in a robot arm control task without any prior knowledge. Policies accept images of the environment as input and output motor actions. However, the high-dimensionality of the policies as well as the large state space makes policy search difficult. This is overcome by ensuring interesting states are explored via intermediate rewards that guide the policy towards higher reward states. Our results demonstrate that deep Q-networks can be used to learn policies for a task that involves locating a cube, grasping, and then finally lifting. The agent is able to learn to deal with a range of starting joint configurations and starting cube positions when tested in simulation. Moreover, we show that policies trained via simulation have the potential to be directly applied to real-world equivalents without any further training. We believe that robot simulations can decrease the dependency on physical robots and ultimately improve productivity of training robot control tasks.

Stochastic computation graphs (SCGs) provide a formalism to represent structured optimization problems arising in artificial intelligence, including supervised, unsupervised, and reinforcement learning. Previous work has shown that an unbiased estimator of the gradient of the expected loss of SCGs can be derived from a single principle. However, this estimator often has high variance and requires a full model evaluation per data point, making this algorithm costly in large graphs. In this work, we address these problems by generalizing concepts from the reinforcement learning literature. We introduce the concepts of value functions, baselines and critics for arbitrary SCGs, and show how to use them to derive lower-variance gradient estimates from partial model evaluations, paving the way towards general and efficient credit assignment for gradient-based optimization. In doing so, we demonstrate how our results unify recent advances in the probabilistic inference and reinforcement learning literature.

Techniques for both exploratory and known item search tend to direct only to more specific subtopics or individual documents, as opposed to allowing directing the exploration of the information space. We present an interactive information retrieval system that combines Reinforcement Learning techniques along with a novel user interface design to allow active engagement of users in directing the search. Users can directly manipulate document features (keywords) to indicate their interests and Reinforcement Learning is used to model the user by allowing the system to trade off between exploration and exploitation. This gives users the opportunity to more effectively direct their search nearer, further and following a direction. A task-based user study conducted with 20 participants comparing our system to a traditional query-based baseline indicates that our system significantly improves the effectiveness of information retrieval by providing access to more relevant and novel information without having to spend more time acquiring the information.

Adversarial examples have been shown to exist for a variety of deep learning architectures. Deep reinforcement learning has shown promising results on training agent policies directly on raw inputs such as image pixels. In this paper we present a novel study into adversarial attacks on deep reinforcement learning polices. We compare the effectiveness of the attacks using adversarial examples vs. random noise. We present a novel method for reducing the number of times adversarial examples need to be injected for a successful attack, based on the value function. We further explore how re-training on random noise and FGSM perturbations affects the resilience against adversarial examples.

A critical component to enabling intelligent reasoning in partially observable environments is memory. Despite this importance, Deep Reinforcement Learning (DRL) agents have so far used relatively simple memory architectures, with the main methods to overcome partial observability being either a temporal convolution over the past k frames or an LSTM layer. More recent work (Oh et al., 2016) has went beyond these architectures by using memory networks which can allow more sophisticated addressing schemes over the past k frames. But even these architectures are unsatisfactory due to the reason that they are limited to only remembering information from the last k frames. In this paper, we develop a memory system with an adaptable write operator that is customized to the sorts of 3D environments that DRL agents typically interact with. This architecture, called the Neural Map, uses a spatially structured 2D memory image to learn to store arbitrary information about the environment over long time lags. We demonstrate empirically that the Neural Map surpasses previous DRL memories on a set of challenging 2D and 3D maze environments and show that it is capable of generalizing to environments that were not seen during training.

Reinforcement Learning (RL) algorithms allow artificial agents to improve their selection of actions so as to increase rewarding experiences in their environments. The learning can become intractably slow as the state space of the environment grows. This has motivated methods that learn internal representations of the agents state by a function approximator. Impressive results have been produced by using deep artificial neural networks to learn the state representations. However, deep reinforcement learning algorithms require solving a non-convex and nonlinear unconstrained optimization problem. Methods for solving the optimization problems in deep RL are restricted to the class of first-order algorithms, like stochastic gradient descent (SGD). The major drawback of the SGD methods is that they have the undesirable effect of not escaping saddle points. Furthermore, these methods require exhaustive trial and error to fine-tune many learning parameters. Using second derivative information can result in improved convergence properties, but computing the Hessian matrix for large-scale problems is not practical. QuasiNewton methods, like SGD, require only first-order gradient information, but they can result in superlinear convergence, which makes them attractive alternatives. The limitedmemory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) approach is one of the most popular quasi-Newton methods that construct positive definite Hessian approximations. In this paper, we introduce an efficient optimization method, based on the limited memory BFGS quasi-Newton method using line search strategy – as an alternative to SGD methods. Our method bridges the disparity between first order methods and second order methods by continuing to use gradient information to calculate a low-rank Hessian approximations. We provide empirical results on variety of the classic ATARI 2600 games. Our results show a robust convergence with preferred generalization characteristics, as well as fast training time and no need for the experience replaying mechanism.


Question: Automated ranking of profane language through the utilization of word embeddings and pseudo-relevance feedback methods.

Introducing the Natural Language Toolkit (NLTK): A Comprehensive Resource for Computational Linguistics

NLTK, the Natural Language Toolkit, is an open-source software suite designed for teaching and research in the field of computational linguistics. It comprises a collection of modules, tutorials, and exercises that serve as a valuable resource for both educators and students. The toolkit offers a wide range of functionalities, encompassing both symbolic and statistical approaches to natural language processing, and provides seamless access to annotated corpora for data analysis and model training.

One of the key features of NLTK is its hands-on approach to learning. Students are encouraged to engage with the material by augmenting and replacing existing components, thereby gaining practical experience in structured programming. This interactive learning method enables students to manipulate complex models from the very beginning, fostering a deeper understanding of computational linguistics concepts and techniques.

Moreover, NLTK's modular structure allows for flexibility in course design, making it suitable for various educational settings and levels of expertise. Whether used as a standalone resource or integrated into existing computational linguistics courses, NLTK provides a solid foundation for students to explore the intricacies of natural language processing and to develop their own computational models and algorithms.

In summary, NLTK is a versatile and comprehensive toolkit that serves as an indispensable resource for anyone interested in the field of computational linguistics. Its open-source nature, combined with its rich set of features and hands-on learning opportunities, makes it an ideal platform for teaching and research in this rapidly evolving field.

As research on hate speech becomes more and more relevant every day, most of it is still focused on hate speech detection. By attempting to replicate a hate speech detection experiment performed on an existing Twitter corpus annotated for hate speech, we highlight some issues that arise from doing research in the field of hate speech, which is essentially still in its infancy. We take a critical look at the training corpus in order to understand its biases, while also using it to venture beyond hate speech detection and investigate whether it can be used to shed light on other facets of research, such as popularity of hate tweets.

Publications in the life sciences are characterized by a large technical vocabulary, with many lexical and semantic variations for expressing the same concept. Towards addressing the problem of relevance in biomedical literature search, we introduce a deep learning model for the relevance of a document’s text to a keyword style query. Limited by a relatively small amount of training data, the model uses pre-trained word embeddings. With these, the model first computes a variable-length Delta matrix between the query and document, representing a difference between the two texts, which is then passed through a deep convolution stage followed by a deep feed-forward network to compute a relevance score. This results in a fast model suitable for use in an online search engine. The model is robust and outperforms comparable state-of-the-art deep learning approaches.

Santa Fe, New Mexico, USA, August 20-26, 2018. 997 Encoding Sentiment Information into Word Vectors for Sentiment Analysis Zhe Ye♠ Fang Li♠ Timothy Baldwin♥ ♠ Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China ♥ School of Computing and Information Systems, The University of Melbourne, Victoria, 3010, Australia {yezhejack, fli}@sjtu.edu.cn, tb@ldwin.net

We propose a graph-based mechanism to extract rich-emotion bearing patterns, which fosters a deeper analysis of online emotional expressions, from a corpus. The patterns are then enriched with word embeddings and evaluated through several emotion recognition tasks. Moreover, we conduct analysis on the emotion-oriented patterns to demonstrate its applicability and to explore its properties. Our experimental results demonstrate that the proposed techniques outperform most stateof-the-art emotion recognition techniques.

0957-4174/$ see front matter 2012 Elsevier Ltd. A http://dx.doi.org/10.1016/j.eswa.2012.12.015 ⇑ Corresponding author. E-mail addresses: juaner@lsi.uned.es (J. MartinezAraujo). Twitter spam detection is a recent area of research in which most previous works had focused on the identification of malicious user accounts and honeypot-based approaches. However, in this paper we present a methodology based on two new aspects: the detection of spam tweets in isolation and without previous information of the user; and the application of a statistical analysis of language to detect spam in trending topics. Trending topics capture the emerging Internet trends and topics of discussion that are in everybody’s lips. This growing microblogging phenomenon therefore allows spammers to disseminate malicious tweets quickly and massively. In this paper we present the first work that tries to detect spam tweets in real time using language as the primary tool. We first collected and labeled a large dataset with 34 K trending topics and 20 million tweets. Then, we have proposed a reduced set of features hardly manipulated by spammers. In addition, we have developed a machine learning system with some orthogonal features that can be combined with other sets of features with the aim of analyzing emergent characteristics of spam in social networks. We have also conducted an extensive evaluation process that has allowed us to show how our system is able to obtain an F-measure at the same level as the best state-ofthe-art systems based on the detection of spam accounts. Thus, our system can be applied to Twitter spam detection in trending topics in real time due mainly to the analysis of tweets instead of user accounts. 2012 Elsevier Ltd. All rights reserved.

Retrieving semantic similar short texts is a crucial issue to many applications, e.g., web search, ads matching, questionanswer system, and so forth. Most of the traditional methods concentrate on how to improve the precision of the similarity measurement, while current real applications need to efficiently explore the top similar short texts semantically related to the query one. We address the efficiency issue in this paper by investigating the similarity strategies and incorporating them into the FAST framework (efficient FrAmework for semantic similar Short Texts retrieval). We conduct comprehensive performance evaluation on real-life data which shows that our proposed method outperforms the state-ofthe-art techniques.

We describe the ATILF-LLF system built for the MWE 2017 Shared Task on automatic identification of verbal multiword expressions. We participated in the closed track only, for all the 18 available languages. Our system is a robust greedy transition-based system, in which MWE are identified through a MERGE transition. The system was meant to accommodate the variety of linguistic resources provided for each language, in terms of accompanying morphological and syntactic information. Using per-MWE Fscore, the system was ranked first1 for all but two languages (Hungarian and Romanian).

Review of Semantic-Free Utterances in Social Human–Robot Interaction Selma Yilmazyildiz, Robin Read, Tony Belpeame & Werner Verhelst To cite this article: Selma Yilmazyildiz, Robin Read, Tony Belpeame & Werner Verhelst (2016) Review of Semantic-Free Utterances in Social Human–Robot Interaction, International Journal of Human–Computer Interaction, 32:1, 63-85, DOI: 10.1080/10447318.2015.1093856 To link to this article: http://dx.doi.org/10.1080/10447318.2015.1093856

In web search, users queries are formulated using only few terms and term-matching retrieval functions could fail at retrieving relevant documents. Given a user query, the technique of query expansion (QE) consists in selecting related terms that could enhance the likelihood of retrieving relevant documents. Selecting such expansion terms is challenging and requires a computational framework capable of encoding complex semantic relationships. In this paper, we propose a novel method for learning, in a supervised way, semantic representations for words and phrases. By embedding queries and documents in special matrices, our model disposes of an increased representational power with respect to existing approaches adopting a vector representation. We show that our model produces high-quality query expansion terms. Our expansion increase IR measures beyond expansion from current word-embeddings models and well-established traditional QE methods.

We report on a system for automatically generating and displaying crosswords from a system manager supplied database of potential clues and corresponding words that index those clues. The system relies on the lexical relations encoded in WordNet to enhance the aesthetics of the resulting crossword by making it easier to automatically identify a grid that may be populated with words and clues that have a thematic focus. The system architecture is provided in overview, as is empirical evaluation.


Question: Automated ranking of profane language through the application of word embeddings and pseudo-relevance feedback.

This paper presents an innovative framework, the Lexical Syntactic Feature (LSF) architecture, aimed at enhancing the detection of offensive language in social media to safeguard adolescent online safety. Given the unstructured, informal nature of online social media content, which often includes misspellings, conventional research on message-level offensive language detection has proven inadequate. In contrast, user-level offensiveness detection presents a more viable strategy, yet remains an underexplored area. To address this gap, we introduce the LSF architecture, which distinguishes between pejoratives/profanities and obscenities in offensive content determination, while also integrating hand-crafted syntactic rules to pinpoint name-calling harassment. Specifically, our framework considers a user's writing style, structure, and specific cyberbullying content as predictive features for offensive content generation. Experimental results demonstrate that the LSF framework significantly outperforms existing methods in offensive content detection, achieving a precision of 98.24% and recall of 94.34% in sentence-level offensive detection, as well as a precision of 77.9% and recall of 77.8% in user-level offensive detection. Notably, the LSF architecture processes sentences at a rate of approximately 10 milliseconds per sentence, indicating its suitability for efficient implementation in social media environments.

Citation sentiment analysis is an important task in scientific paper analysis. Existing machine learning techniques for citation sentiment analysis are focusing on labor-intensive feature engineering, which requires large annotated corpus. As an automatic feature extraction tool, word2vec has been successfully applied to sentiment analysis of short texts. In this work, I conducted empirical research with the question: how well does word2vec work on the sentiment analysis of citations? The proposed method constructed sentence vectors (sent2vec) by averaging the word embeddings, which were learned from Anthology Collections (ACL-Embeddings). I also investigated polarity-specific word embeddings (PS-Embeddings) for classifying positive and negative citations. The sentence vectors formed a feature space, to which the examined citation sentence was mapped to. Those features were input into classifiers (support vector machines) for supervised classification. Using 10-cross-validation scheme, evaluation was conducted on a set of annotated citations. The results showed that word embeddings are effective on classifying positive and negative citations. However, hand-crafted features performed better for the overall classification.

While games are a popular social media for children, there is a real risk that these children are exposed to potential sexual assault. A number of studies have already addressed this issue, however, the data used in previous research did not properly represent the real chats found in multiplayer online games. To address this issue, we obtained real chat data from MovieStarPlanet, a massively multiplayer online game for children. The research described in this paper aimed to detect predatory behaviours in the chats using machine learning methods. In order to achieve a high accuracy on this task, extensive preprocessing was necessary. We describe three different strategies for data selection and preprocessing, and extensively compare the performance of different learning algorithms on the different datasets and features.

Today, a huge amount of text is being generated for social purposes on social networking services on the Web. Unlike traditional documents, such text is usually extremely short and tends to be informal. Analysis of such text benefit many applications such as advertising, search, and content filtering. In this work, we study one traditional text mining task on such new form of text, that is extraction of meaningful keywords. We propose several intuitive yet useful features and experiment with various classification models. Evaluation is conducted on Facebook data. Performances of various features and models are reported and compared.

We discuss the design and usage of ldquoWordle,rdquo a Web-based tool for visualizing text. Wordle creates tag-cloud-like displays that give careful attention to typography, color, and composition. We describe the algorithms used to balance various aesthetic criteria and create the distinctive Wordle layouts. We then present the results of a study of Wordle usage, based both on spontaneous behaviour observed in the wild, and on a large-scale survey of Wordle users. The results suggest that Wordles have become a kind of medium of expression, and that a ldquoparticipatory culturerdquo has arisen around them.

Social networks are persistently generating text-based data that encapsulate vast amounts of knowledge. However, the presence of non-standard terms and misspellings in texts originating from social networks poses a crucial challenge for natural language processing and machine learning systems that attempt to mine this knowledge. To address this problem, we propose a sequential, modular, and hybrid pipeline for social media text normalization. In the first phase, text preprocessing techniques and social media-specific vocabularies gathered from publicly available sources are used to transform, with high precision, out-of-vocabulary terms into in-vocabulary terms. A sequential language model, generated using the partially normalized texts from the first phase, is then utilized to normalize short, high-frequency, ambiguous terms. A supervised learning module is employed to normalize terms based on a manually annotated training corpus. Finally, a tunable, distributed language model-based backoff module at the end of the pipeline enables further customization of the system to specific domains of text. We performed intrinsic evaluations of the system on a publicly available domain-independent dataset from Twitter, and our system obtained an F-score of 0.836, outperforming other benchmark systems for the task. We further performed brief, task-oriented evaluations of the system to illustrate the customizability of the system to domain-specific tasks and the effects of normalization on downstream applications. The modular design enables the easy customization of the system to distinct types domain-specific social media text, in addition to its off-the-shelf application to generic social media text.

Bigrams and trigrams are commonly used in statistical natural language processing; this paper will describe techniques for working with much longer n-grams. Suffix arrays (Manber and Myers 1990) were first introduced to compute the frequency and location of a substring (n-gram) in a sequence (corpus) of length N. To compute frequencies over all N(N+1)/2 substrings in a corpus, the substrings are grouped into a manageable number of equivalence classes. In this way, a prohibitive computation over substrings is reduced to a manageable computation over classes. This paper presents both the algorithms and the code that were used to compute term frequency (tf) and document frequency (df) for all n-grams in two large corpora, an English corpus of 50 million words of Wall Street Journal and a Japanese corpus of 216 million characters of Mainichi Shimbun. The second half of the paper uses these frequencies to find interesting substrings. Lexicographers have been interested in n-grams with high mutual information (MI) where the joint term frequency is higher than what would be expected by chance, assuming that the parts of the n-gram combine independently. Residual inverse document frequency (RIDF) compares document frequency to another model of chance where terms with a particular term frequency are distributed randomly throughout the collection. MI tends to pick out phrases with noncompositional semantics (which often violate the independence assumption) whereas RIDF tends to highlight technical terminology, names, and good keywords for information retrieval (which tend to exhibit nonrandom distributions over documents). The combination of both MI and RIDF is better than either by itself in a Japanese word extraction task.

This paper introduces NEMWEL, a system that performs Never-Ending MultiWord Expressions Learning. Instead of using a static corpus and classifier, NEMWEL applies supervised learning on automatically crawled news texts. Moreover, it uses its own results to periodically retrain the classifier, bootstrapping on its own results. In addition to a detailed description of the system’s architecture and its modules, we report the results of a manual evaluation. It shows that NEMWEL is capable of learning new expressions over time with improved precision.

We propose a new unsupervised learning technique for extracting information from large text collections. We model documents as if they were generated by a two-stage stochastic process. Each author is represented by a probability distribution over topics, and each topic is represented as a probability distribution over words for that topic. The words in a multi-author paper are assumed to be the result of a mixture of each authors' topic mixture. The topic-word and author-topic distributions are learned from data in an unsupervised manner using a Markov chain Monte Carlo algorithm. We apply the methodology to a large corpus of 160,000 abstracts and 85,000 authors from the well-known CiteSeer digital library, and learn a model with 300 topics. We discuss in detail the interpretation of the results discovered by the system including specific topic and author models, ranking of authors by topic and topics by author, significant trends in the computer science literature between 1990 and 2002, parsing of abstracts by topics and authors and detection of unusual papers by specific authors. An online query interface to the model is also discussed that allows interactive exploration of author-topic models for corpora such as CiteSeer.

The paper systematically compares two feature extraction algorithms to mine product features commented on in customer reviews. The first approach [17] identifies candidate features by applying a set of POS patterns and pruning the candidate set based on the log likelihood ratio test. The second approach [11] applies association rule mining for identifying frequent features and a heuristic based on the presence of sentiment terms for identifying infrequent features. We evaluate the performance of the algorithms on five product specific document collections regarding consumer electronic devices. We perform an analysis of errors and discuss advantages and limitations of the algorithms.

This paper introduces a task of identifying and semantically classifying lexical expressions in running text. We investigate the online reviews genre, adding semantic supersense annotations to a 55,000 word English corpus that was previously annotated for multiword expressions. The noun and verb supersenses apply to full lexical expressions, whether singleor multiword. We then present a sequence tagging model that jointly infers lexical expressions and their supersenses. Results show that even with our relatively small training corpus in a noisy domain, the joint task can be performed to attain 70% class labeling F1.


Question: Automated ranking of profane language through the utilization of word embeddings and pseudo-relevance feedback mechanisms.

This comprehensive review explores the landscape of sentiment analysis and opinion mining, two critical areas that have gained significant attention in the era of Web 2.0 and social media proliferation. The widespread use of the internet for sharing personal views and experiences on a global scale has necessitated the development of sophisticated tools and techniques to understand and analyze these digital expressions of sentiment. Over the past fifteen years, scholars, researchers, and industry professionals have dedicated considerable effort to advancing sentiment analysis methodologies, which are crucial for gauging public sentiment and opinions in various contexts. This paper offers an in-depth overview of sentiment analysis, drawing insights from over one hundred studies conducted between 2002 and 2015. It systematically categorizes the research based on the essential sub-tasks involved in sentiment analysis, the application of machine learning and natural language processing techniques, and the diverse applications of sentiment analysis in different domains. By consolidating this extensive body of work, the review highlights the key approaches, methodologies, and trends in sentiment analysis, while also identifying gaps and challenges that remain to be addressed. Accompanied by a detailed summary table encompassing 161 articles, this paper serves as a valuable resource for academics, practitioners, and enthusiasts seeking to deepen their understanding of sentiment analysis and its evolving landscape.

We propose a new unified framework for monolingual (MoIR) and cross-lingual information retrieval (CLIR) which relies on the induction of dense real-valued word vectors known as word embeddings (WE) from comparable data. To this end, we make several important contributions: (1) We present a novel word representation learning model called Bilingual Word Embeddings Skip-Gram (BWESG) which is the first model able to learn bilingual word embeddings solely on the basis of document-aligned comparable data; (2) We demonstrate a simple yet effective approach to building document embeddings from single word embeddings by utilizing models from compositional distributional semantics. BWESG induces a shared cross-lingual embedding vector space in which both words, queries, and documents may be presented as dense real-valued vectors; (3) We build novel ad-hoc MoIR and CLIR models which rely on the induced word and document embeddings and the shared cross-lingual embedding space; (4) Experiments for English and Dutch MoIR, as well as for English-to-Dutch and Dutch-to-English CLIR using benchmarking CLEF 2001-2003 collections and queries demonstrate the utility of our WE-based MoIR and CLIR models. The best results on the CLEF collections are obtained by the combination of the WE-based approach and a unigram language model. We also report on significant improvements in ad-hoc IR tasks of our WE-based framework over the state-of-the-art framework for learning text representations from comparable data based on latent Dirichlet allocation (LDA).

We consider noisy crowdsourced assessments and their impact on learning-to-rank algorithms. Starting with EM-weighted assessments, we modify LambdaMART in order to use smoothed probabilistic preferences over pairs of documents, directly as input to the ranking algorithm.

Large parallel corpora that are automatically obtained from the web, documents or elsewhere often exhibit many corrupted parts that are bound to negatively affect the quality of the systems and models that learn from these corpora. This paper describes frequent problems found in data and such data affects neural machine translation systems, as well as how to identify and deal with them. The solutions are summarised in a set of scripts that remove problematic sentences from input corpora.

Determining semantic similarity between texts is important in many tasks in information retrieval such as search, query suggestion, automatic summarization and image finding. Many approaches have been suggested, based on lexical matching, handcrafted patterns, syntactic parse trees, external sources of structured semantic knowledge and distributional semantics. However, lexical features, like string matching, do not capture semantic similarity beyond a trivial level. Furthermore, handcrafted patterns and external sources of structured semantic knowledge cannot be assumed to be available in all circumstances and for all domains. Lastly, approaches depending on parse trees are restricted to syntactically well-formed texts, typically of one sentence in length.
 We investigate whether determining short text similarity is possible using only semantic features---where by semantic we mean, pertaining to a representation of meaning---rather than relying on similarity in lexical or syntactic representations. We use word embeddings, vector representations of terms, computed from unlabelled data, that represent terms in a semantic space in which proximity of vectors can be interpreted as semantic similarity.
 We propose to go from word-level to text-level semantics by combining insights from methods based on external sources of semantic knowledge with word embeddings. A novel feature of our approach is that an arbitrary number of word embedding sets can be incorporated. We derive multiple types of meta-features from the comparison of the word vectors for short text pairs, and from the vector means of their respective word embeddings. The features representing labelled short text pairs are used to train a supervised learning algorithm. We use the trained model at testing time to predict the semantic similarity of new, unlabelled pairs of short texts
 We show on a publicly available evaluation set commonly used for the task of semantic similarity that our method outperforms baseline methods that work under the same conditions.

We develop latent Dirichlet allocation with WORDNET (LDAWN), an unsupervised probabilistic topic model that includes word sense as a hidden variable. We develop a probabilistic posterior inference algorithm for simultaneously disambiguating a corpus and learning the domains in which to consider each word. Using the WORDNET hierarchy, we embed the construction of Abney and Light (1999) in the topic model and show that automatically learned domains improve WSD accuracy compared to alternative contexts.

Previous approaches to the problem of measuring similarity between automatically generated topics have been based on comparison of the topics’ word probability distributions. This paper presents alternative approaches, including ones based on distributional semantics and knowledgebased measures, evaluated by comparison with human judgements. The best performing methods provide reliable estimates of topic similarity comparable with human performance and should be used in preference to the word probability distribution measures used previously.

With the rise of social media, learning from informal text has become increasingly important. We present a novel semantic lexicon induction approach that is able to learn new vocabulary from social media. Our method is robust to the idiosyncrasies of informal and open-domain text corpora. Unlike previous work, it does not impose restrictions on the lexical features of candidate terms – e.g. by restricting entries to nouns or noun phrases – while still being able to accurately learn multiword phrases of variable length. Starting with a few seed terms for a semantic category, our method first explores the context around seed terms in a corpus, and identifies context patterns that are relevant to the category. These patterns are used to extract candidate terms – i.e. multiword segments that are further analyzed to ensure meaningful term boundary segmentation. We show that our approach is able to learn high quality semantic lexicons from informally written social media text of Twitter, and can achieve accuracy as high as 92% in the top 100 learned category members.

The large scale of Q&A archives accumulated in community based question answering (CQA) servivces are important information and knowledge resource on the web. Question and answer matching task has been attached much importance to for its ability to reuse knowledge stored in these systems: it can be useful in enhancing user experience with recurrent questions. In this paper, a Word Embedding based Correlation (WEC) model is proposed by integrating advantages of both the translation model and word embedding. Given a random pair of words, WEC can score their co-occurrence probability in Q&A pairs, while it can also leverage the continuity and smoothness of continuous space word representation to deal with new pairs of words that are rare in the training parallel text. An experimental study on Yahoo! Answers dataset and Baidu Zhidao dataset shows this new method’s promising

In web search, users queries are formulated using only few terms and term-matching retrieval functions could fail at retrieving relevant documents. Given a user query, the technique of query expansion (QE) consists in selecting related terms that could enhance the likelihood of retrieving relevant documents. Selecting such expansion terms is challenging and requires a computational framework capable of encoding complex semantic relationships. In this paper, we propose a novel method for learning, in a supervised way, semantic representations for words and phrases. By embedding queries and documents in special matrices, our model disposes of an increased representational power with respect to existing approaches adopting a vector representation. We show that our model produces high-quality query expansion terms. Our expansion increase IR measures beyond expansion from current word-embeddings models and well-established traditional QE methods.

Publications in the life sciences are characterized by a large technical vocabulary, with many lexical and semantic variations for expressing the same concept. Towards addressing the problem of relevance in biomedical literature search, we introduce a deep learning model for the relevance of a document’s text to a keyword style query. Limited by a relatively small amount of training data, the model uses pre-trained word embeddings. With these, the model first computes a variable-length Delta matrix between the query and document, representing a difference between the two texts, which is then passed through a deep convolution stage followed by a deep feed-forward network to compute a relevance score. This results in a fast model suitable for use in an online search engine. The model is robust and outperforms comparable state-of-the-art deep learning approaches.


Question: Automated ranking of profane language through word embeddings and pseudo-relevance feedback.

This paper presents an innovative system for the automatic detection of offensive language in text communications, leveraging multi-level classification techniques. With the proliferation of digital communication via the Internet and mobile devices, the prevalence of abusive language, including rants, taunts, and vulgar expressions, has increased, potentially causing harm or discomfort to users. Developing a software tool capable of automatically identifying and filtering out such offensive content, with a customizable sensitivity setting, is crucial. Despite the ease with which humans can distinguish between offensive and benign messages, automating this process poses significant challenges for computational algorithms. Our proposed methodology employs a multi-level classification approach, integrating various statistical models and rule-based patterns to extract features at different conceptual levels. This system is further enhanced by a supplementary weighted pattern database, which improves detection accuracy by cross-referencing text against a graded list of offensive terms. The combination of these elements allows for a more nuanced and effective identification of abusive language in digital communications.

Social media has become an inevitable part of individuals personal and business lives. Its benefits come with various negative consequences. One major concern is the prevalence of detrimental online behavior on social media, such as online harassment and cyberbullying. In this study, we aim to address the computational challenges associated with harassment detection in social media by developing a machine learning framework with three distinguishing characteristics. (1) It uses minimal supervision in the form of expert-provided key phrases that are indicative of bullying or non-bullying. (2) It detects harassment with an ensemble of two learners that co-train one another; one learner examines the language content in the message, and the other learner considers the social structure. (3) It incorporates distributed word and graph-node representations by training nonlinear deep models. The model is trained by optimizing an objective function that balances a co-training loss with a weak-supervision loss. We evaluate the effectiveness of our approach using post-hoc, crowdsourced annotation of Twitter, Ask.fm, and Instagram data, finding that our deep ensembles outperform previous non-deep methods for weakly supervised harassment detection.

Language modeling approaches to information retrieval are attractive and promising because they connect the problem of retrieval with that of language model estimation, which has been studied extensively in other application areas such as speech recognition. The basic idea of these approaches is to estimate a language model for each document, and to then rank documents by the likelihood of the query according to the estimated language model. A central issue in language model estimation is smoothing, the problem of adjusting the maximum likelihood estimator to compensate for data sparseness. In this article, we study the problem of language model smoothing and its influence on retrieval performance. We examine the sensitivity of retrieval performance to the smoothing parameters and compare several popular smoothing methods on different test collections. Experimental results show that not only is the retrieval performance generally sensitive to the smoothing parameters, but also the sensitivity pattern is affected by the query type, with performance being more sensitive to smoothing for verbose queries than for keyword queries. Verbose queries also generally require more aggressive smoothing to achieve optimal performance. This suggests that smoothing plays two different role---to make the estimated document language model more accurate and to "explain" the noninformative words in the query. In order to decouple these two distinct roles of smoothing, we propose a two-stage smoothing strategy, which yields better sensitivity patterns and facilitates the setting of smoothing parameters automatically. We further propose methods for estimating the smoothing parameters automatically. Evaluation on five different databases and four types of queries indicates that the two-stage smoothing method with the proposed parameter estimation methods consistently gives retrieval performance that is close to---or better than---the best results achieved using a single smoothing method and exhaustive parameter search on the test data.

We demonstrate a new research approach to the problem of predicting the reading difficulty of a text passage, by recasting readability in terms of statistical language modeling. We derive a measure based on an extension of multinomial naïve Bayes classification that combines multiple language models to estimate the most likely grade level for a given passage. The resulting classifier is not specific to any particular subject and can be trained with relatively little labeled data. We perform predictions for individual Web pages in English and compare our performance to widely-used semantic variables from traditional readability measures. We show that with minimal changes, the classifier may be retrained for use with French Web documents. For both English and French, the classifier maintains consistently good correlation with labeled grade level (0.63 to 0.79) across all test sets. Some traditional semantic variables such as type-token ratio gave the best performance on commercial calibrated test passages, while our language modeling approach gave better accuracy for Web documents and very short passages (less than 10 words).

While social media empowers freedom of expression and individual voices, it also enables anti-social behavior, online harassment, cyberbullying, and hate speech. In this paper, we deepen our understanding of online hate speech by focusing on a largely neglected but crucial aspect of hate speech – its target: either directed towards a specific person or entity, or generalized towards a group of people sharing a common protected characteristic. We perform the first linguistic and psycholinguistic analysis of these two forms of hate speech and reveal the presence of interesting markers that distinguish these types of hate speech. Our analysis reveals that Directed hate speech, in addition to being more personal and directed, is more informal, angrier, and often explicitly attacks the target (via name calling) with fewer analytic words and more words suggesting authority and influence. Generalized hate speech, on the other hand, is dominated by religious hate, is characterized by the use of lethal words such as murder, exterminate, and kill; and quantity words such as million and many. Altogether, our work provides a data-driven analysis of the nuances of online-hate speech that enables not only a deepened understanding of hate speech and its social implications, but also its detection.

This paper presents a bootstrapping process that learns linguistically rich extraction patterns for subjective (opinionated) expressions. High-precision classifiers label unannotated data to automatically create a large training set, which is then given to an extraction pattern learning algorithm. The learned patterns are then used to identify more subjective sentences. The bootstrapping process learns many subjective patterns and increases recall while maintaining high precision.

We introduce an extended naive Bayes model for word sense induction (WSI) and apply it to a WSI task. The extended model incorporates the idea the words closer to the target word are more relevant in predicting its sense. The proposed model is very simple yet effective when evaluated on SemEval-2010 WSI data.

In a large online advertising system, adversaries may attempt to profit from the creation of low quality or harmful advertisements. In this paper, we present a large scale data mining effort that detects and blocks such adversarial advertisements for the benefit and safety of our users. Because both false positives and false negatives have high cost, our deployed system uses a tiered strategy combining automated and semi-automated methods to ensure reliable classification. We also employ strategies to address the challenges of learning from highly skewed data at scale, allocating the effort of human experts, leveraging domain expert knowledge, and independently assessing the effectiveness of our system.

We introduce a (semi-)automatic OCR post-processing system that utilizes web-scale linguistic corpora in providing high-quality correction. This paper is a comprehensive system overview with the focus on the computational procedures, applied linguistic analysis, and processing optimization.

Query expansion methods have been studied for a long time - with debatable success in many instances. In this paper we present a probabilistic query expansion model based on a similarity thesaurus which was constructed automatically. A similarity thesaurus reflects domain knowledge about the particular collection from which it is constructed. We address the two important issues with query expansion: the selection and the weighting of additional search terms. In contrast to earlier methods, our queries are expanded by adding those terms that are most similar to the concept of the query, rather than selecting terms that are similar to the query terms. Our experiments show that this kind of query expansion results in a notable improvement in the retrieval effectiveness when measured using both recall-precision and usefulness.

Keyphrase extraction is a fundamental task in natural language processing that facilitates mapping of documents to a set of representative phrases. In this paper, we present an unsupervised technique (Key2Vec) that leverages phrase embeddings for ranking keyphrases extracted from scientific articles. Specifically, we propose an effective way of processing text documents for training multi-word phrase embeddings that are used for thematic representation of scientific articles and ranking of keyphrases extracted from them using theme-weighted PageRank. Evaluations are performed on benchmark datasets producing state-of-the-art results.


Question: Automated ranking of profane language through the utilization of word embeddings and pseudo-relevance feedback.

Introducing Movie-DiC: A Comprehensive Movie Dialogue Corpus for Research and Development
This paper presents Movie-DiC, a newly assembled Movie Dialogue Corpus, specifically curated for research and development initiatives. The dataset encompasses a rich collection of 132,229 dialogues, totaling 764,146 conversational turns, meticulously extracted from a diverse range of 753 films. The paper offers an in-depth look at the methodology employed in data collection and the organizational structure of the corpus. Additionally, it delves into key statistical insights and distinctive features that define Movie-DiC, making it a valuable resource for scholars and developers alike.

**Question**: A novel framework for detecting and tracking multiple moving objects in complex scenes

**Documents**: A fast and accurate algorithm for online handwritten Chinese character recognition
In this paper, we propose a fast and accurate algorithm for online handwritten Chinese character recognition. The algorithm uses a new feature extraction method based on stroke direction and length, which is robust to various writing styles and sizes. The recognition is performed using a Hidden Markov Model (HMM) with a Gaussian Mixture Model (GMM) for each state. The algorithm is tested on a large dataset of online handwritten Chinese characters, and the results show that it outperforms the state-of-the-art methods in terms of recognition accuracy and speed.

**Similar Documents**: An Innovative Algorithm for Enhanced Online Handwritten Chinese Character Recognition
This paper introduces a novel and efficient algorithm designed for online handwritten Chinese character recognition. The proposed algorithm features a distinctive approach to feature extraction, focusing on stroke direction and length, which ensures robustness against variations in writing styles and sizes. The recognition process employs a Hidden Markov Model (HMM) coupled with a Gaussian Mixture Model (GMM) for each state, significantly enhancing the accuracy and speed of recognition. The algorithm's performance is rigorously evaluated using a comprehensive dataset of online handwritten Chinese characters, demonstrating its superiority over existing state-of-the-art methods in both recognition accuracy and processing speed.

We describe a method for the automatic acquisition of the hyponymy lexical relation from unrestricted text. Two goals motivate the approach: (i) avoidance of the need for pre-encoded knowledge and (ii) applicability across a wide range of text. We identify a set of lexicosyntactic patterns that are easily recognizable, that occur frequently and across text genre boundaries, and that indisputably indicate the lexical relation of interest. We describe a method for discovering these patterns and suggest that other lexical relations will also be acquirable in this way. A subset of the acquisition algorithm is implemented and the results are used to augment and critique the structure of a large hand-built thesaurus. Extensions and applications to areas such as information retrieval are suggested.

We explore the relation between classical probabilistic models of information retrieval and the emerging language modeling approaches. It has long been recognized that the primary obstacle to effective performance of classical models is the need to estimate a relevance model: probabilities of words in the relevant class. We propose a novel technique for estimating these probabilities using the query alone. We demonstrate that our technique can produce highly accurate relevance models, addressing important notions of synonymy and polysemy. Our experiments show relevance models outperforming baseline language modeling systems on TREC retrieval and TDT tracking tasks. The main contribution of this work is an effective formal method for estimating a relevance model with no training data.

This paper proposes a new model for the detection of clickbait, i.e., short messages that lure readers to click a link. Clickbait is primarily used by online content publishers to increase their readership, whereas its automatic detection will give readers a way of filtering their news stream. We contribute by compiling the first clickbait corpus of 2992 Twitter tweets, 767 of which are clickbait, and, by developing a clickbait model based on 215 features that enables a random forest classifier to achieve 0.79 ROC-AUC at 0.76 precision and 0.76 recall.

In recent years, collocation has been widely acknowledged as an essential characteristic to distinguish native speakers from non-native speakers. Research on academic writing has also shown that collocations are not only common but serve a particularly important discourse function within the academic community. In our study, we propose a machine learning approach to implementing an online collocation writing assistant. We use a data-driven classifier to provide collocation suggestions to improve word choices, based on the result of classification. The system generates and ranks suggestions to assist learners’ collocation usages in their academic writing with satisfactory results. *

0957-4174/$ see front matter 2012 Elsevier Ltd. A http://dx.doi.org/10.1016/j.eswa.2012.12.015 ⇑ Corresponding author. E-mail addresses: juaner@lsi.uned.es (J. MartinezAraujo). Twitter spam detection is a recent area of research in which most previous works had focused on the identification of malicious user accounts and honeypot-based approaches. However, in this paper we present a methodology based on two new aspects: the detection of spam tweets in isolation and without previous information of the user; and the application of a statistical analysis of language to detect spam in trending topics. Trending topics capture the emerging Internet trends and topics of discussion that are in everybody’s lips. This growing microblogging phenomenon therefore allows spammers to disseminate malicious tweets quickly and massively. In this paper we present the first work that tries to detect spam tweets in real time using language as the primary tool. We first collected and labeled a large dataset with 34 K trending topics and 20 million tweets. Then, we have proposed a reduced set of features hardly manipulated by spammers. In addition, we have developed a machine learning system with some orthogonal features that can be combined with other sets of features with the aim of analyzing emergent characteristics of spam in social networks. We have also conducted an extensive evaluation process that has allowed us to show how our system is able to obtain an F-measure at the same level as the best state-ofthe-art systems based on the detection of spam accounts. Thus, our system can be applied to Twitter spam detection in trending topics in real time due mainly to the analysis of tweets instead of user accounts. 2012 Elsevier Ltd. All rights reserved.

This paper introduces NEMWEL, a system that performs Never-Ending MultiWord Expressions Learning. Instead of using a static corpus and classifier, NEMWEL applies supervised learning on automatically crawled news texts. Moreover, it uses its own results to periodically retrain the classifier, bootstrapping on its own results. In addition to a detailed description of the system’s architecture and its modules, we report the results of a manual evaluation. It shows that NEMWEL is capable of learning new expressions over time with improved precision.

We present an analysis of over 26000 projects from Kickstarter, a popular crowdfunding platform. Specifically, we focus on the language used in project pitches, and how it impacts project success/failure. We train a series of discriminative models for binary success/failure prediction of projects with increasing complexity of linguistic features, and show that successful project pitches are, on average, more emotive, thoughtful and colloquial. In addition, we also present an analysis of pledge rewards in Kickstarter, and how these differ across categories and for successful versus unsuccessful projects.

Most keyword search engines return directly matching keyword phrases. However, publishers cannot anticipate all possible ways in which users would search for the items in their documents. In fact, many times, there may be no direct keyword match between a keyword search phrase and descriptions of relevant items that are perfect matches for the search. We present an automated, high precision-based information retrieval solution to boost item find-ability by bridging the semantic gap between item information and popular keyword search phrases. Our solution achieves an average of 80% F-measure for various boosted matches for keyword search phrases in various categories.

In this paper we introduce a novel computational technique of extraction of personality traits (HEXACO) of employees from Enterprise Social Media posts. We deal with challenges such as not being able to use existing survey instruments for scoring and not being able to directly use existing psychological studies on written text due to lack of overlapping words between the existing dictionary and words used in Enterprise Social Media. Using our approach we are able to infer personality traits (HEXACO) from posts and find better coverage and usage of the extended dictionary.

In this paper we introduce the novel task of “word epoch disambiguation,” defined as the problem of identifying changes in word usage over time. Through experiments run using word usage examples collected from three major periods of time (1800, 1900, 2000), we show that the task is feasible, and significant differences can be observed between occurrences of words in different periods of time.


Question: The negative impacts of personality traits in the workplace.

This study delves into the interplay between CEO personality traits, leadership styles, and their strategic impact on organizations, specifically within the context of Major League Baseball over a century. The investigation focuses on how positive personality traits, encapsulated by core self-evaluations, correlate with transformational leadership, while the negative aspects of personality, epitomized by narcissism, are linked to contingent reward leadership. The findings reveal that CEOs with high core self-evaluations tend to exhibit stronger transformational leadership, which in turn positively influences strategic outcomes such as managerial stability, team performance, fan engagement, and perceived influence. Conversely, narcissistic CEOs are more likely to adopt contingent reward leadership, which is associated with higher manager turnover and lower ratings of influence. This study underscores the dual nature of CEO personality, highlighting how the bright and dark sides can distinctly shape leadership approaches and, consequently, organizational success.

**Question**: The impact of renewable energy on global warming

**Documents**: The impact of renewable energy on global warming: A review.
This paper reviews the current state of knowledge on the impact of renewable energy on global warming, focusing on the potential of wind, solar, hydro, and biomass energy to mitigate greenhouse gas emissions. The review covers recent studies on the environmental, economic, and social aspects of renewable energy, and discusses the challenges and opportunities for its wider adoption. The paper concludes that renewable energy can play a significant role in reducing global warming, but that its full potential can only be realized through a combination of technological innovation, policy support, and public engagement.

**Similar Documents**: This article presents an in-depth analysis of the role renewable energy sources—wind, solar, hydro, and biomass—play in combating global warming. By surveying recent research, the paper examines the environmental, economic, and social dimensions of renewable energy, exploring its potential to curtail greenhouse gas emissions. It also identifies the barriers to and prospects for the broader implementation of renewable energy. The analysis confirms that renewable energy is a crucial element in the fight against global warming, yet it emphasizes that to fully harness its benefits, concerted efforts in technology development, supportive policy frameworks, and community involvement are essential.

Past works on personality detection has shown that psycho-linguistic features, frequency based analysis at lexical level , emotive words and other lexical clues such as number of first person or sec ond person words carry major role to identify personality associated with the te xt. In this work, we propose a new architecture for the same task using common sen se knowledge with associated sentiment polarity and affective labels. To extract the common sense knowledge with sentiment polarity scores and affect iv labels we used Senticnet which is one of the most useful resources for opinion mining and sentiment analysis. In particular, we combined comm n sense knowledge based features with phycho-linguistic features and freque ncy based features and later the features were employed in supervised classifier . We designed five SMO based supervised classifiers for five personality t raits. We observe that the use of common sense knowledge with affective and sentim ent information enhances the accuracy of the existing frameworks which use o nly psycho-linguistic features and frequency based analysis at lexical le ve .

Affective influences abound in groups. In this article we propose an organizing model for understanding these affective influences and their effects on group life. We begin with individuallevel affective characteristics that members bring to their groups: moods, emotions, sentiments, and emotional intelligence. These affective characteristics then combine to form a group’s affective composition. We discuss explicit and implicit processes through which this affective combination occurs by examining the research on emotional contagion, entrainment, modeling, and the manipulation of affect. We also explore how elements of the affective context, such as organizationwide emotion norms and the group’s particular emotional history, may serve to constrain or amplify group members’ emotions. The outcome, group emotion, results from the combination of the group’s affective composition and the affective context in which the group is behaving. Last, we focus on the important interaction between nonaffective factors and affective factors in group life and suggest a possible agenda for future research. q 2001 Academic Press

Facebook is quickly becoming one of the most popular tools for social communication. However, Facebook is somewhat different from other Social Networking Sites as it demonstrates an offline-to-online trend; that is, the majority of Facebook Friends are met offline and then added later. The present research investigated how the Five-Factor Model of personality relates to Facebook use. Despite some expected trends regarding Extraversion and Openness to Experience, results indicated that personality factors were not as influential as previous literature would suggest. The results also indicated that a motivation to communicate was influential in terms of Facebook use. It is suggested that different motivations may be influential in the decision to use tools such as Facebook, especially when individual functions of Facebook are being considered. Ó 2008 Elsevier Ltd. All rights reserved. 1. Personality correlates and competency factors associated

In this study, the authors examined the findings and implications of the research on trust in leadership that has been conducted during the past 4 decades. First, the study provides estimates of the primary relationships between trust in leadership and key outcomes, antecedents, and correlates (k = 106). Second, the study explores how specifying the construct with alternative leadership referents (direct leaders vs. organizational leadership) and definitions (types of trust) results in systematically different relationships between trust in leadership and outcomes and antecedents. Direct leaders (e.g., supervisors) appear to be a particularly important referent of trust. Last, a theoretical framework is offered to provide parsimony to the expansive literature and to clarify the different perspectives on the construct of trust in leadership and its operation.

In this article some formal, content-related and procedural considerations towards the sense of humor are articulated and the analysis of both everyday humor behavior and of comic styles leads to the initial proposal of a four factormodel of humor (4FMH). This model is tested in a new dataset and it is also examined whether two forms of comic styles (benevolent humor and moral mockery) do fit in. The model seems to be robust but further studies on the structure of the sense of humor as a personality trait are required.

OBJECTIVE
This study sought to understand affective instability among patients with borderline personality disorder by examining the degree of instability in six affective domains. The authors also examined the subjective intensity with which moods are experienced and the association between instability and intensity of affect.

METHOD
In a group of 152 patients with personality disorders, subjective affective intensity and six dimensions of affective instability were measured. The mean scores for lability and intensity for each affective domain for patients with borderline personality disorder were compared with those of patients with other personality disorders through analyses that controlled for other axis I affective disorders, age, and sex.

RESULTS
Greater lability in terms of anger and anxiety and oscillation between depression and anxiety, but not in terms of oscillation between depression and elation, was associated with borderline personality disorder. Contrary to expectation, the experience of an increase in subjective affective intensity was not more prominent in patients with borderline personality disorder than in those with other personality disorders.

CONCLUSIONS
By applying a finer-grained perspective on affective instability than those of previous personality disorder studies, this study points to patterns of affective experience characteristic of patients with borderline personality disorder.


Question: The negative aspects of personality traits in the workplace.

An Investigative Study on the Nexus of Leadership Personality's 'Dark Side' and Transformational Leadership

In recent times, the focus on dysfunctional personality traits in professional settings, particularly among organizational leaders and managers, has gained significant traction. This paper details an exploratory study aimed at elucidating the connection between the 'dark side' of leadership personality and transformational leadership. The research involved a sample of 80 business leaders and senior executives from New Zealand, who were assessed using the Hogan Development Survey (HDS; Hogan & Hogan, 1997) and the Multifactor Leadership Questionnaire (MLQ; Bass & Avolio, 1995). Statistical analyses through regression techniques highlighted that the 'Colorful' (histrionic) trait of the HDS was positively correlated with transformational leadership. Conversely, the 'Cautious' (avoidant) and 'Bold' (narcissistic) facets of the HDS were identified as negative predictors of transformational leadership. This study marks a pioneering effort in highlighting the association between histrionic personality and transformational leadership, a relationship that has hitherto been overshadowed by the extensive focus on the interplay between leadership and narcissistic traits. © 2007 Elsevier Ltd. All rights reserved.

In this paper, we address the issue of personality and interaction style recognition from profile pictures in Facebook. We recruited volunteers among Facebook users and collected a dataset of profile pictures, labeled with gold standard self-assessed personality and interaction style labels. Then, we exploited a bag-of-visual-words technique to extract features from pictures. Finally, different machine learning approaches were used to test the effectiveness of these features in predicting personality and interaction style traits. Our good results show that this task is very promising, because profile pictures convey a lot of information about a user and are directly connected to impression formation and identity management.

Four meta-analyses were conducted to examine gender differences in personality in the literature (1958-1992) and in normative data for well-known personality inventories (1940-1992). Males were found to be more assertive and had slightly higher self-esteem than females. Females were higher than males in extraversion, anxiety, trust, and, especially, tender-mindedness (e.g., nurturance). There were no noteworthy sex differences in social anxiety, impulsiveness, activity, ideas (e.g., reflectiveness), locus of control, and orderliness. Gender differences in personality traits were generally constant across ages, years of data collection, educational levels, and nations.

Bandura, A. (2000). Cultivate self-efficacy for personal and organizational effectiveness. In E.

This paper targets the automatic detection of personality traits in a meeting environment by means of audio and visual features; information about the relational context is captured by means of acoustic features designed to that purpose. Two personality traits are considered: Extraversion (from the Big Five) and the Locus of Control. The classification task is applied to thin slices of behaviour, in the form of 1-minute sequences. SVM were used to test the performances of several training and testing instance setups, including a restricted set of audio features obtained through feature selection. The outcomes improve considerably over existing results, provide evidence about the feasibility of the multimodal analysis of personality, the role of social context, and pave the way to further studies addressing different features setups and/or targeting different personality traits.

In order for researchers to understand and predict behavior, they must consider both person and situation factors and how these factors interact. Even though organization researchers have developed interactional models, many have overemphasized either person or situation components, and most have failed to consider the effects that persons have on situations. This paper presents criteria for improving interactional models and a model of person-organization fit, which satisfies these criteria. Using a Q-sort methodology, individual value profiles are compared to organizational value profiles to determine fit and to predict changes in values, norms, and behaviors.