Question: Cosine Siamese networks for identifying stance in textual content.

This paper presents an innovative global logbilinear regression model for generating word vector representations, which we term Global Vectors for Word Representation (GloVe). Unlike previous methods that have demonstrated the ability to capture nuanced semantic and syntactic patterns through vector arithmetic, our work elucidates the underlying model characteristics necessary for these patterns to emerge within word vectors. By synthesizing the strengths of both global matrix factorization and local context window techniques, our model achieves a novel balance. It optimizes statistical information usage by focusing the training process exclusively on the non-zero elements found in a word-word co-occurrence matrix, thereby bypassing the need to process the entire sparse matrix or individual context windows from a vast corpus. The outcome is a vector space endowed with meaningful substructures, as substantiated by our model's 75% accuracy on a contemporary word analogy task. Additionally, GloVe outperforms competing models on tasks related to word similarity and named entity recognition, showcasing its versatility and effectiveness in linguistic analysis.

Polarization in social media networks is a fact in several scenarios such as political debates and other contexts such as same-sex marriage, abortion and gun control. Understanding and quantifying polarization is a longterm challenge to researchers from several areas, also being a key information for tasks such as opinion analysis. In this paper, we perform a systematic comparison between social networks that arise from both polarized and non-polarized contexts. This comparison shows that the traditional polarization metric – modularity – is not a direct measure of antagonism between groups, since non-polarized networks may be also divided into fairly modular communities. To bridge this conceptual gap, we propose a novel polarization metric based on the analysis of the boundary of a pair of (potentially polarized) communities, which better captures the notions of antagonism and polarization. We then characterize polarized and non-polarized social networks according to the concentration of high-degree nodes in the boundary of communities, and found that polarized networks tend to exhibit low concentration of popular nodes along the boundary. To demonstrate the usefulness of our polarization measures, we analyze opinions expressed on Twitter on the gun control issue in the United States, and conclude that our novel metrics help making sense of opinions expressed on online media.

Determining semantic similarity between texts is important in many tasks in information retrieval such as search, query suggestion, automatic summarization and image finding. Many approaches have been suggested, based on lexical matching, handcrafted patterns, syntactic parse trees, external sources of structured semantic knowledge and distributional semantics. However, lexical features, like string matching, do not capture semantic similarity beyond a trivial level. Furthermore, handcrafted patterns and external sources of structured semantic knowledge cannot be assumed to be available in all circumstances and for all domains. Lastly, approaches depending on parse trees are restricted to syntactically well-formed texts, typically of one sentence in length.
 We investigate whether determining short text similarity is possible using only semantic features---where by semantic we mean, pertaining to a representation of meaning---rather than relying on similarity in lexical or syntactic representations. We use word embeddings, vector representations of terms, computed from unlabelled data, that represent terms in a semantic space in which proximity of vectors can be interpreted as semantic similarity.
 We propose to go from word-level to text-level semantics by combining insights from methods based on external sources of semantic knowledge with word embeddings. A novel feature of our approach is that an arbitrary number of word embedding sets can be incorporated. We derive multiple types of meta-features from the comparison of the word vectors for short text pairs, and from the vector means of their respective word embeddings. The features representing labelled short text pairs are used to train a supervised learning algorithm. We use the trained model at testing time to predict the semantic similarity of new, unlabelled pairs of short texts
 We show on a publicly available evaluation set commonly used for the task of semantic similarity that our method outperforms baseline methods that work under the same conditions.

The web is a huge source of valuable information. However, in recent times, there is an increasing trend towards false claims in social media, other web-sources, and even in news. Thus, factchecking websites have become increasingly popular to identify such misinformation based on manual analysis. Recent research proposed methods to assess the credibility of claims automatically. However, there are major limitations: most works assume claims to be in a structured form, and a few deal with textual claims but require that sources of evidence or counter-evidence are easily retrieved from the web. None of these works can cope with newly emerging claims, and no prior method can give user-interpretable explanations for its verdict on the claim’s credibility. This paper overcomes these limitations by automatically assessing the credibility of emerging claims, with sparse presence in web-sources, and generating suitable explanations from judiciously selected sources. To this end, we retrieve diverse articles about the claim, and model the mutual interaction between: the stance (i.e., support or refute) of the sources, the language style of the articles, the reliability of the sources, and the claim’s temporal footprint on the web. Extensive experiments demonstrate the viability of our method and its superiority over prior works. We show that our methods work well for early detection of emerging claims, as well as for claims with limited presence on the web and social media.

Most existing approaches to Twitter sentiment analysis assume that sentiment is explicitly expressed through affective words. Nevertheless, sentiment is often implicitly expressed via latent semantic relations, patterns and dependencies among words in tweets. In this paper, we propose a novel approach that automatically captures patterns of words of similar contextual semantics and sentiment in tweets. Unlike previous work on sentiment pattern extraction, our proposed approach does not rely on external and fixed sets of syntactical templates/patterns, nor requires deep analyses of the syntactic structure of sentences in tweets. We evaluate our approach with tweetand entity-level sentiment analysis tasks by using the extracted semantic patterns as classification features in both tasks. We use 9 Twitter datasets in our evaluation and compare the performance of our patterns against 6 state-of-the-art baselines. Results show that our patterns consistently outperform all other baselines on all datasets by 2.19% at the tweet-level and 7.5% at the entity-level in average F-measure.

In online social networks, users openly interact, share content, and endorse each other. Although the data is interconnected, previous research has primarily focused on modeling the social network behavior separately from the textual content. Here we model the data in a holistic way, taking into account connections between social behavior and content. Specifically, we define multiple decision tasks over the relationships between users and the content generated by them. We show, on a real world dataset, that a learning a joint embedding (over user characteristics and language) and using joint prediction (based on intraand inter-task constraints) produces consistent gains over (1) learning specialized embeddings, and (2) predicting locally w.r.t. a single task, with or without constraints.

We propose a graph-based mechanism to extract rich-emotion bearing patterns, which fosters a deeper analysis of online emotional expressions, from a corpus. The patterns are then enriched with word embeddings and evaluated through several emotion recognition tasks. Moreover, we conduct analysis on the emotion-oriented patterns to demonstrate its applicability and to explore its properties. Our experimental results demonstrate that the proposed techniques outperform most stateof-the-art emotion recognition techniques.

Semantic matching is of central importance to many natural language tasks [2, 28]. A successful matching algorithm needs to adequately model the internal structures of language objects and the interaction between them. As a step toward this goal, we propose convolutional neural network models for matching two sentences, by adapting the convolutional strategy in vision and speech. The proposed models not only nicely represent the hierarchical structures of sentences with their layerby-layer composition and pooling, but also capture the rich matching patterns at different levels. Our models are rather generic, requiring no prior knowledge on language, and can hence be applied to matching tasks of different nature and in different languages. The empirical study on a variety of matching tasks demonstrates the efficacy of the proposed model on a variety of matching tasks and its superiority to competitor models.

This paper investigates the effect of various types of linguistic features (lexical, syntactic and semantic) for training classifiers to detect threats of violence in a corpus of YouTube comments. Our results show that combinations of lexical features outperform the use of more complex syntactic and semantic features for this task.

Posts on Twitter allow users to express ideas and opinions in a very dynamic way. The volume of data available is incredibly high in this support, and it may provide relevant clues regarding the judgement of the public on certain product, event, service etc. While in standard sentiment analysis the most common task is to classify the utterances according to their polarity, it is clear that detecting ironic senses represent a big challenge for Natural Language Processing. By observing a corpus constitued by tweets, we propose a set of patterns that might suggest ironic/sarcastic statements. Thus, we developed special clues for irony detection, through the implementation and evaluation of a set of patterns.

The classification of a real text should not be necessarily treated as a binary or multi-class classification, since the text may belong to one or more labels. This type of problem is called multi-label classification. In this paper, we propose the use of latent semantic indexing to text representation, convolutional neural networks to feature extraction and a single multi layer perceptron for multi-label classification in real text data. The experiments show that the model outperforms state of the art techniques when the dataset has long documents, and we observe that the precision is poor when the size of the texts is small.


Question: Cosine Siamese networks for identifying stance in textual data.

This paper presents an innovative approach for paraphrase detection utilizing dynamic pooling and unfolding recursive autoencoders (RAEs). Paraphrase detection involves assessing two sentences to ascertain if they convey the same meaning, necessitating a comprehensive syntactic and semantic analysis. Our method employs unsupervised RAEs, which are distinctive due to a novel unfolding objective. These RAEs learn feature vectors for phrases within syntactic trees, facilitating the measurement of word and phrase similarity between two sentences. Given that sentences can vary in length, the resulting similarity matrix is correspondingly variable in size. To address this, we introduce a dynamic pooling layer that computes a consistent representation from these variable-sized matrices. This pooled representation is subsequently fed into a classifier. Our approach surpasses other cutting-edge techniques on the challenging MSRP paraphrase dataset, demonstrating its efficacy in accurately detecting paraphrases.

In an effort to assist factcheckers in the process of factchecking, we tackle the claim detection task, one of the necessary stages prior to determining the veracity of a claim. It consists of identifying the set of sentences, out of a long text, deemed capable of being factchecked. This paper is a collaborative work between Full Fact, an independent factchecking charity, and academic partners. Leveraging the expertise of professional factcheckers, we develop an annotation schema and a benchmark for automated claim detection that is more consistent across time, topics and annotators than previous approaches. Our annotation schema has been used to crowdsource the annotation of a dataset with sentences from UK political TV shows. We introduce an approach based on universal sentence representations to perform the classification, achieving an F1 score of 0.83, with over 5% relative improvement over the state-of-the-art methods ClaimBuster and ClaimRank. The system was deployed in production and received positive user feedback.

Aspect-level sentiment classification aims at identifying the sentiment polarity of specific target in its context. Previous approaches have realized the importance of targets in sentiment classification and developed various methods with the goal of precisely modeling their contexts via generating target-specific representations. However, these studies always ignore the separate modeling of targets. In this paper, we argue that both targets and contexts deserve special treatment and need to be learned their own representations via interactive learning. Then, we propose the interactive attention networks (IAN) to interactively learn attentions in the contexts and targets, and generate the representations for targets and contexts separately. With this design, the IAN model can well represent a target and its collocative context, which is helpful to sentiment classification. Experimental results on SemEval 2014 Datasets demonstrate the effectiveness of our model.

There is a relationship between what we say and where we say it. Word embeddings are usually trained assuming that semantically-similar words occur within the same textual contexts. We investigate the extent to which semantically-similar words occur within the same geospatial contexts. We enrich a corpus of geolocated Twitter posts with physical data derived from Google Places and OpenStreetMap, and train word embeddings using the resulting geospatial contexts. Intrinsic evaluation of the resulting vectors shows that geographic context alone does provide useful information about semantic relatedness.

We consider the ROC story cloze task (Mostafazadeh et al., 2016) and present several findings. We develop a model that uses hierarchical recurrent networks with attention to encode the sentences in the story and score candidate endings. By discarding the large training set and only training on the validation set, we achieve an accuracy of 74.7%. Even when we discard the story plots (sentences before the ending) and only train to choose the better of two endings, we can still reach 72.5%. We then analyze this “ending-only” task setting. We estimate human accuracy to be 78% and find several types of clues that lead to this high accuracy, including those related to sentiment, negation, and general ending likelihood regardless of the story context.

We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the architecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification.

Document level sentiment classification remains a challenge: encoding the intrinsic relations between sentences in the semantic meaning of a document. To address this, we introduce a neural network model to learn vector-based document representation in a unified, bottom-up fashion. The model first learns sentence representation with convolutional neural network or long short-term memory. Afterwards, semantics of sentences and their relations are adaptively encoded in document representation with gated recurrent neural network. We conduct document level sentiment classification on four large-scale review datasets from IMDB and Yelp Dataset Challenge. Experimental results show that: (1) our neural model shows superior performances over several state-of-the-art algorithms; (2) gated recurrent neural network dramatically outperforms standard recurrent neural network in document modeling for sentiment classification.1

In multi-sense word embeddings, contextual variations in corpus may cause a univocal word to be embedded into different sense vectors. Shi et al. (2016) show that this kind of pseudo multi-senses can be eliminated by linear transformations. In this paper, we show that pseudo multi-senses may come from a uniform and meaningful phenomenon such as subjective and sentimental usage, though they are seemingly redundant. In this paper, we present an unsupervised algorithm to find a linear transformation which can minimize the transformed distance of a group of sense pairs. The major shrinking direction of this transformation is found to be related with subjective shift. Therefore, we can not only eliminate pseudo multi-senses in multisense embeddings, but also identify these subjective senses and tag the subjective and sentimental usage of words in the corpus automatically.

Sentiment analysis is the Natural Language Processing (NLP) task dealing with the detection and classification of sentiments in texts. While some tasks deal with identifying presence of sentiment in text (Subjectivity analysis), other tasks aim at determining the polarity of the text categorizing them as positive, negative and neutral. Whenever there is presence of sentiment in text, it has a source (people, group of people or any entity) and the sentiment is directed towards some entity, object, event or person. Sentiment analysis tasks aim to determine the subject, the target and the polarity or valence of the sentiment. In our work, we try to automatically extract sentiment (positive or negative) from Facebook posts using a machine learning approach. While some works have been done in code-mixed social media data and in sentiment analysis separately, our work is the first attempt (as of now) which aims at performing sentiment analysis of code-mixed social media text. We have used extensive pre-processing to remove noise from raw text. Multilayer Perceptron model has been used to determine the polarity of the sentiment. We have also developed the corpus for this task by manually labelling Facebook posts with their associated sentiments.

Argumentation mining (AM) requires the identification of complex discourse structures and has lately been applied with success monolingually. In this work, we show that the existing resources are, however, not adequate for assessing cross-lingual AM, due to their heterogeneity or lack of complexity. We therefore create suitable parallel corpora by (human and machine) translating a popular AM dataset consisting of persuasive student essays into German, French, Spanish, and Chinese. We then compare (i) annotation projection and (ii) bilingual word embeddings based direct transfer strategies for cross-lingual AM, finding that the former performs considerably better and almost eliminates the loss from cross-lingual transfer. Moreover, we find that annotation projection works equally well when using either costly human or cheap machine translations. Our code and data are available at http://github.com/UKPLab/ coling2018-xling_argument_mining.

Sentiment analysis of microblog texts has drawn lots of attention in both the academic and industrial fields. However, most of the current work only focuses on polarity classification. In this paper, we present an opinion mining system for Chinese microblogs called CMiner. Instead of polarity classification, CMiner focuses on more complicated opinion mining tasks - opinion target extraction and opinion summarization. Novel algorithms are developed for the two tasks and integrated into the end-to-end system. CMiner can help to effectively understand the users' opinion towards different opinion targets in a microblog topic. Specially, we develop an unsupervised label propagation algorithm for opinion target extraction. The opinion targets of all messages in a topic are collectively extracted based on the assumption that similar messages may focus on similar opinion targets. In addition, we build an aspect-based opinion summarization framework for microblog topics. After getting the opinion targets of all the microblog messages in a topic, we cluster the opinion targets into several groups and extract representative targets and summaries for each group. A co-ranking algorithm is proposed to rank both the opinion targets and microblog sentences simultaneously. Experimental results on a benchmark dataset show the effectiveness of our system and the algorithms.


Question: Cosine Siamese networks for identifying stance in text.

This article explores the effectiveness of various vector-based representations for capturing the compositional meaning of phrases and sentences through distributional methods. We compare a range of approaches, from simple to complex, evaluating their ability to model semantic composition. Some strategies are superficial, relying on basic operations, while others delve deeper, utilizing syntactic structures, parameter learning, or extensive corpora for training. Surprisingly, our findings indicate that simpler methods perform comparably to more resource-intensive techniques when applied to two specific tasks: (1) measuring phrase similarity and (2) detecting paraphrases. The critical factor appears to be the compatibility between the meaning representation and the compositional method, rather than the size of the training data or the dimensionality of the generated vectors. This suggests that, in certain scenarios, sophisticated models may not necessarily outperform more straightforward ones, particularly when the goal is to accurately represent and compare semantic meaning.

Nowadays, it is a heated topic for many industries to build automatic question-answering (QA) systems. A key solution to these QA systems is to retrieve from a QA knowledge base the most similar question of a given question, which can be reformulated as a paraphrase identification (PI) or a natural language inference (NLI) problem. However, most existing models for PI and NLI have at least two problems: They rely on a large amount of labeled data, which is not always available in real scenarios, and they may not be efficient for industrial applications. In this paper, we study transfer learning for the PI and NLI problems, aiming to propose a general framework, which can effectively and efficiently adapt the shared knowledge learned from a resource-rich source domain to a resource-poor target domain. Specifically, since most existing transfer learning methods only focus on learning a shared feature space across domains while ignoring the relationship between the source and target domains, we propose to simultaneously learn shared representations and domain relationships in a unified framework. Furthermore, we propose an efficient and effective hybrid model by combining a sentence encoding-based method and a sentence interaction-based method as our base model. Extensive experiments on both paraphrase identification and natural language inference demonstrate that our base model is efficient and has promising performance compared to the competing models, and our transfer learning method can help to significantly boost the performance. Further analysis shows that the inter-domain and intra-domain relationship captured by our model are insightful. Last but not least, we deploy our transfer learning model for PI into our online chatbot system, which can bring in significant improvements over our existing system. Finally, we launch our new system on the chatbot platform Eva in our E-commerce site AliExpress.

The Winograd Schema Challenge targets pronominal anaphora resolution problems which require the application of cognitive inference in combination with world knowledge. These problems are easy to solve for humans but most difficult to solve for machines. Computational models that previously addressed this task rely on syntactic preprocessing and incorporation of external knowledge by manually crafted features. We address the Winograd Schema Challenge from a new perspective as a sequence ranking task, and design a Siamese neural sequence ranking model which performs significantly better than a random baseline, even when solely trained on sequences of words. We evaluate against a baseline and a state-of-the-art system on two data sets and show that anonymization of noun phrase candidates strongly helps our model to generalize.

The technology of opinion extraction allows users to retrieve and analyze people’s opinions scattered over Web documents. We define an opinion unit as a quadruple consisting of the opinion holder, the subject being evaluated, the part or the attribute in which the subject is evaluated, and the value of the evaluation that expresses a positive or negative assessment. We use this definition as the basis for our opinion extraction task. We focus on two important subtasks of opinion extraction: (a) extracting aspect-evaluation relations, and (b) extracting aspect-of relations, and we approach each task using methods which combine contextual and statistical clues. Our experiments on Japanese weblog posts show that the use of contextual clues improve the performance for both tasks.

This article introduces a sentiment analysis approach that adopts the way humans read, interpret, and extract sentiment from text. Our motivation builds on the assumption that human interpretation should lead to the most accurate assessment of sentiment in text. We call this automated process Human Reading for Sentiment (HRS). Previous research in sentiment analysis has produced many frameworks that can fit one or more of the HRS aspects; however, none of these methods has addressed them all in one approach. HRS provides a meta-framework for developing new sentiment analysis methods or improving existing ones. The proposed framework provides a theoretical lens for zooming in and evaluating aspects of any sentiment analysis method to identify gaps for improvements towards matching the human reading process. Key steps in HRS include the automation of humans low-level and high-level cognitive text processing. This methodology paves the way towards the integration of psychology with computational linguistics and machine learning to employ models of pragmatics and discourse analysis for sentiment analysis. HRS is tested with two state-of-the-art methods; one is based on feature engineering, and the other is based on deep learning. HRS highlighted the gaps in both methods and showed improvements for both.

Recent years have brought a symbolic growth in the volume of research in Sentiment Analysis, mostly on highly subjective text types like movie or product reviews. The main difference these texts have with news articles is that their target is apparently defined and unique across the text. Thence while dealing with news articles, we performed three subtasks namely identifying the target; separation of good and bad news content from the good and bad sentiment expressed on the target and analysis of clearly marked opinion that is expressed explicitly, not needing interpretation or the use of world knowledge. On concluding these tasks, we present our work on mining opinions about three different Indian political parties during elections in the year 2009. We built a Corpus of 689 opinion-rich instances from three different English dailies namely The Hindu, Times of India and Economic Times extracted from 02/ 01/ 2009 to 05/ 01/ 2009 (MM/ DD/ YY). In which (a) we tested the relative suitability of various sentiment analysis methods (both machine learning and lexical based) and (b) we attempted to separate positive or negative opinion from good or bad news. Evaluation includes comparison of three sentiment analysis methods (two machine learning based and one lexical based) and analyzing the choice of certain words used in political text which influence the Sentiments of public in polls. This preliminary experiment will benefit in predicting and forecasting the winning party in forthcoming Indian elections 2014.

In online social networks, users openly interact, share content, and endorse each other. Although the data is interconnected, previous research has primarily focused on modeling the social network behavior separately from the textual content. Here we model the data in a holistic way, taking into account connections between social behavior and content. Specifically, we define multiple decision tasks over the relationships between users and the content generated by them. We show, on a real world dataset, that a learning a joint embedding (over user characteristics and language) and using joint prediction (based on intraand inter-task constraints) produces consistent gains over (1) learning specialized embeddings, and (2) predicting locally w.r.t. a single task, with or without constraints.

The Fake News Challenge is a competition to classify articles as agreeing, disagreeing, discussing, or having no relation to their headlines. The competition provides an annotated dataset (FNC-1) for training supervised models as well as a baseline model to compare against. We cast our fake news problem as a textual entailment task by using multiple Bidirectional LSTMs and an attention mechanism to predict the entailment of the articles to their paired headlines. To tune our model, we adjust a multilayer perceptron predictor layer’s width as well as the amount of dropout and regularization to control overfitting. We evaluate the effectiveness of our model against the baseline and compare to implementations of solutions to the same FNC-1 stance classification task.

The paper presents a language dependent model for classification of statements into ironic and non-ironic. The model uses various language resources: morphological dictionaries, sentiment lexicon, lexicon of markers and a WordNet based ontology. This approach uses various features: antonymous pairs obtained using the reasoning rules over the Serbian WordNet ontology (R), antonymous pairs in which one member has positive sentiment polarity (PPR), polarity of positive sentiment words (PSP), ordered sequence of sentiment tags (OSA), Part-of-Speech tags of words (POS) and irony markers (M). The evaluation was performed on two collections of tweets that had been manually annotated according to irony. These collections of tweets as well as the used language resources are in the Serbian language (or one of closely related languages --Bosnian/Croatian/Montenegrin). The best accuracy of the developed classifier was achieved for irony with a set of 5 features -- (PPR, PSP, POS, OSA, M) -- acc = 86.1%, while for sarcasm the best results were achieved with the set (R, PSP, POS, OSA, M) -- acc = 72.8.

The automatic recognition of the rhetorical function of citations in scientific text has many applications, from improvement of impact factor calculations to text summarisation and more informative citation indexers. Citation function is defined as the author’s reason for citing a given paper (e.g. acknowledgement of the use of the cited method). We show that our annotation scheme for citation function is reliable, and present a supervised machine learning framework to automatically classify citation function, which uses several shallow and linguistically-inspired features. We find, amongst other things, a strong relationship between citation function and sentiment classification.

The given paper describes modern approach to the task of sentiment analysis of movie reviews by using deep learning recurrent neural networks and decision trees. These methods are based on statistical models, which are in a nutshell of machine learning algorithms. The fertile area of research is the application of Google's algorithm Word2Vec presented by Tomas Mikolov, Kai Chen, Greg Corrado and Jeffrey Dean in 2013. The main idea of Word2Vec is the representations of words with the help of vectors in such manner that semantic relationships between words preserved as basic linear algebra operations. The extra advantage of the mentioned algorithm above the alternatives is computational efficiency. This paper focuses on using Word2Vec model for text classification by their sentiment type.


Question: Cosine Siamese Networks for stance identification in text.

This paper presents a new method for enhancing text similarity measures through discriminative projections. Unlike conventional measures that treat terms as solely similar to themselves without accounting for semantic relationships, our approach introduces a projection of raw term vectors into a shared, lower-dimensional space. This is achieved by optimizing a matrix that minimizes the loss of a chosen similarity function, such as cosine similarity, between the projected vectors. Our technique is designed to handle extensive training data efficiently in high-dimensional environments. When evaluated on two distinct tasks—cross-lingual document retrieval and ad relevance assessment—our method surpasses current state-of-the-art techniques. Notably, it maintains high accuracy even at reduced dimensions, offering significant computational advantages.

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while ‘multi-sense’ methods have been proposed and tested on artificial wordsimilarity tasks, we don’t know if they improve real natural language understanding tasks. In this paper we introduce a multisense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language un-

We present a novel technique for learning semantic representations, which extends the distributional hypothesis to multilingual data and joint-space embeddings. Our models leverage parallel data and learn to strongly align the embeddings of semantically equivalent sentences, while maintaining sufficient distance between those of dissimilar sentences. The models do not rely on word alignments or any syntactic information and are successfully applied to a number of diverse languages. We extend our approach to learn semantic representations at the document level, too. We evaluate these models on two cross-lingual document classification tasks, outperforming the prior state of the art. Through qualitative analysis and the study of pivoting effects we demonstrate that our representations are semantically plausible and can capture semantic relationships across languages without parallel data.

Given a question and a set of answer candidates, answer triggering determines whether the candidate set contains any correct answers. If yes, it then outputs a correct one. In contrast to existing pipeline methods which first consider individual candidate answers separately and then make a prediction based on a threshold, we propose an end-to-end deep neural network framework, which is trained by a novel group-level objective function that directly optimizes the answer triggering performance. Our objective function penalizes three potential types of error and allows training the framework in an end-to-end manner. Experimental results on the WIKIQA benchmark show that our framework outperforms the state of the arts by a 6.6% absolute gain under F1 measure1.

We propose an effective multitask learning setup for reducing distant supervision noise by leveraging sentence-level supervision. We show how sentence-level supervision can be used to improve the encoding of individual sentences, and to learn which input sentences are more likely to express the relationship between a pair of entities. We also introduce a novel neural architecture for collecting signals from multiple input sentences, which combines the benefits of attention and maxpooling. The proposed method increases AUC by 10% (from 0.261 to 0.284), and outperforms recently published results on the FBNYT dataset.

Identification of intensity ordering among polar (positive or negative) words which have the same semantics can lead to a finegrained sentiment analysis. For example, master, seasoned and familiar point to different intensity levels, though they all convey the same meaning (semantics), i.e., expertise: having a good knowledge of. In this paper, we propose a semisupervised technique that uses sentiment bearing word embeddings to produce a continuous ranking among adjectives that share common semantics. Our system demonstrates a strong Spearman’s rank correlation of 0.83 with the gold standard ranking. We show that sentiment bearing word embeddings facilitate a more accurate intensity ranking system than other standard word embeddings (word2vec and GloVe). Word2vec is the state-of-the-art for intensity ordering task.

Opinion question answering is a challenging task for natural language processing. In this paper, we discuss a necessary component for an opinion question answering system: separating opinions from fact, at both the document and sentence level. We present a Bayesian classifier for discriminating between documents with a preponderance of opinions such as editorials from regular news stories, and describe three unsupervised, statistical techniques for the significantly harder task of detecting opinions at the sentence level. We also present a first model for classifying opinion sentences as positive or negative in terms of the main perspective being expressed in the opinion. Results from a large collection of news stories and a human evaluation of 400 sentences are reported, indicating that we achieve very high performance in document classification (upwards of 97% precision and recall), and respectable performance in detecting opinions and classifying them at the sentence level as positive, negative, or neutral (up to 91% accuracy).

A key challenge in entity linking is making effective use of contextual information to disambiguate mentions that might refer to different entities in different contexts. We present a model that uses convolutional neural networks to capture semantic correspondence between a mention’s context and a proposed target entity. These convolutional networks operate at multiple granularities to exploit various kinds of topic information, and their rich parameterization gives them the capacity to learn which n-grams characterize different topics. We combine these networks with a sparse linear model to achieve state-of-the-art performance on multiple entity linking datasets, outperforming the prior systems of Durrett and Klein (2014) and Nguyen et al. (2014).1

Document-level multi-aspect sentiment classification aims to predict user’s sentiment polarities for different aspects of a product in a review. Existing approaches mainly focus on text information. However, the authors (i.e. users) and overall ratings of reviews are ignored, both of which are proved to be significant on interpreting the sentiments of different aspects in this paper. Therefore, we propose a model called Hierarchical User Aspect Rating Network (HUARN) to consider user preference and overall ratings jointly. Specifically, HUARN adopts a hierarchical architecture to encode word, sentence, and document level information. Then, user attention and aspect attention are introduced into building sentence and document level representation. The document representation is combined with user and overall rating information to predict aspect ratings of a review. Diverse aspects are treated differently and a multi-task framework is adopted. Empirical results on two real-world datasets show that HUARN achieves state-of-the-art performances.

This paper describes an ensemble system submitted as part of the LSDSem Shared Task 2017 the Story Cloze Test. The main conclusion from our results is that an approach based on semantic similarity alone may not be enough for this task. We test various approaches and compare them with two ensemble systems. One is based on voting and the other on logistic regression based classifier. Our final system is able to outperform the previous state of the art for the Story Cloze test. Another very interesting observation is the performance of sentiment based approach which works almost as well on its own as our final en-

Social media for news consumption is becoming popular nowadays. The low cost, easy access and rapid information dissemination of social media bring benefits for people to seek out news timely. However, it also causes the widespread of fake news, i.e., low-quality news pieces that are intentionally fabricated. The fake news brings about several negative effects on individual consumers, news ecosystem, and even society trust. Previous fake news detection methods mainly focus on news contents for deception classification or claim fact-checking. Recent Social and Psychology studies show potential importance to utilize social media data: 1) Confirmation bias effect reveals that consumers prefer to believe information that confirms their existing stances; 2) Echo chamber effect suggests that people tend to follow likeminded users and form segregated communities on social media. Even though users social engagements towards news on social media provide abundant auxiliary information for better detecting fake news, but existing work exploiting social engagements is rather limited. In this paper, we explore the correlations of publisher bias, news stance, and relevant user engagements simultaneously, and propose a Tri-Relationship Fake News detection framework (TriFN). We also provide two comprehensive real-world fake news datasets to facilitate fake news research. Experiments on these datasets demonstrate the effectiveness of the proposed approach. Introduction People nowadays tend to seek out and consume news from social media rather than traditional news organizations. For example, 62% of U.S. adults get news on social media in 2016, while in 2012, only 49 percent reported seeing news on social media. However, social media for news consumption is a double-edged sword. The quality of news on social media is much lower than traditional news organizations. Large volumes of “fake news”, i.e., those news articles with intentionally false information, are produced online for a variety of purposes, such as financial and political gain (Klein and Wueller 2017; Allcott and Gentzkow 2017). Fake news can have detrimental effects on individuals and the society. First, people may be misled by fake Copyright c © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. http://www.journalism.org/2016/05/26/news-use-acrosssocial-media-platforms-2016/ news and accept false beliefs (Nyhan and Reifler 2010; Paul and Matthews 2016). Second, fake news could change the way people respond to true news. Third, the widespread of fake news could break the trustworthiness of entire news ecosystem. Thus, it is important to detect fake news on social media. Fake news is intentionally written to mislead consumers, which makes it nontrivial to detect simply based on news content. Thus, it is necessary to explore auxiliary information to improve detection. For example, several style-based approaches try to capture the deceptive manipulators originated from the particular writing style of fake news (Rubin and Lukoianova 2015; Potthast et al. 2017). In addition, previous approaches try to aggregate users’ responses from relevant social engagements to infer the veracity of original news (Castillo, Mendoza, and Poblete 2011; Gupta, Zhao, and Han 2012). The news ecosystem on social media involves three basic entities, i.e., news publisher, news and social media users. Figure 1 gives an illustration of such ecosystem. In Figure 1, p1, p2 and p3 are news publishers who publish news a1, . . . , a4 and u1, . . . , u6 are users who have engaged in posting these news. In addition, users with similar interests can also form social links. The tri-relationship among publisher, news, and social engagements contains additional information to help detect fake news. First, sociallogical studies on journalism have theorized the correlation between the partisan bias of publisher and news contents veracity (Gentzkow, Shapiro, and Stone 2014; Entman 2007), where partisan means the perceived bias of the publisher in the selection of how news is reported and covered. For example, in Figure 1, for p1 with extreme left partisan bias and p2 with extreme right partisan bias, to support their own partisan, they have high degree to report fake news, such as a1 and a3; while for a mainstream publisher p3 that has least partisan bias, she has lower degree to manipulate original news events, and is more likely to write true news a4. Thus, exploiting publisher partisan information can bring additional benefits to predict fake news. Second, mining user engagements on social media towards the news also help fake news detection. Different users have different credibility levels on social media, and https://www.nytimes.com/2016/11/28/opinion/fake-newsand-the-internet-shell-game.html? Figure 1: Tri-relationship among publishers, news pieces, and social media users for news dissemination ecosystem. the credibility score which means “the quality of being trustworthy” (Abbasi and Liu 2013) has a strong indication of whether the user is more likely to engage fake news or not. Those less credible users, such as malicious accounts or normal users who are vulnerable to fake news, are more likely to spread fake news. For example, u2 and u4 are users with low credibility scores, and they tend to spread fake news more than other highly credible users. In addition, users tend to form relationships with like-minded people. For example, user u5 and u6 are friends on social media, so they tend to engage those news that confirm their own views, such as a4. Publisher partisan information can bridge the publishernews relationship, while social engagements can capture the news-user relationship. In other words, they provide complementary information that has potential to improve fake news prediction. Thus, it’s important to integrate these two components and model the tri-relationship simultaneously. In this paper, we study the novel problem of exploiting trirelationship for fake news detection. In essence, we need to address the following challenges (1) how to mathematically model the tri-relationship to extract news feature representations; and (2) how to take the advantage of tri-relationship learning for fake news detection. In an attempt to address these challenges, we propose a novel framework TriFN that captures the Tri-relationship for Fake News detection. The main contributions are: • We provide a principled way to model tri-relationship among publisher, news, and relevant user engagements simultaneously; • We propose a novel framework TriFN that exploits trirelationship for fake news prediction; and • We evaluate the effectiveness of the proposed framework for fake news detection through extensive experiments on newly collected real-world datasets. Problem Statement Even though fake news has been existed for long time, there is no agreed definition. In this paper, we follow the definition of fake news that is widely used in recent research (Shu et al. 2017; Zubiaga et al. 2017; Allcott and Gentzkow 2017), which has been shown to be able to 1) provide theoretical and practical values for fake news topic; and 2) eliminate the ambiguities between fake news and related concepts. DEFINITION 1 (FAKE NEWS) Fake news is a news article that is intentionally and verifiably false. Let A = {a1, a2, ..., an} be the set of n news articles and U = {u1, u2, ..., um} be the set of m users on social media engaging the news spreading process. We denote X ∈ R as the news feature matrix. Users can become friends with other users and we use A ∈ {0, 1} to denote the user-user adjacency matrix. On social media sites, users can easily share, comment and discuss about the news pieces. This kind of social engagements provide auxiliary information for fake news detection. We denote the social news engagement matrix as W ∈ {0, 1}, where Wij = 1 indicate that user ui has engaged in the spreading process of the news piece aj ; otherwise Wij = 0. It’s worth mentioning that we focus on those engagements that show that users agree with the news. For example, For example, we only utilize those users that directly post the news, or repost the news without adding comments. More details will introduced in Section . We also denote P = {p1, p2, ..., pl} as the set of l news publishers. In addition, we denote B ∈ R as the publisher-news relation matrix, and Bkj = 1 means news publisher pk publishes the news article aj ; otherwise Bkj = 0. We assume that the partisan labels of some publishers are given and available. We define o ∈ {−1, 0, 1} as the partisan label vectors, where -1, 0, 1 represents left-, neutral-, and right-partisan bias. Similar to previous research (Shu et al. 2017), we treat fake news detection problem as a binary classification problem. In other words, each news piece can be true or fake, and we use y = {y1;y2; ...;yn} ∈ R n×1 to represent the labels, and yj = 1 means news piece aj is fake news; yj = −1 means true news. With the notations given above, the problem is formally defined as, Given news article feature matrix X, user adjacency matrix A, user social engagement matrix W, publishernews publishing matrix B, publisher partisan label vector o, and partial labeled news vector yL, we aim to predict remaining unlabeled news label vector yU . A Tri-Relationship Embedding Framework In this section, we propose a semi-supervised detection framework by exploiting tri-relationship. The idea of modeling tri-relationship is demonstrated in Figure 2. Specifically, we first introduce the news latent feature embedding from news content, and then show how to model user social engagements and publisher partisan separately; At last, we integrate the components to model tri-relationship and provide a semi-supervised detection framework. A Basic Model for News Content Embedding The inherent manipulators of fake news can be reflected in the news content. Thus, it’s important to extract


Question: An inverse kinematic method for infrared optical finger tracking.

This paper introduces an innovative, efficient technique for fitting ellipses to scattered datasets, addressing limitations of previous methods that either focused on general conics or were computationally intensive. The new approach minimizes the algebraic distance while enforcing the ellipticity constraint through a normalization factor, ensuring that the output is always an ellipse, even with poor quality data. The technique boasts several key advantages: (i) It is specifically tailored for ellipse fitting, ensuring that the output remains an ellipse regardless of data quality; (ii) It can be naturally solved using a generalized eigensystem, making it highly robust; (iii) It is exceptionally efficient and straightforward to implement. The paper contrasts this method with alternative approaches, highlighting its resilience through various examples where non-ellipse-specific methods would either fail or necessitate time-consuming iterative adjustments. To facilitate its adoption, the source code for the algorithm is provided, and a demonstration is accessible online. This technique represents a significant advancement in ellipse fitting, particularly in scenarios where data quality is uncertain or suboptimal.

In this paper, we concentrate on the design of a new whole-sensitive robot arm enabling torque measurement in each joint by means of developed optical torque sensors. When the contact of arm with an object occurs, local impedance algorithm provides active compliance of corresponding robot arm joint. Thus, the whole structure of the manipulator can safely interact with unstructured environment. The detailed design procedure of the 4-DOF robot arm and optical torque sensors is described in the paper. The gravity compensation algorithm was elaborated and verified by implementation of the local impedance control of a simple two-link arm manipulator

In Proc. of IEEE Conf. on CVPR’2000, Vol.II, pp.88-94, Hilton Head Island, SC, 2000 Since human hand is highly articulated and deformable, hand posture recognition is a challenging example in the research of view-independent object recognition. Due to the difficulties of the modelbased approach, the appearance-based learning approach is promising to handle large variation in visual inputs. However, the generalization of many proposed supervised learning methods to this problem often suffers from the insufficiency of labeled training data. This paper describes an approach to alleviate this difficulty by adding a large unlabeled training set. Combining supervised and unsupervised learning paradigms, a novel and powerful learning approach, the Discriminant-EM (D-EM) algorithm, is proposed in this paper to handle the case of small labeled training set. Experiments show that D-EM outperforms many other learning methods. Based on this approach, we implement a gesture interface to recognize a set of predefined gesture commands, and it is also extended to hand detection. This algorithm can also apply to other object recognition tasks.

Gesture recognition is a well-establish topic of research that is widely adopted for a broad range of applications. For instance, it can be exploited for the command of a smart environment without any remote control unit or even for the recognition of human activities from a set of video cameras deployed in strategic position. Many researchers working on assistive smart home, such as our team, believe that the intrusiveness of that technology will prevent the future adoption and commercialization of smart homes. In this paper, we propose a novel gesture recognition algorithm that is solely based on passive RFID technology. This technology enables the localization of small tags that can be embedded in everyday life objects (a cup or a book, for instance) while remaining non intrusive. However, until now, this technology has been largely ignored by researchers on gesture recognition, mostly because it is easily disturbed by noise (metal, human, etc.) and offer limited precision. Despite these issues, the localization algorithms have improved over the years, and our recent efforts resulted in a real-time tracking algorithm with a precision approaching 14cm. With this, we developed a gesture recognition algorithm able to perform segmentation of gestures and prediction on a spatio-temporal data series. Our new model, exploiting works on qualitative spatial reasoning, achieves recognition of 91%. Our goal is to ultimately use that knowledge for both human activity recognition and errors detection.

The Kinect has primarily been used as a gesture-driven device for motion-based controls. To date, Kinect-based research has predominantly focused on improving tracking and gesture recognition across a wide base of users. In this paper, we propose to use the Kinect for biometrics; rather than accommodating a wide range of users we exploit each user's uniqueness in terms of gestures. Unlike pure biometrics, such as iris scanners, face detectors, and fingerprint recognition which depend on irrevocable biometric data, the Kinect can provide additional revocable gesture information. We propose a dynamic time-warping (DTW) based framework applied to the Kinect's skeletal information for user access control. Our approach is validated in two scenarios: user identification, and user authentication on a dataset of 20 individuals performing 8 unique gestures. We obtain an overall 4.14%, and 1.89% Equal Error Rate (EER) in user identification, and user authentication, respectively, for a gesture and consistently outperform related work on this dataset. Given the natural noise present in the real-time depth sensor this yields promising results.

In this paper, a new method for eye gaze tracking is proposed under natural head movement. In this method, Local-Binary-Pattern Texture Feature (LBP) is adopted to calculate the eye features according to the characteristic of the eye, and a precise Binocular Vision approach is used to detect the space coordinate of the eye. The combined features of space coordinates and LBP features of the eyes are fed into Support Vector Regression (SVR) to match the gaze mapping function, in the hope of tracking gaze direction under natural head movement. The experimental results prove that the proposed method can determine the gaze direction accurately.

Grabbing and manipulating virtual objects is an important user interaction for immersive virtual environments. We present implementations and discussion of six techniques which allow manipulation of remote objects. A user study of these techniques was performed which revealed their characteristics and deficiencies, and led to the development of a new class of techniques. These hybrid techniques provide istinct advantages in terms of ease of use and efficiency because they consider the tasks of grabbing and manipulation separately. CR Categories and Subject Descr iptors: I.3.7 [Computer Graphics]:Three-Dimensional Graphics and Realism Virtual Reality; I.3.6 [Computer Graphics]:Methodology and Techniques Interaction Techniques.

We constructed an acoustic, gesture-based recognition system called Multiwave, which leverages the Doppler Effect to translate multidimensional movements into user interface commands. Our system only requires the use of two speakers and a microphone to be operational. Since these components are already built in to most end user systems, our design makes gesture-based input more accessible to a wider range of end users. By generating a known high frequency tone from multiple speakers and detecting movement using changes in the sound waves, we are able to calculate a Euclidean representation of hand velocity that is then used for more natural gesture recognition and thus, more meaningful interaction mappings. We present the results of a user study of Multiwave to evaluate recognition rates for different gestures and report accuracy rates comparable to or better than the current state of the art. We also report subjective user feedback and some lessons learned from our system that provide additional insight for future applications of multidimensional gesture recognition.

The recent introduction of depth cameras like Leap Motion Controller allows researchers to exploit the depth information to recognize hand gesture more robustly. This paper proposes a novel hand gesture recognition system with Leap Motion Controller. A series of features are extracted from Leap Motion tracking data, we feed these features along with HOG feature extracted from sensor images into a multiclass SVM classifier to recognize performed gesture, dimension reduction and feature weighted fusion are also discussed. Our results show that our model is much more accurate than previous work.

Three-dimensional hand gesture recognition has attracted increasing research interests in computer vision, pattern recognition, and human-computer interaction. The emerging depth sensors greatly inspired various hand gesture recognition approaches and applications, which were severely limited in the 2D domain with conventional cameras. This paper presents a survey of some recent works on hand gesture recognition using 3D depth sensors. We first review the commercial depth sensors and public data sets that are widely used in this field. Then, we review the state-of-the-art research for 3D hand gesture recognition in four aspects: 1) 3D hand modeling; 2) static hand gesture recognition; 3) hand trajectory gesture recognition; and 4) continuous hand gesture recognition. While the emphasis is on 3D hand gesture recognition approaches, the related applications and typical systems are also briefly summarized for practitioners.

Person-independent and pose-invariant estimation of eye-gaze is important for situation analysis and for automated video annotation. We propose a fast cascade regression based method that first estimates the location of a dense set of markers and their visibility, then reconstructs face shape by fitting a part-based 3D model. Next, the reconstructed 3D shape is used to estimate a canonical view of the eyes for 3D gaze estimation. The model operates in a feature space that naturally encodes local ordinal properties of pixel intensities leading to photometric invariant estimation of gaze. To evaluate the algorithm in comparison with alternative approaches, three publicly-available databases were used, Boston University Head Tracking, Multi-View Gaze and CAVE Gaze datasets. Precision for head pose and gaze averaged 4 degrees or less for pitch, yaw, and roll. The algorithm outperformed alternative methods in both datasets.


Question: An inverse kinematic method for infrared optical finger tracking.

This paper offers an in-depth exploration of glove-based input devices, highlighting their pivotal role in enhancing human-computer interaction. Traditionally, the use of intermediary devices has often limited the fluidity and naturalness of such interactions. Glove-based input devices, however, capitalize on the inherent manual dexterity of humans, bridging the gap between human capabilities and technological applications. To provide a comprehensive understanding of the field, this document delves into the key hand-tracking technologies and their applications, particularly focusing on the recent advancements that have not been extensively documented in existing literature. The development of glove-based input technologies has surged in recent times, with a multitude of approaches and devices now available. The technologies encompassed within this domain include position tracking, optical tracking, marker systems, silhouette analysis, magnetic tracking, and acoustic tracking. Notable glove technologies that have been commercialized include the Sayre glove, MIT LED glove, Digital Data Entry Glove, DataGlove, Dexterous HandMaster, Power Glove, CyberGlove, and Space Glove. These devices have found diverse applications, ranging from the development of natural interfaces, systems for interpreting signed languages, teleoperation and robotic control, computer-based puppetry, to innovative uses in musical performance. This overview serves as a valuable resource for researchers, developers, and enthusiasts seeking to understand the current landscape of glove-based input devices and their potential future directions.

In this letter, we present a novel gripper, whose design was inspired by chuck clamping devices, for transferring heavy objects and assembling parts precisely in industrial applications. The developed gripper is lightweight (0.9 kg), can manipulate heavy payloads (over 23 kgf), and can automatically align its position and posture via a grasping motion. A fingertip design criterion is presented for the position alignment, while a control strategy is presented for the posture alignment. With one actuator, this gripper realized the above features. This letter describes the mathematical analyses and experiments used to validate these key metrics.

For many years, fingerprints have been captured by pressing a finger against a paper or hard surface. This touch-based fingerprint acquisition introduces some problems such as distortions and deformations in the acquired images, which arise due to the contact of the fingerprint surface with the sensor platen, and degrades the recognition performance. A new touch-less fingerprint technology has been recently introduced to the market, which can address the problems with the contact-based fingerprint systems. In this paper, we propose a new algorithm for unwrapping the acquired 3-D scan of the subject's finger into a 2-D rolled equivalent image. Therefore, The resulting image can be matched with the conventional 2-D scans; it also can be used for matching unwrapped 3-D fingerprints among themselves with the 2-D fingerprint matching algorithms. The algorithm is based on curvature analysis of the 3-D surface. The quality of the resulting image is evaluated and analyzed using NIST fingerprint image software.

Robots using ball(s) as spherical wheels have the advantage of omnidirectional motion. Some of these robots use only a single ball as their wheel and dynamically balance on it. However, driving a ball wheel is not straightforward. These robots usually use one or more motor-driven rollers or wheels in frictional contact with the ball wheel. Some slippage can occur with these schemes, so it is important to measure the actual rotation of the ball wheel for good control. This paper proposes one method for measuring the three dimensional rotation of a sphere, which is applicable to the case of a ball wheel. The system measures surface speed by using two or more optical mouse sensors and transforms them into the angular velocity vector of the ball, followed by integration to give the rotational angle. Experiments showed the correctness of this method, yielding an error of approximately 1%, with a 10 ms response.

In this paper, we have developed a method to tele-control a robot arm to imitate human writing skills using electromyography (EMG) signals. The control method is implemented on the Baxter® robot with a brush attached on the endpoint of its arm to imitate human writing skills, and a MYO sensor is used to measure the surface electromyographic (sEMG) signals generated by contractions of human muscles involved in writing tasks. A haptic device Sensable® Omni is used to control the motion of the Baxter® robot arm, and V-Rep® is employed to simulate the movement of the arm and to calculate the inverse kinematic of Baxter® robot arm. All the communications for Baxter® robot, V-Rep simulator, Omni device and MYO are integrated in MATLAB®/Simulink®. The main test is to make the Baxter® arm following the movement of a human subject when writing with Omni stylus, and the EMG signals are processed using low pass filter and moving average technique to extract the smoothed envelope which is utilised to control the variation of position instead of variation of force along the vertical Z axis, so when the operator writes with force variations, the Baxter® can draw lines with variable thickness. This imitation system is successfully tested to make the Baxter® arm to follow the writing movement of the operator's hand and would have potential applications in more complicated human robot teleoperation tasks such as tele-rehabilitation and tele-surgery.

Recent improvements in ultrasound imaging enable new opportunities for hand pose detection using wearable devices. Ultrasound imaging has remained under-explored in the HCI community despite being non-invasive, harmless and capable of imaging internal body parts, with applications including smart-watch interaction, prosthesis control and instrument tuition. In this paper, we compare the performance of different forearm mounting positions for a wearable ultrasonographic device. Location plays a fundamental role in ergonomics and performance since the anatomical features differ among positions. We also investigate the performance decrease due to cross-session position shifts and develop a technique to compensate for this misalignment. Our gesture recognition algorithm combines image processing and neural networks to classify the flexion and extension of 10 discrete hand gestures with an accuracy above 98%. Furthermore, this approach can continuously track individual digit flexion with less than 5% NRMSE, and also differentiate between digit flexion at different joints.

2D and 3D computer vision systems are frequently being used in automated production to detect and determine the position of objects. Accuracy is important in the production industry, and computer vision systems require structured environments to function optimally. For 2D vision systems, a change in surfaces, lighting and viewpoint angles can reduce the accuracy of a method, maybe even to a degree that it will be erroneous, while for 3D vision systems, the accuracy mainly depends on the 3D laser sensors. Commercially available 3D cameras lack the precision found in high-grade 3D laser scanners, and are therefore not suited for accurate measurements in industrial use. In this paper, we show that it is possible to identify and locate objects using a combination of 2D and 3D cameras. A rough estimate of the object pose is first found using a commercially available 3D camera. Then, a robotic arm with an eye-in-hand 2D camera is used to determine the pose accurately. We show that this increases the accuracy to < 1 mm and < 1◦. This was demonstrated in a real industrial assembly task where high accuracy is required.

In this paper, we use Kinect Xbox 360 sensor to implement the motion control of Baxter robot, a semi humanoid robot with limbs of 7 DOF joints with collision avoidance capabilities. Two different methods using vector approach and inverse kinematics approach have been designed to achieve a given task. Primitive experiments have been carried out to verify the effectiveness of the developed methods. Human motions is captured by Kinect sensor and calculated with Processing Software using SimpleOpenNI wrapper for OpenNI and NITE. UDP protocol is adopted to send reference motion to Baxter robot. Python and RosPy script programming kit is used to calculate joint angles of Baxter robot based on vector approach and inverse kinematics approach. The experimental results demonstrate that both of our proposed approaches have achieved satisfactory performance.

This paper presents an augmented reality based system for hand rehabilitation. In this system, digital data gloves are used to detect the movements of the patients' hands and collect the physical information from the patients. Using augmented reality technology, a highly controllable environment with tasks of different levels of difficulty is provided to the patients for them to perform the rehabilitation exercises gradually. The targets of the exercises can be adjusted dynamically with respect to the physical conditions and progress of the patients. Multimodal feedbacks are provided to facilitate and encourage the patients during the rehabilitation sessions.

In this paper, a graphical, flexible, interactive, and systematic 3D simulation helps facilitate analyzing and previewing kinematics of PA10-7C robot arm in terms of forward kinematics, inverse kinematics, and the Denavit-Hartenberg convention. Modeling and control are of critical importance when the robot arm is used for practical applications. In the paper, the D-H model of PA10-7C robot arm is given first to describe the relationship between two consecutive frames of joints. Based on this D-H model, forward kinematics is calculated efficiently. To describe the relationship between joint angular velocities in the joint space and the end effector's velocities in Cartesian space, Jacobian matrix for PA10-7C robot arm is derived. By Jacobian matrix, inverse kinematics approach is obtained in the paper. VRML-based model of PA10-7C is built in the paper for the simulation. Finally, simulation results are discussed and the paper is concluded.

Many robotic hands or prosthetic hands have been developed in the last several decades, and many use tendon-driven mechanisms for their transmissions. Robotic hands are now built with underactuated mechanisms, which have fewer actuators than degrees of freedom, to reduce mechanical complexity or to realize a biomimetic motion such as flexion of an index finger. The design is heuristic and it is useful to develop design methods for the underactuated mechanisms. This paper classifies mechanisms driven by tendons into three classes, and proposes a design method for them. The two classes are related to underactuated tendon-driven mechanisms, and these have been used without distinction so far. An index finger robot, which has four active tendons and two passive tendons, is developed and controlled with the proposed method.


Question: An infrared optical method for inverse kinematic finger tracking.

This document introduces CHARADE, a novel application designed to enable the remote control of objects and computer interfaces through the interpretation of free-hand gestures. The objective of CHARADE is to facilitate user interaction in computerized reality environments, specifically during presentations, by defining a comprehensive interaction model. This model encompasses a gesture notation system and a set of design guidelines for creating intuitive gestural command sets. The application not only focuses on enhancing computer interaction but also considers the communicative aspect of gestures in social settings and their potential for operating additional devices. By integrating gesture recognition technology, CHARADE empowers users to manipulate digital content and control various devices without the need for physical contact, thereby enriching the user experience in virtual and augmented reality environments. This innovative approach to gesture-based interaction is poised to redefine the way we engage with technology in everyday scenarios, making the control of objects and devices more intuitive and accessible.

This paper describes a novel actuation system for very compact and light-weight robotic devices, like artificial hands. The actuation concept presented here allows the implementation of powerful tendon-based driving systems, using as actuators small-size DC motors characterized by high speed and low torque. After the presentation of the basic concept of this novel actuation system, the constitutive equations of the system are given, validated by means of laboratory tests. Moreover, the problem of tracking a desired actuation force profile is taken into account, considering as load a mass-spring-damper system. A control algorithm based on a second-order sliding manifold has been firstly evaluated by means of simulations, and then validated by experiments. This output-feedback controller has been chosen to guarantee a high level of robustness against disturbances, parameter variations and uncertainties while maintaining a low computational burden.

This work evaluates the feasibility of a motor imagery-based optical brain-computer interface (BCI) for humanoid robot control. The functional near-infrared spectroscopy (fNIRS) based BCI-robot system developed in this study operates through a high-level control mechanism where user specifies a target action through the BCI and the robot performs the set of micro operations necessary to fulfill the identified goal. For the evaluation of the system, four motor imagery tasks (left hand, right hand, left foot, and right foot) were mapped to operational commands (turn left, turn right, walk forward, walk backward) that were sent to the robot in real time to direct the robot navigating a small room. An ecologically valid offline analysis with minimal preprocessing shows that seven subjects could achieve an average accuracy of 32.5 %. This was increased to 43.6 % just by including calibration data from the same day of the robot control using the same cap setup, indicating that day-of calibration following the initial training may be important for BCI control.

Robot manipulator designs are increasingly focused on low cost approaches, especially those envisioned for use in unstructured environments such as households, office spaces and hazardous environments. The cost of angular sensors varies based on the precision offered. For tasks in these environments, millimeter order manipulation errors are unlikely to cause drastic reduction in performance. In this paper, estimates the joint angles of a manipulator using low cost triaxial accelerometers by taking the difference between consecutive acceleration vectors. The accelerometer-based angle is compensated with a uniaxial gyroscope using a complementary filter to give robust measurements. Three compensation strategies are compared: complementary filter, time varying complementary filter, and extended Kalman filter. This sensor setup can also accurately track the joint angle even when the joint axis is parallel to gravity and the accelerometer data does not provide useful information. In order to analyze this strategy, accelerometers and gyroscopes were mounted on one arm of a PR2 robot. The arm was manually moved smoothly through different trajectories in its workspace while the joint angle readings from the on-board optical encoders were compared against the joint angle estimates from the accelerometers and gyroscopes. The low cost angle estimation strategy has a mean error 1.3° over the three joints estimated, resulting in mean end effector position errors of 6.1 mm or less. This system provides an effective angular measurement as an alternative to high precision encoders in low cost manipulators and as redundant measurements for safety in other manipulators.

2D and 3D computer vision systems are frequently being used in automated production to detect and determine the position of objects. Accuracy is important in the production industry, and computer vision systems require structured environments to function optimally. For 2D vision systems, a change in surfaces, lighting and viewpoint angles can reduce the accuracy of a method, maybe even to a degree that it will be erroneous, while for 3D vision systems, the accuracy mainly depends on the 3D laser sensors. Commercially available 3D cameras lack the precision found in high-grade 3D laser scanners, and are therefore not suited for accurate measurements in industrial use. In this paper, we show that it is possible to identify and locate objects using a combination of 2D and 3D cameras. A rough estimate of the object pose is first found using a commercially available 3D camera. Then, a robotic arm with an eye-in-hand 2D camera is used to determine the pose accurately. We show that this increases the accuracy to < 1 mm and < 1◦. This was demonstrated in a real industrial assembly task where high accuracy is required.

We present a survey of the most significant techniques, used in the last few years, concerning the coded structured light methods employed to get 3D information. In fact, depth perception is one of the most important subjects in computer vision. Stereovision is an attractive and widely used method, but, it is rather limited to make 3D surface maps, due to the correspondence problem. The correspondence problem can be improved using a method based on structured light concept, projecting a given pattern on the measuring surfaces. However, some relations between the projected pattern and the reflected one must be solved. This relationship can be directly found codifying the projected light, so that, each imaged region of the projected pattern carries the needed information to solve the correspondence problem. ( 1998 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved. Pattern projection Correspondence problem Active stereo Depth perception Range data Computer vision.

In this letter, using the active dual-mode twisted string actuation (TSA) mechanism and tiny tension sensors on the tendon strings, an anthropomorphic robot hand is newly designed in a compact manner. Thanks to the active dual-mode TSA mechanism, which is a miniaturized transmission, the proposed robot hand dose has a wide range of operation in terms of the grasping force and speed. It experimentally produces maximally the fingertip force 31.3 N and minimally the closing time of 0.5 s in average. Also, tiny tension sensor with the dimension of 4.7 (width) × 4.0 (height) × 10.75 (length) mm is newly presented and embedded at the fingertips in order to measure the tension on the tendon strings, which would allow the grasping force control. The kinetic and kinematic analyses are performed and the performance is verified by experiments.

This report documents the design and implementation of a binocular, foveated active vision system as part of the Cog project at the MIT Artificial Intelligence Laboratory. The active vision system features a 3 degree of freedom mechanical platform that supports four color cameras, a motion control system, and a parallel network of digital signal processors for image processing. To demonstrate the capabilities of the system, we present results from four sample visual-motor tasks. The author receives support from a National Defense Science and Engineering Graduate Fellowship. Support for this project is provided in part by an ONR/ARPA Vision MURI Grant (No. N00014-95-1-0600).

This paper presents a new design of a wireless sensor glove developed for American Sign Language finger spelling gesture recognition. Five contact sensors are installed on the glove, in addition to five flex sensors on the fingers and a 3D accelerometer on the back of the hand. Each pair of flex and contact sensors are combined into the same input channel on the BSN node in order to save the number of channels and the installation area. After which, the signal is analyzed and separated back into flex and contact features by software. With electrical contacts and wirings made of conductive fabric and threads, the glove design has become thinner and more flexible. For validation, ASL finger spelling gesture recognition experiments have been performed on signals collected from six speech-impaired subjects and a normal subject. With the new sensor glove design, the experimental results have shown a significant increase in classification accuracy.

The kinematic analyses, of manipulators and other robotic devices composed of mechanical links, usually depend on the solution of sets of nonlinear equations. There are a variety of both numerical and algebraic techniques available to solve such systems of equations and to give bounds on the number of solutions. These solution methods have also led to an understanding of how special choices of the various structural parameters of a mechanism influence the number of solutions inherent to the kinematic geometry of a given structure. In this paper, results from studying the kinematic geometry of such systems are reviewed, and the three most useful solution techniques are summarized. The solution techniques are polynomial continuation, Gröbner bases, and elimination. We then discuss the results that have been obtained with these techniques in the solution of two basic problems, namely, the inverse kinematics for serial-chain manipulators, and the direct kinematics of in-parallel platform devices. KEY WORDS—inverse kinematics, direct kinematics, manipulators, in-parallel manipulators, series manipulators, polynomial continuation, characteristic polynomial

We present a new motion tracking technique to robustly reconstruct non-rigid geometries and motions from a single view depth input recorded by a consumer depth sensor. The idea is based on the observation that most non-rigid motions (especially human-related motions) are intrinsically involved in articulate motion subspace. To take this advantage, we propose a novel <inline-formula><tex-math notation="LaTeX">$L_0$</tex-math><alternatives> <inline-graphic xlink:href="xu-ieq2-2688331.gif"/></alternatives></inline-formula> based motion regularizer with an iterative solver that implicitly constrains local deformations with articulate structures, leading to reduced solution space and physical plausible deformations. The <inline-formula><tex-math notation="LaTeX">$L_0$</tex-math> <alternatives><inline-graphic xlink:href="xu-ieq3-2688331.gif"/></alternatives></inline-formula> strategy is integrated into the available non-rigid motion tracking pipeline, and gradually extracts articulate joints information online with the tracking, which corrects the tracking errors in the results. The information of the articulate joints is used in the following tracking procedure to further improve the tracking accuracy and prevent tracking failures. Extensive experiments over complex human body motions with occlusions, facial and hand motions demonstrate that our approach substantially improves the robustness and accuracy in motion tracking.


Question: An inverse kinematic method for infrared optical finger tracking.

"Place-it-here": Integrating Voice and Gesture for Enhanced Graphics Interface Interaction

The rapid progression in connected-speech recognition and spatial position sensing technologies has invigorated the belief that voice and gesture inputs can harmoniously merge to create a more intuitive and natural user interaction modality at the graphics interface. This research delves into scenarios where the user directs basic geometric shapes across a large-screen graphics display. By enabling concurrent voice and pointing inputs, the use of pronouns in commands becomes fluid and natural, significantly enhancing the efficiency and expressiveness of user interaction. Simultaneously, gesture input, when complemented by voice, gains a level of accuracy and referencing capability, elevating its potential as a powerful control mechanism. This synergy between voice and gesture not only enriches the user experience but also paves the way for more sophisticated and intuitive interfaces in various application domains.

In this paper, we demonstrate an underactuated finger design and grasping method for precision grasping and manipulation of small objects. Taking inspiration from the human grasping strategy for picking up objects from a flat surface, we introduce the flip-and-pinch task, in which the hand picks up a thin object by flipping it into a stable configuration between two fingers. Despite the fact that finger motions are not fully constrained by the hand actuators, we demonstrate that the hand and fingers can interact with the table surface to produce a set of constraints that result in a repeatable quasi-static motion trajectory. Even when utilizing only open-loop kinematic playback, this approach is shown to be robust to variation in object size and hand position. Variation of up to 20° in orientation and 10 mm in hand height still result in experimental success rates of 80% or higher. These results suggest that the advantages of underactuated, adaptive robot hands can be carried over from basic grasping tasks to more dexterous tasks.

We describea markerlesscamera tracking systemfor augmentedreality thatoperatesin environmentswhich contain oneor more planes. This is a commonspecialcase, which we showsignificantlysimplifiestracking. The result is a practical, reliable, vision-basedtracker. Furthermore, thetrackedplaneimposesa natural referenceframe, sothat thealignmentof thereal andvirtual coordinatesystemsis rather simplerthan would be the casewith a general structure-and-motionsystem.Multiple planescan be tracked, and additional data such as 2D point tracks are easilyincorporated.

Three-dimensional hand gesture recognition has attracted increasing research interests in computer vision, pattern recognition, and human-computer interaction. The emerging depth sensors greatly inspired various hand gesture recognition approaches and applications, which were severely limited in the 2D domain with conventional cameras. This paper presents a survey of some recent works on hand gesture recognition using 3D depth sensors. We first review the commercial depth sensors and public data sets that are widely used in this field. Then, we review the state-of-the-art research for 3D hand gesture recognition in four aspects: 1) 3D hand modeling; 2) static hand gesture recognition; 3) hand trajectory gesture recognition; and 4) continuous hand gesture recognition. While the emphasis is on 3D hand gesture recognition approaches, the related applications and typical systems are also briefly summarized for practitioners.

This paper proposes a static hand gesture recognition method with low computation and memory consumption for wearable applications. The hand contour is chosen as the hand gesture feature and support vector machine is used to classify the feature. A multiscale weighted histogram of contour direction-based direction normalization is proposed to ensure good recognition performance. In order to improve efficiency, the proposed histogram only counts the direction of the contour point to focus on the most significant hand feature in the first-person view of wearable devices. Based on the hand’s anatomy, the proposed histogram is weighted by considering each contour point’s position and direction jointly using the direction-angle map, to ensure robustness. Experimental results show that the proposed method can give a recognition accuracy of 97.1% with a frame rate of 30 fps on a PC.

Previous work on muscle activity sensing has leveraged specialized sensors such as electromyography and force sensitive resistors. While these sensors show great potential for detecting finger/hand gestures, they require additional hardware that adds to the cost and user discomfort. Past research has utilized sensors on commercial devices, focusing on recognizing gross hand gestures. In this work we present Serendipity, a new technique for recognizing unremarkable and fine-motor finger gestures using integrated motion sensors (accelerometer and gyroscope) in off-the-shelf smartwatches. Our system demonstrates the potential to distinguish 5 fine-motor gestures like pinching, tapping and rubbing fingers with an average f1-score of 87%. Our work is the first to explore the feasibility of using solely motion sensors on everyday wearable devices to detect fine-grained gestures. This promising technology can be deployed today on current smartwatches and has the potential to be applied to cross-device interactions, or as a tool for research in fields involving finger and hand motion.

Articulated robots with multiple degrees of freedom, such as humanoid robots, have become popular research platforms in robotics and artificial intelligence. Such robots can perform complex motions, including the balancing, walking, and kicking skills required in the RoboCup robot soccer competition. The design of complex dynamic motions is achievable only through the use of robot kinematics, which is an application of geometry to the study of arbitrary robotic chains. This thesis studies the problems of forward and inverse kinematics for the Aldebaran NAO humanoid robot and presents for the first time a complete analytical solution to both problems with no approximations, including an implementation of a software library for real-time execution. The forward kinematics allow NAO developers to map any configuration of the robot from its own joint space to the three-dimensional physical space, whereas the inverse kinematics provide closedform solutions to finding joint configurations that drive the end effectors of the robot to desired points in the three-dimensional space. The proposed solution was made feasible through a decomposition into five independent problems (head, two arms, two legs), the use of the Denavit-Hartenberg method, and the analytical solution of a non-linear system of equations. The main advantage of the proposed inverse kinematics compared to existing numerical approaches is its accuracy, its efficiency, and the elimination of singularities. The implemented NAO kinematics library, which additionally offers centerof-mass calculation, is demonstrated in two motion design tasks: pointing to the ball and basic balancing. The library has been integrated into the software architecture of the RoboCup team “Kouretes” and is currently being used in various motion design problems, such as dynamic balancing, trajectory following, dynamic kicking, and omnidirectional walking.

For many applications a laser beam needs to be scanned across a plane. Several methods exist to accomplish this.One method, dual galvanometer pre-objective scanning, is particularly well suited to precise laser marking applications. The laser beam impinges on a first mirror mounted on a galvanometer motor, then on a second mirror with its rota tion axis parallel to the laser axis, and passes then through the flat-field objective lens, which focuses the beam onto the image plane. Figure 1 shows a photograph of the SILAMATIK laser marker. This letter investigates what amount of distortion is intro duced and how it can be corrected electronically. We as sume that the lens is axial symmetric. The geometry is that of Fig. 2. Say the first mirror is rotated a mechanical angle a/2 from its rest position and the second mirror an angle 6/2. The direction vector lR of the beam that falls on the lens is

Marc Rioux National Research Council, Canada Institute for Information Technology Visual Information Technology Ottawa, Canada, K1A 0R6 Abstract. We review some usual laser range finding techniques for industrial applications. After outlining the basic principles of triangulation and time of flight [pulsed, phase-shift and frequency modulated continuous wave (FMCW)], we discuss their respective fundamental limitations. Selected examples of traditional and new applications are also briefly presented. © 2001 Society of Photo-Optical Instrumentation Engineers. [DOI: 10.1117/1.1330700]

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon's navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D optical imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions.

This paper presents an augmented reality based system for hand rehabilitation. In this system, digital data gloves are used to detect the movements of the patients' hands and collect the physical information from the patients. Using augmented reality technology, a highly controllable environment with tasks of different levels of difficulty is provided to the patients for them to perform the rehabilitation exercises gradually. The targets of the exercises can be adjusted dynamically with respect to the physical conditions and progress of the patients. Multimodal feedbacks are provided to facilitate and encourage the patients during the rehabilitation sessions.