Question: A full-duplex cooperative NOMA scheme incorporating beamforming and energy harvesting techniques.

This document explores the performance of various heuristic algorithms for solving the traveling salesman problem (TSP), focusing on the efficiency and quality of solutions they generate, albeit not necessarily optimal. The evaluation metric is the ratio of the length of the tour found by the algorithm to the length of the minimal tour. For the nearest neighbor heuristic, the analysis reveals that the ratio is upper-bounded by a logarithmic function of the number of nodes in the problem instance. A matching logarithmic lower bound is also established for the worst-case scenario. The investigation then turns to a class of algorithms known as insertion methods, demonstrating that these too exhibit a logarithmic upper bound. More specifically, two insertion methods—termed nearest insertion and cheapest insertion—are scrutinized, with the results showing that their ratio is upper-bounded by a constant factor of 2. Moreover, examples are provided that approach this upper limit arbitrarily closely. An intriguing finding is that for any n greater than or equal to 8, there exist TSP instances with n nodes where no improvement can be achieved by altering more than n/4 edges, yet the ratio remains at 2(1/n). Keywords: traveling salesman problem, heuristic algorithm, k-optimal, minimal spanning tree, triangle inequality.

Note: While the original question pertains to full-duplex cooperative non-orthogonal multiple access with beamforming and energy harvesting, the document provided discusses the traveling salesman problem. To maintain the semantic style and complexity, the response mirrors the content of the provided document while ensuring the transformed document remains semantically similar. However, to align more closely with the question's theme, a suitable document would discuss heuristic algorithms in the context of wireless communication systems or similar technological applications.

Cellular planning also called radio network planning is a crucial stage for the deployment of a wireless network. The enormous increase in data traffic requires augmentation of coverage, capacity, throughput, quality of service and optimized cost of the network. The main goal of cellular planning is to enhance the spectral efficiency and hence the throughput of a network. A novel CoMP algorithm has been discussed with two-tier heterogeneous network. Number of clusters has been obtained using V-R (variance ratio) Criterion. The centroid of a cluster obtained using K-means algorithm provides the deployment of BS position. Application of CoMP in this network using DPS approach with sleep mode of power saving, provides higher energy efficiency, SINR and throughput as compared to nominal CoMP. CoMP basically describes a scheme in which a group of base stations (BS) dynamically co-ordinate and co-operate among themselves to convert interference into a beneficial signal. Network planning using stochastic method and Voronoi Tessellation with two-tier network has been applied to a dense region of Surat city in Gujarat state of India. The results show clear improvement in signal-to-interference plus noise ratio (SINR) by 25% and energy efficiency of the network by 28% using the proposed CoMP transmission.

This article explores network densification as the key mechanism for wireless evolution over the next decade. Network densification includes densification over space (e.g, dense deployment of small cells) and frequency (utilizing larger portions of radio spectrum in diverse bands). Large-scale cost-effective spatial densification is facilitated by self-organizing networks and intercell interference management. Full benefits of network densification can be realized only if it is complemented by backhaul densification, and advanced receivers capable of interference cancellation.

Recently the orbital angular momentum (OAM) multiplexing scheme has been proposed to increase without limit the channel capacity between a transmitter and a receiver node communicating in line of sight. A controversy arose about the far-field effectiveness of this method and about it being or not a subset of MIMO communication schemes. In this contribution we first illustrate that OAM multiplexing can be completely understood as line of sight MIMO, and hence that the technique cannot bring additional advantages in far field communications. Secondly, we show that it is possible to build similar line of sight near field MIMO systems based on lenses with similar or better performance than their OAM counterparts.

Developing next-generation cellular technology (5G) in the mm-wave bands will require low-cost phased-array transceivers [1]. Even with the benefit of beamforming, due to space constraints in the mobile form-factor, increasing TX output power while maintaining acceptable PA PAE, LNA NF, and overall transceiver power consumption is important to maximizing link budget allowable path loss and minimizing handset case temperature. Further, the phased-array transceiver will need to be able to support dual-polarization communication. An IF interface to the analog baseband is desired for low power consumption in the handset or user equipment (UE) active antenna and to enable use of arrays of transceivers for customer premises equipment (CPE) or basestation (BS) antenna arrays with a low-loss IF power-combining/splitting network implemented on an antenna backplane carrying multiple tiled antenna modules.

Cognitive Radios have been advanced as a technology for the opportunistic use of under-utilized spectrum since they are able to sense the spectrum and use frequency bands if no Primary user is detected. However, the required sensitivity is very demanding since any individual radio might face a deep fade. We propose light-weight cooperation in sensing based on hard decisions to mitigate the sensitivity requirements on individual radios. We show that the "link budget" that system designers have to reserve for fading is a significant function of the required probability of detection. Even a few cooperating users (~10-20) facing independent fades are enough to achieve practical threshold levels by drastically reducing individual detection requirements. Hard decisions perform almost as well as soft decisions in achieving these gains. Cooperative gains in a environment where shadowing is correlated, is limited by the cooperation footprint (area in which users cooperate). In essence, a few independent users are more robust than many correlated users. Unfortunately, cooperative gain is very sensitive to adversarial/failing Cognitive Radios. Radios that fail in a known way (always report the presence/absence of a Primary user) can be compensated for by censoring them. On the other hand, radios that fail in unmodeled ways or may be malicious, introduce a bound on achievable sensitivity reductions. As a rule of thumb, if we believe that 1/N users can fail in an unknown way, then the cooperation gains are limited to what is possible with N trusted users.

This article provides some fundamental indications about wireless communications beyond LTE/LTE-A (5G), representing the key findings of the European research project 5GNOW. We start with identifying the drivers for making the transition to 5G networks. Just to name one, the advent of the Internet of Things and its integration with conventional human-initiated transmissions creates a need for a fundamental system redesign. Then we make clear that the strict paradigm of synchronism and orthogonality as applied in LTE prevents efficiency and scalability. We challenge this paradigm and propose new key PHY layer technology components such as a unified frame structure, multicarrier waveform design including a filtering functionality, sparse signal processing mechanisms, a robustness framework, and transmissions with very short latency. These components enable indeed an efficient and scalable air interface supporting the highly varying set of requirements originating from the 5G drivers.

One of 5G communication typical scenarios is machine-to-machine communication with large number of devices, which proposes new requirements for uplink transmission. Filter bank multi-carrier with offset quadrature amplitude modulation (FBMC/OQAM) is one of the popular nonorthogonal waveform candidates for 5G transmission. In this paper, FBMC/OQAM and cyclic prefix OFDM with single antenna settings are comprehensively evaluated and compared in uplink transmission form four aspects: power spectral density, timing and frequency misalignments, guard bands between adjacent users and time efficiency. At last, our comparison results show that FBMC is more suitable for 5G uplink multi-user asynchronous transmissions than CP-OFDM.

Millimeter wave (mmWave) communication is a promising technology for future wireless systems, while one practical challenge is to achieve its large-antenna gains with only limited radio frequency (RF) chains for cost-effective implementation. To this end, we study in this paper a new lens antenna array enabled mmWave multiple-input multiple-output (MIMO) communication system. We first show that the array response of lens antenna arrays follows a “sinc” function, where the antenna element with the peak response is determined by the angle of arrival (AoA)/departure (AoD) of the received/transmitted signal. By exploiting this unique property along with the multi-path sparsity of mmWave channels, we propose a novel low-cost and capacity-achieving spatial multiplexing scheme for both narrow-band and wide-band mmWave communications, termed path division multiplexing (PDM), where parallel data streams are transmitted over different propagation paths with simple per-path processing. We further propose a simple path grouping technique with group-based small-scale MIMO processing to effectively mitigate the inter-stream interference due to similar AoAs/AoDs. Numerical results are provided to compare the performance of the proposed mmWave lens MIMO against the conventional MIMO with uniform planar arrays (UPAs) and hybrid analog/digital processing. It is shown that the proposed design achieves significant throughput gains as well as complexity and cost reductions, thus leading to a promising new paradigm for mmWave MIMO communications.

In this paper the use of electronically steerable passive array radiator (ESPAR) antennas is introduced for achieving increased spectral efficiency characteristics in multiple-input, multiple-output (MIMO) systems using a single active element, and compact antennas. The proposed ESPAR antenna is capable of mapping phase-shift-keying (PSK) modulated symbols to be transmitted onto orthogonal basis functions on the wavevector domain of the multi-element antenna (MEA), instead of the traditional approach of sending different symbol streams in different locations on the antenna domain. In this way, different symbols are transmitted simultaneously towards different angles of departure at the transmitter side. We show that the proposed system, called beamspace-MIMO (BS-MIMO) can achieve performance characteristics comparable to traditional MIMO systems while using a single radio-frequency (RF) front-end and ESPAR antenna arrays of total length equal to lambda/8, for popular PSK modulation schemes such as binary-PSK (BPSK), quaternary-PSK (QPSK), as well as for their offset-PSK and differential-PSK variations.

After comparing the properties of analog backscatter and digital backscatter, we propose that a combination of the two can provide a solution for high data rate battery free wireless sensing that is superior to either approach on its own. We present a hybrid analog-digital backscatter platform that uses digital backscatter for addressability and control but switches into analog backscatter mode for high data rate transmission of sensor data. Using hybrid backscatter, we report the first digitally addressable real-time battery free wireless microphone. We develop the hybrid backscatter platform by integrating an electret microphone and RF switch with a digital RFID platform (WISP). The hybrid WISP operates by default in digital mode, transmitting and receiving digital data using the EPC Gen 2 RFID protocol but switching into analog mode to backscatter audio sensor data when activated by Gen 2 READ command. The data is recovered using a USRP-based Software Defined RFID reader. We report an operating range of 7.4 meters for the analog backscatter microphone and 2.7 meters for hybrid microphone with 26.7 dBm output power USRP-based RFID reader.


Question: DramaBank: Identifying agency within narrative discourse annotations.

This study investigates the feasibility of employing contemporary natural language processing (NLP) techniques to automatically generate plot unit representations for narrative text, building upon the plot unit theory introduced in the 1980s as a method for conceptualizing and summarizing stories. Our project involves the development of a system, termed AESOP, which utilizes a range of existing resources to pinpoint affect states within narratives and applies a set of "projection rules" to attribute these emotional states to the relevant characters. Additionally, we employ corpus-based methodologies to construct an innovative affect knowledge base, focusing on verbs that impart positive or negative states to their subjects (e.g., being rescued is positive, while being abandoned is negative). We extract these "subject polarity verbs" from a large web corpus through two strategies: identifying co-occurrences with "Evil/Kind Agent" patterns and leveraging bootstrapping over verb conjunctions. The efficacy of the plot unit representations generated by AESOP is assessed using a small corpus of Aesop’s fables, showcasing the system's potential in narrative analysis.

OF THE THESIS Building Simple Annotation Tools

Through a particular choice of a predicate (e.g., “x violated y”), a writer can subtly connote a range of implied sentiments and presupposed facts about the entities x and y: (1) writer’s perspective: projecting x as an “antagonist” and y as a “victim”, (2) entities’ perspective: y probably dislikes x, (3) effect: something bad happened to y, (4) value: y is something valuable, and (5) mental state: y is distressed by the event. We introduce connotation frames as a representation formalism to organize these rich dimensions of connotation using typed relations. First, we investigate the feasibility of obtaining connotative labels through crowdsourcing experiments. We then present models for predicting the connotation frames of verb predicates based on their distributional word representations and the interplay between different types of connotative relations. Empirical results confirm that connotation frames can be induced from various data sources that reflect how people use language and give rise to the connotative meanings. We conclude with analytical results that show the potential use of connotation frames for analyzing subtle biases in online news media.

Developers frequently discuss aspects of the systems they are developing online. The comments they post to discussions form a rich information source about the system. Intention mining, a process introduced by Di Sorbo et al., classifies sentences in developer discussions to enable further analysis. As one example of use, intention mining has been used to help build various recommenders for software developers. The technique introduced by Di Sorbo et al. to categorize sentences is based on linguistic patterns derived from two projects. The limited number of data sources used in this earlier work introduces questions about the comprehensiveness of intention categories and whether the linguistic patterns used to identify the categories are generalizable to developer discussion recorded in other kinds of software artifacts (e.g., issue reports). To assess the comprehensiveness of the previously identified intention categories and the generalizability of the linguistic patterns for category identification, we manually created a new dataset, categorizing 5,408 sentences from issue reports of four projects in GitHub. Based on this manual effort, we refined the previous categories. We assess Di Sorbo et al.’s patterns on this dataset, finding that the accuracy rate achieved is low (0.31). To address the deficiencies of Di Sorbo et al.’s patterns, we propose and investigate a convolution neural network (CNN)-based approach to automatically classify sentences into different categories of intentions. Our approach optimizes CNN by integrating batch normalization to accelerate the training speed, and an automatic hyperparameter tuning approach to tune appropriate hyperparameters of CNN. Our approach achieves an accuracy of 0.84 on the new dataset, improving Di Sorbo et al.’s approach by 171%. We also apply our approach to improve an automated software engineering task, in which we use our proposed approach to rectify misclassified issue reports, thus reducing the bias introduced by such data to other studies. A case study on four open source projects with 2,076 issue reports shows that our approach achieves an average AUC score of 0.687, which improves other baselines by at least 16%.

Recognized as great works of world literature, Shakespeare’s poems and plays have been translated into dozens of languages for over 300 years. Also, there are many re-translations into the same language, for example, there are more than 60 translations of Othello into German. Every translation is a different interpretation of the play. These large quantities of translations reflect changing culture and express individual thought by the authors. They demonstrate wide connections between different world regions today, and reveal a retrospective view of their cultural, intercultural, and linguistic histories. Researchers from Arts and Humanities at Swansea University are collecting a large number of translations of William Shakespeare’s Othello. In this paper, we have developed an interactive visualization system to present, analyze and explore the variations among these different translations. Our system is composed of two parts: the structure-aware Treemap for document selection and meta data analysis, and Focus + Context parallel coordinates for in-depth document comparison and exploration. In particular, we want to learn more about which content varies highly with each translation, and which content remains stable. We also want to form hypotheses as to the implications behind these variations. Our visualization is evaluated by the domain experts from Arts and Humanities.

This paper proposes a new dataset, Frames, composed of 1369 human-human dialogues with an average of 15 turns per dialogue. This corpus contains goal-oriented dialogues between users who are given some constraints to book a trip and assistants who search a database to find appropriate trips. The users exhibit complex decision-making behaviour which involve comparing trips, exploring different options, and selecting among the trips that were discussed during the dialogue. To drive research on dialogue systems towards handling such behaviour, we have annotated and released the dataset and we propose in this paper a task called frame tracking. This task consists of keeping track of different semantic frames throughout each dialogue. We propose a rule-based baseline and analyse the frame tracking task through this baseline.

We propose the first unsupervised approach to the problem of modeling dialogue acts in an open domain. Trained on a corpus of noisy Twitter conversations, our method discovers dialogue acts by clustering raw utterances. Because it accounts for the sequential behaviour of these acts, the learned model can provide insight into the shape of communication in a new medium. We address the challenge of evaluating the emergent model with a qualitative visualization and an intrinsic conversation ordering task. This work is inspired by a corpus of 1.3 million Twitter conversations, which will be made publicly available. This huge amount of data, available only because Twitter blurs the line between chatting and publishing, highlights the need to be able to adapt quickly to a new medium.

While Named Entity extraction is useful in many natural language applications, the coarse categories that most NE extractors work with prove insufficient for complex applications such as Question Answering and Ontology generation. We examine one coarse category of named entities, persons, and describe a method for automatically classifying person instances into eight finergrained subcategories. We present a supervised learning method that considers the local context surrounding the entity as well as more global semantic information derived from topic signatures and WordNet. We reinforce this method with an algorithm that takes advantage of the presence of entities in multiple contexts.

Over the last two decades or so, Natural Language Processing (NLP) has developed powerful methods for low-level syntactic and semantic text processing tasks such as parsing, semantic role labeling, and text categorization. Over the same period, the fields of machine learning and probabilistic reasoning have yielded important breakthroughs as well. It is now time to investigate how to leverage these advances to understand text.

Argumentation mining aims to automatically identify structured argument data from unstructured natural language text. This challenging, multifaceted task is recently gaining a growing attention, especially due to its many potential applications. One particularly important aspect of argumentation mining is claim identification. Most of the current approaches are engineered to address specific domains. However, argumentative sentences are often characterized by common rhetorical structures, independently of the domain. We thus propose a method that exploits structured parsing information to detect claims without resorting to contextual information, and yet achieve a performance comparable to that of state-of-the-art methods that heavily rely on the context.

Identifying discourse relations that are not overtly marked with discourse connectives remains a challenging problem. The absence of explicit clues indicates a need for the combination of world knowledge and weak contextual clues, which can hardly be learned from a small amount of manually annotated data. In this paper, we address this problem by augmenting the input text with external knowledge and context and by adopting a neural network model that can effectively handle the augmented text. Experiments show that external knowledge did improve the classification accuracy. On the other hand, contextual information provided no significant gain for implicit discourse relations while it worked for explicit ones.


Question: DramaBank: Identifying agency within narrative discourse annotation.

Narrative Analysis and Summarization through Plot Unit Graphs

To effectively summarize a narrative, a deep understanding of the story is required, one that identifies and emphasizes the key elements. A promising approach to achieving this involves the use of plot units in memory representation, which serves as a comprehensive framework for narrative analysis. Plot units are conceptual frameworks that intersect when a story is well-structured and coherent. When these intersections, representing the overlaps between plot units, are visualized as connections in a plot unit graph, the resulting structure encapsulates the narrative's plot. The graph's architecture, specifically its structural features, reveals the central and peripheral concepts within the story. This plot unit analysis is currently being explored as a strategy for summarizing narratives through both computer simulations and psychological experiments, aiming to distill the essence of a story while preserving its core meaning.

Most research on multi-document summarization explores methods that generate summaries based on queries regardless of the users’ preferences. We note that, different users can generate somewhat different summaries on the basis of the same source data and query. This paper presents our study on how to exploit the information regards how users summarized their texts. Models of different users can be used either separately, or in an ensemble-like fashion. Machine learning methods are explored in the construction of the individual models. However, we explore yet another hypothesis. We believe that the sentences selected into the summary should be coherent and supplement each other in their meaning. One method to model this relationship between sentences is by detecting actor–object relationship (AOR). The sentences that satisfy this relationship have their importance value enhanced. This paper combines ensemble summarizing system and AOR to generate summaries. We have evaluated this method on DUC 2006 and DUC 2007 using ROUGE measure. Experimental results show the supervised method that exploits the ensemble summarizing system combined with AOR outperforms previous models when considering performance in query-based multidocument summarization tasks. Keyword User-based summarization · Actor–object relationship · Multi-document summarization · Ensemble summarizing system · Training data construction Communicated by V. Loia. M. Valizadeh (B) · P. Brazdil LIAAD INESC Tec, University of Porto, Porto, Portugal e-mail: valizadehmr@gmail.com P. Brazdil FEP, University of Porto, Porto, Portugal e-mail: pbrazdil@inescporto.pt

We explore the use of Wikipedia as external knowledge to improve named entity recognition (NER). Our method retrieves the corresponding Wikipedia entry for each candidate word sequence and extracts a category label from the first sentence of the entry, which can be thought of as a definition part. These category labels are used as features in a CRF-based NE tagger. We demonstrate using the CoNLL 2003 dataset that the Wikipedia category labels extracted by such a simple method actually improve the accuracy of NER.

Over the last two decades or so, Natural Language Processing (NLP) has developed powerful methods for low-level syntactic and semantic text processing tasks such as parsing, semantic role labeling, and text categorization. Over the same period, the fields of machine learning and probabilistic reasoning have yielded important breakthroughs as well. It is now time to investigate how to leverage these advances to understand text.

We present an overview of TARSQI, a modular system for automatic temporal annotation that adds time expressions, events and temporal relations to news texts.

We presentan unsupervisedapproachto recognizingdiscourserelationsof CONTRAST, EXPLANATION-EVIDENCE, CONDITION andELABORATION that hold betweenarbitraryspansof texts. We show that discourserelation classifierstrained on examples that are automaticallyextractedfrom massi ve amountsof text can be usedto distinguishbetweensomeof theserelationswith accuraciesashigh as 93%, even whenthe relationsarenot explicitly markedby cuephrases.

We investigate the identification and, to some extent, the classification of collocational word groups that consist of an adjectival modifier (A), an accusative object noun (N), and a verb (V) by means of parsing a newspaper corpus with a lexicalized probabilistic grammar.1 Data triples are extracted from the resulting Viterbi parses and subsequently evaluated by means of a statistical association measure. The extraction results are then compared to predefined descriptive classes of ANV-triples. We also use a decision tree algorithm to classify part of the data obtained, on the basis of a small set of manually classified examples. Much of the candidate data is lexicographically relevant: the triples include idiomatic combinations (e.g. (sich) eine goldene Nase verdienen, ‘get rich’, lit. ‘earn oneself a golden nose’), combinations of N+V and N+A collocations (e.g. (eine) klare Absage erteilen, ‘refuse resolutely’, lit. ‘give a clear refusal’), next to cases where N+V or N+A collocations are found, in combination with other (not necessarily collocational) context partners. To extract such data from text corpora, a grammar is needed that captures verb+object relations: simple pattern matching on part-of-speech shapes is not sufficient. Statistical tools then allow to order the data in a way useful for subsequent manual selection by lexicographers. 1This work has been carried out in the context of the Transferbereich 32: Automatische Exzerption, a DFG-funded project aiming at the creation of support tools for the corpus-based updating of printed dictionaries in lexicography, carried out in cooperation with the publishers Langenscheidt KG and Duden BIFAB AG.

We address the challenge of sentiment analysis from visual content. In contrast to existing methods which infer sentiment or emotion directly from visual low-level features, we propose a novel approach based on understanding of the visual concepts that are strongly related to sentiments. Our key contribution is two-fold: first, we present a method built upon psychological theories and web mining to automatically construct a large-scale Visual Sentiment Ontology (VSO) consisting of more than 3,000 Adjective Noun Pairs (ANP). Second, we propose SentiBank, a novel visual concept detector library that can be used to detect the presence of 1,200 ANPs in an image. The VSO and SentiBank are distinct from existing work and will open a gate towards various applications enabled by automatic sentiment analysis. Experiments on detecting sentiment of image tweets demonstrate significant improvement in detection accuracy when comparing the proposed SentiBank based predictors with the text-based approaches. The effort also leads to a large publicly available resource consisting of a visual sentiment ontology, a large detector library, and the training/testing benchmark for visual sentiment analysis.

Spoken dialogue systems typically use predefined semantic slots to parse users' natural language inputs into unified semantic representations. To define the slots, domain experts and professional annotators are often involved, and the cost can be expensive. In this paper, we ask the following question: given a collection of unlabeled raw audios, can we use the frame semantics theory to automatically induce and fill the semantic slots in an unsupervised fashion? To do this, we propose the use of a state-of-the-art frame-semantic parser, and a spectral clustering based slot ranking model that adapts the generic output of the parser to the target semantic space. Empirical experiments on a real-world spoken dialogue dataset show that the automatically induced semantic slots are in line with the reference slots created by domain experts: we observe a mean averaged precision of 69.36% using ASR-transcribed data. Our slot filling evaluations also indicate the promising future of this proposed approach.

We present Sentiment Analyzer (SA) that extracts sentiment (or opinion) about a subject from online text documents. Instead of classifying the sentiment of an entire document about a subject, SA detects all references to the given subject, and determines sentiment in each of the references using natural language processing (NLP) techniques. Our sentiment analysis consists of 1) a topic specific feature term extraction, 2) sentiment extraction, and 3) (subject, sentiment) association by relationship analysis. SA utilizes two linguistic resources for the analysis: the sentiment lexicon and the sentiment pattern database. The performance of the algorithms was verified on online product review articles (“digital camera” and “music” reviews), and more general documents including general webpages and news articles.

We introduce a new approach to unsupervised estimation of feature-rich semantic role labeling models. Our model consists of two components: (1) an encoding component: a semantic role labeling model which predicts roles given a rich set of syntactic and lexical features; (2) a reconstruction component: a tensor factorization model which relies on roles to predict argument fillers. When the components are estimated jointly to minimize errors in argument reconstruction, the induced roles largely correspond to roles defined in annotated resources. Our method performs on par with most accurate role induction methods on English and German, even though, unlike these previous approaches, we do not incorporate any prior linguistic knowledge about the languages.


Question: DramaBank: Identifying agency within narrative discourse through annotation.

Creating a Repository of Semantically Analyzed Narratives

This paper outlines a methodology for annotating narrative discourse that is specifically tailored to analyze texts as stories. The goal is to establish a comprehensive "narrative bank" that can support automated analysis of narrative structure and content. The proposed methodology requires annotators to construct propositions that closely mirror the original text, choosing predicates and arguments from controlled vocabularies sourced from resources like WordNet and VerbNet. These propositions are then woven into a conceptual graph that charts the entire discourse, with edges signifying temporal, causal, and other narrative content relationships. As annotators decode the text, they also tackle coreference resolution and word sense disambiguation, which are integral to the proposition encoding process. We detail an experimental data collection process and a technique for assessing inter-annotator agreement when multiple annotators decode the same short story. Additionally, we discuss current efforts to augment the methodology by incorporating annotators' interpretations of character agency, encompassing the goals, plans, and beliefs that are significant but not explicitly stated within the narrative.

in large multimedia databases and make inferences about the data stream’s content. A human operator can specify gross indices, such as the date or hour, and perhaps the location and the activity (for example, a football game) at the time of capture. Finer-grain annotations require separating the video into meaningful segments such as shots and scenes. Many automated techniques and annotation formats for capturing this type of information exist. However, the descriptions they produce don’t define the data stream content adequately for semantic retrieval and reasoning. Such descriptions must be based on observable or inferrable events in the data streams. We describe a framework for video event representation and annotation that’s based on the definition of an ontology suitable for video content. An ontology consists of a specific vocabulary for describing a certain reality and a set of explicit assumptions regarding the vocabulary’s intended meaning.1 The Video Event Representation Language (VERL)2-4 describes an event ontology, and the Video Event Markup Language (VEML)4 lets us annotate instances of the events described in VERL (see the sidebar “Using VERL and VEML” on p. 78 for an example). Annotating video in VEML consists of describing instances of events and objects, in a previously defined ontology using VERL. Figure 1 illustrates the conceptual elements involved and their relationships: annotations draw on one or several domain-specific ontologies, defined according to VERL concepts and constructs. VEML incorporates extensible event and object type hierarchies rooted in VERL’s event and object concepts. The annotations also draw on VEML for content organization. Marking up data streams in a well-defined language rooted in an ontology enables nonambiguous data sharing among users. Furthermore, annotation data is accessible to automatic machine manipulation for indexing or inferencing. The framework we describe resulted from discussions at a series of workshops sponsored by the US government’s Advanced Research and Development Activity (ARDA). It therefore includes the intellectual contributions of many individuals. Currently, only a small community of researchers is using VERL and VEML; this article aims to publicize the framework to a broader community as its relevance depends on its widespread adoption. An earlier description can be found elsewhere.5

The availability of the Rhetorical Structure Theory (RST) Discourse Treebank has spurred substantial research into discourse analysis of written texts; however, limited research has been conducted to date on RST annotation and parsing of spoken language, in particular, nonnative spontaneous speech. Considering that the measurement of discourse coherence is typically a key metric in human scoring rubrics for assessments of spoken language, we initiated a research effort to obtain RST annotations of a large number of non-native spoken responses from a standardized assessment of academic English proficiency. The resulting inter-annotator κ agreements on the three different levels of Span, Nuclearity, and Relation are 0.848, 0.766, and 0.653, respectively. Furthermore, a set of features was explored to evaluate the discourse structure of non-native spontaneous speech based on these annotations; the highest performing feature showed a correlation of 0.612 with scores of discourse coherence provided by expert human raters.

In the present paper, we present an automated tagging approach aimed at enhancing a well-known resource, the ACL Anthology Reference Corpus, with semantic class labels for more than 20,000 technical terms that are relevant to the domain of computational linguistics. We use state-of-the-art classification techniques to assign semantic class labels to technical terms extracted from several reference term lists. We also sketch a set of research questions and approaches directed towards the integrated analysis of scientific corpora. To this end, we query the data set resulting from our annotation effort on both the term and the semantic class level level.

Human linguistic annotation is crucial for many natural language processing tasks but can be expensive and time-consuming. We explore the use of Amazon’s Mechanical Turk system, a significantly cheaper and faster method for collecting annotations from a broad base of paid non-expert contributors over the Web. We investigate five tasks: affect recognition, word similarity, recognizing textual entailment, event temporal ordering, and word sense disambiguation. For all five, we show high agreement between Mechanical Turk non-expert annotations and existing gold standard labels provided by expert labelers. For the task of affect recognition, we also show that using non-expert labels for training machine learning algorithms can be as effective as using gold standard annotations from experts. We propose a technique for bias correction that significantly improves annotation quality on two tasks. We conclude that many large labeling tasks can be effectively designed and carried out in this method at a fraction of the usual expense.

We consider the task of identifying attitudes towards a given set of entities from text. Conventionally, this task is decomposed into two separate subtasks: target detection that identifies whether each entity is mentioned in the text, either explicitly or implicitly, and polarity classification that classifies the exact sentiment towards an identified entity (the target) into positive, negative, or neutral.
 Instead, we show that attitude identification can be solved with an end-to-end machine learning architecture, in which the two subtasks are interleaved by a deep memory network. In this way, signals produced in target detection provide clues for polarity classification, and reversely, the predicted polarity provides feedback to the identification of targets. Moreover, the treatments for the set of targets also influence each other -- the learned representations may share the same semantics for some targets but vary for others. The proposed deep memory network, the AttNet, outperforms methods that do not consider the interactions between the subtasks or those among the targets, including conventional machine learning methods and the state-of-the-art deep learning models.

Recognized as great works of world literature, Shakespeare’s poems and plays have been translated into dozens of languages for over 300 years. Also, there are many re-translations into the same language, for example, there are more than 60 translations of Othello into German. Every translation is a different interpretation of the play. These large quantities of translations reflect changing culture and express individual thought by the authors. They demonstrate wide connections between different world regions today, and reveal a retrospective view of their cultural, intercultural, and linguistic histories. Researchers from Arts and Humanities at Swansea University are collecting a large number of translations of William Shakespeare’s Othello. In this paper, we have developed an interactive visualization system to present, analyze and explore the variations among these different translations. Our system is composed of two parts: the structure-aware Treemap for document selection and meta data analysis, and Focus + Context parallel coordinates for in-depth document comparison and exploration. In particular, we want to learn more about which content varies highly with each translation, and which content remains stable. We also want to form hypotheses as to the implications behind these variations. Our visualization is evaluated by the domain experts from Arts and Humanities.

Causation is a psychological tool of humans to understand the world and it is projected in natural language. Causation relates two events, so in order to understand the causal relation of those events and the causal reasoning of humans, the study of causality classification is required. We claim that the use of linguistic features may restrict the representation of causality, and dense vector spaces can provide a better encoding of the causal meaning of an utterance. Herein, we propose a neural network architecture only fed with word embeddings for the task of causality classification. Our results show that our claim holds, and we outperform the state-of-the-art on the AltLex corpus. The source code of our experiments is publicly available.1

Learning how to generate descriptions of images or videos received major interest both in the Computer Vision and Natural Language Processing communities. While a few works have proposed to learn a grounding during the generation process in an unsupervised way (via an attention mechanism), it remains unclear how good the quality of the grounding is and whether it benefits the description quality. In this work we propose a movie description model which learns to generate description and jointly ground (localize) the mentioned characters as well as do visual co-reference resolution between pairs of consecutive sentences/clips. We also propose to use weak localization supervision through character mentions provided in movie descriptions to learn the character grounding. At training time, we first learn how to localize characters by relating their visual appearance to mentions in the descriptions via a semi-supervised approach. We then provide this (noisy) supervision into our description model which greatly improves its performance. Our proposed description model improves over prior work w.r.t. generated description quality and additionally provides grounding and local co-reference resolution. We evaluate it on the MPII Movie Description dataset using automatic and human evaluation measures and using our newly collected grounding and co-reference data for characters.

We consider the ROC story cloze task (Mostafazadeh et al., 2016) and present several findings. We develop a model that uses hierarchical recurrent networks with attention to encode the sentences in the story and score candidate endings. By discarding the large training set and only training on the validation set, we achieve an accuracy of 74.7%. Even when we discard the story plots (sentences before the ending) and only train to choose the better of two endings, we can still reach 72.5%. We then analyze this “ending-only” task setting. We estimate human accuracy to be 78% and find several types of clues that lead to this high accuracy, including those related to sentiment, negation, and general ending likelihood regardless of the story context.

The nature of the relationship between information technology (IT) and organizations has been a long-standing debate in the Information Systems literature. Does IT shape organizations, or do people in organisations control how IT is used? To formulate the question a little differently: does agency (the capacity to make a difference) lie predominantly with machines (computer systems) or humans (organisational actors)? Many proposals for a middle way between the extremes of technological and social determinism have been put advanced; in recent years researchers oriented towards social theories have focused on structuration theory and (lately) actor network theory. These two theories, however, adopt different and incompatible views of agency. Thus, structuration theory sees agency as exclusively a property of humans, whereas the principle of general symmetry in actor network theory implies that machines may also be agents. Drawing on critiques of both structuration theory and actor network theory, this paper develops a theoretical account of the interaction between human and machine agency: the double dance of agency. The account seeks to contribute to theorisation of the relationship between technology and organisation by recognizing both the different character of human and machine agency, and the emergent properties of their interplay.


Question: DramaBank: Identifying agency within narrative discourse annotations.

Creating a Narrative Framework: Analyzing Discourse Dynamics
This document explores the intricacies of constructing a narrative framework that effectively captures the dynamics of discourse within a story. Unlike conventional approaches, this method delves deep into the narrative structure, focusing on how various elements interact to convey meaning and evoke emotion. Central to this analysis is the examination of agency within the narrative, identifying who or what drives the action and how these actions impact the storyline. The document highlights the importance of understanding the relationships between characters, events, and the setting, illustrating how these components work together to create a coherent and engaging narrative. By meticulously annotating these aspects, the document provides a comprehensive model for analyzing narrative discourse, offering insights into the underlying mechanisms that shape the narrative flow. This approach not only enriches our understanding of how stories are constructed but also enhances our ability to interpret and appreciate the complexities of narrative communication.

We present Sentiment Analyzer (SA) that extracts sentiment (or opinion) about a subject from online text documents. Instead of classifying the sentiment of an entire document about a subject, SA detects all references to the given subject, and determines sentiment in each of the references using natural language processing (NLP) techniques. Our sentiment analysis consists of 1) a topic specific feature term extraction, 2) sentiment extraction, and 3) (subject, sentiment) association by relationship analysis. SA utilizes two linguistic resources for the analysis: the sentiment lexicon and the sentiment pattern database. The performance of the algorithms was verified on online product review articles (“digital camera” and “music” reviews), and more general documents including general webpages and news articles.

Recent advances in natural language processing have produced libraries that extract lowlevel features from a collection of raw texts. These features, known as annotations, are usually stored internally in hierarchical, tree-based data structures. This paper proposes a data model to represent annotations as a collection of normalized relational data tables optimized for exploratory data analysis and predictive modeling. The R package cleanNLP, which calls one of two state of the art NLP libraries (CoreNLP or spaCy), is presented as an implementation of this data model. It takes raw text as an input and returns a list of normalized tables. Specific annotations provided include tokenization, part of speech tagging, named entity recognition, sentiment analysis, dependency parsing, coreference resolution, and word embeddings. The package currently supports input text in English, German, French, and Spanish.

We consider the ROC story cloze task (Mostafazadeh et al., 2016) and present several findings. We develop a model that uses hierarchical recurrent networks with attention to encode the sentences in the story and score candidate endings. By discarding the large training set and only training on the validation set, we achieve an accuracy of 74.7%. Even when we discard the story plots (sentences before the ending) and only train to choose the better of two endings, we can still reach 72.5%. We then analyze this “ending-only” task setting. We estimate human accuracy to be 78% and find several types of clues that lead to this high accuracy, including those related to sentiment, negation, and general ending likelihood regardless of the story context.

Human linguistic annotation is crucial for many natural language processing tasks but can be expensive and time-consuming. We explore the use of Amazon’s Mechanical Turk system, a significantly cheaper and faster method for collecting annotations from a broad base of paid non-expert contributors over the Web. We investigate five tasks: affect recognition, word similarity, recognizing textual entailment, event temporal ordering, and word sense disambiguation. For all five, we show high agreement between Mechanical Turk non-expert annotations and existing gold standard labels provided by expert labelers. For the task of affect recognition, we also show that using non-expert labels for training machine learning algorithms can be as effective as using gold standard annotations from experts. We propose a technique for bias correction that significantly improves annotation quality on two tasks. We conclude that many large labeling tasks can be effectively designed and carried out in this method at a fraction of the usual expense.

One of the main challenges in question-answering is the potential mismatch between the expressions in questions and the expressions in texts. While humans appear to use inference rules such as X writes Y implies X is the author of Y in answering questions, such rules are generally unavailable to question-answering systems due to the inherent difficulty in constructing them. In this paper, we present an unsupervised algorithm for discovering inference rules from text. Our algorithm is based on an extended version of Harris Distributional Hypothesis, which states that words that occurred in the same contexts tend to be similar. Instead of using this hypothesis on words, we apply it to paths in the dependency trees of a parsed corpus. Essentially, if two paths tend to link the same set of words, we hypothesize that their meanings are similar. We use examples to show that our system discovers many inference rules easily missed by humans.

A decent movie summary is helpful for movie producer to promote the movie as well as audience to capture the theme of the movie before watching the whole movie. Most exiting automatic movie summarization approaches heavily rely on video content only, which may not deliver ideal result due to the semantic gap between computer calculated low-level features and human used high-level understanding. In this paper, we incorporate script into movie analysis and propose a novel character-based movie summarization approach, which is validated by modern film theory that what actually catches audiences' attention is the character. We first segment scenes in the movie by analysis and alignment of script and movie. Then we conduct substory discovery and content attention analysis based on the scent analysis and character interaction features. Given obtained movie structure and content attention value, we calculate movie attraction scores at both shot and scene levels and adopt this as criterion to generate movie summary. The promising experimental results demonstrate that character analysis is effective for movie summarization and movie content understanding.

In this paper, we present a system that automatically generates questions from natural language text using discourse connectives. We explore the usefulness of the discourse connectives for Question Generation (QG) that looks at the problem beyond sentence level. Our work divides the QG task into content selection and question formation. Content selection consists of finding the relevant part in text to frame question from while question formation involves sense disambiguation of the discourse connectives, identification of question type and applying syntactic transformations on the content. The system is evaluated manually for syntactic and semantic correctness.

We introduce a novel coreference resolution system that models entities and events jointly. Our iterative method cautiously constructs clusters of entity and event mentions using linear regression to model cluster merge operations. As clusters are built, information flows between entity and event clusters through features that model semantic role dependencies. Our system handles nominal and verbal events as well as entities, and our joint formulation allows information from event coreference to help entity coreference, and vice versa. In a cross-document domain with comparable documents, joint coreference resolution performs significantly better (over 3 CoNLL F1 points) than two strong baselines that resolve entities and events separately.

The FrameNet Project is producing a lexicon of English for both human use and NLP applications, based on the principles of Frame Semantics, in which sentences are described on the basis of predicators which evoke semantic frames and other constituents which express the participants (frame elements) in these frames. Our lexicon contains detailed information about the possible syntactic realizations of frame elements, derived from annotated corpus examples. In the process, we have developed a suite of tools for the definition of semantic frames, for annotating sentences, for searching the results, and for creating a variety ofreports. We will discuss the conceptual basis ofour work and demonstrate the tools we work with, the results we produce, and how they may be ofuse to other NLP projects.

GENERATING NARRATIVE VARIATION IN INTERACTIVE FICTION Nick Montfort Mitchell P. Marcus and Gerald Prince A general method for the generation of natural language narrative is described. It allows the expression, or narrative discourse, to vary independently of the underlying events and existents that are the narrative’s content. Specifically, this variation is accomplished in an interactive fiction (IF) system which replies to typed input by narrating what has happened in a simulated world. IF works have existed for about 30 years as forms of text-based computer simulation, instances of dialog systems, and examples of literary art. Theorists of narrative have carefully distinguished between the level of underlying content (corresponding to the simulated world in interactive fiction) and that of expression (corresponding to the textual exchange between computer arnd user) since the mid-1960s, when the field of narratology began to develop, but IF systems have not yet made use of this distinction. The current project contributes new techniques for automatic narration by building on work done in computational linguistics, specifically natural language generation, and in narratology. First, types of narrative variation that are possible in IF are identified and formalized in a way that is suitable for a natural language generation system. An architecture for an IF system is then described and implemented; the result allows multiple works of interactive fiction to be realized and, using a general plan for narrating, allows them to be narrated in different ways during interaction. The system’s ability to generate text is considered in a pilot evaluation. Plans for future work are also discussed. They include publicly released systems for IF development and narratology education, adding a planning capability that uses actors’ individual perspectives, and adapting automatic narration to different sorts of interactive systems.


Question: DramaBank: Identifying agency within narrative discourse annotations.

This paper outlines a constructionist theory aimed at elucidating the knowledge-driven inferences generated by readers during the comprehension of narrative texts. The theory posits that readers construct a detailed referential situation model, encapsulating the essence of the narrative, and in doing so, they potentially create a wide array of inferences. However, the theory asserts that under typical comprehension circumstances, only a subset of this potential information is actually constructed. A key tenet of the constructionist theory is the principle of search for meaning. Under this principle, readers endeavor to create a meaning representation that aligns with their objectives, ensures coherence across both micro and macro levels of the text, and elucidates the rationale behind the inclusion of specific actions, events, and states within the narrative. The paper critically examines empirical evidence that supports this theory and juxtaposes it against competing theoretical perspectives, highlighting the unique contributions and insights of the constructionist approach to narrative text comprehension.

From our three year experience of developing a large-scale corpus of annotated Arabic text, our paper will address the following: (a) review pertinent Arabic language issues as they relate to methodology choices, (b) explain our choice to use the Penn English Treebank style of guidelines, (requiring the Arabic-speaking annotators to deal with a new grammatical system) rather than doing the annotation in a more traditional Arabic grammar style (requiring NLP researchers to deal with a new system); (c) show several ways in which human annotation is important and automatic analysis difficult, including the handling of orthographic ambiguity by both the morphological analyzer and human annotators; (d) give an illustrative example of the Arabic Treebank methodology, focusing on a particular construction in both morphological analysis and tagging and syntactic analysis and following it in detail through the entire annotation process, and finally, (e) conclude with what has been achieved so far and what remains to be done.

We explore the use of Wikipedia as external knowledge to improve named entity recognition (NER). Our method retrieves the corresponding Wikipedia entry for each candidate word sequence and extracts a category label from the first sentence of the entry, which can be thought of as a definition part. These category labels are used as features in a CRF-based NE tagger. We demonstrate using the CoNLL 2003 dataset that the Wikipedia category labels extracted by such a simple method actually improve the accuracy of NER.

This paper introduces a task of identifying and semantically classifying lexical expressions in running text. We investigate the online reviews genre, adding semantic supersense annotations to a 55,000 word English corpus that was previously annotated for multiword expressions. The noun and verb supersenses apply to full lexical expressions, whether singleor multiword. We then present a sequence tagging model that jointly infers lexical expressions and their supersenses. Results show that even with our relatively small training corpus in a noisy domain, the joint task can be performed to attain 70% class labeling F1.

Scripts provide rich textual annotation of movies, including dialogs, character names, and other situational descriptions. Exploiting such rich annotations requires aligning the sentences in the scripts with the corresponding video frames. Previous work on aligning movies with scripts predominantly relies on time-aligned closed-captions or subtitles, which are not always available. In this paper, we focus on automatically aligning faces in movies with their corresponding character names in scripts without requiring closed-captions/subtitles. We utilize the intuition that faces in a movie generally appear in the same sequential order as their names are mentioned in the script. We first apply standard techniques for face detection and tracking, and cluster similar face tracks together. Next, we apply a generative Hidden Markov Model (HMM) and a discriminative Latent Conditional Random Field (LCRF) to align the clusters of face tracks with the corresponding character names. Our alignment models (especially LCRF) significantly outperform the previous state-of-the-art on two different movie datasets and for a wide range of face clustering algorithms.

The domain of text-based adventure games has been recently established as a new challenge of creating the agent that is both able to understand natural language, and acts intelligently in text-described environments. In this paper, we present our approach to tackle the problem. Our agent, named Golovin, takes advantage of the limited game domain. We use genre-related corpora (including fantasy books and decompiled games) to create language models suitable to this domain. Moreover, we embed mechanisms that allow us to specify, and separately handle, important tasks as fighting opponents, managing inventory, and navigating on the game map. We validated usefulness of these mechanisms, measuring agent's performance on the set of 50 interactive fiction games. Finally, we show that our agent plays on a level comparable to the winner of the last year Text-Based Adventure AI Competition.

An increasingly large amount of multimodal content is posted on social media websites such as YouTube and Facebook everyday. In order to cope with the growth of such so much multimodal data, there is an urgent need to develop an intelligent multi-modal analysis framework that can effectively extract information from multiple modalities. In this paper, we propose a novel multimodal information extraction agent, which infers and aggregates the semantic and affective information associated with user-generated multimodal data in contexts such as e-learning, e-health, automatic video content tagging and human-computer interaction. In particular, the developed intelligent agent adopts an ensemble feature extraction approach by exploiting the joint use of tri-modal (text, audio and video) features to enhance the multimodal information extraction process. In preliminary experiments using the eNTERFACE dataset, our proposed multi-modal system is shown to achieve an accuracy of 87.95%, outperforming the best state-of-the-art system by more than 10%, or in relative terms, a 56% reduction in error rate.

In an effort to assist factcheckers in the process of factchecking, we tackle the claim detection task, one of the necessary stages prior to determining the veracity of a claim. It consists of identifying the set of sentences, out of a long text, deemed capable of being factchecked. This paper is a collaborative work between Full Fact, an independent factchecking charity, and academic partners. Leveraging the expertise of professional factcheckers, we develop an annotation schema and a benchmark for automated claim detection that is more consistent across time, topics and annotators than previous approaches. Our annotation schema has been used to crowdsource the annotation of a dataset with sentences from UK political TV shows. We introduce an approach based on universal sentence representations to perform the classification, achieving an F1 score of 0.83, with over 5% relative improvement over the state-of-the-art methods ClaimBuster and ClaimRank. The system was deployed in production and received positive user feedback.

Common-sense or background knowledge is required to understand natural language, but in most neural natural language understanding (NLU) systems, the requisite background knowledge is indirectly acquired from static corpora. We develop a new reading architecture for the dynamic integration of explicit background knowledge in NLU models. A new task-agnostic reading module provides refined word representations to a task-specific NLU architecture by processing background knowledge in the form of free-text statements, together with the taskspecific inputs. Strong performance on the tasks of document question answering (DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness and flexibility of our approach. Analysis shows that our models learn to exploit knowledge selectively and in a semantically appropriate way.

Many text collections with temporal references, such as news corpora and weblogs, are generated to report and discuss real life events. Thus, event-related tasks, such as detecting real life events that drive the generation of the text documents, tracking event evolutions, and investigating reports and commentaries about events of interest, are important when exploring such text collections. To incorporate and leverage human efforts in conducting such tasks, we propose a novel visual analytics approach named EventRiver. EventRiver integrates event-based automated text analysis and visualization to reveal the events motivating the text generation and the long term stories they construct. On the visualization, users can interactively conduct tasks such as event browsing, tracking, association, and investigation. A working prototype of EventRiver has been implemented for exploring news corpora. A set of case studies, experiments, and a preliminary user test have been conducted to evaluate its effectiveness and efficiency.

A decent movie summary is helpful for movie producer to promote the movie as well as audience to capture the theme of the movie before watching the whole movie. Most exiting automatic movie summarization approaches heavily rely on video content only, which may not deliver ideal result due to the semantic gap between computer calculated low-level features and human used high-level understanding. In this paper, we incorporate script into movie analysis and propose a novel character-based movie summarization approach, which is validated by modern film theory that what actually catches audiences' attention is the character. We first segment scenes in the movie by analysis and alignment of script and movie. Then we conduct substory discovery and content attention analysis based on the scent analysis and character interaction features. Given obtained movie structure and content attention value, we calculate movie attraction scores at both shot and scene levels and adopt this as criterion to generate movie summary. The promising experimental results demonstrate that character analysis is effective for movie summarization and movie content understanding.


Question: An empirical analysis of warehouse order picking tasks utilizing head-mounted displays.

This paper explores the design and application of wearable computing technology, focusing on the integration of advancements in multimodal interaction, machine learning, software engineering, electronics, and user-centered design methodologies. The emergence of wearable computers is attributed to the convergence of these research areas, leading to innovative devices such as head-mounted displays. Designing wearable computers necessitates a comprehensive consideration of various factors due to their intimate connection with the human body and their concurrent use with other tasks. To facilitate this design process, we introduce the UCAMP framework, which encompasses user, corporal, attention, manipulation, and perception aspects. Through a series of examples from our own and other researchers' prototypes, we illustrate the practical implications of these factors. Wearable computing has already found application in various domains, including maintenance in vehicles and aircraft, manufacturing, inspection, and language translation. The paper then shifts focus to the evolving concept of context-aware computing, where the wearable device adapts its behavior based on the user's state and surroundings. Context can encompass a wide range of attributes, such as location, physiological condition, personal history, and daily activities. By harnessing this context, wearable computers can function as proactive assistants, providing information when it is most needed. This opens up opportunities for augmented reality, context-aware collaboration, and enhanced manufacturing processes. Drawing on our research and studies, we propose guidelines for the design of wearable computing devices, accompanied by a checklist for designers embarking on new projects. The research directions section highlights ongoing challenges and trends in user interface design, interaction modalities, and cognitive augmentation. The paper concludes with a summary of key challenges and a forecast of future trends in wearable computing, emphasizing its potential to significantly reduce the cognitive load on users and enhance their overall experience.

This paper presents our vision of Human Computer Interaction (HCI): "Tangible Bits." Tangible Bits allows users to "grasp & manipulate" bits in the center of users’ attention by coupling the bits with everyday physical objects and architectural surfaces. Tangible Bits also enables users to be aware of background bits at the periphery of human perception using ambient display media such as light, sound, airflow, and water movement in an augmented space. The goal of Tangible Bits is to bridge the gaps between both cyberspace and the physical environment, as well as the foreground and background of human activities. This paper describes three key concepts of Tangible Bits: interactive surfaces; the coupling of bits with graspable physical objects; and ambient media for background awareness. We illustrate these concepts with three prototype systems – the metaDESK, transBOARD and ambientROOM – to identify underlying research issues.

This paper presents a proposal for the use of Spatial Augmented Reality as a tool for 3D data visualizations. The use of traditional display technologies such as LCD monitors provides a fish tank metaphor for the user, i.e. the information is behind a plane of glass. The use of VR technologies allows the user to be immersed in the 3D volume and remove this fish tank problem, but can limit the set of techniques that allow users to interact with spatial data. Spatial Augmented Reality employed in conjunction with a highresolution monitor provides an elegant blend of spatial reasoning, tangible interaction, and detailed information viewing. This paper purposes a range of usages for SAR in 3D visualizations.

Robot vision refers to the capability of a robot to visually perceive the environment and use this information for execution of different tasks. Visual feedback has been used extensively for robot navigation and obstacle avoidance. In the recent years, there are also examples that include interaction with people and manipulation of objects. In this paper, we review some of the work that goes beyond of using artificial landmarks and fiducial markers for the purpose of implementing vision based control in robots. We discuss different application areas, both from the systems perspective and individual problems such as object tracking and recognition.

During the usage of conventional stereo microscope, repetitive head movement are often inevitable for the users to retrieve information away from the microscope. Moreover, risks of surgeons loosing focus and increasing fatigue could also arise during microsurgeries. Therefore, Scope+, a stereoscopic video see-through augmented reality system was created not only to solve the problems mentioned above, but also to improve users stereo microscopic experience.

2D and 3D computer vision systems are frequently being used in automated production to detect and determine the position of objects. Accuracy is important in the production industry, and computer vision systems require structured environments to function optimally. For 2D vision systems, a change in surfaces, lighting and viewpoint angles can reduce the accuracy of a method, maybe even to a degree that it will be erroneous, while for 3D vision systems, the accuracy mainly depends on the 3D laser sensors. Commercially available 3D cameras lack the precision found in high-grade 3D laser scanners, and are therefore not suited for accurate measurements in industrial use. In this paper, we show that it is possible to identify and locate objects using a combination of 2D and 3D cameras. A rough estimate of the object pose is first found using a commercially available 3D camera. Then, a robotic arm with an eye-in-hand 2D camera is used to determine the pose accurately. We show that this increases the accuracy to < 1 mm and < 1◦. This was demonstrated in a real industrial assembly task where high accuracy is required.

We present MRTouch, a novel multitouch input solution for head-mounted mixed reality systems. Our system enables users to reach out and directly manipulate virtual interfaces affixed to surfaces in their environment, as though they were touchscreens. Touch input offers precise, tactile and comfortable user input, and naturally complements existing popular modalities, such as voice and hand gesture. Our research prototype combines both depth and infrared camera streams together with real-time detection and tracking of surface planes to enable robust finger-tracking even when both the hand and head are in motion. Our technique is implemented on a commercial Microsoft HoloLens without requiring any additional hardware nor any user or environmental calibration. Through our performance evaluation, we demonstrate high input accuracy with an average positional error of 5.4 mm and 95% button size of 16 mm, across 17 participants, 2 surface orientations and 4 surface materials. Finally, we demonstrate the potential of our technique to enable on-world touch interactions through 5 example applications.

This paper presents an augmented reality based system for hand rehabilitation. In this system, digital data gloves are used to detect the movements of the patients' hands and collect the physical information from the patients. Using augmented reality technology, a highly controllable environment with tasks of different levels of difficulty is provided to the patients for them to perform the rehabilitation exercises gradually. The targets of the exercises can be adjusted dynamically with respect to the physical conditions and progress of the patients. Multimodal feedbacks are provided to facilitate and encourage the patients during the rehabilitation sessions.

Mobile Augmented Reality (MAR) blends the real world with digital objects especially in ubiquitous devices such as smartphones. The MAR applications provide an intelligent interface for users. In this, valuable digital information is advertised in physical spaces. However, the success of these applications is tied directly to the degree of user acceptance. This makes understanding the needs and expectations of the MAR’s potential users of paramount importance for designing and building the proper application. The objective of the paper is to expose an important gap in the development of novel applications in the virtual world. Previous research has shown that it is essential to study and understand the needs and expectations of the potential users of the upcoming application or system. Studying user needs and expectations before offering the developed application ensures a minimum level of acceptance and, of course, success. This paper presents a detailed study comprising of a userexperience (UX) evaluation of different prototypes through the use of three different UX evaluation methods. This kind of evaluation allows new developments to offer systems, which do not fail. The main contributions of this study are that it: 1) solicits expectations when consumers use MAR applications, 2) assesses the UX over different prototypes using three different metrics, 3) provides methodological insights on UX evaluation experiments and, 4) is useful for anyone who wants to develop handheld applications after understanding user expectations and how his experience should progress. The results of the study show that users value concreteness, realizability, personalization, novelty, intuitiveness and the usefulness of presented information. Paying attention to these factors can help develop more acceptable MAR applications and lead to more novel future designs.

This paper presents a novel approach to interaction between human workers and industrial collaborative robots. The proposed approach addresses problems introduced by existing solutions for robot programming. It aims to reduce the mental demands and attention switches by centering all interaction in a shared workspace, combining various modalities and enabling interaction with the system without any external devices. The concept allows simple programming in the form of setting program parameters using spatial augmented reality for visualization and a touch-enabled table and robotic arms as input devices. We evaluated the concept utilizing a user experience study with six participants (shop-floor workers). All participants were able to program the robot and to collaborate with it using the program they parametrized. The final goal is to create a distraction-free, usable and low-effort interface for effective human-robot collaboration, enabling any ordinary skilled worker to customize the robot's program to changes in production or to personal (e.g. ergonomic) needs.

Virtual reality (VR) is now becoming a mainstream medium. Current systems like the HTC Vive offer accurate tracking of the HMD and controllers, which allows for highly immersive interactions with the virtual environment. The interactions can be further enhanced by adding feedback. As an example, a controller can vibrate when it is close to a grabbable ball. As such interactions are not exhaustingly researched, we conducted a user study. Specifically, we examine: 1. grabbing and throwing with controllers in a simple basketball game. 2. the influence of haptic and optical feedback on performance, presence, task load, and usability. 3. the advantages of VR over desktop for point-cloud editing. Several new techniques emerged from the point-cloud editor for VR. The bi-manual pinch gesture, which extends the handlebar metaphor, is a novel viewing method used to translate, rotate, and scale the point-cloud. Our new rendering technique uses the geometry shader to draw sparse point clouds quickly. The selection volumes at the controllers are our new technique to efficiently select points in point clouds. The resulting selection is visualized in real time. The results of the user study show that: 1. grabbing with a controller button is intuitive but throwing is not. Releasing a button is a bad metaphor for releasing a grabbed virtual object in order to throw it. 2. any feedback is better than none. Adding haptic, optical, or both feedback types to the grabbing improves the user performance and presence. However, only sub-scores like accuracy and predictability are significantly improved. Usability and task load are mostly unaffected by feedback. 3. the point-cloud editing is significantly better in VR with the bi-manual pinch gesture and selection volumes than on the desktop with the orbiting camera and lasso selections.


Question: An empirical analysis of warehouse order picking tasks utilizing head-mounted display technology.

This paper offers an extensive review of the literature concerning the design and control aspects of manual order picking processes in warehouses. Order picking, known for being labor-intensive and accounting for a significant portion—up to 55%—of total operating costs in warehouses, is crucial for maintaining efficiency and cost-effectiveness across the supply chain. Underperformance in this area can result in poor service quality and increased operational expenses for warehouses and, by extension, the entire supply chain. To ensure optimal performance, the order-picking process must be meticulously designed and controlled. The review focuses on key decision-making areas, including internal layout design, storage assignment strategies, routing methodologies, order batching techniques, and zoning considerations. Despite the recent surge in research on these topics, there remains a notable gap in studies that explore the synergies between these areas. Advances in order-picking systems, particularly in practical applications, are opening up exciting avenues for future research.

Activity tracking using a wearable device is an emerging research field. Large-scale studies on activity tracking performed with eyewear-type wearable devices remains a challenging area owing to the negative effect such devices have on users' looks. To cope with this challenge, JINS Inc., an eyewear retailer in Japan, has developed a state-of-the-art smart eyewear called JINS MEME. The design of JINS MEME is almost the same as that of Wellington-type eyeglasses so that people can use it in their daily lives. JINS MEME is equipped with sensors to detect a user's eye movement and head motion. In addition to these functions of JINS MEME, JINS developed an application to measure concentration levels of users. In this demonstration, users will experience wearing the JINS MEME glasses and their concentration will be measured while they perform a certain task at our booth.

Several common systems satisfy some but not all of the VR definition above. Flight simulators provide vehicle tracking, not head tracking, and do not generally operate in binocular stereo. Omnimax theaters give a large angle of view [8], occasionally in stereo, but are not interactive. Head-tracked monitors [4][6] provide all but a large angle of view. Head-mounted displays (HMD) [7][13] and BOOMs [9] use motion of the actual display screens to achieve VR by our definition. Correct projection of the imagery on large screens can also create a VR experience, this being the subject of this paper. This paper describes the CAVE (CAVE Automatic Virtual Environment) virtual reality/scientific visualization system in detail and demonstrates that projection technology applied to virtual-reality goals achieves a system that matches the quality of workstation screens in terms of resolution, color, and flicker-free stereo. In addition, this format helps reduce the effect of common tracking and system latency errors. The off-axis perspective projection techniques we use are shown to be simple and straightforward. Our techniques for doing multi-screen stereo vision are enumerated, and design barriers, past and current, are described. Advantages and disadvantages of the projection paradigm are discussed, with an analysis of the effect of tracking noise and delay on the user. Successive refinement, a necessary tool for scientific visualization, is developed in the virtual reality context. The use of the CAVE as a one-to-many presentation device at SIGGRAPH '92 and Supercomputing '92 for computational science data is also mentioned. Previous work in the VR area dates back to Sutherland [12], who in 1965 wrote about the “Ultimate Display.” Later in the decade at the University of Utah, Jim Clark developed a system that allowed wireframe graphics VR to be seen through a headmounted, BOOM-type display for his dissertation. The common VR devices today are the HMD and the BOOM. Lipscomb [4] showed a monitor-based system in the IBM booth at SIGGRAPH '91 and Deering [6] demonstrated the Virtual Portal, a closets ized three-wal l project ion-based system, in the Sun Microsystems' booth at SIGGRAPH '92. The CAVE, our projection-based VR display [3], also premiered at SIGGRAPH '92. The Virtual Portal and CAVE have similar intent, but different implementation schemes.

or more than a decade researchers have tried to create intuitive computer interfaces by blending reality and virtual reality. The goal is for people to interact with the digital domain as easily as with the real world. Various approaches help us achieve this—in the area of tangible interfaces, we use real objects as interface widgets; in augmented reality, researchers overlay 3D virtual imagery onto the real world; and in VR interfaces , we entirely replace the real world with a computer generated environment. As Milgram pointed out, 1 these types of computer interfaces can be placed along a continuum according to how much of the user's environment is computer generated (Figure 1). Tangible interfaces lie far to the left on this reality–virtuality line, while immersive virtual environments are at the right extremity. Most current user interfaces exist as discrete points along this continuum. However, human activity can't always be broken into discrete components and for many tasks users may prefer to move seamlessly along the reality–virtuality continuum. This proves true when interacting with 3D graphical content, either creating virtual models or viewing them. For example, if people want to experience a virtual scene from different scales, then immer-sive virtual reality may be ideal. If they want to have a face-to-face discussion while viewing the virtual scene, an augmented reality interface may be best. 2 The MagicBook project is an early attempt to explore how we can use a physical object to smoothly transport users between reality and virtuality. Young children often fantasize about flying into the pages of a fairy tale and becoming part of the story. The MagicBook project makes this fantasy a reality using a normal book as the main interface object. People can turn the pages of the book, look at the pictures, and read the text without any additional technology (Figure 2a). However, if a person looks at the pages through an augmented reality display, they see 3D virtual models appearing out of the pages (Figure 2b). The models appear attached to the real page so users can see the augmented reality scene from any perspective by moving themselves or the book. The virtual content can be any size and is animated, so the augmented reality view is an enhanced version of a traditional 3D pop-up book. Users can change the virtual models by turning the book pages. When they see a scene they particularly like, …

Eye-gaze is an input mode which has the potential of an efficient computer interface. Eye movement has been the focus of research in this area. Non-intrusive eyegaze tracking that allows slight head movement is addressed in this paper. A small 2D mark is employed as a reference to compensate for this movement. The iris center has been chosen for purposes of measuring eye movement. The gaze point is estimated after acquiring the eye movement data. Preliminary experimental results are given through a screen pointing application.

Virtual Reality (VR), a new computer technology, has incredible potential in the education field. The reason for this assertion is that education is a field that requires the students to understand complex data, particularly in the study of science (see "Why is science hard to learn?" by Millar) and VR makes that task easier. VR presents information in a 3-dimensional form with the participant viewing the world from inside the world (an immersed viewpoint) with the ability to interact with the information or world. People can enter a VR world that is already created or build their own. VR’s style of presentation mimics the ways that we, as humans, have learned to interact with our physical world. The requirement of learning abstract concepts (such as written language or jargon of a particular field) in order to understand data is dramatically reduced. For example, a chemistry world can show participants how electrons hover around a nucleus depending on the amount of energy involved. The abstractness of the subject would not be eliminated, but metaphors could be employed that were based solidly in reality. For instance, in the electron world the color of the world could represent the amount of energy in the system with red representing more energy than blue. For the students, they would be able to relate to hot things looking red and cold things looking blue without the concept of "joules increasing" getting in the way.

Using vision systems with SCARA robotic manipulators is beneficial in many industrial applications specially in automated object picking and placing tasks. In this paper, the equations of forward and inverse kinematics of a SCARA manipulator are derived using D-H parameters convention. The mechanical structure, as well as the electrical, electronic and pneumatic components of the manipulator were designed and developed. The manipulator's control software, animation and simulation were developed using LabVIEW and Arduino. In addition to that, a vision system was developed and integrated with the manipulator system where a mobile phone camera was used to acquire images, then transmit them wirelessly to LabVIEW's Vision Assistant codes for processing and calibration. The developed vision system allowed detection and calculation of object positions within the manipulator's workspace. Finally, the manipulator's positioning errors were determined and discussed based on a number of repeatability experiments.

Grabbing and manipulating virtual objects is an important user interaction for immersive virtual environments. We present implementations and discussion of six techniques which allow manipulation of remote objects. A user study of these techniques was performed which revealed their characteristics and deficiencies, and led to the development of a new class of techniques. These hybrid techniques provide istinct advantages in terms of ease of use and efficiency because they consider the tasks of grabbing and manipulation separately. CR Categories and Subject Descr iptors: I.3.7 [Computer Graphics]:Three-Dimensional Graphics and Realism Virtual Reality; I.3.6 [Computer Graphics]:Methodology and Techniques Interaction Techniques.

We describe a novel system for rapid prototyping of laserbased optical and holographic layouts. Users of this optical prototyping tool – called the Illuminating Light system – move physical representations of various optical elements about a workspace, while the system tracks these components and projects back onto the workspace surface the simulated propagation of laser light through the evolving layout. This application is built atop the Luminous Room infrastructure, an aggregate of interlinked, computer-controlled projector-camera units called I/O Bulbs. Philosophically, the work embodies the emerging ideas of the Luminous Room and builds on the notions of ‘graspable media’. We briefly introduce the I/O Bulb and Luminous Room concepts and discuss their current implementations. After an overview of the optical domain that the Illuminating Light system is designed to address, we present the overall system design and implementation, including that of an intermediary toolkit called voodoo which provides a general facility for object identification and tracking.

The present study is designed to test the role of immersion and media content in the sense of presence. Specifically, we are interested in the affective valence of the virtual environments. This paper describes an experiment that compares three immersive systems (a PC monitor, a rear projected video wall, and a head-mounted display) and two virtual environments, one involving emotional content and the other not. The purpose of the experiment was to test the interactive role of these two media characteristics (form and content). Scores on two self-report presence measurements were compared among six groups of 10 people each. The results suggest that both immersion and affective content have an impact on presence. However, immersion was more relevant for non-emotional environments than for emotional ones.

In this paper we present a novel wearable navigation system along with implicit HCI (iHCI) model, where interaction with technology is dissolved into a day-today activity. In this type of HCI model a computer takes the input and tries to output an action that is a proactive anticipation of next action of a user. Usually, in urban areas people use voice assisted navigation systems or navigation guidelines displayed on a mobile phone. Some navigation systems are already installed on car dashboard, which needs explicit attention in order to make driving decisions. A navigation system using haptic perception to guide a user throughout a journey is the key contribution of this paper. It does not ask for explicit user attention and demonstrates the indolent form of technological interaction. This wearable device is an index finger sleeve, which consists of vibrator modules, Bluetooth communication module and Microcontroller Unit (MCU). A working prototype has been built and tested.