Question: Using affordances as a framework for controlling robots.

In pursuit of understanding the complexities of developmental structure, physical embodiment, integration of diverse sensory and motor systems, and social engagement, our research team has developed an advanced humanoid robot, named Cog. This robot is equipped with twenty-one degrees of freedom and an array of sensory capabilities, encompassing vision, audition, vestibular, kinesthetic, and tactile perceptions. This paper delves into the methodology employed in our studies, elucidates the research questions that have emerged during the project's progression, and offers an overview of the project's current status and our ambitious long-term objectives. We detail a range of implemented visual-motor functions, such as smooth-pursuit tracking, saccades, binocular vergence, and vestibular-ocular and opto-kinetic reflexes. Additionally, we discuss orientation behaviors, motor control strategies, and social interactions, including pointing to a visual target, recognizing joint attention through facial and ocular cues, imitating head nods, and managing interactions through expressive feedback mechanisms. We also delineate several promising avenues for future research aimed at creating a fully functional embodied system, capable of navigating complex social environments and demonstrating sophisticated motor skills.

This research underscores the significance of employing a humanoid robot like Cog to investigate the intricate interplay between physical embodiment and cognitive development, highlighting the potential for robots to serve as valuable tools in understanding human behavior and cognition. Through the implementation of various motor and social routines, we aim to shed light on the foundational mechanisms underlying human-like intelligence and social adeptness. The Cog project, with its focus on the integration of multiple sensory systems and motor control techniques, serves as a stepping stone towards building robots capable of more nuanced and human-like interactions, potentially revolutionizing fields such as robotics, psychology, and cognitive science.

Effective physical Human-Robot Interaction (pHRI) needs to account for variable human dynamics and also predict human intent. Recently, there has been a lot of progress in adaptive impedance and admittance control for human-robot interaction. Not as many contributions have been reported on online adaptation schemes that can accommodate users with varying physical strength and skill level during interaction with a robot. The goal of this paper is to present and evaluate a novel adaptive admittance controller that can incorporate human intent, nominal task models, as well as variations in the robot dynamics. An outer-loop controller is developed using an ARMA model which is tuned using an adaptive inverse control technique. An inner-loop neuroadaptive controller linearizes the robot dynamics. Working in conjunction and online, this two-loop technique offers an elegant way to decouple the pHRI problem. Experimental results are presented comparing the performance of different types of admittance controllers. The results show that efficient online adaptation of the robot admittance model for different human subjects can be achieved. Specifically, the adaptive admittance controller reduces jerk which results in a smooth human-robot interaction.

Progress robot development very varied almost all fields seize the attention of the world's scientists and researchers. For some cases, robots are generally used to help humans in relieving job. The robot can replace human mobilization for move so that the efficiency of wasted time will be less. Many things need to be note what kind of robot control reliable to replace human work in mobilization. Virtual world as in android smartphone greatly benefit humans to creating control tools easy to use anywhere and could represent a desire humans. By making control robot using virtual paths on the Android smartphone which have an accuracy of more than 70% to a scale of 1:50 with the actual track in an environment that is not wide enough, the advantage can be used to control the robot move easily and efficiently compared to the usual manual control robot that will take more time to control The robot moves its place as desired.

Interpretation of images and videos containing humans interacting with different objects is a daunting task. It involves understanding scene or event, analyzing human movements, recognizing manipulable objects, and observing the effect of the human movement on those objects. While each of these perceptual tasks can be conducted independently, recognition rate improves when interactions between them are considered. Motivated by psychological studies of human perception, we present a Bayesian approach which integrates various perceptual tasks involved in understanding human-object interactions. Previous approaches to object and action recognition rely on static shape or appearance feature matching and motion analysis, respectively. Our approach goes beyond these traditional approaches and applies spatial and functional constraints on each of the perceptual elements for coherent semantic interpretation. Such constraints allow us to recognize objects and actions when the appearances are not discriminative enough. We also demonstrate the use of such constraints in recognition of actions from static images without using any motion information.

Care issues and costs associated with an increasing elderly population are becoming a major concern for many countries. The use of assistive robots in “smart-home” environments has been suggested as a possible partial solution to these concerns. A challenge is the personalization of the robot to meet the changing needs of the elderly person over time. One approach is to allow the elderly person, or their carers or relatives, to make the robot learn activities in the smart home and teach it to carry out behaviors in response to these activities. The overriding premise being that such teaching is both intuitive and “nontechnical.” To evaluate these issues, a commercially available autonomous robot has been deployed in a fully sensorized but otherwise ordinary suburban house. We describe the design approach to the teaching, learning, robot, and smart home systems as an integrated unit and present results from an evaluation of the teaching component with 20 participants and a preliminary evaluation of the learning component with three participants in a human-robot interaction experiment. Participants reported findings using a system usability scale and ad-hoc Likert questionnaires. Results indicated that participants thought that this approach to robot personalization was easy to use, useful, and that they would be capable of using it in real-life situations both for themselves and for others.

A framework for the analysis and control of manipulator systems with respect to the dynamic behavior of their end-effectors is developed. First, issues related to the description of end-effector tasks that involve constrained motion and active force control are discussed. The fundamentals of the operational space formulation are then presented, and the unified approach for motion and force control is developed. The extension of this formulation to redundant manipulator systems is also presented, constructing the end-effector equations of motion and describing their behavior with respect to joint forces. These results are used in the development of a new and systematic approach for dealing with the problems arising at kinematic singularities. At a singular configuration, the manipulator is treated as a mechanism that is redundant with respect to the motion of the end-effector in the subspace of operational space orthogonal to the singular direction.

We present the novel kinematics, workspace characterization, functional prototype and impedance control of a six degrees of freedom haptic interface designed to train surgeons for laparoscopic procedures, through virtual reality simulations. The parallel kinematics of the device is constructed by connecting a 3RRP planar parallel mechanism to a linearly actuated modified delta mechanism with a connecting link. The configuration level forward and inverse kinematics of the device assume analytic solutions, while its workspace can be shaped to enable large end-effector translations and rotations, making it well-suited for laparoscopy operations. Furthermore, the haptic interface features a low apparent inertia with high structural stiffness, thanks to its parallel kinematics with grounded actuators. A model-based open-loop impedance controller with feed-forward gravity compensation has been implemented for the device and various virtual tissue/organ stiffness levels have been rendered.

This work proposes a framework that enables arbitrary robots with unknown kinematics models to imitate human demonstrations to acquire a skill, and reproduce it in real-time. The diversity of robots active in non-laboratory environments is growing constantly, and to this end we present an approach for users to be able to easily teach a skill to a robot with any body configuration. Our proposed method requires a motion trajectory obtained from human demonstrations via a Kinect sensor, which is then projected onto a corresponding human skeleton model. The kinematics mapping between the robot and the human model is learned by employing Local Procrustes Analysis, which enables the transfer of the demonstrated trajectory from the human model to the robot. Finally, the transferred trajectory is modeled using Dynamic Movement Primitives, allowing it to be reproduced in real time. Experiments in simulation on a 4 degree of freedom robot show that our method is able to correctly imitate various skills demonstrated by a human.

Grabbing and manipulating virtual objects is an important user interaction for immersive virtual environments. We present implementations and discussion of six techniques which allow manipulation of remote objects. A user study of these techniques was performed which revealed their characteristics and deficiencies, and led to the development of a new class of techniques. These hybrid techniques provide istinct advantages in terms of ease of use and efficiency because they consider the tasks of grabbing and manipulation separately. CR Categories and Subject Descr iptors: I.3.7 [Computer Graphics]:Three-Dimensional Graphics and Realism Virtual Reality; I.3.6 [Computer Graphics]:Methodology and Techniques Interaction Techniques.

We describe the architecture, algorithms, and experiments with HERB, an autonomous mobile manipulator that performs useful manipulation tasks in the home. We present new algorithms for searching for objects, learning to navigate in cluttered dynamic indoor scenes, recognizing and registering objects accurately in high clutter using vision, manipulating doors and other constrained objects using caging grasps, grasp planning and execution in clutter, and manipulation on pose and torque constraint manifolds. S.S. Srinivasa ( ) · D. Ferguson · C.J. Helfrich Intel Research Pittsburgh, 4720 Forbes Avenue, Suite 410, Pittsburgh, PA 15213, USA e-mail: siddhartha.srinivasa@intel.com C.J. Helfrich e-mail: casey.j.helfrich@intel.com D. Berenson · A. Collet · R. Diankov · G. Gallagher · G. Hollinger · J. Kuffner · M.V. Weghe The Robotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA D. Berenson e-mail: dberenso@ri.cmu.edu A. Collet e-mail: acollet@ri.cmu.edu R. Diankov e-mail: rdiankov@ri.cmu.edu G. Gallagher e-mail: ggallagh@ri.cmu.edu G. Hollinger e-mail: gholling@ri.cmu.edu J. Kuffner e-mail: kuffner@ri.cmu.edu M.V. Weghe e-mail: vandeweg@ri.cmu.edu We also present numerous severe real-world test results from the integration of these algorithms into a single mobile manipulator.

Hand drawn sketches are natural means by which a high level description of an environment can be provided. They can be exploited to impart coarse prior information about the scene to a robot, thereby enabling it to perform autonomous navigation and exploration when a full metrical description of the scene is not available beforehand. In this paper, we present a navigation system supplemented by a tablet interface that allows a user to sketch a rough map of an indoor environment and a desired trajectory for the robot to follow. We propose a novel theoretical framework for sketch interpretation based upon the manifold formalism in which associations between the sketch and the real world are modeled as local deformation of a suitable metric manifold. We also present empirical results from experimental evaluations of our approach in real world scenarios both from the perspective of the navigation capability and the usability of the interface.


Question: Affordances serving as a framework for the control of robotic systems.

This article delves into the essence of tutoring as a facilitator in the problem-solving journey, particularly focusing on the dynamics between a knowledgeable adult or "expert" and a less experienced or younger individual. The exploration is grounded in a specific scenario where a tutor aims to guide children aged 3 to 5 years in constructing a three-dimensional structure, a task that initially exceeds their capabilities. This scenario mirrors the common tutoring context where one party possesses the solution while the other does not, akin to a practical session where only the instructor has the know-how. The evolving interaction between the tutor and children forms the empirical foundation of the study.

In the early stages of development, children frequently engage in problem-solving activities that are assisted and nurtured by more skilled individuals. From the moment of birth, children exhibit innate problem-solving abilities (Bruner, 1973), but their efforts are often bolstered by the guidance of more adept companions (Kaye, 1970). This guidance is crucial in learning various skills, including attending, communicating, manipulating objects, locomotion, and even refining problem-solving strategies themselves. Tutorial interactions are a defining characteristic of the infancy and childhood experience. Notably, humans appear to be the only species engaging in intentional tutoring (Bruner, 1972; Hinde, 1971), although higher primates do learn by observing their elders (Hamburg, 1968; van Lawick-Goodall, 1968), without evidence of active instruction from their seniors. The ability to teach, in addition to learning, sets humans apart as a species.

The central objective of this paper is to scrutinize the significant implications of the interactive and instructional relationship between the developing child and their elders for the study of skill acquisition and problem-solving. Skill acquisition in children can be conceptualized as a hierarchical process, where foundational skills are integrated into more complex skills through appropriate orchestration to address increasingly sophisticated task demands (Bruner, 1973). This process mirrors problem-solving, where mastery of "lower order" problems is a prerequisite for tackling larger, more intricate challenges, with each level impacting the other. A prime example is reading, where deciphering words enables the understanding of sentences, and in turn, sentences aid in deciphering specific words (F. Smith, 1971). Given the learner's persistent intent and an arsenal of constituent skills, the pivotal challenge often lies in the orchestration of these skills to achieve a common goal.

The paper describes the system developed by researchers from MIT for the Defense Advanced Research Projects Agency's (DARPA) Virtual Robotics Challenge (VRC), held in June 2013. The VRC was the first competition in the DARPA Robotics Challenge (DRC), a program that aims to “develop ground robotic capabilities to execute complex tasks in dangerous, degraded, human-engineered environments”. The VRC required teams to guide a model of Boston Dynamics' humanoid robot, Atlas, through driving, walking, and manipulation tasks in simulation. Team MIT's user interface, the Viewer, provided the operator with a unified representation of all available information. A 3D rendering of the robot depicted its most recently estimated body state with respect to the surrounding environment, represented by point clouds and texture-mapped meshes as sensed by on-board LIDAR and fused over time.

Localizing functional regions of objects or affordances is an important aspect of scene understanding and relevant for many robotics applications. In this work, we introduce a pixel-wise annotated affordance dataset of 3090 images containing 9916 object instances. Since parts of an object can have multiple affordances, we address this by a convolutional neural network for multilabel affordance segmentation. We also propose an approach to train the network from very few keypoint annotations. Our approach achieves a higher affordance detection accuracy than other weakly supervised methods that also rely on keypoint annotations or image annotations as weak supervision.

Recent advances in technology are delivering robots of reduced size and cost. A natural outgrowth of these advances are systems comprised of large numbers of robots that collaborate autonomously in diverse applications. Research on effective autonomous control of such systems, commonly called swarms, has increased dramatically in recent years and received attention from many domains, such as bioinspired robotics and control theory. These kinds of distributed systems present novel challenges for the effective integration of human supervisors, operators, and teammates that are only beginning to be addressed. This paper is the first survey of human-swarm interaction (HSI) and identifies the core concepts needed to design a human-swarm system. We first present the basics of swarm robotics. Then, we introduce HSI from the perspective of a human operator by discussing the cognitive complexity of solving tasks with swarm systems. Next, we introduce the interface between swarm and operator and identify challenges and solutions relating to human-swarm communication, state estimation and visualization, and human control of swarms. For the latter, we develop a taxonomy of control methods that enable operators to control swarms effectively. Finally, we synthesize the results to highlight remaining challenges, unanswered questions, and open problems for HSI, as well as how to address them in future works.

Brain Computer Interfaces (BCI) provide the opportunity to control external devices using the brain ElectroEncephaloGram (EEG) signals. In this paper we propose two software framework in order to control a 5 degree of freedom robotic and prosthetic hand. Results are presented where an Emotiv Cognitive Suite (i.e. the 1 st framework) combined with an embedded software system (i.e. an open source Arduino board) is able to control the hand through character input associated with the taught actions of the suite. This system provides evidence of the feasibility of brain signals being a viable approach to controlling the chosen prosthetic. Results are then presented in the second framework. This latter one allowed for the training and classification of EEG signals for motor imagery tasks. When analysing the system, clear visual representations of the performance and accuracy are presented in the results using a confusion matrix, accuracy measurement and a feedback bar signifying signal strength. Experiments with various acquisition datasets were carried out and with a critical evaluation of the results given. Finally depending on the classification of the brain signal a Python script outputs the driving command to the Arduino to control the prosthetic. The proposed architecture performs overall good results for the design and implementation of economically convenient BCI and prosthesis.

Visual servoing in telerobotics provides information to the operator about the remote location and assists in the task execution to reduce stress on the user. Occlusions in this scenario can, on the one hand, lead to the visual servoing failure, and, on the other hand, degrade the user experience and navigation performance due to the obstructed vision of elements of interest. In this letter, we consider a teleoperation system composed of two robot arms where one is remotely operated, while the other is autonomous and equipped with an eye-in-hand camera sensor. We propose a reactive unified convex optimization based controller that allows the execution of occlusion-free tasks by autonomously adjusting the camera robot, so as to keep the teleoperated one in the field of view. The occlusion avoidance is formulated inside the optimization as a constraint in the image space, it is formally derived from a collision avoidance analogy and made robust against noisy measurements and dynamic environment. A state machine is used to switch the control policy whenever an occlusion might occur. We validate our approach with experiments on a 14 d.o.f. dual-arm ABB YuMi robot equipped with an red, green, blue (RGB) camera and teleoperated by a 3 d.o.f. Novint Falcon device.

There have been several recent efforts to build behavior-based autonomous creatures. While competent autonomous action is highly desirable, there is an important need to integrate autonomy with “directability”. In this paper we discuss the problem of building autonomous animated creatures for interactive virtual environments which are also capable of being directed at multiple levels. We present an approach to control which allows an external entity to “direct” an autonomous creature at the motivational level, the task level, and the direct motor level. We also detail a layered architecture and a general behavioral model for perception and action-selection which incorporates explicit support for multi-level direction. These ideas have been implemented and used to develop several autonomous animated creatures.

We describe the architecture, algorithms, and experiments with HERB, an autonomous mobile manipulator that performs useful manipulation tasks in the home. We present new algorithms for searching for objects, learning to navigate in cluttered dynamic indoor scenes, recognizing and registering objects accurately in high clutter using vision, manipulating doors and other constrained objects using caging grasps, grasp planning and execution in clutter, and manipulation on pose and torque constraint manifolds. S.S. Srinivasa ( ) · D. Ferguson · C.J. Helfrich Intel Research Pittsburgh, 4720 Forbes Avenue, Suite 410, Pittsburgh, PA 15213, USA e-mail: siddhartha.srinivasa@intel.com C.J. Helfrich e-mail: casey.j.helfrich@intel.com D. Berenson · A. Collet · R. Diankov · G. Gallagher · G. Hollinger · J. Kuffner · M.V. Weghe The Robotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA D. Berenson e-mail: dberenso@ri.cmu.edu A. Collet e-mail: acollet@ri.cmu.edu R. Diankov e-mail: rdiankov@ri.cmu.edu G. Gallagher e-mail: ggallagh@ri.cmu.edu G. Hollinger e-mail: gholling@ri.cmu.edu J. Kuffner e-mail: kuffner@ri.cmu.edu M.V. Weghe e-mail: vandeweg@ri.cmu.edu We also present numerous severe real-world test results from the integration of these algorithms into a single mobile manipulator.

A flexible microactuator (FMA) driven by an electropneumatic (or electrohydraulic) system has been developed. It has three degrees of freedom-pitch, yaw, and stretch-making it suitable for robotic mechanisms such as fingers, arms, or legs. It is made of fiber-reinforced rubber, and the mechanism is very simple, enabling miniature robots without conventional link mechanisms to be designed. Serially connected FMAs act as a miniature robot manipulator. The kinematics and control algorithm for this type of robot are presented. FMAs combined in parallel act as a multifingered robot hand, with each FMA representing a finger. An algorithm for the cooperative control for such FMAs, the stable region for holding, and its performance have been developed.<<ETX>>

Abstmct-Designing robots that learn by themselves to perform complex real-world tasks is a still-open challenge for the field of Robotics and Artificial Intelligence. In this paper we present the robot learning problem as a lifelong problem, in which a robot faces a collection of tasks over its entire lifetime. Such a scenario provides the opportunity to gather general-purpose knowledge that transfers across tasks. We illustrate a particular learning mechanism, explanation-based neural network learning, that transfers knowledge between related tasks via neural network action models. The learning approach is illustrated using a mobile robot, equipped with visual, ultrasonic and laser sensors. In less than 10 minutes operation time, the robot is able to learn to navigate to a marked target object in a natural office environment.

Robotic surgery systems are highly complex and expensive pieces of equipment. Demands for lower cost of care can be met if these systems are employable in a flexible manner for a large variety of procedures. To protect the initial investment the capabilities of a robotic system need to be expandable as new tasks arise.


Question: Odin's Runes: A rule-based language for extracting information.

This paper introduces an efficient and precise dependency parser that utilizes neural networks, addressing the limitations of current parsers which often rely on numerous sparse indicator features. These features, while prevalent, suffer from poor generalization and significantly impact parsing speed due to the high cost of feature computation. We present a novel methodology that involves training a neural network classifier for implementation within a greedy, transition-based dependency parser. This classifier is designed to learn and utilize a limited set of dense features, resulting in enhanced processing speed and a notable 2% increase in both unlabeled and labeled attachment scores on datasets in English and Chinese. Specifically, our parser can process over 1000 sentences per second, achieving an impressive 92.2% unlabeled attachment score on the English Penn Treebank, demonstrating its superior performance compared to traditional methods.

The paper systematically compares two feature extraction algorithms to mine product features commented on in customer reviews. The first approach [17] identifies candidate features by applying a set of POS patterns and pruning the candidate set based on the log likelihood ratio test. The second approach [11] applies association rule mining for identifying frequent features and a heuristic based on the presence of sentiment terms for identifying infrequent features. We evaluate the performance of the algorithms on five product specific document collections regarding consumer electronic devices. We perform an analysis of errors and discuss advantages and limitations of the algorithms.

This paper discusses the use of our information extraction (IE) system, Textract, in the questionanswering (QA) track of the recently held TREC-8 tests. One of our major objectives is to examine how IE can help IR (Information Retrieval) in applications like QA. Our study shows: (i) IE can provide solid support for QA; (ii ) low-level IE like Named Entity tagging is often a necessary component in handling most types of questions; (iii ) a robust natural language shallow parser provides a structural basis for handling questions; (iv) high-level domain independent IE, i.e. extraction of multiple relationships and general events, is expected to bring about a breakthrough in QA.

The 2010 i2b2/VA Workshop on Natural Language Processing Challenges for Clinical Records presented three tasks: a concept extraction task focused on the extraction of medical concepts from patient reports; an assertion classification task focused on assigning assertion types for medical problem concepts; and a relation classification task focused on assigning relation types that hold between medical problems, tests, and treatments. i2b2 and the VA provided an annotated reference standard corpus for the three tasks. Using this reference standard, 22 systems were developed for concept extraction, 21 for assertion classification, and 16 for relation classification. These systems showed that machine learning approaches could be augmented with rule-based systems to determine concepts, assertions, and relations. Depending on the task, the rule-based systems can either provide input for machine learning or post-process the output of machine learning. Ensembles of classifiers, information from unlabeled data, and external knowledge sources can help when the training data are inadequate.

In this paper we present the data manipulation facility for a structured English query language (SEQUEL) which can be used for accessing data in an integrated relational data base. Without resorting to the concepts of bound variables and quantifiers SEQUEL identifies a set of simple operations on tabular structures, which can be shown to be of equivalent power to the first order predicate calculus. A SEQUEL user is presented with a consistent set of keyword English templates which reflect how people use tables to obtain information. Moreover, the SEQUEL user is able to compose these basic templates in a structured manner in order to form more complex queries. SEQUEL is intended as a data base sublanguage for both the professional programmer and the more infrequent data base user.

In recent years patents have become increasingly important for businesses to protect their intellectual capital and as a valuable source of information. Patent information is, however, not employed to its full potential and the interpretation of structured and unstructured patent information in large volumes remains a challenge. We address this by proposing an integrated interdisciplinary approach that uses natural language processing and machine learning techniques to formalize multilingual patent information in an ontology. The ontology further contains patent and domain specific knowledge, which allows for aligning patents with technological fields of interest and other business-related artifacts. Our empirical evaluation shows that for categorizing patents according to well-known technological fields of interest, the approach achieves high accuracy with selected feature sets compared to related work focussing on monolingual patents. We further show that combining OWL RL reasoning with SPARQL querying over the patent knowledge base allows for answering complex business queries and illustrate this with real-world use cases from the automotive domain.

Ontology have become a relevant representation formalism and many application domains are  considering adopting them. This attention claims for methods for reusing domain knowledge resources in the development of domain ontologies. Accordingly, in this paper we discuss a general methodology to create domain ontology for more than one object oriented language (OOP) like Java, PHP and C++. A lot of software development methods specially Web applications have presented most of these methods that are focusing on the structure of distributed systems and security, in which they are connected through networks and the internet; resulting in more valuable business and critical assets stored, searched and manipulated by World Wide Web. The aims of this study building domain ontology for OOP language classes for different OOP languages or different versions of the same language is an exciting opportunity for researchers to access the information required under the constant increase in the volume of information disseminated on the Internet. By creating Ontology domain for OOP, we can Improve methods of viewing and 1 E-mail: iabuhassan@newsoft.ps 2 E-mail: akram.othman@gmail.com Article Info: Received : October 14, 2012. Revised : November 19, 2012 Published online : December 30, 2012 76 Domain Ontology for Programming Languages organizing information, improve the way of processing, in addition to increasing the vocabulary and their relationship to terminology as well as the rules used in natural language with OOP languages. The clear identification of the properties and relations of terms is the starting point to become Ontology domain. The importance of the domain Ontology among object oriented programming languages is that through the synthesis of these relationships or Ontology an OOP can be achieved through web by any junior programmers.

As a typical type of structured documents, table drawings are widely used in engineering fields. Knowledge extraction of such structured documents plays an important role in automatic interpretation systems. In this paper, we propose a new knowledge extraction method based on automatically analyzing drawing layout and extracting physical or logical structures from the given engineering table drawings. Then based on the automatic interpretation results, we further propose normalization method to integrate varied types of engineering tables with other engineering drawings and extract implied domain knowledge.

A semantic file system is an information storage system that provides flexible associative access to the system's contents by automatically extracting attributes from files with file type specific transducers. Associative access is provided by a conservative extension to existing tree-structured file system protocols, and by protocols that are designed specifically for content based access. Compatiblity with existing file system protocols is provided by introducing the concept of a virtual directory. Virtual directory names are interpreted as queries, and thus provide flexible associative access to files and directories in a manner compatible with existing software. Rapid attribute-based access to file system contents is implemented by automatic extraction and indexing of key properties of file system objects. The automatic indexing of files and directories is called "semantic" because user programmable transducers use information about the semantics of updated file system objects to extract the properties for indexing. Experimental results from a semantic file system implementation support the thesis that semantic file systems present a more effective storage abstraction than do traditional tree structured file systems for information sharing and command level programming.

When animation of a humanoid figure is to be generated at run-time, instead of by replaying precomposed motion clips, some method is required of specifying the avatar’s movements in a form from which the required motion data can be automatically generated. This form must be of a more abstract nature than raw motion data: ideally, it should be independent of the particular avatar’s proportions, and both writable by hand and suitable for automatic generation from higher-level descriptions of the required actions. We describe here the development and implementation of such a scripting language for the particular area of sign languages of the deaf, called SiGML (Signing Gesture Markup Language), based on the existing HamNoSys notation for sign languages. We conclude by suggesting how this work may be extended to more general animation for interactive virtual reality applications.

This research proposes an expanded semantic graph definition that serves as a basis for an expanded semantic graph representation and graph matching approach. This representation separates the content and context and adds a number of semantic structures that encapsulate inferred information. The expanded semantic graph approach facilitates finding additional matches, identifying and eliminating poor matches, and prioritizing matches based on how much new knowledge is provided. By focusing on information of interest, doing pre-processing, and reducing processing requirements, the approach is applicable to problems where related information of interest is sought across a massive body of free text documents. Key aspects of the approach include (1) expanding the nodes and edges through inference using DL-Safe rules, abductive hypotheses, and syntactic patterns, (2) separating semantic content into nodes and semantic context into edges, and (3) applying relatedness measures on a node, edge, and sub graph basis. Results from tests using a ground-truthed subset of a large dataset of law enforcement investigator emails illustrate the benefits of these approaches.


Question: Odin's Runes: A rule-based language for the extraction of information.

This paper outlines the CoNLL 2008 Shared Task, which was centered on the concurrent parsing of syntactic and semantic dependencies. The annual Conference on Computational Natural Language Learning features a shared task designed to foster advancements and standardize the evaluation of natural language processing tools. In 2008, the focus was on integrating syntactic and semantic dependency parsing, consolidating and expanding upon the themes of the previous four years. This task marked a notable advancement by enriching syntactic dependencies with additional details, such as named-entity boundaries, and by encompassing roles associated with both verbal and nominal predicates in its semantic dependencies. Here, we delineate the parameters of the shared task and detail the methodology used in the creation of the data sets. Additionally, we present and scrutinize the outcomes, alongside an overview of the strategies employed by the participating systems.

We describe efforts to adapt the Tesseract open source OCR engine for multiple scripts and languages. Effort has been concentrated on enabling generic multi-lingual operation such that negligible customization is required for a new language beyond providing a corpus of text. Although change was required to various modules, including physical layout analysis, and linguistic post-processing, no change was required to the character classifier beyond changing a few limits. The Tesseract classifier has adapted easily to Simplified Chinese. Test results on English, a mixture of European languages, and Russian, taken from a random sample of books, show a reasonably consistent word error rate between 3.72% and 5.78%, and Simplified Chinese has a character error rate of only 3.77%.

Background Since Big Data mainly aims to explore the correlation between surface features but not their underlying causality relationship, the Big Mechanism 2 program has been proposed by DARPA to find out “why” behind the “Big Data”. However, the pre-requisite for it is that the machine can read each document and learn its associated knowledge, which is the task of Machine Reading (MR). Since a domain-independent MR system is complicated and difficult to build, the math word problem (MWP) [1] is frequently chosen as the first test case to study MR (as it usually uses less complicated syntax and requires less amount of domain knowledge). According to the framework for making the decision while there are several candidates, previous MWP algebra solvers can be classified into: (1) Rule-based approaches with logic inference [2-7], which apply rules to get the answer (via identifying entities, quantities, operations, etc.) with a logic inference engine. (2) Rule-based approaches without logic inference [8-13], which apply rules to get the answer without a logic inference engine. (3) Statistics-based approaches [14, 15], which use statistical models to identify entities, quantities, operations, and get the answer. To our knowledge, all the statistics-based approaches do not adopt logic inference. The main problem of the rule-based approaches mentioned above is that the coverage rate problem is serious, as rules with wide coverage are difficult and expensive to construct. Also, since they adopt Go/No-Go approach (unlike statistical approaches which can adopt a large Top-N to have high including rates), the error accumulation problem would be severe. On the other hand, the main problem of those approaches without adopting logic inference is that they usually need to implement a new handling procedure for each new type of problems (as the general logic inference mechanism is not adopted). Also, as there is no inference engine to generate the reasoning chain [16], additional effort would be required for

We describe a new formalism for storing linguistic data in a text to speech system. Linguistic entities such as words and phones are stored as feature structures in a general object called an linguistic item. Items are configurable at run time and via the feature structure can contain arbitrary information. Linguistic relations are used to store the relationship between items of the same linguistic type. Relations can take any graph structure but are commonly trees or lists. Utterance structures contain all the items and relations contained in a single utterance. We first describe the design goals when building a synthesis architecture, and then describe some problems with previous architectures. We then discuss our new formalism in general along with the implementation details and consequences of our approach.

MOTIVATION
Natural language processing (NLP) methods are regarded as being useful to raise the potential of text mining from biological literature. The lack of an extensively annotated corpus of this literature, however, causes a major bottleneck for applying NLP techniques. GENIA corpus is being developed to provide reference materials to let NLP techniques work for bio-textmining.

RESULTS
GENIA corpus version 3.0 consisting of 2000 MEDLINE abstracts has been released with more than 400,000 words and almost 100,000 annotations for biological terms.

Populating large-scale structured databases from unstructured sources is a critical and challenging task in data analytics. As automated feature engineering methods grow increasingly prevalent, constructing sufficiently large labeled training sets has become the primary hurdle in building machine learning information extraction systems. In light of this, we have taken a new approach called data programming [7]. Rather than hand-labeling data, in the data programming paradigm, users generate large amounts of noisy training labels by programmatically encoding domain heuristics as simple rules. Using this approach over more traditional distant supervision methods and fully supervised approaches using labeled data, we have been able to construct knowledge base systems more rapidly and with higher quality. Since the ability to quickly prototype, evaluate, and debug these rules is a key component of this paradigm, we introduce DDLite, an interactive development framework for data programming. This paper reports feedback collected from DDLite users across a diverse set of entity extraction tasks. We share observations from several DDLite hackathons in which 10 biomedical researchers prototyped information extraction pipelines for chemicals, diseases, and anatomical named entities. Initial results were promising, with the disease tagging team obtaining an F1 score within 10 points of the state-of-the-art in only a single day-long hackathon's work. Our key insights concern the challenges of writing diverse rule sets for generating labels, and exploring training data. These findings motivate several areas of active data programming research.

When animation of a humanoid figure is to be generated at run-time, instead of by replaying precomposed motion clips, some method is required of specifying the avatar’s movements in a form from which the required motion data can be automatically generated. This form must be of a more abstract nature than raw motion data: ideally, it should be independent of the particular avatar’s proportions, and both writable by hand and suitable for automatic generation from higher-level descriptions of the required actions. We describe here the development and implementation of such a scripting language for the particular area of sign languages of the deaf, called SiGML (Signing Gesture Markup Language), based on the existing HamNoSys notation for sign languages. We conclude by suggesting how this work may be extended to more general animation for interactive virtual reality applications.

Recent advances in natural language processing have produced libraries that extract lowlevel features from a collection of raw texts. These features, known as annotations, are usually stored internally in hierarchical, tree-based data structures. This paper proposes a data model to represent annotations as a collection of normalized relational data tables optimized for exploratory data analysis and predictive modeling. The R package cleanNLP, which calls one of two state of the art NLP libraries (CoreNLP or spaCy), is presented as an implementation of this data model. It takes raw text as an input and returns a list of normalized tables. Specific annotations provided include tokenization, part of speech tagging, named entity recognition, sentiment analysis, dependency parsing, coreference resolution, and word embeddings. The package currently supports input text in English, German, French, and Spanish.

Medication information is one of the most important types of clinical data in electronic medical records. It is critical for healthcare safety and quality, as well as for clinical research that uses electronic medical record data. However, medication data are often recorded in clinical notes as free-text. As such, they are not accessible to other computerized applications that rely on coded data. We describe a new natural language processing system (MedEx), which extracts medication information from clinical notes. MedEx was initially developed using discharge summaries. An evaluation using a data set of 50 discharge summaries showed it performed well on identifying not only drug names (F-measure 93.2%), but also signature information, such as strength, route, and frequency, with F-measures of 94.5%, 93.9%, and 96.0% respectively. We then applied MedEx unchanged to outpatient clinic visit notes. It performed similarly with F-measures over 90% on a set of 25 clinic visit notes.

Ontologies are knowledge constructs essential for creation of the Web of Data. Good documentation is required to permit people to understand ontologies and thus employ them correctly, but this is costly to create by tradition authorship methods, and is thus inefficient to create in this way until an ontology has matured into a stable structure. The authors describe three tools, LODE, Parrot and the OWLDoc-based Ontology Browser, that can be used automatically to create documentation from a well-formed OWL ontology at any stage of its development. They contrast their properties and then report on the authors’ evaluation of their effectiveness and usability, determined by two task-based user testing sessions.

What would it take to teach a computer to play a game entirely through language and sketching? In this paper, we present an implemented program through which an instructor can teach the rules of simple board games using such input. We describe the architecture, information flow, and vocabulary of instructional events and walk through an annotated example. In our approach, the instructional and communication events guide abductive reasoning for language interpretation and help to integrate information from sketching and language. Having a general target representation enables the learning process to be viewed more as translation and problem solving than as induction. Lastly, learning by demonstration complements and extends instruction, resulting in concrete, operational rules.


Question: Odin's Runes: A rule-based language for extracting information.

This paper delves into the Stanford typed dependencies representation, a framework devised to offer a clear depiction of grammatical relationships for users seeking to enhance their capabilities in automated text comprehension. We assert that for this purpose, dependency schemes should adhere to a minimalist design philosophy, convey semantically meaningful data, and enable a systematic method for extracting these relationships. From this standpoint, we scrutinize the foundational design tenets of the Stanford scheme and juxtapose it with the GR and PARC representations. Moreover, we tackle the issue of the Stanford scheme's appropriateness for evaluating parsers. The Stanford typed dependencies representation, in essence, aims to serve as a bridge between the complexities of linguistic structures and the demands of automatic text analysis, ensuring that the extracted information is both comprehensible and useful for a wide range of applications. By comparing it with alternative schemes, we underscore its potential as a robust tool for parser assessment and text understanding tasks.

The field of Spatial Humanities has advanced substantially in the past years. The identification and extraction of toponyms and spatial information mentioned in historical text collections has allowed its use in innovative ways, making possible the application of spatial analysis and the mapping of these places with geographic information systems. For instance, automated place name identification is possible with Named Entity Recognition (NER) systems. Statistical NER methods based on supervised learning, in particular, are highly successful with modern datasets. However, there are still major challenges to address when dealing with historical corpora. These challenges include language changes over time, spelling variations, transliterations, OCR errors, and sources written in multiple languages among others. In this article, considering a task of place name recognition over two collections of historical correspondence, we report an evaluation of five NER systems and an approach that combines these through a voting system. We found that although individual performance of each NER system was corpus dependent, the ensemble combination was able to achieve consistent measures of precision and recall, outperforming the individual NER systems. In addition, the results showed that these NER systems are not strongly dependent on preprocessing and translation to Modern English.

This paper describes an approach and our experiences from the development, deployment and usability testing of a Natural Language Processing (NLP) and Information Retrieval system that supports the moderation of user comments on a large newspaper website. We highlight some of the differences between industry-oriented and academic research settings and their influence on the decisions made in the data collection and annotation processes, selection of document representation and machine learning methods. We report on classification results, where the problems to solve and the data to work with come from a commercial enterprise. In this context typical for NLP research, we discuss relevant industrial aspects. We believe that the challenges faced as well as the solutions proposed for addressing them can provide insights to others working in a similar setting. Data and experiment code related to this paper are available for download at https://ofai.github.io/million-post-corpus.

Dynamic Bayesian networks (DBNs) offer an elegant way to integrate various aspects of language in one model. Many existing algorithms developed for learning and inference in DBNs are applicable to probabilistic language modeling. To demonstrate the potential of DBNs for natural language processing, we employ a DBN in an information extraction task. We show how to assemble wealth of emerging linguistic instruments for shallow parsing, syntactic and semantic tagging, morphological decomposition, named entity recognition etc. in order to incrementally build a robust information extraction system. Our method outperforms previously published results on an established benchmark domain. 1 Information Extraction Information extraction ( IE) is the task of filling in template information from previously unseen text which belongs to a pre-defined domain. The resulting database is suited for formal queries and filtering.IE systems generally work by detecting patterns in the text that help identify significant information. Researchers have shown [Freitag and McCallum, 1999; Ray and Craven, 2001 ] that a probabilistic approach allows the construction of robust and well-performing systems. However, the existing probabilistic systems are generally based on Hidden Markov Models ( HMMs). Due to this relatively impoverished representation, they are unable to take advantage of the wide array of linguistic information used by many non-probabilistic IE systems. In addition, existing HMM -based systems model each target category separately, failing to capture relational information, such as typical target order, or the fact that each element only belongs to a single category. This paper shows how to incorporate a wide array of knowledge into a probabilistic IE system, based on dynamic Bayesian networks ( DBN)—a rich probabilistic representation that generalizes HMMs. Let us illustrateIE by describing seminar announcements which got established as one of the most popular benchmark domains in the field[Califf and Mooney, 1999; Freitag and McCallum, 1999; Soderland, 1999; Roth and Yih, 2001; Ciravegna, 2001 ]. People receive dozens of seminar announcements weekly and need to manually extract information and paste it into personal organizers. The goal of an IE system is to automatically identify target fields such as location and topic of a seminar, date and starting time, ending time and speaker. Announcements come in many formats, but usually follow some pattern. We often find a header with a gist in the form “ PostedBy: john@host.domain; Who: Dr. Steals; When: 1 am; ” and so forth. Also in the body of the message, the speaker usually precedes both location and starting time, which in turn precedes ending time as in: ‘‘Dr. Steals presents in Dean Hall at one am.’’ The task is complicated since some fields may be missing or may contain multiple values. This kind of data falls into the so-called semi-structured text category. Instances obey certain structure and usually contain information for most of the expected fields in some order. There are two other categories: free text and structured text. Instructured text , the positions of the information fields are fixed and values are limited to pre-defined set. Consequently, theIE systems focus on specifying the delimiters and order associated with each field. At the opposite end lies the task of extracting information fromfree textwhich, although unstructured, is assumed to be grammatical. Here IE systems rely more on syntactic, semantic and discourse knowledge in order to assemble relevant information potentially scattered all over a large document. IE algorithms face different challenges depending on the extraction targets and the kind of the text they are embedded in. In some cases, the target is uniquely identifiable (singleslot), while in others, the targets are linked together in multislot association frames. For example, a conference schedule has several slots for related speaker, topic and time of the presentation, while a seminar announcement usually refers to a unique event. Sometimes it is necessary to identify each word in a target slot, while some benefit may be reaped from partial identification of the target, such as labeling the beginning or end of the slot separately. Many applications involve processing of domain-specific jargon like Internetese —a style of writing prevalent in news groups, e-mail messages, bulletin boards and online chat rooms. Such documents do not follow a good grammar, spelling or literary style. Often these are more like a stream-of-consciousness ranting in which asciiart and pseudo-graphic sketches are used and emphasis is provided by all-capitals, or using multiple exclamation signs. As we exemplify below, syntactic analysers easily fail on such Position 1 2 3 4 5 6 7 8 9 10 Tag <speaker > <location> <s-time> Phrase Doctor Steals Presents in Dean Hall at 1 am . Lemma Dr. present in hall at am PoS NNP NNP(VB) VB(NNS) IN NNP NN(NNP) IN CD NN(RB) . Syn.segm. NP NP (VP) VP PP NP NP PP NP NP(VP) Semantic Title LstName LstName Location Time Length 3 6 8 2 4 4 2 1 2 1 Case Upper Upper Upper lower Upper Upper lower lower Table 1: Sample phrase and its representation in multiple feature values for ten tokens.

Acronyms are widely used in biomedical and other technical texts. Understanding their meaning constitutes an important problem in the automatic extraction and mining of information from text. Here we present a system called ACROMED that is part of a set of Information Extraction tools designed for processing and extracting information from abstracts in the Medline database. In this paper, we present the results of two strategies for finding the long forms for acronyms in biomedical texts. These strategies differ from previous automated acronym extraction methods by being tuned to the complex phrase structures of the biomedical lexicon and by incorporating shallow parsing of the text into the acronym recognition algorithm. The performance of our system was tested with several data sets obtaining a performance of 72 % recall with 97 % precision. These results are found to be better for biomedical texts than the performance of other acronym extraction systems designed for unrestricted text.

While much health data is available online, patients who are not technically astute may be unable to access it because they may not know the relevant resources, they may be reluctant to confront an unfamiliar interface, and they may not know how to compose an answer from information provided by multiple heterogeneous resources. We describe ongoing research in using natural English text queries and automated deduction to obtain answers based on multiple structured data sources in a specific subject domain. Each English query is transformed using natural language technology into an unambiguous logical form; this is submitted to a theorem prover that operates over an axiomatic theory of the subject domain. Symbols in the theory are linked to relations in external databases known to the system. An answer is obtained from the proof, along with an English language explanation of how the answer was obtained. Answers need not be present explicitly in any of the databases, but rather may be deduced or computed from the information they provide. Although English is highly ambiguous, the natural language technology is informed by subject domain knowledge, so that readings of the query that are syntactically plausible but semantically impossible are discarded. When a question is still ambiguous, the system can interrogate the patient to determine what meaning was intended. Additional queries can clarify earlier ones or ask questions referring to previously computed answers. We describe a prototype system, Quadri, which answers questions about HIV treatment using the Stanford HIV Drug Resistance Database and other resources. Natural language processing is provided by PARC’s Bridge, and the deductive mechanism is SRI’s SNARK theorem prover. We discuss some of the problems that must be faced to make this approach work, and some of our solutions.

Memory-Based Learning (MBL) is based on the storage of all available training data, and similarity-based reasoning for handling new cases. By interpreting tasks such as POS tagging and shallow parsing as classification tasks, the advantages of MBL (implicit smoothing of sparse data, automatic integration and relevance weighting of information sources, handling exceptional data) contribute to state-of-the-art accuracy. However, Hidden Markov Models (HMM) typically achieve higher accuracy than MBL (and other Machine Learning approaches) for tasks such as POS tagging and chunking. In this paper, we investigate how the advantages of MBL, such as its potential to integrate various sources of information, come to play when we compare our approach to HMMs on two Information Extraction (IE) datasets: the well-known Seminar Announcement data set and a new German Curriculum Vitae data set. 1 Memory-Based Language Processing Memory-Based Learning (MBL) is a supervised classification-based learning method. A vector of feature values (an instance) is associated with a class by a classifier that lazily extrapolates from the most similar set (nearest neighbors) selected from all stored training examples. This is in contrast to eager learning methods like decision tree learning [26], rule induction [9], or Inductive Logic Programming [7], which abstract a generalized structure from the training set beforehand (forgetting the examples themselves), and use that to derive a classification for a new instance. In MBL, a distance metric on the feature space defines what are the nearest neighbors of an instance. Metrics with feature weights based on information-theory or other relevance statistics allow us to use rich representations of instances and their context, and to balance the influences of diverse information sources in computing distance. Natural Language Processing (NLP) tasks typically concern the mapping of an input representation (e.g., a series of words) into an output representation (e.g., the POS tags corresponding to each word in the input). Most NLP tasks can therefore easily be interpreted as sequences of classification 34 Zavrel, Daelemans tasks: e.g., given a word and some representation of its context, decide what tag to assign to each word in its context. By creating a separate classification instance (a “moving window” approach) for each word and its context, shallow syntactic or semantic structures can be produced for whole sentences or texts. In this paper, we argue that more semantic and complex input-output mappings, such as Information Extraction, can also effectively be modeled by such a Memory-based classification-oriented framework, and that this approach has a number of very interesting advantages over rivalling methods, most notably that each classification decision can be made dependent on a very rich and diverse set of features. The properties of MBL as a lazy, similarity-based learning method seem make a good fit to the properties of typical disambiguation problems in NLP: • Similar input representations lead to similar output. E.g., words occurring in a similar context in general have the same POS tag. Similarity-based reasoning is the core of MBL. • Many sub-generalizations and exceptions. By keeping in memory all training instances, exceptions included, an MBL approach can capture generalization from exceptional or low-frequency cases according to [12]. • Need for integration of diverse types of information. E.g., in Information Extraction, lexical features, spelling features, syntactic as well as phrasal context features, global text structure, and layout features can potentially be very relevant. • Automatic smoothing in very rich event spaces. Supervised learning of NLP tasks regularly runs into problems of sparse data; not enough training data is available to extract reliable parameters for complex models. MBL incorporates an implicit robust form of smoothing by similarity [33]. In the remainder of this Section, we will show how a memory-, similarity-, and classification-based approach can be applied to shallow syntactic parsing, and can lead to state-of-the-art accuracy. Most of the tasks discussed here can also easily be modeled using Hidden Markov Models (HMM), and often with surprising accuracy. We will discuss the strengths of the HMMs and draw a comparison between the classification-based MBL method and the sequence-optimizing HMM approach (Section 1.2). 1.1 Memory-Based Shallow Parsing Shallow parsing is an important component of most text analysis systems in Text Mining applications such as information extraction, summary generation, and question answering. It includes discovering the main constituents of sentences (NPs, VPs, PPs) and their heads, and determining syntactic relationships like subject, object, adjunct relations between verbs and heads of other constituents. This is an important first step to understanding the who, what, when, and where of sentences in a text. Feature-Rich Memory-Based Classification for Information Extraction 35 In our approach to memory-based shallow parsing, we carve up the syntactic analysis process into a number of classification tasks with input vectors representing a focus item and a dynamically selected surrounding context. These classification tasks can be segmentation tasks (e.g., decide whether a focus word or tag is the start or end of an NP) or disambiguation tasks (e.g., decide whether a chunk is the subject NP, the object NP or neither). Output of some memory-based modules is used as input by other memory-based modules (e.g., a tagger feeds a chunker and the latter feeds a syntactic relation assignment module). Similar ideas about cascading of processing steps have also been explored in other approaches to text analysis: e.g., finite state partial parsing [1,18], statistical decision tree parsing [23], and maximum entropy parsing [30]. The approach briefly described here is explained and evaluated in more detail in [10,11,6] . Chunking The phrase chunking task can be defined as a classification task by generalizing the approach of [28], who proposed to convert NP-chunking to tagging each word with I for a word inside an NP, O for outside an NP, and B for between two NPs). The decision on these so called IOB tags for a word can be made by looking at the Part-of-Speech tag and the identity of the focus word and its local context. For the more general task of chunking other non-recursive phrases, we simply extend the tag set with IOB tags for each type of phrase. To illustrate this encoding with the extended IOB tag set, we can tag the sentence: But/CC [NP the/DT dollar/NN NP] [ADVP later/RB ADVP] [VP rebounded/VBD VP] ,/, [VP finishing/VBG VP] [ADJP slightly/RB higher/R ADJP] [Prep against/IN Prep] [NP the/DT yen/NNS NP] [ADJP although/IN ADJP] [ADJP slightly/RB lower/JJR ADJP] [Prep against/IN Prep] [NP the/DT mark/NN NP] ./. as: But/CCO the/DTI−NP dollar/NNI−NP later/RBI−ADV P rebounded/VBDI−V P ,/,O finishing/VBGI−V P slightly/RBI−ADV P higher/RBRI−ADV P against/INI−Prep the/DTI−NP yen/NNSI−NP although/INI−ADJP slightly/RBB−ADJP lower/JJRI−ADJP against/INI−Prep the/DTI−NP mark/NNI−NP ./.O Table 1 (from [6]) shows the accuracy of this memory-based chunking approach when training and testing on Wall Street Journal material. We report Precision, Recall, and Fβ=1 scores, a weighted harmonic mean of Recall and Precision (Fβ = (β+1)∗P∗R β2∗P+R) ). 1 An online demonstration of the Memory-Based Shallow Parser can be found at http://ilk.kub.nl . 36 Zavrel, Daelemans type precision recall Fβ=1 NPchunks 92.5 92.2 92.3 VPchunks 91.9 91.7 91.8 ADJPchunks 68.4 65.0 66.7 ADVPchunks 78.0 77.9 77.9 Prepchunks 95.5 96.7 96.1 PPchunks 91.9 92.2 92.0 ADVFUNCs 78.0 69.5 73.5 Table 1. Results of chunking–labeling experiments. Reproduced from [6]. Grammatical Relation Finding After POS tagging, phrase chunking and labeling, the last step of the shallow parsing consists of resolving the (types of) attachment between labeled phrases. This is done by using a classifier to assign a grammatical relation (GR) between pairs of words in a sentence. In our approach, one of these words is always a verb, since this yields the most important GRs. The other word (focus) is the head of the phrase which is annotated with this grammatical relation in the treebank (e.g., a noun as head of an NP). An instance for such a pair of words is constructed by extracting a set of feature values from the sentence. The instance contains information about the verb and the focus: a feature for the word form and a feature for the POS of both. It also has similar features for the local context of the focus. Experiments on the training data suggest an optimal context width of two words to the left and one to the right. In addition to the lexical and the local context information, superficial information about clause structure was included: the distance from the verb to the focus, counted in words. A negative distance means that the focus is to the left of the verb. Other features contain the number of other verbs between the verb and the focus, and the number of intervening commas. These features were chosen by manual “feature engineering”. Table 2 shows some of the feature-value instances corresponding to the following sentence (POS tags after the slash, chunks denoted with square and curly brackets, and adverbial functions after the dash): [ADVP Not /RB surprisingly /RB ADVP] ,/, [NP Peter /NNP Miller /NNP NP] ,/, [NP who /WP NP] [VP organized /VBD VP] [NP the /DT conference /NN NP] {PP-LOC [Prep in /IN Prep] [NP New /NNP York /NNP NP] PP-LOC} ,/, [VP does /VBZ not /RB want /VB to /TO come /VB VP] {PP-DIR [Prep to /IN Prep] [NP Paris /NNP NP] PP-DIR} [Prep without /IN Prep] [VP bringing /VBG VP] [NP his /PRP$ wife /NN NP]. Table 3 shows the results of the experiments. In the first row, only POS tag features are used. Other rows show the results when adding several types of chunk informati

We present a question answering system (CASIA@V2) over Linked Data (DBpedia), which translates natural language questions into structured queries automatically. Existing systems usually adopt a pipeline framework, which contains four major steps: 1) Decomposing the question and detecting candidate phrases; 2) mapping the detected phrases into semantic items of Linked Data; 3) grouping the mapped semantic items into semantic triples; and 4) generating the rightful SPARQL query. We present a jointly learning framework using Markov Logic Network(MLN) for phrase detection, phrases mapping to semantic items and semantic items grouping. We formulate the knowledge for resolving the ambiguities in three steps of QALD as first-order logic clauses in a MLN. We evaluate our approach on QALD-4 test dataset and achieve an F-measure score of 0.36, an average precision of 0.32 and an average recall of 0.40 over 50 questions.

We describe new applications of the theory of automata to natural language processing: the representation of very large scale dictionaries and the indexation of natural language texts. They are based on new algorithms that we introduce and describe in detail. In particular, we give pseudocodes for the de-terminization of string to string transducers, the deterministic union of p-subsequential string to string transducers, and the indexation by automata. We report several experiments illustrating the applications.

The 2010 i2b2/VA Workshop on Natural Language Processing Challenges for Clinical Records presented three tasks: a concept extraction task focused on the extraction of medical concepts from patient reports; an assertion classification task focused on assigning assertion types for medical problem concepts; and a relation classification task focused on assigning relation types that hold between medical problems, tests, and treatments. i2b2 and the VA provided an annotated reference standard corpus for the three tasks. Using this reference standard, 22 systems were developed for concept extraction, 21 for assertion classification, and 16 for relation classification. These systems showed that machine learning approaches could be augmented with rule-based systems to determine concepts, assertions, and relations. Depending on the task, the rule-based systems can either provide input for machine learning or post-process the output of machine learning. Ensembles of classifiers, information from unlabeled data, and external knowledge sources can help when the training data are inadequate.

STRAND (Resnik, 1998) is a languageindependent system for automatic discovery of text in parallel translation on the World Wide Web. This paper extends the preliminary STRAND results by adding automatic language identification, scaling up by orders of magnitude, and formally evaluating performance. The most recent end-product is an automatically acquired parallel corpus comprising 2491 English-French document pairs, approximately 1.5 million words per language. 1 I n t r o d u c t i o n Text in parallel translation is a valuable resource in natural language processing. Statistical methods in machine translation (e.g. (Brown et al., 1990)) typically rely on large quantities of bilingual text aligned at the document or sentence level, and a number of approaches in the burgeoning field of crosslanguage information retrieval exploit parallel corpora either in place of or in addition to mappings between languages based on information from bilingual dictionaries (Davis and Dunning, 1995; Landauer and Littman, 1990; Hull and Oard, 1997; Oard, 1997). Despite the utility of such data, however, sources of bilingual text are subject to such limitations as licensing restrictions, usage fees, restricted domains or genres, and dated text (such as 1980's Canadian politics); or such sources simply may not exist for * This work was supported by Department of Defense contract MDA90496C1250, DARPA/ITO Contract N66001-97-C-8540, and a research grant from Sun Microsystems Laboratories. The author gratefully acknowledges the comments of the anonymous reviewers, helpful discussions with Dan Melamed and Doug Oard, and the assistance of Jeff Allen in the French-English experimental evaluation. language pairs of interest. Although the majority of Web content is in English, it also shows great promise as a source of multilingual content. Using figures from the Babel survey of multilinguality on the Web (h tZp : / /www. i s o c . o r g / ) , it is possible to estimate that as of June, 1997, there were on the order of 63000 primarily non-English Web servers, ranging over 14 languages. Moreover, a followup investigation of the non-English servers suggests that nearly a third contain some useful cross-language data, such as parallel English on the page or links to parallel English pages the follow-up also found pages in five languages not identified by the Babel study (Catalan, Chinese, Hungarian, Icelandic, and Arabic; Michael Littman, personal communication). Given the continued explosive increase in the size of the Web, the trend toward business organizations that cross national boundaries, and high levels of competition for consumers in a global marketplace, it seems impossible not to view multilingual content on the Web as an expanding resource. Moreover, it is a dynamic resource, changing in content as the world changes. For example, Diekema et al., in a presentation at the 1998 TREC-7 conference (Voorhees and Harman, 1998), observed that the performance of their cross-language information retrieval was hurt by lexical gaps such as Bosnia/Bosniethis illustrates a highly topical missing pair in their static lexical resource (which was based on WordNet 1.5). And Gey et al., also at TREC-7, observed that in doing cross-language retrieval using commercial machine translation systems, gaps in the lexicon (their example was acupuncture/Akupunktur) could make the difference between precision of 0.08 and precision of 0.83 on individual queries. ttesnik (1998) presented an algorithm called


Question: Odin's Runes: A rule-based language for the extraction of information.

Introducing the brat rapid annotation tool (BRAT), a web-based platform designed to streamline text annotation processes with the integration of Natural Language Processing (NLP) technologies. BRAT is tailored to facilitate complex, structured annotation across diverse NLP tasks, with a focus on enhancing manual curation efficiency and boosting annotator productivity through the application of NLP methodologies. The tool's efficacy is underscored through several practical annotation projects conducted using pre-release versions of BRAT. A case in point is the evaluation of annotation time reduction when employing semantic class disambiguation for a multicategory entity mention annotation task, which resulted in a significant 15% decrease in total annotation time. To foster collaborative research and development, BRAT is made available under an open-source license at http://brat.nlplab.org, offering the NLP community access to a powerful annotation tool.

This thesis discusses the problems and the methods of finding relevant information in large collections of documents. The contribution of this thesis to this problem is to develop better content analysis methods which can be used to describe document content with

ANIMATE, HUMAN, etc. and encode type restrictions on nouns and adjectives and on the arguments of verbs. Subject codes use another set of primitives to classify senses of words by subject (ECONOMICS, ENGINEERING, etc.). Guthrie et al. (1991) demonstrate a typical use of this information: in addition to using the Lesk-based method of counting overlaps between definitions and contexts, they impose a correspondence of subject codes in an iterative process. No quantitative evaluation of this method is available, but Cowie et al. (1992) improve the method using simulated annealing and report results of 47% for sense distinctions and 72% for homographs. The use of LDOCE box codes, however, is problematic: the codes are not systematic (see, for example, Fontenelle, 1990); in later work, Braden-Harder (1993) showed that simply matching box or subject codes is not sufficient for disambiguation. For example, in I tipped the driver, the codes for several senses of the words in the sentence satisfy the necessary constraints (e.g. tip-money + human object or tip-tilt + movable solid object). In many ways, the supple7 Note that the assumptions underlying this method are very similar to Quillian’s: Thus one may think of a full concept analogically as consisting of all the information one would have if he looked up what will be called the “patriarch” word in a dictionary, then looked up every word in each of its definitions, then looked up every word found in each of these, and so on, continually branching outward[...] (Quillian, 1968, p. 238). However, Quillian’s network also keeps track of semantic relationships among the words encountered along the path between two words, which are encoded in his semantic network; the neural network avoids the overhead of creating the semantic network but loses this relational information. 13 mentary information in the LDOCE, and in particular the subject codes, are similar to those in a thesaurus, which, however, are more systematically structured. Inconsistencies in dictionaries, noted earlier, are not the only and perhaps not the major source of their limitations for WSD. While dictionaries provide detailed information at the lexical level, they lack pragmatic information that enters into sense determination (see, e.g., Hobbs, 1987). For example, the link between ash and tobacco, cigarette or tray in a network such as Quillian’s is very indirect, whereas in the Brown Corpus, the word ash co-occurs frequently with one of these words. It is therefore not surprising that corpora have become a primary source of information for WSD; this development is outlined below in section 2.3. 1.3.2 Thesauri. Thesauri provide information about relationships among words, most notably synonymy. Roget's International Thesaurus, which was put into machine-tractable form in the 1950's8 and has been used in a variety of applications including machine translation (Masterman, 1957), information retrieval (Sparck Jones, 1964, 1986), and content analysis (Sedelow and Sedelow, 1969; see also Sedelow and Sedelow, 1986, 1992), also supplies an explicit concept hierarchy consisting of up to eight increasingly refined levels. Typically, each occurrence of the same word under different categories of the thesaurus represent different senses of that word; i.e., the categories correspond roughly to word senses (Yarowsky, 1992). A set of words in the same category are semantically related. The earliest known use of Roget’s for WSD is the work of Masterman (1957), described above in section 2.1. Several years later, Patrick (1985) used Roget’s to discriminate among verb senses, by examining semantic clusters formed by “e-chains” derived from the thesaurus (Bryan, 1973, 1974; see also Sedelow and Sedelow, 1986). He uses “word-strong neighborhoods,” comprising word groups in low-level semicolon groups, which are the most closely related semantically in the thesaurus, and words connected to the group via chains. He is able to discriminate the correct sense of verbs such as inspire (to raise the spirits vs. to inhale, breathe in, sniff, etc.), question (to doubt vs. to ask a question) with “high reliability.” Bryan's earlier work had already demonstrated that homographs can be distinguished by applying a metric based on relationships defined by his chains (Bryan, 1973, 1974). Similar work is described in Sedelow and Mooney (1988). Yarowsky (1992) derives classes of words by starting with words in common categories in Roget's (4th ed.). A 100-word context of each word in the category is extracted from a corpus (the 1991 electronic text of Grolier's Ency8 The work of Masterman (1957) and Sparck Jones (1964) relied on a version of Roget’s that was hand-punched onto cards in the 1950’s; the Sedelow’s (1969) work relied on a machine readable version of the 3rd Edition. Roget’s is now widely available via anonymous ftp from various sites.

This Doctoral Consortium paper discusses how NLP can be applied in the domain of BPM in order to automatically generate business process models from existing documentation within the organization. The main idea is that from the syntactic and grammatical structure of a sentence, the components of a business process model can be derived (i.e. activities, resources, tasks, patterns). The result would be a business process model depicted using BPMN - a dedicated business process modeling technique

MOTIVATION
The importance of extracting biomedical information from scientific publications is well recognized. A number of information extraction systems for the biomedical domain have been reported, but none of them have become widely used in practical applications. Most proposals to date make rather simplistic assumptions about the syntactic aspect of natural language. There is an urgent need for a system that has broad coverage and performs well in real-text applications.

RESULTS
We present a general biomedical domain-oriented NLP engine called MedScan that efficiently processes sentences from MEDLINE abstracts and produces a set of regularized logical structures representing the meaning of each sentence. The engine utilizes a specially developed context-free grammar and lexicon. Preliminary evaluation of the system's performance, accuracy, and coverage exhibited encouraging results. Further approaches for increasing the coverage and reducing parsing ambiguity of the engine, as well as its application for information extraction are discussed.

Most Web-based Q/A systems work by finding pages that contain an explicit answer to a question. These systems are helpless if the answer has to be inferred from multiple sentences, possibly on different pages. To solve this problem, we introduce the HOLMES system, which utilizes textual inference (TI) over tuples extracted from text. Whereas previous work on TI (e.g., the literature on textual entailment) has been applied to paragraph-sized texts, HOLMES utilizes knowledge-based model construction to scale TI to a corpus of 117 million Web pages. Given only a few minutes, HOLMES doubles recall for example queries in three disparate domains (geography, business, and nutrition). Importantly, HOLMES’s runtime is linear in the size of its input corpus due to a surprising property of many textual relations in the Web corpus—they are “approximately” functional in a well-defined sense.

IDP is a knowledge base system based on first order logic. It is finding its way to a larger public but is still facing practical challenges. Adoption of new languages requires a newcomer-friendly way for users to interact with it. Both an online presence to try to convince potential users to download the system and offline availability to develop larger applications are essential. We developed an IDE which can serve both purposes through the use of web technology. It enables us to provide the user with a modern IDE with relatively little effort.

IBM Research has over 200 people working on Unstructured Information Management (UIM) technologies with a strong focus on Natural Language Processing (NLP). These researchers are engaged in activities ranging from natural language dialog, information retrieval, topic-tracking, named-entity detection, document classification and machine translation to bioinformatics and open-domain question answering. An analysis of these activities strongly suggested that improving the organization’s ability to quickly discover each other's results and rapidly combine different technologies and approaches would accelerate scientific advance. Furthermore, the ability to reuse and combine results through a common architecture and a robust software framework would accelerate the transfer of research results in NLP into IBM’s product platforms. Market analyses indicating a growing need to process unstructured information, specifically multilingual, natural language text, coupled with IBM Research’s investment in NLP, led to the development of middleware architecture for processing unstructured information dubbed UIMA. At the heart of UIMA are powerful search capabilities and a data-driven framework for the development, composition and distributed deployment of analysis engines. In this paper we give a general introduction to UIMA focusing on the design points of its analysis engine architecture and we discuss how UIMA is helping to accelerate research and technology transfer. 1 Motivation and Objectives We characterize a UIM application as a software system that analyzes large volumes of unstructured information while consulting structured information sources to discover, organize and present knowledge relevant to the application’s end-user. We define structured information as information whose intended meaning is unambiguous and explicitly represented in the structure or format of the data. The canonical example of structured information is a relational database table. We define unstructured information as information whose intended meaning is only loosely implied by its form. The canonical example is a natural language document; other examples include voice, audio, image and video. To appear in a special issue of the Journal of Natural Language Engineering 2004 2 Ferrucci and Lally Do Not Distirubte without written permission from authors Market indicators suggest a rapid increase in the use of text and voice analytics to a growing class of commercially viable applications (Roush 2003). National security interests are driving applications to involve an even broader set of UIM technologies including image and video analytics. Emerging and commercially important UIM application areas include life sciences, e-commerce, technical support, advanced search, and national and business intelligence. In six major labs spread out over the globe, IBM Research has over 200 people working on Unstructured Information Management (UIM) technologies with a primary focus on Natural Language Processing (NLP). These researchers are engaged in activities ranging from natural language dialog, information retrieval, topic-tracking, named-entity detection, document classification and machine translation to bioinformatics and open-domain question answering. Each group is developing different technical and engineering approaches to process unstructured information in pursuit of their specific research agenda in the UIM space. While IBM Research’s independent UIM technologies fare well in scientific venues, accelerated development of integrated, robust applications is becoming increasingly important to IBM’s commercial interests. Furthermore the rapid integration of different alogorthims and techniques is also a means to advance scientific results. For example, experiments in statistical name-entity detection or machine translation may benefit from using a deep parser as a feature generator. Facilitating the rapid integration of an independently produced deep parser with statistical algorithm implementations can accelerate the research cycle time as well as the time to product. Some of the challenges that face a large research organization in efficiently leveraging and integrating its (and third-party) UIM assets include: • Organizational Structure. IBM Research is largely organized in three to five person teams geographically dispersed among different labs throughout world. This structure makes it more difficult to jointly develop and reuse technologies. • Skills Alignment. There are inefficiencies in having PhDs trained and specialized in, for example, computational linguistics or statistical machine learning, develop and apply the requisite systems expertise to deliver their work into product architectures and/or make it easily usable by other researchers. To appear in a special issue of the Journal of Natural Language Engineering 2004 3 Ferrucci and Lally Do Not Distirubte without written permission from authors • Development Inefficiencies. Research results are delivered in a myriad of interfaces and technologies. Reuse and integration often requires too much effort dedicated to technical adaptation. Too often work is replicated. • Speed to product. Software Products introduce their own architectures, interfaces and implementation technologies. Technology transfer from the lab to the products often becomes a significant engineering or rewrite effort for researchers. A common framework and engineering discipline could help address the challenges listed above. The Unstructured Information Management Architecture (UIMA) project was initiated within IBM Research based on the premise that a common software architecture for developing, composing and delivering UIM technologies, if adopted by the organization, would facilitate technology reuse and integration, enable quicker scientific experimentation and speed the path to product. IBM’s Unstructured Information Management Architecture (UIMA) high-level objectives are two fold: 1) Accelerate scientific advances by enabling the rapid combination of UIM technologies (e.g., natural language processing, audio and video analysis, information retrieval, etc.). 2) Accelerate transfer of UIM technologies to product by providing an architecture and associated framework implementations that promote reuse and support flexible deployment options. 2 The Unstructured Information Management Architecture (UIMA) UIMA is a software architecture for developing UIM applications. The UIMA high-level architecture, illustrated in Figure 1, defines the roles, interfaces and communications of large-grained components essential for UIM applications. These include components capable of analyzing unstructured artifacts, integrating and accessing structured sources and storing, indexing and searching for artifacts based on discovered semantic content resulting from analyses. A primary design point for UIMA is that it should admit the implementation of middleware frameworks that provide a component-based infrastructure for developing UIM applications suitable for vastly different deployment environments. These should range from lightweight and embeddable implementations to highly scalable implementations that are meant to exploit clusters of machines and provide high throughput, high availability serviced-based offerings. While the primary focus of current UIMA framework implementations is squarely on natural language To appear in a special issue of the Journal of Natural Language Engineering 2004 4 Ferrucci and Lally Do Not Distirubte without written permission from authors UIM Application Analysis Collection Level Analysis Document, Collection & Metadata Store Knowledge Source Access Collection Processing Manager (CPM) Knowledge & Data Bases Unstructured Information (Text) Analysis Engines Crawlers Analysis Engine Directory Acquisition Unstructured Information Analysis Structured Information Access Semantic Search Engine Indices Indices Documents Collections Metadata Knowledge Source Adapter Directory Access

This paper presents a large-scale system for the recognition and semantic disambiguation of named entities based on information extracted from a large encyclopedic collection and Web search results. It describes in detail the disambiguation paradigm employed and the information extraction process from Wikipedia. Through a process of maximizing the agreement between the contextual information extracted from Wikipedia and the context of a document, as well as the agreement among the category tags associated with the candidate entities, the implemented system shows high disambiguation accuracy on both news stories and Wikipedia articles.

In this paper, we address the tasks of recognition and interpretation of affect communicated through text messaging in online communication environments. Specifically, we focus on Instant Messaging (IM) or blogs, where people use an informal or garbled style of writing. We introduced a novel rule-based linguistic approach for affect recognition from text. Our Affect Analysis Model (AAM) was designed to deal with not only grammatically and syntactically correct textual input, but also informal messages written in an abbreviated or expressive manner. The proposed rule-based approach processes each sentence in stages, including symbolic cue processing, detection and transformation of abbreviations, sentence parsing and word/phrase/sentence-level analyses. Our method is capable of processing sentences of different complexity, including simple, compound, complex (with complement and relative clauses) and complex–compound sentences. Affect in text is classified into nine emotion categories (or neutral). The strength of the resulting emotional state depends on vectors of emotional words, relations among them, tense of the analysed sentence and availability of first person pronouns. The evaluation of the Affect Analysis Model algorithm showed promising results regarding its capability to accurately recognize fine-grained emotions reflected in sentences from diary-like blog posts (averaged accuracy is up to 77 per cent), fairy tales (averaged accuracy is up to 70.2 per cent) and news headlines (our algorithm outperformed eight other systems on several measures).

The DBpedia project is a community effort to extract structured information from Wikipedia and to make this information accessible on the Web. The resulting DBpedia knowledge base currently describes over 2.6 million entities. For each of these entities, DBpedia defines a globally unique identifier that can be dereferenced over the Web into a rich RDF description of the entity, including human-readable definitions in 30 languages, relationships to other resources, classifications in four concept hierarchies, various facts as well as data-level links to other Web data sources describing the entity. Over the last year, an increasing number of data publishers have begun to set data-level links to DBpedia resources, making DBpedia a central interlinking hub for the emerging Web of data. Currently, the Web of interlinked data sources around DBpedia provides approximately 4.7 billion pieces of information and covers domains such as geographic information, people, companies, films, music, genes, drugs, books, and scientific publications. This article describes the extraction of the DBpedia knowledge base, the current status of interlinking DBpedia with other data sources on the Web, and gives an overview of applications that facilitate the Web of Data around DBpedia.