Question: An underactuated propeller design for attitude control in micro air vehicles.

This paper introduces the application of reinforcement learning in achieving complex aerobatic maneuvers with a radio-controlled (RC) helicopter, marking a significant advancement in the field of autonomous helicopter flight. Successfully executing maneuvers such as forward flips, sideways rolls at low speeds, tail-in funnels, and nose-in funnels, our experimental outcomes represent a notable extension of the current capabilities in autonomous helicopter operation. The methodology we employed involved a series of steps: Initially, a skilled pilot's expertise was utilized to guide the identification of a helicopter dynamics model and to define a reward (cost) function that accurately reflects the desired performance criteria. Subsequently, we applied a reinforcement learning algorithm, specifically designed for optimal control, to derive a controller that maximizes performance based on the developed model and reward function. The algorithm we utilized was differential dynamic programming (DDP), a sophisticated variant of the linear quadratic regulator (LQR) technique, which is particularly adept at handling non-linear systems and optimizing control policies. This approach allowed us to train the RC helicopter to perform aerobatic maneuvers autonomously with unprecedented precision and reliability.

Designing and manufacturing actuators using soft materials are among the most important subjects for future robotics. In nature, animals made by soft tissues such as the octopus have attracted the attention of the robotics community in the last years. Suckers (or suction cups) are one of the most important and peculiar organs of the octopus body, giving it the ability to apply high forces on the external environment. The integration of suction cups in soft robots can enhance their ability to manipulate objects and interact with the environment similarly to what the octopus does. However, artificial suction cups are currently actuated using fluid pressure so most of them require external compressors, which will greatly increase the size of the soft robot. In this work, we proposed the use of the ElectroHydroDynamics (EHD) principle to actuate a suction cup. EHD is a fluidic phenomenon coupled with electrochemical reaction that can induce pressure through the application of a high-intensity electric field. We succeeded in developing a suction cup driven by EHD keeping the whole structure extremely simple, fabricated by using a 3D printer and a cutting plotter. We can control the adhesion of the suction cup by controlling the direction of the fluidic flow in our EHD pump. Thanks to a symmetrical arrangement of the electrodes, composed by plates parallel to the direction of the channel, we can change the direction of the flow by changing the sign of the applied voltage. We obtained the pressure of 643 Pa in one unit of EHD pump and pressure of 1428 Pa in five units of EHD pump applying 6 kV. The suction cup actuator was able to hold and release a 2.86 g piece of paper. We propose the soft actuator driven by the EHD pump, and expand the possibility to miniaturize the size of soft robots.

In this paper, the authors describe the design and control of RHex, a power autonomous, untethered, compliant-legged hexapod robot. RHex has only six actuators—one motor located at each hip—achieving mechanical simplicity that promotes reliable and robust operation in real-world tasks. Empirically stable and highly maneuverable locomotion arises from a very simple clock-driven, openloop tripod gait. The legs rotate full circle, thereby preventing the common problem of toe stubbing in the protraction (swing) phase. An extensive suite of experimental results documents the robot’s significant “intrinsic mobility”—the traversal of rugged, broken, and obstacle-ridden ground without any terrain sensing or actively controlled adaptation. RHex achieves fast and robust forward locomotion traveling at speeds up to one body length per second and traversing height variations well exceeding its body clearance.

A path-tracking criterion for the so-called LHD (load-haul-dump) truck used in underground mining is proposed in this paper. It exploits the particular configuration of this vehicle, composed of two units connected by an actuated articulation. The task is to follow the path represented by the middle of the tunnel, maintaining the whole vehicle at a reduced distance from the path itself, to decrease the risk of crashes against the walls of the tunnel. This is accomplished via feedback through the synthesis of an appropriate path-tracking criterion. The criterion is based on monitoring the distances of the midpoints of both axles of the vehicle from their orthogonal projections on the path, using two different moving frames simultaneously. Local asymptotic stability to paths of constant curvature is achieved by means of linear-state feedback. KEY WORDS—mobile robots, articulated vehicles, mining trucks, path tracking, off-tracking, feedback stabilization

The compliance of muscles with external forces and the structural stability given by biarticular muscles are important features of animals to realize dynamic whole body motions such as running and hopping in various environments. For this reason, we have been studying an electromagnetic linear actuator. This actuator can emulate the behavior of a human muscle such as the spring-damper characteristics by quick control of the output force (i.e. impedance control) and it is expected to be used as an artificial muscle. In this paper, we develop a monopedal robot possessing bi- and mono-articular muscles implemented by linear actuators. Thanks to the biarticular muscle, the bouncing direction of the robot can be controlled by changing the stiffness ellipse at the endpoint (i.e. foot) of the robot. We confirm that the bouncing direction of the robot can be controlled and hopping can be achieved by changing the stiffness ellipse.

The latest research in mesoscale drive systems is targeting rotational speeds toward 1 million r/min for a power range of 1-1 kW. Emerging applications for megaspeed drives (MegaNdrives) are to be found in future turbo compressor systems for fuel cells and heat pumps, generators/starters for portable nanoscale gas turbines, printed circuit board drilling and machining spindles, and electric power generation from pressurized gas flow. The selection of the machine type and the challenges involved in designing a machine for megaspeed operation such as the winding concepts, a mechanical rotor design capable of 1 000 000 r/min, the selection of magnetic materials for the stator, and the optimization concerning high-frequency losses and torque density are presented. Furthermore, a review of the advantageous inverter topologies, taking into account the extremely low stator inductance and possible high-speed bearing types such as ball bearings, air bearings, foil bearings, and magnetic bearings, are given. Finally, prototypes and experimental results originating from MegaNdrive research at Swiss Federal Institute of Technology Zurich are discussed and extreme temperature operation and power microelectricalmechanical system are identified as targets for future research.

This paper is about the use of control algorithms to animate dynamic legged locomotion. Control could free the animator from specifying the details of joint and limb motion while producing both physically realistic and natural looking results. We implemented computer animations of a biped robot, a quadruped robot, and a kangaroo. Each creature was modeled as a linked set of rigid bodies with compliant actuators at its joints. Control algorithms regulated the running speed, organized use of the legs, and maintained balance. All motions were generated by numerically integrating equations of motion derived from the physical models. The resulting behavior included running at various speeds, traveling with several gaits (run, trot, bound, gallop, and hop), jumping, and traversing simple paths. Whereas the use of control permitted a variety of physically realistic animated behavior to be generated with limited human intervention, the process of designing the control algorithms was not automated: the algorithms were "tweaked" and adjusted for each new creature.

The recent developments of soft robots have inspired a variety of applications that require compliance for safety and softness for biomimetics. With such prevalence and advantages of soft robots, researchers are diverting their interests toward applications like micromanipulation of biological tissue. However, the progress has been thwarted by the difficulty in producing soft robots in miniature scale. In this paper, we present a new kind of microscale soft pneumatic actuators (SPA) made from streamlined and standardized fabrication procedure with customizable bending modalities inspired by shape engineering. Preliminary mathematical models are given to interpret width-based shape engineering for customization and to compare the bending angle and radius of curvature measured from the characterization experiments. The fabricated SPA was tested on the sciatic nerve of a rat in order to show its future potentials in biomedical field. Ultimately, this paper will contribute to the diversification of SPA fabrication and customization, as well as biomedical applications that demand smaller dimensions and higher precision.

With the availability of low-cost micro aerial vehicles (MAVs), unmanned aerial vehicles (UAVs) quickly gain popularity and application potential. This requires techniques that can be understood by non-experts and flexibly applied for rapid prototyping. Visual tracking is an essential task with many applications, such as autonomous navigation and scene acquisition. While marker-less methods emerge, marker-based methods still have major advantages including simplicity, robustness, accuracy and performance. In practice, however, multi-marker setups introduce complexity and calibration efforts that can void the advantages. This work proposes a solution for practical, robust and easy-to-use marker-based tracking with independent compound targets. We introduce two novel target designs and describe pose estimation, noise removal and geometric transformations. The concepts are implemented in a tracking library for the Parrot AR. Drone 2.0. We explain its video access and camera calibration, and provide a first set of intrinsic parameters, jointly estimated from 14 units with high accuracy and low variance. The library is applied in a one-day contest on automatic visual navigation of UAVs, where students without technical background and programming skills achieved learning by experience and rapid development. This shows the effectiveness of combining capability with simplicity, and provides a case study on robotics in interdisciplinary education.

The paper is inspired from a challenging experiment conducted with undergraduate students for bachelor project within Automatic Control and Computer Engineering Faculty, Iaşi. A bicycle type system was implemented, analyzed and some control strategies were implemented and tested in order to self-stabilize the system. The bicycle study is enjoyable both to undergraduate students and researchers because it uses a “hi-tech” relevant concrete example. This work encompasses and uses a full range of system-theoretic aspects which covers knowledge from analytical, computational and experimental point of view. The dynamic description of the bicycle is done using a set of dynamic equations that describes the location and orientation of the body, considering four rigid subsystems that form the bicycle. The control issues were conducted in order to follow a desired forward speed using classical PI controller, followed by the LQR approach for steering angle.

The paper investigates the development of an adaptive dynamic nonlinear model inversion control law for a twin rotor MIMO system (TRMS) utilizing artificial neural networks and genetic algorithms. The TRMS is an aerodynamic test rig representing the control challenges of modern air vehicles. A highly nonlinear 1DOF mathematical model of the TRMS is considered in this study and a nonlinear inverse model is developed for the pitch channel. In the absence of model inversion errors, a genetic algorithm-tuned PD controller is used to enhance the tracking characteristics of the system. An adaptive neural network element is integrated thereafter with the feedback control system to compensate for model inversion errors. In order to show the effectiveness of the proposed method in the simulation environment an inversion error has deliberately been provided as an uncertainty in the real situation. Square and sinusoidal reference command signals are used to test the control system performance, and it is noted that an excellent tracking response is exhibited in the presence of inversion errors caused by model uncertainty.


Question: An underactuated propeller utilized for attitude control in micro air vehicles.

The Emergence of the GRASP Multiple Micro-UAV Testbed in Recent Years
Over the past half-decade, the proliferation of micro-unmanned aerial vehicles (MAVs) has been significantly accelerated by advancements in materials science, electronics, sensor technology, and battery efficiency [1]. MAVs, typically ranging from 0.1 to 0.5 meters in length and 0.1 to 0.5 kilograms in weight, have become a focus of research and development. Several research groups have successfully designed and analyzed MAVs within the 10-centimeter size category [2], [3]. Among the smallest MAVs is the Picoftyer, boasting a propeller diameter of 60 millimeters and a featherweight of 3.3 grams [4]. Meanwhile, MAV platforms in the 50-centimeter range are more commonly encountered, with numerous research teams having constructed and tested systems of this size [5]-[7]. Notably, the market for these MAVs has also expanded, with an array of commercially available radio-controlled (RC) helicopters and high-quality research helicopters now accessible in this size category [8].

This paper proposes a new bending technique with a screwdrive mechanism that allows for omnidirectional bending motion by rotating two linkages, each consisting of a right-handed screw, a universal joint, and a left-handed screw. The new screwdrive mechanism, termed double-screw-drive (DSD) mechanism, is utilized in a robotic forceps manipulator for laparoscopic surgery. A robotic forceps manipulator incorporating the DSD mechanism (DSD forceps) can bend without using wires. Without wires, it has high rigidity, and can bend at 90° in any arbitrary direction. In addition, the gripper of the DSD forceps can perform rotational motion, which is achieved by rotating a third linkage in the DSD mechanism. Opening and closing motions of the gripper are attained by wire actuation. Fundamental experiments examining the bending force and the accuracy of the DSD forceps were conducted, and an analysis of the accuracy was performed. Control of the DSD forceps through a teleoperation system was achieved via a joystick-type manipulator. A servo system was constructed for each linkage and the wire actuation mechanism, and tracking control experiments as well as a suturing experiment were conducted. The results of the experiments showed that the required design specifications were fulfilled. Thus, the validity of the DSD forceps was demonstrated.

The ability to use all of the available tire force is essential to road vehicles for emergency maneuvers and racing. As the front tires of an understeering vehicle saturate while cornering at the limit of tire-road friction, steering is lost as a control input for path-tracking. Experimental data from an autonomous vehicle show that for path-tracking at the limit of friction through steering the value of friction needs to be known to within approximately 2%. This requirement exceeds the capabilities of existing real-time friction estimation algorithms. Data collected with a professional race car driver inspire a novel control framework, with a slip angle-based control strategy of maintaining the front tires at the slip angle for which maximum tire force is attained, and longitudinal speed control for path-tracking. This approach has significantly less demanding requirements on the accuracy of friction estimation. A controller is presented to explore this concept, and experimental results demonstrate successful tracking of a circular path at the friction limit without a priori friction information.

This chapter presents an introduction to the dynamics and control of robot manipulators. We derive the equations of motion for a general open-chain manipulator and, using the structure present in the dynamics , construct control laws for asymptotic tracking of a desired trajectory. In deriving the dynamics, we will make explicit use of twists for representing the kinematics of the manipulator and explore the role that the kinematics play in the equations of motion. We assume some familiarity with dynamics and control of physical systems.

This paper describes an intelligent autonomous airborne flight capability that is being used as a test bed for future technology development. The unmanned air vehicles (UAVs) fly under autonomous control of both an onboard computer and an autopilot. The onboard computer provides the mission control and runs the autonomous Intelligent Controller (IC) software while the autopilot controls the vehicle navigation and flight control. The autonomous airborne flight system is described in detail. An IC architecture directly applicable to the design of unmanned vehicles is also presented. The UAVs may operate independently or in cooperation with one another to carry out a specified mission. The intelligent UAV flight system is used to evaluate and study autonomous UAV control as well as multi-vehicle collaborative control.

In recent years impressive results have been presented illustrating the potential of quadrotors to solve challenging tasks. Generally, the derivation of the controllers involve complex analytical manipulation of the dynamics and are very specific to the task at hand. In addition, most approaches construct a trajectory and then design a stabilizing controller in a separate step, whereas a fully optimal solution requires finding both simultaneously. In this paper, a generalized approach is presented using an iterative optimal control algorithm. A series of complex tasks are thus solved using the same algorithm without the need for manual manipulation of the system dynamics, heuristic simplifications, or manual trajectory generation. First, aggressive maneuvers are performed by requiring the quadrotor to pass with a slung load through a window not high enough for the load to pass while hanging straight down. Second, go-to-goal tasks with single and double rotor failure are demonstrated. The adaptability and applicability of this unified approach to such diverse tasks with a nonlinear, underactuated, constrained, and in the case of the slung load, hybrid quadrotor systems is thus shown.

The VelociRoACH is a 10 cm long, 30 gram hexapedal millirobot capable of running at 2.7 m/s, making it the fastest legged robot built to date, relative to scale. We present the design by dynamic similarity technique and the locomotion adaptations which have allowed for this highly dynamic performance. In addition, we demonstrate that rotational dynamics become critical for stability as the scale of a robotic system is reduced. We present a new method of experimental dynamic tuning for legged millirobots, aimed at finding stable limit cycles with minimal rotational energy. By implementing an aerodynamic rotational damper, we further reduced the rotational energy in the system, and demonstrated that stable limit cycles with lower rotational energy are more robust to disturbances. This method increased the stability of the system without detracting from forward speed.

This paper deals with the modelling, simulation-based controller design and path planning of a four rotor helicopter known as a quadrotor. All the drags, aerodynamic, coriolis and gyroscopic effect are neglected. A Newton-Euler formulation is used to derive the mathematical model. A smart self-tuning fuzzy PID controller based on an EKF algorithm is proposed for the attitude and position control of the quadrotor. The PID gains are tuned using a self-tuning fuzzy algorithm. The self-tuning of fuzzy parameters is achieved based on an EKF algorithm. A smart selection technique and exclusive tuning of active fuzzy parameters is proposed to reduce the computational time. Dijkstra’s algorithm is used for path planning in a closed and known environment filled with obstacles and/or boundaries. The Dijkstra algorithm helps avoid obstacle and find the shortest route from a given initial position to the final position.

Robots using ball(s) as spherical wheels have the advantage of omnidirectional motion. Some of these robots use only a single ball as their wheel and dynamically balance on it. However, driving a ball wheel is not straightforward. These robots usually use one or more motor-driven rollers or wheels in frictional contact with the ball wheel. Some slippage can occur with these schemes, so it is important to measure the actual rotation of the ball wheel for good control. This paper proposes one method for measuring the three dimensional rotation of a sphere, which is applicable to the case of a ball wheel. The system measures surface speed by using two or more optical mouse sensors and transforms them into the angular velocity vector of the ball, followed by integration to give the rotational angle. Experiments showed the correctness of this method, yielding an error of approximately 1%, with a 10 ms response.

We propose a new method for the localization of a Micro Aerial Vehicle (MAV) with respect to a ground robot. We solve the problem of registering the 3D maps computed by the robots using different sensors: a dense 3D reconstruction from the MAV monocular camera is aligned with the map computed from the depth sensor on the ground robot. Once aligned, the dense reconstruction from the MAV is used to augment the map computed by the ground robot, by extending it with the information conveyed by the aerial views. The overall approach is novel, as it builds on recent developments in live dense reconstruction from moving cameras to address the problem of air-ground localization. The core of our contribution is constituted by a novel algorithm integrating dense reconstructions from monocular views, Monte Carlo localization, and an iterative pose refinement. In spite of the radically different vantage points from which the maps are acquired, the proposed method achieves high accuracy whereas appearance-based, state-of-the-art approaches fail. Experimental validation in indoor and outdoor scenarios reported an accuracy in position estimation of 0.08 meters and real time performance. This demonstrates that our new approach effectively overcomes the limitations imposed by the difference in sensors and vantage points that negatively affect previous techniques relying on matching visual features.

This paper presents the mechanical design and control system design of an electromagnetic actuator-based microdisplacement module. The microdisplacement module composed of a symmetrical leaf-spring parallelogram mechanism and an electromagnetic actuator. The characteristics of the mechanism in terms of stiffness and natural frequencies are derived and verified. Both leakage flux and core inductance are taken into consideration during modeling the mathematic model of the electromagnetic actuator, and based on which, the system dynamic model is established. Due to the nonlinearity characteristics of the system, a dynamic sliding-mode controller is designed without linearizing the system dynamics. A prototype of the microdisplacement module is fabricated, and the parameters of the system are identified and calibrated. Finally, the designed dynamic sliding-mode controller is applied; step response and tracking performance are studied. Experimental results demonstrate that a submicrometer accuracy can be achieved by the module, which validate the effectiveness of the proposed mechanism and controller design as well. The research results show that the electromagnetic actuator-based module can be extended to wide applications in the field of micro/nanopositioning and manipulation.


Question: An underactuated propeller utilized for attitude control in micro air vehicles.

This paper explores the design, analysis, and performance evaluation of a micro air vehicle (MAV) featuring a coaxial rotor configuration. The focus is on utilizing custom manufacturing methods alongside readily available, off-the-shelf components to create a viable micro-coaxial rotorcraft. The paper delves into the challenges and potential solutions for achieving stable hover and comprehensive flight control at miniature scales, specifically for coaxial rotor systems. Based on preliminary studies, the authors assert that it's feasible to construct a micro rotorcraft weighing around 100 grams that can effectively control roll, yaw, and lateral movements. A test model was constructed and its rotor performance was assessed using a custom-built hover stand, which measured thrust and power output. The radio-controlled MAV was successfully flown untethered, powered by its own energy source, demonstrating commendable flight stability and control characteristics. The maximum rotor performance recorded was 42%, indicating promising capabilities for micro air vehicles in this design category.

**Question**: A Multi-Agent System for Energy Management in Smart Grids

**Documents**: Multi-Agent Systems for the Smart Grid: A Survey
In this paper, we present a survey of multi-agent systems (MAS) for smart grid applications. We describe the main concepts and technologies used in MAS and discuss how they can be applied to the smart grid. We also review the literature on MAS for smart grid applications and identify the main challenges and open issues in this area. The paper is organized as follows: Section 2 provides a brief overview of MAS and their main characteristics. Section 3 describes the main technologies used in MAS, including agent communication languages, agent programming languages, and agent platforms. Section 4 discusses how MAS can be applied to the smart grid, focusing on the main tasks and functions of MAS in this context. Section 5 presents a review of the literature on MAS for smart grid applications, including a discussion of the main challenges and open issues in this area. Finally, Section 6 concludes the paper and provides some directions for future research.

**Similar Documents**: This comprehensive survey delves into the application of multi-agent systems (MAS) within the realm of smart grids. It offers a concise explanation of the foundational principles and technological underpinnings of MAS, alongside an exploration of their potential integration into smart grid operations. The survey is methodically structured: Section 2 offers a succinct introduction to MAS, highlighting their key attributes. Section 3 details the technological frameworks that underpin MAS, encompassing agent communication languages, programming languages, and platforms. Section 4

This letter describes dynamic self-assembly of two-component rotors floating at the interface between liquid and air into simple, reconfigurable mechanical systems ~‘‘machines’’!. The rotors are powered by an external, rotating magnetic field, and their positions within the interface are controlled by:~i! repulsive hydrodynamic interactions between them and ~ii ! by localized magnetic fields produced by an array of small electromagnets located below the plane of the interface. The mechanical functions of the machines depend on the spatiotemporal sequence of activation of the electromagnets. ©2004 American Institute of Physics. @DOI: 10.1063/1.1664019 #

This paper details the design, modeling, and experimental validation of a novel mobile robot capable of both aerial and terrestrial locomotion. Flight is achieved through a quadrotor configuration; four actuators provide the required thrust. Adding a rolling cage to the quadrotor makes terrestrial locomotion possible using the same actuator set and control system. Thus, neither the mass nor the system complexity is increased by inclusion of separate actuators for terrestrial and aerial locomotion. An analysis of the system's energy consumption demonstrates that during terrestrial locomotion, the robot only needs to overcome rolling resistance and consumes much less energy compared to the aerial mode. This solves one of the most vexing problems of quadrotors and rotorcraft in general - their short operation time. Experimental results show that the hybrid robot can travel a distance four times greater and operate almost six times longer than an aerial only system. It also solves one of the most challenging problems in terrestrial robot design - obstacle avoidance. When an obstacle is encountered, the system simply flies over it.

This paper presents the mechanical design of an ultra-lightweight, biomimetically actuated anthropomorphic hand for prosthetic purposes. The proposed design is based on an underactuated configuration of 16 joints and 7 active degrees of freedom. Shape Memory Alloy wires are used as motive elements of a specially designed actuation system installed within the envelope of the hand and forearm; due to their inherent contraction ability when heated, these innovative micro-actuators produce linear motion which is imparted to the fingers via a tendon transmission system. The overall design is completed with the integration of the necessary locking mechanisms for power saving. An experimental prototype of the suggested prosthesis has been fabricated using rapid prototyping techniques and it will be used as an evaluation platform for further research towards the development of a truly multifunctional, silent and cosmetically appealing hand for upper limb amputees.

The combination of on-board sensors measurements with different statistical characteristics can be employed in robotics for localization and control, especially in GPS-denied environments. In particular, most aerial vehicles are packaged with low cost sensors, important for aerial robotics, such as camera, a gyroscope, and an accelerometer. In this work, we develop a visual inertial odometry system based on the Unscented Kalman Filter (UKF) acting on the Lie group SE(3), such to obtain an unique, singularity-free representation of a rigid body pose. We model this pose with the Lie group SE(3) and model the noise on the corresponding Lie algebra. Moreover, we extend the concepts used in the standard UKF formulation, such as state uncertainty and modeling, to correctly incorporate elements that do not belong to an Euclidean space such as the Lie group members. In this analysis, we use the parallel transport, which requires us to explicitly consider SE(3) as representing rigid bodies though the use of the affine connection. We present experimental results to show the effectiveness of the proposed approach for state estimation of a quadrotor platform.

Scout II, an autonomous four-legged robot with only one actuator per compliant leg is described. We demonstrate the need to model the actuators and the power source of the robot system carefully in order to obtain experimentally valid models for simulation and analysis. We describe a new, simple running controller that requires minimal task level feedback, yet achieves reliable and fast running up to 1.2 m/s. These results contribute to the increasing evidence that apparently complex dynamically dexterous tasks may be controlled via simple control laws. In addition, the simple mechanical design of our robot may be used as a template for the control of higher degree of freedom quadrupeds. An energetics analysis reveals a highly efficient system with a specific resistance of 0.32 when based on mechanical power dissipation and of 1.0 when based on total electrical power dissipation.

This paper presents a novel orientation algorithm designed to support a computationally efficient, wearable inertial human motion tracking system for rehabilitation applications. It is applicable to inertial measurement units (IMUs) consisting of tri-axis gyroscopes and accelerometers, and magnetic angular rate and gravity (MARG) sensor arrays that also include tri-axis magnetometers. The MARG implementation incorporates magnetic distortion compensation. The algorithm uses a quaternion representation, allowing accelerometer and magnetometer data to be used in an analytically derived and optimised gradient descent algorithm to compute the direction of the gyroscope measurement error as a quaternion derivative. Performance has been evaluated empirically using a commercially available orientation sensor and reference measurements of orientation obtained using an optical measurement system. Performance was also benchmarked against the propriety Kalman-based algorithm of orientation sensor. Results indicate the algorithm achieves levels of accuracy matching that of the Kalman based algorithm; < 0.8° static RMS error, < 1.7° dynamic RMS error. The implications of the low computational load and ability to operate at small sampling rates significantly reduces the hardware and power necessary for wearable inertial movement tracking, enabling the creation of lightweight, inexpensive systems capable of functioning for extended periods of time.

This article presents an image-based visual servoing system for indoor visual tracking of 3D moving objects by an Unmanned Aerial Vehicle. This system autonomously follows a 3D moving target object, maintaining it with a fixed distance and centered on its image plane. The initial setup is tested in a detailed simulation environment. The system is then validated on flights in indoor scenarios using the Parrot AR.Drone and the CMT tracker, demonstrating the robustness of the system to differences in object features, environmental clutter, and target trajectory. The obtained results indicate that the proposed system is suitable for complex controls task, such object surveillance and pursuit.

The aim of this work is to describe an intelligent system based on fuzzy logic that is developed to control a quadrotor. A quadrotor is a helicopter with four rotors, that make the vehicle more stable but more complex to model and to control. The quadrotor has been used as a testing platform in the last years for various universities and research centres. A quadrotor has six degrees of freedom, three of them regarding the position: height, horizontal and vertical motions; and the other three are related to the orientation: pitch, roll and yaw. A fuzzy control is designed and implemented to control a simulation model of the quadrotor. The inputs are the desired values of the height, roll, pitch and yaw. The outputs are the power of each of the four rotors that is necessary to reach the specifications. Simulation results prove the efficiency of this intelligent control strategy.

This paper presents the mechanical design and control system design of an electromagnetic actuator-based microdisplacement module. The microdisplacement module composed of a symmetrical leaf-spring parallelogram mechanism and an electromagnetic actuator. The characteristics of the mechanism in terms of stiffness and natural frequencies are derived and verified. Both leakage flux and core inductance are taken into consideration during modeling the mathematic model of the electromagnetic actuator, and based on which, the system dynamic model is established. Due to the nonlinearity characteristics of the system, a dynamic sliding-mode controller is designed without linearizing the system dynamics. A prototype of the microdisplacement module is fabricated, and the parameters of the system are identified and calibrated. Finally, the designed dynamic sliding-mode controller is applied; step response and tracking performance are studied. Experimental results demonstrate that a submicrometer accuracy can be achieved by the module, which validate the effectiveness of the proposed mechanism and controller design as well. The research results show that the electromagnetic actuator-based module can be extended to wide applications in the field of micro/nanopositioning and manipulation.

This paper reviews some main results and progress in distributed multi-agent coordination, focusing on papers published in major control systems and robotics journals since 2006. Distributed coordination of multiple vehicles, including unmanned aerial vehicles, unmanned ground vehicles, and unmanned underwater vehicles, has been a very active research subject studied extensively by the systems and control community. The recent results in this area are categorized into several directions, such as consensus, formation control, optimization, and estimation. After the review, a short discussion section is included to summarize the existing research and to propose several promising research directions along with some open problems that are deemed important for further investigations.


Question: Probability-based text structuring: trials in sentence sequencing.

This paper recounts the process and outcomes of creating a significant annotated corpus of English language texts, known as the Penn Treebank. This corpus, encompassing more than 4.5 million words of American English, was developed over the initial three-year phase of the Penn Treebank Project (1989-1992). During this period, the corpus was annotated with part-of-speech (POS) information, and over half of it was further annotated to include skeletal syntactic structure. The University of Pennsylvania's Department of Computer and Information Science produced this technical report, which can be accessed at ScholarlyCommons under the reference MSCIS-93-87. The report, titled "Building A Large Annotated Corpus of English: The Penn Treebank," was authored by Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz from the University of Pennsylvania's School of Engineering and Applied Science, specifically the Computer and Information Science Department, located in Philadelphia, PA 19104-6389. The creation of this corpus represents a substantial contribution to the field of computational linguistics and natural language processing, providing a valuable resource for linguistic research and the development of language processing algorithms.

Discourse segmentation, which segments texts into Elementary Discourse Units, is a fundamental step in discourse analysis. Previous discourse segmenters rely on complicated hand-crafted features and are not practical in actual use. In this paper, we propose an endto-end neural segmenter based on BiLSTMCRF framework. To improve its accuracy, we address the problem of data insufficiency by transferring a word representation model that is trained on a large corpus. We also propose a restricted self-attention mechanism in order to capture useful information within a neighborhood. Experiments on the RST-DT corpus show that our model is significantly faster than previous methods, while achieving new stateof-the-art performance. 1

We propose a new unsupervised learning technique for extracting information from large text collections. We model documents as if they were generated by a two-stage stochastic process. Each author is represented by a probability distribution over topics, and each topic is represented as a probability distribution over words for that topic. The words in a multi-author paper are assumed to be the result of a mixture of each authors' topic mixture. The topic-word and author-topic distributions are learned from data in an unsupervised manner using a Markov chain Monte Carlo algorithm. We apply the methodology to a large corpus of 160,000 abstracts and 85,000 authors from the well-known CiteSeer digital library, and learn a model with 300 topics. We discuss in detail the interpretation of the results discovered by the system including specific topic and author models, ranking of authors by topic and topics by author, significant trends in the computer science literature between 1990 and 2002, parsing of abstracts by topics and authors and detection of unusual papers by specific authors. An online query interface to the model is also discussed that allows interactive exploration of author-topic models for corpora such as CiteSeer.

In this paper, we review our experience with constructing one such large annotated corpus--the Penn Treebank, a corpus consisting of over 4.5 million words of American English. During the first three-year phase of the Penn Treebank Project (1989-1992), this corpus has been annotated for part-of-speech (POS) information. In addition, over half of it has been annotated for skeletal syntactic structure. Comments University of Pennsylvania Department of Computer and Information Science Technical Report No. MSCIS-93-87. This technical report is available at ScholarlyCommons: http://repository.upenn.edu/cis_reports/237 Building A Large Annotated Corpus of English: The Penn Treebank MS-CIS-93-87 LINC LAB 260 Mitchell P. Marcus Beatrice Santorini Mary Ann Marcinkiewicz University of Pennsylvania School of Engineering and Applied Science Computer and Information Science Department Philadelphia, PA 19104-6389

A citing sentence is one that appears in a scientific article and cites previous work. Citing sentences have been studied and used in many applications. For example, they have been used in scientific paper summarization, automatic survey generation, paraphrase identification, and citation function classification. Citing sentences that cite multiple papers are common in scientific writing. This observation should be taken into consideration when using citing sentences in applications. For instance, when a citing sentence is used in a summary of a scientific paper, only the fragments of the sentence that are relevant to the summarized paper should be included in the summary. In this paper, we present and compare three different approaches for identifying the fragments of a citing sentence that are related to a given target reference. Our methods are: word classification, sequence labeling, and segment classification. Our experiments show that segment classification achieves the best results.

GENERATING NARRATIVE VARIATION IN INTERACTIVE FICTION Nick Montfort Mitchell P. Marcus and Gerald Prince A general method for the generation of natural language narrative is described. It allows the expression, or narrative discourse, to vary independently of the underlying events and existents that are the narrative’s content. Specifically, this variation is accomplished in an interactive fiction (IF) system which replies to typed input by narrating what has happened in a simulated world. IF works have existed for about 30 years as forms of text-based computer simulation, instances of dialog systems, and examples of literary art. Theorists of narrative have carefully distinguished between the level of underlying content (corresponding to the simulated world in interactive fiction) and that of expression (corresponding to the textual exchange between computer arnd user) since the mid-1960s, when the field of narratology began to develop, but IF systems have not yet made use of this distinction. The current project contributes new techniques for automatic narration by building on work done in computational linguistics, specifically natural language generation, and in narratology. First, types of narrative variation that are possible in IF are identified and formalized in a way that is suitable for a natural language generation system. An architecture for an IF system is then described and implemented; the result allows multiple works of interactive fiction to be realized and, using a general plan for narrating, allows them to be narrated in different ways during interaction. The system’s ability to generate text is considered in a pilot evaluation. Plans for future work are also discussed. They include publicly released systems for IF development and narratology education, adding a planning capability that uses actors’ individual perspectives, and adapting automatic narration to different sorts of interactive systems.

Machine reading calls for programs that read and understand text, but most current work only attempts to extract facts from redundant web-scale corpora. In this paper, we focus on a new reading comprehension task that requires complex reasoning over a single document. The input is a paragraph describing a biological process, and the goal is to answer questions that require an understanding of the relations between entities and events in the process. To answer the questions, we first predict a rich structure representing the process in the paragraph. Then, we map the question to a formal query, which is executed against the predicted structure. We demonstrate that answering questions via predicted structures substantially improves accuracy over baselines that use shallower representations.

An adaptive statistical language model is described, which successfully integrates long distance linguistic information with other knowledge sources. Most existing statistical language models exploit only the immediate history of a text. To extract information from further back in the document’s history, we propose and use trigger pairs as the basic information bearing elements. This allows the model to adapt its expectations to the topic of discourse. Next, statistical evidence from multiple sources must be combined. Traditionally, linear interpolation and its variants have been used, but these are shown here to be seriously deficient. Instead, we apply the principle of Maximum Entropy (ME). Each information source gives rise to a set of constraints, to be imposed on the combined estimate. The intersection of these constraints is the set of probability functions which are consistent with all the information sources. The function with the highest entropy within that set is the ME solution. Given consistent statistical evidence, a unique ME solution is guaranteed to exist, and an iterative algorithm exists which is guaranteed to converge to it. The ME framework is extremely general: any phenomenon that can be described in terms of statistics of the text can be readily incorporated. An adaptive language model based on the ME approach was trained on the Wall Street Journal corpus, and showed a 32–39% perplexity reduction over the baseline. When interfaced to SPHINX-II, Carnegie Mellon’s speech recognizer, it reduced its error rate by 10–14%. This thus illustrates the feasibility of incorporating many diverse knowledge sources in a single, unified statistical framework.  1996 Academic Press Limited

This paper develops a model that addresses sentence embedding, a hot topic in current natural language processing research, using recurrent neural networks (RNN) with Long Short-Term Memory (LSTM) cells. The proposed LSTM-RNN model sequentially takes each word in a sentence, extracts its information, and embeds it into a semantic vector. Due to its ability to capture long term memory, the LSTM-RNN accumulates increasingly richer information as it goes through the sentence, and when it reaches the last word, the hidden layer of the network provides a semantic representation of the whole sentence. In this paper, the LSTM-RNN is trained in a weakly supervised manner on user click-through data logged by a commercial web search engine. Visualization and analysis are performed to understand how the embedding process works. The model is found to automatically attenuate the unimportant words and detect the salient keywords in the sentence. Furthermore, these detected keywords are found to automatically activate different cells of the LSTM-RNN, where words belonging to a similar topic activate the same cell. As a semantic representation of the sentence, the embedding vector can be used in many different applications. These automatic keyword detection and topic allocation abilities enabled by the LSTM-RNN allow the network to perform document retrieval, a difficult language processing task, where the similarity between the query and documents can be measured by the distance between their corresponding sentence embedding vectors computed by the LSTM-RNN. On a web search task, the LSTM-RNN embedding is shown to significantly outperform several existing state of the art methods. We emphasize that the proposed model generates sentence embedding vectors that are specially useful for web document retrieval tasks. A comparison with a well known general sentence embedding method, the Paragraph Vector, is performed. The results show that the proposed method in this paper significantly outperforms Paragraph Vector method for web document retrieval task.

Language acquisition in both natural and artificial language learning settings crucially depends on extracting information from ordered sequences. A shared sequence learning mechanism is thus assumed to underlie both natural and artificial language learning. A growing body of empirical evidence is consistent with this hypothesis. By means of artificial language learning experiments, we may therefore gain more insight in this shared mechanism. In this paper, we review empirical evidence from artificial language learning and computational modeling studies, as well as natural language data, and suggest that there are two key factors that help determine processing complexity in sequence learning, and thus in natural language processing. We propose that the specific ordering of non-adjacent dependencies (i.e. nested or crossed), as well as the number of non-adjacent dependencies to be resolved simultaneously (i.e. two or three) are important factors in gaining more insight into the boundaries of human sequence learning; and thus, also in natural language processing. The implications for theories of linguistic competence are discussed.

This paper proposes a framework for automatically engineering features for two important tasks of question answering: answer sentence selection and answer extraction. We represent question and answer sentence pairs with linguistic structures enriched by semantic information, where the latter is produced by automatic classifiers, e.g., question classifier and Named Entity Recognizer. Tree kernels applied to such structures enable a simple way to generate highly discriminative structural features that combine syntactic and semantic information encoded in the input trees. We conduct experiments on a public benchmark from TREC to compare with previous systems for answer sentence selection and answer extraction. The results show that our models greatly improve on the state of the art, e.g., up to 22% on F1 (relative improvement) for answer extraction, while using no additional resources and no manual feature engineering.


Question: Probability-based text structuring: trials with sentence sequencing.

This study delves into the intricacies of sentence ordering for multidocument news summarization, a task that has not garnered substantial research focus despite its significance in generating coherent summaries. Unlike single document summarization, where sentence order can be derived from the input article's structure, multidocument summarization presents a more complex challenge due to the potential for summary sentences to originate from disparate sources. To address this, we have devised a method to analyze and understand the characteristics of ordering information within the news domain. Our approach was validated through experiments conducted on a specially curated corpus featuring multiple valid sentence orderings. Inspired by these findings, we have developed a strategy that integrates constraints from both the chronological sequence of events and topical coherence. This combined strategy has been incorporated into our summarization algorithm, resulting in a notable enhancement in summary coherence compared to baseline methods. Our empirical evaluation underscores the efficacy of this novel approach in improving the logical flow and relevance of multidocument news summaries.

We address the problem of query segmentation: given a keyword query submitted to a search engine, the task is to group the keywords into phrases, if possible. Previous approaches to the problem achieve good segmentation performance on a gold standard but are fairly intricate. Our method is easy to implement and comes with a comparable accuracy.

The ACL Anthology Network (AAN)1 is a comprehensive manually curated networked database of citations and collaborations in the field of Computational Linguistics. Each citation edge in AAN is associated with one or more citing sentences. A citing sentence is one that appears in a scientific article and contains an explicit reference to another article. In this paper, we shed the light on the usefulness of AAN citing sentences for understanding research trends and summarizing previous discoveries and contributions. We also propose and motivate several different uses and applications of citing sentences.

Statistical methods have long been the dominant approach in speech recognition and probabilistic modelling in ASR is now a mature technology. The use of statistical methods in other areas of spoken dialogue is however more recent and rather less mature. This paper reviews spoken dialogue systems from a statistical modelling perspective. The complete system is first presented as a partially observable Markov decision process. The various sub-components are then exposed by introducing appropriate intermediate variables. Samples of existing work are reviewed within this framework, including dialogue control and optimisation, semantic interpretation, goal detection, natural language generation and synthesis.

Since sentences in patent texts are long, they are difficult to translate by a machine. Although statistical machine translation is one of the major streams of the field, long patent sentences are difficult to translate not using syntactic analysis. We propose the combination of a rule based method and a statistical method. It is a rule based machine translation (RMT) with a statistical based post editor (SPE). The evaluation by the NIST score shows RMT+SPE is more accurate than RMT only. Manual checks, however, show the outputs of RMT+SPE often have strange expressions in the target language. So we propose a new evaluation measure NMG (normalized mean grams). Although NMG is based on n-gram, it counts the number of words in the longest word sequence matches between the test sentence and the target language reference corpus. We use two reference corpora. One is the reference translation only the other is a large scaled target language corpus. In the former case, RMT+SPE wins in the later case, RMT wins.

In this paper we develop a story generator that leverages knowledge inherent in corpora without requiring extensive manual involvement. A key feature in our approach is the reliance on a story planner which we acquire automatically by recording events, their participants, and their precedence relationships in a training corpus. Contrary to previous work our system does not follow a generate-and-rank architecture. Instead, we employ evolutionary search techniques to explore the space of possible stories which we argue are well suited to the story generation task. Experiments on generating simple children’s stories show that our system outperforms previous data-driven approaches.

Language acquisition in both natural and artificial language learning settings crucially depends on extracting information from ordered sequences. A shared sequence learning mechanism is thus assumed to underlie both natural and artificial language learning. A growing body of empirical evidence is consistent with this hypothesis. By means of artificial language learning experiments, we may therefore gain more insight in this shared mechanism. In this paper, we review empirical evidence from artificial language learning and computational modeling studies, as well as natural language data, and suggest that there are two key factors that help determine processing complexity in sequence learning, and thus in natural language processing. We propose that the specific ordering of non-adjacent dependencies (i.e. nested or crossed), as well as the number of non-adjacent dependencies to be resolved simultaneously (i.e. two or three) are important factors in gaining more insight into the boundaries of human sequence learning; and thus, also in natural language processing. The implications for theories of linguistic competence are discussed.

Current research in natural language generation is situated in a computational linguistics tradition that was founded several decades ago. We critically analyse some of the architectural assumptions underlying existing systems and point out some problems in the domains of text planning and lexicalization. Guided by the identification of major generation challenges viewed from the angles of knowledge-based systems and cognitive psychology, we sketch some new directions for future research.

Conventional QA systems cannot answer to the questions composed of two or more sentences. Therefore, we aim to construct a QA system that can answer such multiple-sentence questions. As the first stage, we propose a method for classifying multiple-sentence questions into question types. Specifically, we first extract the core sentence from a given question text. We use the core sentence and its question focus in question classification. The result of experiments shows that the proposed method improves F-measure by 8.8% and accuracy by 4.4%.

Sentiment analysis seeks to identify the viewpoint(s) underlying a text span; an example application is classifying a movie review as “thumbs up” or “thumbs down”. To determine this sentiment polarity, we propose a novel machine-learning method that applies text-categorization techniques to just the subjective portions of the document. Extracting these portions can be implemented using efficient techniques for finding minimum cuts in graphs; this greatly facilitates incorporation of cross-sentence contextual constraints. Publication info: Proceedings of the ACL, 2004.

Language is used to communicate ideas. Ideas are mental tools for coping with a complex and uncertain world. Thus human conceptual structures should be key to language meaning, and probability—the mathematics of uncertainty— should be indispensable for describing both language and thought. Indeed, probabilistic models are enormously useful in modeling human cognition (Tenenbaum et al., 2011) and aspects of natural language (Bod et al., 2003; Chater et al., 2006). With a few early exceptions (e.g. Adams, 1975; Cohen, 1999b), probabilistic tools have only recently been used in natural language semantics and pragmatics. In this chapter we synthesize several of these modeling advances, exploring a formal model of interpretation grounded, via lexical semantics and pragmatic inference, in conceptual structure. Flexible human cognition is derived in large part from our ability to imagine possibilities (or possible worlds). A rich set of concepts, intuitive theories, and other mental representations support imagining and reasoning about possible worlds—together we will call these the conceptual lexicon. We posit that this collection of concepts also forms the set of primitive elements available for lexical semantics: word meanings can be built from the pieces of conceptual structure. Larger semantic structures are then built from word meanings by composition, ultimately resulting in a sentence meaning which is a phrase in the “language of thought” provided by the conceptual lexicon. This expression is truth-functional in that it takes on a Boolean value for each imagined world, and it can thus be used as the basis for belief updating. However, the connection between cognition, semantics, and belief is not direct: because language must flexibly adapt to the context of communication, the connection between lexical representation and interpreted meaning is mediated by pragmatic inference.


Question: Probability-based text structuring: trials with sentence sequencing.

This paper introduces a novel parsing technique that achieves high accuracy in generating Penn treebank-style parse trees, specifically for sentences up to 40 words in length, with a 90.1% average precision/recall, and for sentences up to 100 words, with an 89.5% average precision/recall. These results were obtained when the parser was trained and tested on the established sections of the Wall Street Journal treebank, as delineated in previous studies [5,9,10,15,17]. The new parser outperforms existing single-parser approaches by reducing the error rate by 13% on this corpus [9]. The key innovation lies in the adoption of a "maximum-entropy-inspired" model for conditioning and smoothing, which enables the successful testing and integration of a variety of conditioning events. Additionally, the paper offers preliminary findings that highlight the impact of different conditioning information, notably a 2% enhancement in accuracy from predicting the lexical head's pre-terminal before identifying the lexical head itself. This approach offers a significant advancement in parsing efficiency and accuracy, particularly in handling complex sentence structures within the specified word length.

Statistical learning problems in many fields involve sequential data. This paper formalizes the principal learning tasks and describes the methods that have been developed within the machine learning research community for addressing these problems. These methods include sliding window methods, recurrent sliding windows, hidden Markov models, conditional random fields, and graph transformer networks. The paper also discusses some open research issues.

Syntactic parsing deals with syntactic structure of a sentence. The word 'syntax' refers to the grammatical arrangement of words in a sentence and their relationship with each other. The objective of syntactic analysis is to find syntactic structure of a sentence which is usually depicted as a tree. Identifying the syntactic structure is useful in determining the meaning of a sentence. Natural language processing is an arena of computer science and linguistics, concerned with the dealings amongst computers and human languages. It processes the data through lexical analysis, Syntax analysis, Semantic analysis, Discourse processing, Pragmatic analysis. This paper gives various parsing methods. The algorithm in this paper splits the English sentences into parts using POS tagger, It identifies the type of sentence (Facts, active, passive etc.) and then parses these sentences using grammar rules of Natural language. The algorithm has been tested on real sentences of English and it accomplished an accuracy of 81%.

We present a novel approach to recognizing Textual Entailment. Structural features are constructed from abstract tree descriptions, which are automatically extracted from syntactic dependency trees. These features are then applied in a subsequence-kernel-based classifier to learn whether an entailment relation holds between two texts. Our method makes use of machine learning techniques using a limited data set, no external knowledge bases (e.g. WordNet), and no handcrafted inference rules. We achieve an accuracy of 74.5% for text pairs in the Information Extraction and Question Answering task, 63.6% for the RTE-2 test data, and 66.9% for the RET-3 test data.

We propose a new unsupervised learning technique for extracting information from large text collections. We model documents as if they were generated by a two-stage stochastic process. Each author is represented by a probability distribution over topics, and each topic is represented as a probability distribution over words for that topic. The words in a multi-author paper are assumed to be the result of a mixture of each authors' topic mixture. The topic-word and author-topic distributions are learned from data in an unsupervised manner using a Markov chain Monte Carlo algorithm. We apply the methodology to a large corpus of 160,000 abstracts and 85,000 authors from the well-known CiteSeer digital library, and learn a model with 300 topics. We discuss in detail the interpretation of the results discovered by the system including specific topic and author models, ranking of authors by topic and topics by author, significant trends in the computer science literature between 1990 and 2002, parsing of abstracts by topics and authors and detection of unusual papers by specific authors. An online query interface to the model is also discussed that allows interactive exploration of author-topic models for corpora such as CiteSeer.

This paper presents a grammatical error correction (GEC) system that provides corrective feedback for essays. We apply the neural sequence-to-sequence model, which is frequently used in machine translation and text summarization, to this GEC task. The model is trained on EF-Cambridge Open Language Database (EFCAMDAT), a large learner corpus annotated with grammatical errors and corrections. Evaluation shows that our system achieves competitive performance on a number of publicly available testsets.

Implicit discourse relation recognition is a challenging task as the relation prediction without explicit connectives in discourse parsing needs understanding of text spans and cannot be easily derived from surface features from the input sentence pairs. Thus, properly representing the text is very crucial to this task. In this paper, we propose a model augmented with different grained text representations, including character, subword, word, sentence, and sentence pair levels. The proposed deeper model is evaluated on the benchmark treebank and achieves stateof-the-art accuracy with greater than 48% in 11-way and F1 score greater than 50% in 4-way classifications for the first time according to our best knowledge.

Introduction Statistical language models estimate the probability for a given sequence of words. Given a sentence s with n words such as s = (w1,w2 . . .wn), the language model assigns P(s). Statistical language models assess good word sequence estimations based on the sequence probability estimation. Building robust, fast and accurate language models is one of the main factors in the success of building systems such as machine translation systems and automatic speech recognition systems. Statistical language models can be estimated based on various approaches. Classical language models are estimated based on n-gram word sequences such as P(s) = P(w1,w2 . . .wn) = ∏n i=1 P(wi|wi−1) and can be approximated based on the Markov concept for shorter contexts (for example, bigram if n = 2 or trigram if n = 3 and so on). Recent researchers have applied neural networks different architectures to build and estimate language models. The classical feed-forward neural network-based language models have been continuously reporting good results among the traditional n-gram language modeling techniques [1]. It Abstract

In this thesis, we present a novel approach for modeling textual entailment using lexicalsemantic information on the level of predicate-argument structure. To this end, we adopt information provided by the Berkeley FrameNet repository and embed it into an implemented end-to-end system. The two main goals of this thesis are the following: (i) to provide an analysis of the potential contribution of frame semantic information to the recognition textual entailment and (ii) to present a robust system architecture that can serve as basis for future experiments, research, and improvement. Our work was carried out in the context of the textual entailment initiative, which since 2005 has set the stage for the broad investigation of inference in natural-language processing tasks, including empirical evaluation of its coverage and reliability. In short, textual entailment describes inferential relations between (entailing) texts and (entailed) hypotheses as interpreted by typical language users. This pre-theoretic notion captures a natural range of inferences as compared to logical entailment, which has traditionally been used within theoretical approaches to natural language semantics. Various methods for modeling textual entailment have been proposed in the literature, ranging from shallow techniques like lexical overlap to shallow syntactic parsing and the exploitation of WordNet relations. Recently, there has been a move towards more structured meaning representations. In particular, the level of predicate-argument structure has gained much attention, which seems to be a natural and straightforward choice. Predicate-argument structure allows annotating sentences or texts with nuclear meaning representations (“who did what to whom”), which are of obvious relevance for this task. For example, it can account for paraphrases like “Ghosts scare John” vs. “John is scared by ghosts”. In this thesis, we present an approach to textual entailment that is centered around the analysis of predicate-argument structure. It combines LFG grammatical analysis, predicate-argument structure in the FrameNet paradigm, and taxonomic information from WordNet into tripartite graph structures. By way of a declarative graph matching algorithm, the “structural and semantic” similarity of hypotheses and texts is computed and the result is represented as feature vectors. A supervised machine learning architecture trained on entailment corpora is used to check textual entailment for new text/hypothesis pairs. The approach is implemented in the SALSA RTE system, which successfully participated in the second and third RTE challenge. While system performance is on a par with that of comparable systems, the intuitively expected strong positive effect of using FrameNet information has not yet been confirmed. In order to evaluate different system components and to assess the potential contribution of FrameNet information for checking textual entailment, we conducted a number of experiments. For example, with the help of a gold-standard corpus, we

Processing language requires the retrieval of concepts from memory in response to an ongoing stream of information. This retrieval is facilitated if one can infer the gist of a sentence, conversation, or document and use that gist to predict related concepts and disambiguate words. This article analyzes the abstract computational problem underlying the extraction and use of gist, formulating this problem as a rational statistical inference. This leads to a novel approach to semantic representation in which word meanings are represented in terms of a set of probabilistic topics. The topic model performs well in predicting word association and the effects of semantic association and ambiguity on a variety of language-processing and memory tasks. It also provides a foundation for developing more richly structured statistical models of language, as the generative process assumed in the topic model can easily be extended to incorporate other kinds of semantic and syntactic structure.

Statistical language models estimate the distribution of various natural language phenomena for the purpose of speech recognition and other language technologies. Since the first significant model was proposed in 1980, many attempts have been made to improve the state of the art. We review them, point to a few promising directions, and argue for a Bayesian approach to integration of linguistic theories with data.


Question: Probability-based text structuring: trials in sentence sequencing.

A new method for estimating probabilities from sparse data is introduced, tailored for the language model component of a speech recognizer. This innovative n-gram language model offers a computationally and memory-efficient solution to the estimation problem through a non-linear recursive procedure, outperforming alternative methods. Originally developed for IBM's Real Time Speech Recognizers, the method's versatility extends to other fields where probability estimation from sparse data is required. Sparse data, a common feature of real texts, presents a challenge in gathering frequency statistics on words and word sequences (n-grams) from finite-sized texts. This leads to issues with maximum likelihood estimation, as it fails to accurately estimate probabilities of infrequent yet possible word sequences. The proposed solution involves reducing the uncertainty of probability estimates based on observed frequencies and reallocating the "released" probability "mass" among n-grams not observed in the text. This reduction is achieved by replacing maximum likelihood estimates for n-grams with low counts with renormalized Turing's estimates, and the reallocation is facilitated through recursive use of lower-level conditional distributions. Turing's method is chosen for its simplicity and its characterization as the optimal empirical Bayes' estimator of a multinomial probability. The empirical Bayes' methodology, introduced by Robbins and further explored by Nadas, provides a solid foundation for this estimation technique. Let N denote the sample text size, and n, represent the number of words (n-grams) that occurred exactly r times in the text, such that N = Σ rn,. Turing's probability estimate PT for a word (n-gram) that occurred r times in the sample is given by r, but with adjustments to account for the sparse data challenge and improve estimation accuracy.

In this context, the proposed method effectively addresses the shortcomings of maximum likelihood estimation in handling sparse data, offering a more reliable and efficient approach to probability estimation for language models. This method not only enhances the performance of speech recognizers but also has broader applications in areas where probability estimation from limited data is necessary.

Introduction Statistical language models estimate the probability for a given sequence of words. Given a sentence s with n words such as s = (w1,w2 . . .wn), the language model assigns P(s). Statistical language models assess good word sequence estimations based on the sequence probability estimation. Building robust, fast and accurate language models is one of the main factors in the success of building systems such as machine translation systems and automatic speech recognition systems. Statistical language models can be estimated based on various approaches. Classical language models are estimated based on n-gram word sequences such as P(s) = P(w1,w2 . . .wn) = ∏n i=1 P(wi|wi−1) and can be approximated based on the Markov concept for shorter contexts (for example, bigram if n = 2 or trigram if n = 3 and so on). Recent researchers have applied neural networks different architectures to build and estimate language models. The classical feed-forward neural network-based language models have been continuously reporting good results among the traditional n-gram language modeling techniques [1]. It Abstract

Neural vector representations are ubiquitous throughout all subfields of NLP. While word vectors have been studied in much detail, thus far only little light has been shed on the properties of sentence embeddings. In this paper, we assess to what extent prominent sentence embedding methods exhibit select semantic properties. We propose a framework that generate triplets of sentences to explore how changes in the syntactic structure or semantics of a given sentence affect the similarities obtained between their sentence embeddings.

Being able to model long-term dependencies in sequential data, such as text, has been among the long-standing challenges of recurrent neural networks (RNNs). This issue is strictly related to the absence of explicit planning in current RNN architectures, more explicitly, the network is trained to predict only the next token given previous ones. In this paper, we introduce a simple way of biasing the RNNs towards planning behavior. Particularly, we introduce an additional neural network which is trained to generate the sequence in reverse order, and we require closeness between the states of the forward RNN and backward RNN that predict the same token. At each step, the states of the forward RNN are required to match the future information contained in the backward states. We hypothesize that the approach eases modeling of long-term dependencies thus helping in generating more globally consistent samples. The model trained with conditional generation achieved 4% relative improvement (CER of 7.3 compared to a baseline of 7.6).

In contrast to expectation-based, predictive views of discourse comprehension, a model is developed in which the initial processing is strictly bottom-up. Word meanings are activated, propositions are formed, and inferences and elaborations are produced without regard to the discourse context. However, a network of interrelated items is created in this manner, which can be integrated into a coherent structure through a spreading activation process. Data concerning the time course of word identification in a discourse context are examined. A simulation of arithmetic word-problem understanding provides a plausible account for some well-known phenomena in this area.

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach is demonstrated for text (where the data are discrete) and online handwriting (where the data are real-valued). It is then extended to handwriting synthesis by allowing the network to condition its predictions on a text sequence. The resulting system is able to generate highly realistic cursive handwriting in a wide variety of styles.

Statistical learning problems in many fields involve sequential data. This paper formalizes the principal learning tasks and describes the methods that have been developed within the machine learning research community for addressing these problems. These methods include sliding window methods, recurrent sliding windows, hidden Markov models, conditional random fields, and graph transformer networks. The paper also discusses some open research issues.

We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.

The field of computational narratology has produced many efforts aimed at generating narrative by computational means. In recent times, a number of such efforts have considered the task of modelling how a reader might consume the story. Whereas all these approaches are clearly different aspects of the task of generating narrative, so far the efforts to model them have occurred as separate and disjoint initiatives. There is an enormous potential for improvement if a way was found to combine results from these initiatives with one another. The present position paper provides a breakdown of the activity of creating stories into five stages that are conceptually different from a computational point of view and represent important aspects of the overall process as observed either in humans or in existing systems. These stages include a feedback loop that builds interpretations of an ongoing composition and provides feedback based on these to inform the composition process. This model provides a theoretical framework that can be employed first to understand how the various aspects of the task of generating narrative relate to one another, second to identify which of these aspects are being addressed by the different existing research efforts, and finally to point the way towards possible integrations of these aspects within progressively more complex systems.

The machine learning approach is represented with two methods: the maximum entropy method and support vector machine. Text representation for the maximum entropy method includes the information about the proportion of positive and negative words and collocations, the quantity of interrogation and exclamation marks, emoticons, obscene language. For the support vector machine binary vectors with cosine normalization are built on texts.

We present a novel approach to recognizing Textual Entailment. Structural features are constructed from abstract tree descriptions, which are automatically extracted from syntactic dependency trees. These features are then applied in a subsequence-kernel-based classifier to learn whether an entailment relation holds between two texts. Our method makes use of machine learning techniques using a limited data set, no external knowledge bases (e.g. WordNet), and no handcrafted inference rules. We achieve an accuracy of 74.5% for text pairs in the Information Extraction and Question Answering task, 63.6% for the RTE-2 test data, and 66.9% for the RET-3 test data.


Question: Probabilistic text organization: trials with sentence sequencing.

This paper explores the concept of learning to order instances, a task distinct from traditional classification, where the goal is to rank instances based on preference judgments. We tackle the challenge of learning how to rank items effectively, given feedback in the form of preference judgments that indicate which instance should precede another. Our approach is twofold: first, we utilize conventional learning techniques to develop a preference function, denoted as PREF, which guides the ranking of instances. This function helps determine whether should be ranked ahead of . Subsequently, new instances are ordered in a way that maximizes alignment with the learned preference function. Despite the NP-completeness of finding the optimal ordering that best matches a preference function, even under stringent conditions, we propose a straightforward greedy algorithm that can effectively approximate a good solution. Additionally, we introduce an online learning method, inspired by the "Hedge" algorithm, to identify a well-performing linear combination of ranking "experts." These experts represent domain-specific query expansion strategies for a web search engine. By combining our ordering algorithm with the online learning technique, we are able to select a combination of search experts. Our experimental findings substantiate the effectiveness of our approach, showcasing its potential in various applications.

Question: The Role of Semantic Similarity in Text Summarization

Documents: Semantic Similarity in a Variable-Size Space
The problem of measuring the semantic similarity between text documents is central to many natural language processing applications, including text summarization, information retrieval, and machine translation. In this paper, we propose a novel approach to calculate semantic similarity that takes into account the variable size of the document space. Our method is based on the assumption that the meaning of a word is context-dependent and can change depending on the surrounding words. To address this, we employ a dynamic representation of documents, where the vector space model is expanded or contracted based on the context. This allows us to capture the nuanced meaning of words and their relationships within the text, leading to a more accurate measure of semantic similarity. We evaluate our method on a benchmark dataset and compare it to several state-of-the-art approaches. Our results show that our method outperforms existing techniques in terms of precision and recall, making it a promising tool for applications that rely on accurate semantic similarity measurements.

Similar Documents: This work presents an innovative method for measuring semantic similarity between text documents, a critical component in various natural language processing tasks such as text summarization, information retrieval, and machine translation. Our approach is designed to accommodate the variable size of the document space, acknowledging that

Representation learning is a fundamental problem in natural language processing. This paper studies how to learn a structured representation for text classification. Unlike most existing representation models that either use no structure or rely on pre-specified structures, we propose a reinforcement learning (RL) method to learn sentence representation by discovering optimized structures automatically. We demonstrate two attempts to build structured representation: Information Distilled LSTM (ID-LSTM) and Hierarchically Structured LSTM (HS-LSTM). ID-LSTM selects only important, taskrelevant words, and HS-LSTM discovers phrase structures in a sentence. Structure discovery in the two representation models is formulated as a sequential decision problem: current decision of structure discovery affects following decisions, which can be addressed by policy gradient RL. Results show that our method can learn task-friendly representations by identifying important words or task-relevant structures without explicit structure annotations, and thus yields competitive performance.

While natural languages are compositional, how state-of-the-art neural models achieve compositionality is still unclear. We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior. That is, while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words. In contrast, the higher layers compose meaningful phrases and clauses, whose lengths increase as the networks get deeper until fully composing the sentence.

This document provides additional details about the experiments described in (Heilman and Smith, 2010). Note that while this document provides information about the datasets and experimental methods, it does not provide further results. If you have any further questions, please feel free to contact the first author. The preprocessed datasets (i.e., tagged and parsed) will be made available for research purposes upon request.

Crowdsourcing lets us collect multiple annotations for an item from several annotators. Typically, these are annotations for non-sequential classification tasks. While there has been some work on crowdsourcing named entity annotations, researchers have largely assumed that syntactic tasks such as part-of-speech (POS) tagging cannot be crowdsourced. This paper shows that workers can actually annotate sequential data almost as well as experts. Further, we show that the models learned from crowdsourced annotations fare as well as the models learned from expert annotations in downstream tasks.

We present a method for authorship discrimination that is based on the frequency of bigrams of syntactic labels that arise from partial parsing of the text. We show that this method, alone or combined with other classification features, achieves a high accuracy on discrimination of the work of Anne and Charlotte Brontë, which is very difficult to do by traditional methods. Moreover, high accuracies are achieved even on fragments of text little more than 200 words long. .................................................................................................................................................................................

Multi-document summarization aims to create a condensed summary while retaining the main characteristics of the original set of documents. Under such background, sentence ranking has hitherto been the issue of most concern. Since documents often cover a number of topic themes with each theme represented by a cluster of highly related sentences, sentence clustering has been explored in the literature in order to provide more informative summaries. For each topic theme, the rank of terms conditional on this topic theme should be very distinct, and quite different from the rank of terms in other topic themes. Existing cluster-based summarization approaches apply clustering and ranking in isolation, which leads to incomplete, or sometimes rather biased, analytical results. A newly emerged framework uses sentence clustering results to improve or refine the sentence ranking results. Under this framework, we propose a novel approach that directly generates clusters integrated with ranking in this paper. The basic idea of the approach is that ranking distribution of sentences in each cluster should be quite different from each other, which may serve as features of clusters and new clustering measures of sentences can be calculated accordingly. Meanwhile, better clustering results can achieve better ranking results. As a result, ranking and clustering by mutually and simultaneously updating each other so that the performance of both can be improved. The effectiveness of the proposed approach is demonstrated by both the cluster quality analysis and the summarization evaluation conducted on the DUC 2004-2007 datasets.

We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.

Conventional sentiment analysis usually neglects semantic information between (sub-)clauses, as it merely implements so-called bag-of-words approaches, where the sentiment of individual words is aggregated independently of the document structure. Instead, we advance sentiment analysis by the use of rhetoric structure theory (RST), which provides a hierarchical representation of texts at document level. For this purpose, texts are split into elementary discourse units (EDU). These EDUs span a hierarchical structure in the form of a binary tree, where the branches are labeled according to their semantic discourse. Accordingly, this paper proposes a novel combination of weighting and grid search to aggregate sentiment scores from the RST tree, as well as feature engineering for machine learning. We apply our algorithms to the especially hard task of predicting stock returns subsequent to financial disclosures. As a result, machine learning improves the balanced accuracy by 8.6 percent compared to the baseline.

This paper presents strategies and lessons learned from the use of natural language annotations to facilitate question answering in the START information access system.

In this paper we develop a story generator that leverages knowledge inherent in corpora without requiring extensive manual involvement. A key feature in our approach is the reliance on a story planner which we acquire automatically by recording events, their participants, and their precedence relationships in a training corpus. Contrary to previous work our system does not follow a generate-and-rank architecture. Instead, we employ evolutionary search techniques to explore the space of possible stories which we argue are well suited to the story generation task. Experiments on generating simple children’s stories show that our system outperforms previous data-driven approaches.