Question: VR-STEP: A hands-free navigation method using inertial sensing for walking-in-place in mobile virtual reality environments.

This research introduces an innovative technique, termed Redirected Walking in Place (RWP), enabling users to traverse virtual environments within a CAVETM setup without the necessity to physically turn and confront the absent rear wall. The method is designed to facilitate an immersive experience by minimizing visual disruptions caused by the virtual space's limitations. A pilot study involving 28 participants was conducted to evaluate the effectiveness of RWP. Half of the participants navigated the virtual world using a conventional handheld controller, while the other half adopted the RWP technique. Although the current implementation of RWP did not significantly decrease the frequency of participants looking towards the missing wall, the findings revealed a crucial correlation. The study demonstrated that a participant's sense of presence within the virtual environment is notably and negatively related to the extent of exposure to the blank rear wall. There is suggestive evidence that RWP can diminish the likelihood of encountering this visual interruption for a subset of users. The enhanced sense of immersion achieved by avoiding the view of the blank wall, alongside the outcomes of the pilot study, indicate that RWP holds substantial potential and warrants further refinement and exploration.

A robot perceives space in order to navigate, map, search and investigate its surroundings. One application area where humans interact with robots is search and rescue. The robot may have unique capabilities such as seeing outside the visible spectrum and being able to navigate in tight and hazardous spaces. In such an operation, the human may also be an effective search agent. Thus, it would be beneficial if the robot’s spatial perception was conveyed to the human via a secondary peripheral modality. We have worked on developing a visual to tactile substitution.device for people who are visually impaired or blind. We propose to use this similar technique to the remote robot in order that it convey its spatial perception to the human operator or co-worker.

Clumsy intermediary devices constrain our interaction with computers and their applications. Glove-based input devices let us apply our manual dexterity to the task. We provide a basis for understanding the field by describing key hand-tracking technologies and applications using glove-based input. The bulk of development in glove-based input has taken place very recently, and not all of it is easily accessible in the literature. We present a cross-section of the field to date. Hand-tracking devices may use the following technologies: position tracking, optical tracking, marker systems, silhouette analysis, magnetic tracking or acoustic tracking. Actual glove technologies on the market include: Sayre glove, MIT LED glove, Digital Data Entry Glove, DataGlove, Dexterous HandMaster, Power Glove, CyberGlove and Space Glove. Various applications of glove technologies include projects into the pursuit of natural interfaces, systems for understanding signed languages, teleoperation and robotic control, computer-based puppetry, and musical performance.<<ETX>>

Recent introduction of low-cost 3D sensing and affordable immersive virtual reality have lowered the barriers for creating and maintaining 3D virtual worlds. In this paper, we propose a way to combine these technologies with discrete-event simulation to improve the use of simulation in decision making in manufacturing. This work will describe how feedback is possible from real world systems directly into a simulation model to guide smart behaviors. Technologies included in the research include feedback from RGBD images of shop floor motion and human interaction within full immersive virtual reality that includes the latest headset technologies.

This article describes research that aims to encourage physical activity through a novel pervasive gaming paradigm. Data from a wearable accelerometer are logged wirelessly to a cell phone and control the animation of an avatar that represents the player in a virtual race game with other players over the cellular network. Winners are declared every day and players with an excess of activity points can spend some to get hints in mental games of the suite, like Sudoku. The racing game runs in the background throughout the day and every little move counts. As the gaming platform is embedded in the daily routine of players, it may act as a strong behavioral modifier and increase everyday physical activity other than volitional sporting exercise. Such physical activity (e.g., taking the stairs), is termed NEAT and was shown to play a major role in obesity prevention and intervention. A pilot experiment demonstrates that players are engaged in NEAT-o-Games and become more physically active while having a good dosage of fun.

Augmented Reality (AR) based navigation has emerged as an impressive, yet seamless way of guiding users in unknown environments. Its quality of experience depends on many factors, including the accuracy of camera pose estimation, response delay, and energy consumption. In this paper, we present SeeNav - a seamless and energy-efficient AR navigation system for indoor environments. SeeNav combines image-based localization and inertial tracking to provide an accurate and robust camera pose estimation. As vision processing is much more compute intensive than the processing of inertial sensor data, SeeNav offloads the former one from resource-constrained mobile devices to a cloud to improve tracking performance and reduce power consumption. More than that, SeeNav implements a context-aware task scheduling algorithm that further minimizes energy consumption while maintaining the accuracy of camera pose estimation. Our experimental results, including a user study, show that SeeNav provides seamless navigation experience and reduces the overall energy consumption by 21.56% with context-aware task scheduling.

INTRODUCTION
Brain Computer Interface (BCI) systems based on Movement Imagination (MI) are widely used in recent decades. Separate feature extraction methods are employed in the MI data sets and classified in Virtual Reality (VR) environments for real-time applications.

METHODS
This study applied wide variety of features on the recorded data using Linear Discriminant Analysis (LDA) classifier to select the best feature sets in the offline mode. The data set was recorded in 3-class tasks of the left hand, the right hand, and the foot motor imagery.

RESULTS
The experimental results showed that Auto-Regressive (AR), Mean Absolute Value (MAV), and Band Power (BP) features have higher accuracy values,75% more than those for the other features.

DISCUSSION
These features were selected for the designed real-time navigation. The corresponding results revealed the subject-specific nature of the MI-based BCI system; however, the Power Spectral Density (PSD) based α-BP feature had the highest averaged accuracy.

Interactions within virtual environments (VE) often require manipulation of 3D virtual objects. For this purpose, several research have been carried out focusing on mouse, touch and gestural input. While existing mid-air gestures in immersive VE (IVE) can offer natural manipulations, mimicking interactions with physical objects, they cannot achieve the same levels of precision attained with mouse-based applications for computer-aided design (CAD). Following prevailing trends in previous research, such as degrees-offreedom (DOF) separation, virtual widgets and scaled user motion, we intend to explore techniques for IVEs that combine the naturalness of mid-air gestures and the precision found in traditional CAD systems. In this document we survey and discuss the state-of-the-art in 3D object manipulation. With challenges identified, we present a research proposal and corresponding work plan.

Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center. They often create discomfort with marker suits, and their recording volume is severely restricted and often constrained to indoor scenes with controlled backgrounds. Alternative suit-based systems use several inertial measurement units or an exoskeleton to capture motion with an inside-in setup, i.e. without external sensors. This makes capture independent of a confined volume, but requires substantial, often constraining, and hard to set up body instrumentation. Therefore, we propose a new method for real-time, marker-less, and egocentric motion capture: estimating the full-body skeleton pose from a lightweight stereo pair of fisheye cameras attached to a helmet or virtual reality headset - an optical inside-in method, so to speak. This allows full-body motion capture in general indoor and outdoor scenes, including crowded scenes with many people nearby, which enables reconstruction in larger-scale activities. Our approach combines the strength of a new generative pose estimation framework for fisheye views with a ConvNet-based body-part detector trained on a large new dataset. It is particularly useful in virtual reality to freely roam and interact, while seeing the fully motion-captured virtual body.

We present an approach for real-time, robust and accurate hand pose estimation from moving egocentric RGB-D cameras in cluttered real environments. Existing methods typically fail for hand-object interactions in cluttered scenes imaged from egocentric viewpoints—common for virtual or augmented reality applications. Our approach uses two subsequently applied Convolutional Neural Networks (CNNs) to localize the hand and regress 3D joint locations. Hand localization is achieved by using a CNN to estimate the 2D position of the hand center in the input, even in the presence of clutter and occlusions. The localized hand position, together with the corresponding input depth value, is used to generate a normalized cropped image that is fed into a second CNN to regress relative 3D hand joint locations in real time. For added accuracy, robustness and temporal stability, we refine the pose estimates using a kinematic pose tracking energy. To train the CNNs, we introduce a new photorealistic dataset that uses a merged reality approach to capture and synthesize large amounts of annotated data of natural hand interaction in cluttered scenes. Through quantitative and qualitative evaluation, we show that our method is robust to self-occlusion and occlusions by objects, particularly in moving egocentric perspectives.

An important and troublesome problem with current virtual environment (VE) technology is the tendency for some users to exhibit symptoms that parallel symptoms of classical motion sickness both during and after the VE experience. This type of sickness, cybersickness, is distinct from motion sickness in that the user is often stationary but has a compelling sense of self motion through moving visual imagery. Unfortunately, there are many factors that can cause cybersickness and there is no foolproof method for eliminating the problem. In this paper, I discuss a number of the primary factors that contribute to the cause of cybersickness, describe three conflicting cybersickness theories that have been postulated, and discuss some possible methods for reducing cybersickness in VEs.


Question: VR-STEP: A hands-free navigation method using inertial sensing for walking-in-place in mobile virtual reality environments.

In a recent study, researchers explored the comparative effects of different locomotion methods in virtual environments, focusing on walking-in-place (virtual walking), real walking, and flying. The results, consistent with previous findings by Slater et al. [1995], revealed that naive subjects felt a more profound sense of presence when using walking-in-place as their mode of movement compared to push-button flying. This study further expanded on the original research by including real walking as an additional condition, which emerged as the most preferred and natural method of locomotion, outperforming both virtual walking and flying in terms of ease and simplicity.

The study also highlighted the significant disparity in subjective presence between flyers and both types of walkers, with real walkers experiencing a slightly higher sense of presence than virtual walkers, although this difference was not always statistically significant. Subsequent investigations demonstrated that the experience of virtual walking could be notably enhanced by incorporating head accelerometer data to detect footfalls, which in turn significantly correlated with a subject's sense of embodiment within their virtual avatar. This finding suggests that there is considerable potential to increase presence by tracking all body parts and personalizing avatar appearances to better align with the user's physical self.

An interesting outcome of the study was the heightened virtual experience achieved through real walking in an enhanced version of Slater's visual-cliff virtual environment, which proved to be the most compelling and immersive experience for both researchers and participants. The study underscored the necessity for wireless technology to replace all physical connections to the user, thereby enhancing the realism and immersion of virtual environments.

In conclusion, the study's findings reinforce the importance of considering locomotion methods in the design of virtual reality experiences, with real walking and enhanced virtual walking offering the most promising avenues for achieving high levels of presence and immersion. Future research should focus on further refining these methods and integrating wireless technology to create seamless, hands-free navigation in mobile VR environments.

Recent technological advances in connected-speech recognition and position sensing in space have encouraged the notion that voice and gesture inputs at the graphics interface can converge to provide a concerted, natural user modality.
 The work described herein involves the user commanding simple shapes about a large-screen graphics display surface. Because voice can be augmented with simultaneous pointing, the free usage of pronouns becomes possible, with a corresponding gain in naturalness and economy of expression. Conversely, gesture aided by voice gains precision in its power to reference.

This paper describes an experiment to assess the influence of immersion on performance in immersive virtual environments. The task involved Tri-Dimensional Chess, and required subjects to reproduce on a real chess board the state of the board learned from a sequence of moves witnessed in a virtual environment. Twenty four subjects were allocated to a factorial design consisting of two levels of immersion (exocentric screen based, and egocentric HMD based), and two kinds of environment (plain and realistic. The results suggest that egocentric subjects performed better than exocentric, and those in the more realistic environment performed better than those in the less realistic environment. Previous knowledge of chess, and amount of virtual practice were also significant, and may be considered as control variables to equalise these factors amongst the subjects. Other things being equal, males remembered the moves better than females, although female performance improved with higher spatial ability test score. The paper also attempts to clarify the relationship between immersion, presence and performance, and locates the experiment within such a theoretical framework.

We describea markerlesscamera tracking systemfor augmentedreality thatoperatesin environmentswhich contain oneor more planes. This is a commonspecialcase, which we showsignificantlysimplifiestracking. The result is a practical, reliable, vision-basedtracker. Furthermore, thetrackedplaneimposesa natural referenceframe, sothat thealignmentof thereal andvirtual coordinatesystemsis rather simplerthan would be the casewith a general structure-and-motionsystem.Multiple planescan be tracked, and additional data such as 2D point tracks are easilyincorporated.

In this study, we present our implementation of using a motion controller to control the motion of a drone via simple human gestures. We have used the Leap as the motion controller and the Parrot AR DRONE 2.0 for this implementation. The Parrot AR DRONE is an off the shelf quad rotor having an on board Wi-Fi system. The AR DRONE is connected to the ground station via Wi-Fi and the Leap is connected to the ground station via USB port. The LEAP Motion Controller recognizes the hand gestures and relays it on to the ground station. The ground station runs ROS (Robot Operating System) in Linux which is used as the platform for this implementation. Python is the programming language used for interaction with the AR DRONE in order to convey the simple hand gestures. In our implementation, we have written python codes to interpret the hand gestures captured by the LEAP, and transmit them in order to control the motion of the AR DRONE via these gestures.

We present the novel kinematics, workspace characterization, functional prototype and impedance control of a six degrees of freedom haptic interface designed to train surgeons for laparoscopic procedures, through virtual reality simulations. The parallel kinematics of the device is constructed by connecting a 3RRP planar parallel mechanism to a linearly actuated modified delta mechanism with a connecting link. The configuration level forward and inverse kinematics of the device assume analytic solutions, while its workspace can be shaped to enable large end-effector translations and rotations, making it well-suited for laparoscopy operations. Furthermore, the haptic interface features a low apparent inertia with high structural stiffness, thanks to its parallel kinematics with grounded actuators. A model-based open-loop impedance controller with feed-forward gravity compensation has been implemented for the device and various virtual tissue/organ stiffness levels have been rendered.

Future computing environments will free the user from the constraints of the desktop. Applications for a mobile environment should take advantage of contextual information, such as position, to o er greater services to the user. In this paper, we present the Cyberguide project, in which we are building prototypes of a mobile context-aware tour guide. Knowledge of the user's current location, as well as a history of past locations, are used to provide more of the kind of services that we come to expect from a real tour guide. We describe the architecture and features of a variety of Cyberguide prototypes developed for indoor and outdoor use on a number of di erent hand-held platforms. We also discuss the general research issues that have emerged in our context-aware applications development in a mobile environment.

In this paper we have proposed and designed FindingHuMo (Finding Human Motion), a real-time user tracking system for Smart Environments. FindingHuMo can perform device-free tracking of multiple (unknown and variable number of) users in the Hallway Environments, just from non-invasive and anonymous (not user specific) binary motion sensor data stream. The significance of our designed system are as follows: (a) fast tracking of individual targets from binary motion data stream from a static wireless sensor network in the infrastructure. This needs to resolve unreliable node sequences, system noise and path ambiguity, (b) Scaling for multi-user tracking where user motion trajectories may crossover with each other in all possible ways. This needs to resolve path ambiguity to isolate overlapping trajectories, FindingHumo applies the following techniques on the collected motion data stream: (i) a proposed motion data driven adaptive order Hidden Markov Model with Viterbi decoding (called Adaptive-HMM), and then (ii) an innovative path disambiguation algorithm (called CPDA). Using this methodology the system accurately detects and isolates motion trajectories of individual users. The system performance is illustrated with results from real-time system deployment experience in a Smart Environment.

Various interaction techniques have been developed in the field of virtual and augmented reality. Whereas techniques for object selection, manipulation, travel, and wayfinding have already been covered in existing taxonomies in some detail, application control techniques have not yet been sufficiently considered. However, they are needed by almost every mixed reality application, e.g. for choosing from alternative objects or options. For this purpose a great variety of distinct three-dimensional (3D) menu selection techniques is available. This paper surveys existing 3D menus from the corpus of literature and classifies them according to various criteria. The taxonomy introduced here assists developers of interactive 3D applications to better evaluate their options when choosing, optimizing, and implementing a 3D menu technique. Since the taxonomy spans the design space for 3D menu solutions, it also aids researchers in identifying opportunities to improve or create novel virtual menu techniques. r 2006 Elsevier Ltd. All rights reserved.

The Tele-immersive Dance Environment (TED) is a geographically distributed, real-time 3-D virtual room where multiple participants interact independent of physical distance. TED, a highly interactive collaborative environment, offers digital options with multiple viewpoints, enhancing the creative movement composition involved with dance choreography. A symbiotic relationship for creativity and design exists between dance artists and computer scientists as the tele-immersive environment is analyzed as a creativity and learning tool. We introduce the advancements of the interactive digital options, new interface developments, user study results, and the possibility of a computational model for human creativity through Laban Movement Analysis.

Virtual reality systems are widely believed to be the next major computing platform. There are, however, some barriers to adoption that must be addressed, such as that of motion sickness — which can lead to undesirable symptoms including postural instability, headaches, and nausea. Motion sickness in virtual reality occurs as a result of moving visual stimuli that cause users to perceive self-motion while they remain stationary in the real world. There are several contributing factors to both this perception of motion and the subsequent onset of sickness, including field of view, motion velocity, and stimulus depth. We verify first that differences in vection due to relative stimulus depth remain correlated with sickness. Then, we build a dataset of stereoscopic 3D videos and their corresponding sickness ratings in order to quantify their nauseogenicity, which we make available for future use. Using this dataset, we train a machine learning algorithm on hand-crafted features (quantifying speed, direction, and depth as functions of time) from each video, learning the contributions of these various features to the sickness ratings. Our predictor generally outperforms a naïve estimate, but is ultimately limited by the size of the dataset. However, our result is promising and opens the door to future work with more extensive datasets. This and further advances in this space have the potential to alleviate developer and end user concerns about motion sickness in the increasingly commonplace virtual world.


Question: VR-STEP: A hands-free navigation method using inertial sensing for walking-in-place in mobile virtual reality environments.

In the realm of virtual reality (VR) environments, achieving temporal-spatial realism is crucial for enhancing user immersion and interaction. The Polhemus Isotrak, a popular orientation and position tracking device, is frequently utilized for dynamically updating a user's viewpoint and line of sight, particularly when employing head-mounted displays. However, this system's inherent noise and latency in measurement data can lead to temporal-spatial distortions, manifesting as image jitter and delays between head movements and visual responses. To mitigate these issues, we conducted an in-depth analysis to identify the primary sources of distortion. Our findings revealed that the user-perceived lag is predominantly caused by delays in orientation data, whereas image jitter is chiefly attributed to noise in position data. Based on these insights, we engineered a predictive Kalman filter to address the delay in orientation data, and an anisotropic low-pass filter to minimize position data noise. The efficacy and constraints of both filtering methods were thoroughly evaluated, with outcomes demonstrating significant improvements in temporal-spatial realism within VR environments. This research underscores the importance of refined tracking mechanisms for enhancing the overall user experience in virtual reality settings.

Controlling a virtual vehicle is a sensory-motor activity with a specific rendering methodology that depends on the hardware technology and the software in use. We propose a method that computes haptic feedback for the steering wheel. It is best suited for low-cost, fixed-base driving simulators but can be ported to any driving simulator platform. The goal of our method is twofold. 1) It provides an efficient yet simple algorithm to model the steering mechanism using a quadri-polar representation. 2) This model is used to compute the haptic feedback on top of which a tunable haptic augmentation is adjusted to overcome the lack of presence and the unavoidable simulation loop latencies. This algorithm helps the driver to laterally control the virtual vehicle. We also discuss the experimental results that demonstrate the usefulness of our haptic feedback method.

JackIn is a new human-human communication framework for connecting two or more people. With first-person view video streaming from a person (called Body) wearing a transparent head-mounted display and a head-mounted camera, the other person (called Ghost) participates in the shared first-person view. With JackIn, people's activities can be shared and assistance or guidance can be given through other peoples expertise. This can be applied to daily activities such as cooking lessons, shopping navigation, education in craft-work or electrical work, and sharing experiences of sporting and live events. For a better viewing experience with frist-person view, we developed the out-of-body view in which first-person images are integrated to construct a scene around a Body, and a Ghost can virtually control the viewpoint to look around the space surrounding the Body. We also developed a tele-pointing gesture interface. We conducted an experiment to evaluate how effective this framework is and found that Ghosts can understand the spatial situation of the Body.

Several common systems satisfy some but not all of the VR definition above. Flight simulators provide vehicle tracking, not head tracking, and do not generally operate in binocular stereo. Omnimax theaters give a large angle of view [8], occasionally in stereo, but are not interactive. Head-tracked monitors [4][6] provide all but a large angle of view. Head-mounted displays (HMD) [7][13] and BOOMs [9] use motion of the actual display screens to achieve VR by our definition. Correct projection of the imagery on large screens can also create a VR experience, this being the subject of this paper. This paper describes the CAVE (CAVE Automatic Virtual Environment) virtual reality/scientific visualization system in detail and demonstrates that projection technology applied to virtual-reality goals achieves a system that matches the quality of workstation screens in terms of resolution, color, and flicker-free stereo. In addition, this format helps reduce the effect of common tracking and system latency errors. The off-axis perspective projection techniques we use are shown to be simple and straightforward. Our techniques for doing multi-screen stereo vision are enumerated, and design barriers, past and current, are described. Advantages and disadvantages of the projection paradigm are discussed, with an analysis of the effect of tracking noise and delay on the user. Successive refinement, a necessary tool for scientific visualization, is developed in the virtual reality context. The use of the CAVE as a one-to-many presentation device at SIGGRAPH '92 and Supercomputing '92 for computational science data is also mentioned. Previous work in the VR area dates back to Sutherland [12], who in 1965 wrote about the “Ultimate Display.” Later in the decade at the University of Utah, Jim Clark developed a system that allowed wireframe graphics VR to be seen through a headmounted, BOOM-type display for his dissertation. The common VR devices today are the HMD and the BOOM. Lipscomb [4] showed a monitor-based system in the IBM booth at SIGGRAPH '91 and Deering [6] demonstrated the Virtual Portal, a closets ized three-wal l project ion-based system, in the Sun Microsystems' booth at SIGGRAPH '92. The CAVE, our projection-based VR display [3], also premiered at SIGGRAPH '92. The Virtual Portal and CAVE have similar intent, but different implementation schemes.

People with visual impairment face enormous difficulties in terms of their mobility as they do not have enough information about their location and orientation with respect to traffic and obstacles on their route. Visually impaired people can navigate unknown areas by relying on the assistance of canes, other people, or specially trained guide dogs. The traditional ways of using guide dogs and long-cane only help to avoid obstacles, not to know where they are. The research presented in this paper introduces a mobile assistant navigation prototype to locate and direct blind people indoors. Since most of the existing navigation systems developed so far for blind people employ a complex conjunction of positioning systems, video cameras, locationbased and image processing algorithms, we designed an affordable low-cost prototype navigation system for orienting and tracking the position of blind people in complex environments. The prototype system is based on the inertial navigation system and experiments have been performed on NXT Mindstorms platform.

This paper introduces a novel prosthetic hand control architecture using inertial information for grasp prediction in order to reduce the cognitive burden of amputees. A pair of inertial measurement sensors (IMUs) are fitted on the wrist and bicep to record arm trajectory when reaching to grasp an object. Each object class can be associated with different methods for grasping and manipulation. An observation experiment was conducted to find the most common grasping methods for generic object classes: Very Small (VS), Small (S), and Medium (M). A Cup (CP) class was also examined to find differences in grasping habits for pick and place, and drinking applications. The resulting grasps were used to test the discriminatory ability of inertial motion features in the upper limb for VS, S and CP object classes. Subject experiments demonstrated an average classification success rate of 90.8%, 69.2% and 88.1% for VS, S and CP classes respectively when using a k-nearest neighbors algorithm with a Euclidean distance metric. The results suggest that inertial motion features have great potential to predict the grasp pattern during reach, and to the authors' knowledge, is the first IMU-based control strategy to utilize natural motion that is aimed at hand prosthesis control.

In this paper we present a low-cost and open-source brain-computer interface (BCI) virtual-reality (VR) Game Control Development Environment prototype, which we demonstrate using real-time signal processing of Electroencephalography (EEG) event-related desynchronization and synchronization changes (ERD/ERS) within the Precentral Gyrus (Motor Cortex), allowing a user to control a 3D object within a Virtual Reality Environment. This BCI-VR system prototype was functionally tested on multiple participants and demonstrated live before an audience during the 2017 ‘Hack the Brain’ at the Dublin Science Gallery. The availability of such an open-source, effective, BCI-VR Game Control Development Environment, at a level acceptable for industry experimentation, has the potential to open up this field to a much wider range of researchers and games developers and to assist the investigation of gaming experiences which both incorporate the specific control features available through BCI as a core element of the game play and the potential for its use in neurorehabilitation.

This essay addresses ethical aspects of the design and use of virtual reality (VR) systems, focusing on the behavioral options made available in such systems and the manner in which reality is represented or simulated in them. An assessment is made of the morality of ‘immoral’ behavior in virtual reality, and of the virtual modeling of such behavior. Thereafter, the ethical aspects of misrepresentation and biased representation in VR applications are discussed.

A new method for researching haptic interaction styles is presented, based on a layered interaction model and a classification of existing devices. The method is illustrated by designing a new foot interaction device. The aim of which is to enhance non-verbal communication over a computer network. A layered protocols interaction model allows to consider all aspects of the haptic communication process: the intention to perform an action, limitations of the human body, and specifications of the communication device and the network. We demonstrate how this model can be used to derive design-guidelines by analyzing and classifying existing communication devices. By designing and evaluating a foot interaction device, we not only demonstrate that feet are suited for personal, concealed communication over a network, but also show the added value of the design-guidelines. Results of user tests provide clues for designing stimuli for foot interaction and indicate applications of foot communication devices.

Clumsy intermediary devices constrain our interaction with computers and their applications. Glove-based input devices let us apply our manual dexterity to the task. We provide a basis for understanding the field by describing key hand-tracking technologies and applications using glove-based input. The bulk of development in glove-based input has taken place very recently, and not all of it is easily accessible in the literature. We present a cross-section of the field to date. Hand-tracking devices may use the following technologies: position tracking, optical tracking, marker systems, silhouette analysis, magnetic tracking or acoustic tracking. Actual glove technologies on the market include: Sayre glove, MIT LED glove, Digital Data Entry Glove, DataGlove, Dexterous HandMaster, Power Glove, CyberGlove and Space Glove. Various applications of glove technologies include projects into the pursuit of natural interfaces, systems for understanding signed languages, teleoperation and robotic control, computer-based puppetry, and musical performance.<<ETX>>

In this paper we explore the combination of a latest generation mobile device and a micro quadrotor platform to perform indoor autonomous navigation for the purpose of autonomous photography. We use the Yellowstone tablet from Google's Tango project [1], equipped with onboard, fully integrated sensing platform and with significant computational capability. To the best of our knowledge we are the first to exploit the Google's Tango tablet as source of pose estimate to control the quadrotor's motion. Using the tablet's onboard camera the system is able to detect people and generate a desired pose that the quadrotor will have to reach in order to take a well framed picture of the detected subject. The experimental results and live video demonstrate the capabilities of the autonomous flying robot photographer using the system described throughout this manuscript.


Question: VR-STEP: A hands-free navigation method using inertial sensing for walking-in-place in mobile virtual reality environments.

This paper delves into an innovative user interface concept that integrates a miniature, hand-held replica of the virtual environment with a head-tracked immersive display system, which we term the Worlds in Miniature (WIM) approach. Beyond the conventional first-person perspective provided by virtual reality setups, the WIM introduces a supplementary, dynamic viewpoint onto the virtual world. This dual perspective enables direct manipulation of objects either through the primary immersive view or via the three-dimensional window offered by the miniature environment. Beyond object manipulation, the paper investigates the versatility of the WIM as a cohesive metaphor for a variety of interaction techniques, including object selection, navigation, path planning, and visualization, all of which are application-independent. A key advantage of the WIM metaphor is its ability to offer diverse vantage points and operational scales, eliminating the need for explicit modes or commands. Observations from informal user testing suggest that individuals rapidly acclimate to the WIM metaphor, and physical props are found to be beneficial in handling the WIM and other virtual objects.

**Question**: The Role of Renewable Energy in Achieving Carbon Neutrality in Urban Areas

**Documents**: Renewable Energy Integration in Urban Planning: A Case Study of Copenhagen
The integration of renewable energy sources into urban planning is a crucial step towards achieving carbon neutrality in cities. This paper presents a case study of Copenhagen, a city that aims to become the world's first carbon-neutral capital by 2025. The analysis focuses on the role of wind energy, solar power, and biomass in the city's energy mix. The city's ambitious goals include the installation of extensive solar panels on rooftops and the expansion of wind farms in offshore locations. Additionally, the paper discusses the implementation of district heating systems that utilize waste heat from power plants and biomass as a sustainable heating source. The study also highlights the importance of energy-efficient building designs and the promotion of electric vehicles in reducing the city's carbon footprint. The successful integration of renewable energy in Copenhagen's urban planning demonstrates the feasibility of achieving carbon neutrality in densely populated areas.

**Similar Documents**: This study explores the pivotal role of renewable energy in urban planning, particularly in the context of Copenhagen's pursuit to become the first carbon-neutral capital by 2025. By examining the city's strategies for incorporating wind, solar, and biomass energy, the paper offers insights into the practical steps towards carbon neutrality. Copenhagen's vision includes a significant expansion of solar energy infrastructure, with a focus on rooftop solar panel installations, and the development of offshore wind energy projects. The paper also del

The massive diffusion of smartphones, the growing interest in wearable devices and the Internet of Things, and the exponential rise of location based services (LBSs) have made the problem of localization and navigation inside buildings one of the most important technological challenges of recent years. Indoor positioning systems have a huge market in the retail sector and contextual advertising; in addition, they can be fundamental to increasing the quality of life for citizens if deployed inside public buildings such as hospitals, airports, and museums. Sometimes, in emergency situations, they can make the difference between life and death. Various approaches have been proposed in the literature. Recently, thanks to the high performance of smartphones’ cameras, marker-less and marker-based computer vision approaches have been investigated. In a previous paper, we proposed a technique for indoor localization and navigation using both Bluetooth low energy (BLE) and a 2D visual marker system deployed into the floor. In this paper, we presented a qualitative performance evaluation of three 2D visual markers, Vuforia, ArUco marker, and AprilTag, which are suitable for real-time applications. Our analysis focused on specific case study of visual markers placed onto the tiles, to improve the efficiency of our indoor localization and navigation approach by choosing the best visual marker system.

This paper presents a robust and flexible framework for augmented reality which does not require instrumenting either the environment or the workpiece. A model-based visual tracking system is combined with with rate gyroscopes to produce a system which can track the rapid camera rotations generated by a head-mounted camera, even if images are substantially degraded by motion blur. This tracking yields estimates of head position at video field rate (50Hz) which are used to align computer-generated graphics on an optical see-through display. Nonlinear optimisation is used for the calibration of display parameters which include a model of optical distortion. Rendered visuals are pre-distorted to correct the optical distortion of the display.

While intelligent environments are often cited as a reason for doing work on visual person-tracking, really making an intelligent environment exposes many realworld problems in visual tracking that must be solved to make the technology practical. In the context of our EasyLiving project in intelligent environments, we created a practical person-tracking system that solves most of the real-world problems. It uses two sets of color stereo cameras for tracking multiple people during live demonstrations in a living room. The stereo images are used for locating people, and the color images are used for maintaining their identities. The system runs quickly enough to make the room feel responsive, and it tracks multiple people standing, walking, sitting, occluding, and entering and leaving the

This thesis presents WiTrack, a system that tracks the 3D motion of a user from the radio signals reflected off her body. It works even if the person is occluded from the WiTrack device or in a different room. WiTrack does not require the user to carry any wireless device, yet its accuracy exceeds current RF localization systems, which require the user to hold a transceiver. Empirical measurements with a WiTrack prototype show that, on average, it localizes the center of a human body to within a median of 10 to 13 cm in the x and y dimensions, and 21 cm in the z dimension. It also provides coarse tracking of body parts, identifying the direction of a pointing hand with a median of 11.20. WiTrack bridges a gap between RF-based localization systems which locate a user through walls and occlusions, and human-computer interaction systems like Kinect, which can track a user without instrumenting her body, but require the user to stay within the direct line of sight of the device. Thesis Supervisor: Dina Katabi Title: Professor of Computer Science and Engineering

This article deals with educational opportunities for mixed reality games and related scenarios for learning. It discusses several issues and educational challenges to be tackled when linking augmented reality and augmented virtuality. Second, the paper describes the architecture of the ARLearn system which offers highly flexible support for different educational settings. Three prototypical use cases implemented based on the underlying ARLearn framework are discussed, which are a field trip system, an augmented Google StreetView client called StreetLearn, and a real time crisis intervention game. ARLearn combines real time notification and mixed reality games across Mobile Augmented Reality and Virtual Reality and the authors aim to use the underlying (open source) framework for further case studies and mixed reality applications for learning support.

The advent of Oculus Rift indicates the start of a booming era of virtual reality. In order to increase the immersive feeling of interaction with the virtual world, haptic devices allow us to touch and manipulate virtual objects in an intuitive way. In this paper, we introduce a portable and low-cost haptic glove that provides both force and tactile feedback using a direct-control pneumatic concept. To produce force feedback, two inlet ports of a double acting pneumatic cylinder are opened and closed via solenoid DC valves through Pulse-width modulation (PWM) technique. For tactile feedback, an air bladder is actuated using a diaphragm pump via PWM operated solenoid valve. Experiments on a single finger prototype validated that the glove can provide force and tactile feedback with sufficient moving range of the finger joints. The maximum continuous force is 9 Newton and the response time is less than 400ms. The glove is light weighted and easy to be mounted on the index finger. The proposed glove could be potentially used for virtual reality grasping scenarios and for teleoperation of a robotic hand for handling hazardous objects.

In this paper we present a low-cost and open-source brain-computer interface (BCI) virtual-reality (VR) Game Control Development Environment prototype, which we demonstrate using real-time signal processing of Electroencephalography (EEG) event-related desynchronization and synchronization changes (ERD/ERS) within the Precentral Gyrus (Motor Cortex), allowing a user to control a 3D object within a Virtual Reality Environment. This BCI-VR system prototype was functionally tested on multiple participants and demonstrated live before an audience during the 2017 ‘Hack the Brain’ at the Dublin Science Gallery. The availability of such an open-source, effective, BCI-VR Game Control Development Environment, at a level acceptable for industry experimentation, has the potential to open up this field to a much wider range of researchers and games developers and to assist the investigation of gaming experiences which both incorporate the specific control features available through BCI as a core element of the game play and the potential for its use in neurorehabilitation.

In this paper we have proposed and designed FindingHuMo (Finding Human Motion), a real-time user tracking system for Smart Environments. FindingHuMo can perform device-free tracking of multiple (unknown and variable number of) users in the Hallway Environments, just from non-invasive and anonymous (not user specific) binary motion sensor data stream. The significance of our designed system are as follows: (a) fast tracking of individual targets from binary motion data stream from a static wireless sensor network in the infrastructure. This needs to resolve unreliable node sequences, system noise and path ambiguity, (b) Scaling for multi-user tracking where user motion trajectories may crossover with each other in all possible ways. This needs to resolve path ambiguity to isolate overlapping trajectories, FindingHumo applies the following techniques on the collected motion data stream: (i) a proposed motion data driven adaptive order Hidden Markov Model with Viterbi decoding (called Adaptive-HMM), and then (ii) an innovative path disambiguation algorithm (called CPDA). Using this methodology the system accurately detects and isolates motion trajectories of individual users. The system performance is illustrated with results from real-time system deployment experience in a Smart Environment.

When compared to image recognition, object detection is a much more challenging task because it requires the accurate real-time localization of an object in the target image. In interaction scenarios, this pipeline can be simplified by incorporating the users’ point of regard. Wearable eye trackers can estimate the gaze direction, but lack own processing capabilities. We enable mobile gaze-aware applications by developing an open-source platform which supports mobile eye tracking based on the Pupil headset and a smartphone running Android OS. Through our platform, we offer researchers and developers a rapid prototyping environment for gaze-enabled applications. We describe the concept, our current progress, and research implications. ACM Classification

Progress robot development very varied almost all fields seize the attention of the world's scientists and researchers. For some cases, robots are generally used to help humans in relieving job. The robot can replace human mobilization for move so that the efficiency of wasted time will be less. Many things need to be note what kind of robot control reliable to replace human work in mobilization. Virtual world as in android smartphone greatly benefit humans to creating control tools easy to use anywhere and could represent a desire humans. By making control robot using virtual paths on the Android smartphone which have an accuracy of more than 70% to a scale of 1:50 with the actual track in an environment that is not wide enough, the advantage can be used to control the robot move easily and efficiently compared to the usual manual control robot that will take more time to control The robot moves its place as desired.


Question: Acquiring the ability to resolve geometric problems from natural language examples provided in textbooks.

This paper introduces an in-depth examination of the knowledge and inference requirements necessary for success in elementary science exams, specifically focusing on the role of explanations in understanding these requirements. We propose a method for analyzing QA systems in this domain by characterizing the types of knowledge and inferences they must handle. Our approach involves identifying relevant sentences from a knowledge base that support the correct answer to a question, and then constructing explanations that incorporate any missing information. This process allows us to categorize questions based on their knowledge and inference requirements. We apply this categorization to compare the performance of a retrieval-based and an inference-based solver on a set of 212 science exam questions. Our analysis confirms the effectiveness of the inference-based solver in addressing questions that require complex inferences. Moreover, it offers valuable insights into the strengths of each solver and the role of various knowledge sources. To facilitate further research in science exam QA, we are making available the annotated questions and explanations as a resource, which can aid in refining knowledge base construction and enhancing automated inference processes.

THIS PAPER is concerned with the nature of the tutorial process; the means whereby an adult or "expert" helps somebody who is less adult or less expert. Though its aim is general, it is expressed in terms of a particular task: a tutor seeks to teach children aged 3, 4 and 5 yr to build a particular three-dimensional structure that requires a degree of skill that is initially beyond them. It is the usual type of tutoring situation in which one member "knows the answer" and the other does not, rather like a "practical" in which only the instructor "knows how". The changing interaction of tutor and children provide our data. A great deal of early problem solving by the developing child is of this order. Although from the earliest months of life he is a "natural" problem solver in his own right (e.g. Bruner, 1973) it is often the ease that his efforts are assisted and fostered by others who are more skilful than he is (Kaye, 1970). Whether he is learning the procedures that constitute the skills of attending, communicating, manipulating objects, locomoting, or, indeed, a more effective problem solving procedure itself, there are usually others in attendance who help him on his way. Tutorial interactions are, in short, a crucial feature of infancy and childhood. Our species, moreover, appears to be the only one in which any "intentional" tutoring goes on (Bruner, 1972; Hinde, 1971). For although it is true that many of the higher primate species learn by observation of their elders (Hamburg, 1968; van Lawick-Goodall, 1968), there is no evidence that those elders do anything to instruct their charges in the performance of the skill in question. What distinguishes man as a species is not only his capacity for learning, but for teaching as well. It is the main aim of this paper to examine some of the major implications of this interactive, instructional relationship between the developing child and his elders for the study of skill acquisition and problem solving. The acquisition of skill in the human child can be fruitfully conceived as a hierarchical program in which component skills are combined into "higher skills" by appropriate orchestration to meet new, more complex task requirements (Bruner, 1973). The process is analogous to problem solving in which mastery of "lower order" or constituent problems in a sine qua non for success with a larger jjroblcm, each level influencing the other—as with reading where the deciphering of words makes possible the deciphering of sentences, and sentences then aid in the deciphering of particular words (F. Smith, 1971). Given persistent intention in the young learner, given a "lexicon" of constituent skills, the crucial task is often one of com-

Current research in natural language generation is situated in a computational linguistics tradition that was founded several decades ago. We critically analyse some of the architectural assumptions underlying existing systems and point out some problems in the domains of text planning and lexicalization. Guided by the identification of major generation challenges viewed from the angles of knowledge-based systems and cognitive psychology, we sketch some new directions for future research.

We propose a method to create process flow graphs automatically from textbooks for cooking programs. This is realized by understanding context by narrowing down the domain to cooking, and making use of domain specific constraints and knowledge. Since it is relatively easy to extract significant keywords from cooking procedures, we create a domain specific dictionary by statistical methods, and propose a structural analysis method using the dictionary. In order to evaluate the ability of the proposed method, we applied the method to actual procedures as an experiment, which showed effective results. The same experiment was also performed on a different program, which showed lower accuracy but also showed realistic results.

Automatically solving algebra word problems has raised considerable interest recently. Existing state-of-the-art approaches mainly rely on learning from human annotated equations. In this paper, we demonstrate that it is possible to efficiently mine algebra problems and their numerical solutions with little to no manual effort. To leverage the mined dataset, we propose a novel structured-output learning algorithm that aims to learn from both explicit (e.g., equations) and implicit (e.g., solutions) supervision signals jointly. Enabled by this new algorithm, our model gains 4.6% absolute improvement in accuracy on the ALG514 benchmark compared to the one without using implicit supervision. The final model also outperforms the current state-of-the-art approach by 3%.

We present an account of semantics that is not construed as a mapping of language to the world but rather as a mapping between individual meaning spaces. The meanings of linguistic entities are established via a “meeting of minds.” The concepts in the minds of communicating individuals are modeled as convex regions in conceptual spaces. We outline a mathematical framework, based on fixpoints in continuous mappings between conceptual spaces, that can be used to model such a semantics. If concepts are convex, it will in general be possible for interactors to agree on joint meaning even if they start out from different representational spaces. Language is discrete, while mental representations tend to be continuous—posing a seeming paradox. We show that the convexity assumption allows us to address this problem. Using examples, we further show that our approach helps explain the semantic processes involved in the composition of expressions.

As part of the information fusion task we wish to automatically fuse information derived from the text extraction process with data from a structured knowledge base. This process will involve resolving, aggregating, integrating and abstracting information via the methodologies of Knowledge Representation and Reasoning into a single comprehensive description of an individual or event. This report surveys the key principles underlying research in the field of Knowledge Representation and Reasoning. It represents an initial step in deciding upon a Knowledge Representation and Reasoning system for our information fusion task. APPROVED FOR PUBLIC RELEASE

been developed during the last 20 years have proven to be quite successful, particularly in the domains of mathematics, science, and technology. They produce significant learning gains beyond classroom environments. They are capable of engaging most students’ attention and interest for hours. We have been working on a new generation of intelligent tutoring systems that hold mixedinitiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AUTOTUTOR is a conversational agent, with a talking head, that helps college students learn about computer literacy. ANDES, ATLAS, AND WHY2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations.

The amount of readily available on-line text has reached hundreds of billions of words and continues to grow. Yet for most core natural language tasks, algorithms continue to be optimized, tested and compared after training on corpora consisting of only one million words or less. In this paper, we evaluate the performance of different learning methods on a prototypical natural language disambiguation task, confusion set disambiguation, when trained on orders of magnitude more labeled data than has previously been used. We are fortunate that for this particular application, correctly labeled training data is free. Since this will often not be the case, we examine methods for effectively exploiting very large corpora when labeled data comes at a cost.

To assist learners during problem-solving activities, an intelligent tutoring system (ITS) has to be equipped with domain knowledge that can support appropriate tutoring services. Providing domain knowledge is usually done by adopting one of the following paradigms: building a cognitive model, specifying constraints, integrating an expert system, and using data mining algorithms to learn domain knowledge. However, for some ill-defined domains, each single paradigm may present some advantages and limitations in terms of the required resources for deploying it, and tutoring support that can be offered. To address this issue, we propose using a multiparadigm approach. In this paper, we explain how we have applied this idea in CanadarmTutor, an ITS for learning to operate the Canadarm2 robotic arm. To support tutoring services in this ill-defined domain, we have developed a multiparadigm model combining: 1) a cognitive model to cover well-defined parts of the task and spatial reasoning, 2) a data mining approach for automatically building a task model from user solutions for ill-defined parts of the task, and 3) a 3D path-planner to cover other parts of the task for which no user data are available. The multiparadigm version of CanadarmTutor allows providing a richer set of tutoring services than what could be offered with previous single paradigm versions of CanadarmTutor.

Language-endowed intelligent agents benefit from leveraging lexical knowledge falling at different points along a spectrum of compositionality. This means that robust computational lexicons should include not only the compositional expectations of argument-taking words, but also non-compositional collocations (idioms), semi-compositional collocations that might be difficult for an agent to interpret (e.g., standard metaphors), and even collocations that could be compositionally analyzed but are so frequently encountered that recording their meaning increases the efficiency of interpretation. In this paper we argue that yet another type of stringto-meaning mapping can also be useful to intelligent agents: remembered semantic analyses of actual text inputs. These can be viewed as super-specific multi-word expressions whose recorded interpretations mimic a person’s memories of knowledge previously learned from language input. These differ from typical annotated corpora in two ways. First, they provide a full, context-sensitive semantic interpretation rather than select features. Second, they are are formulated in the ontologically-grounded metalanguage used in a particular agent environment, meaning that the interpretations contribute to the dynamically evolving cognitive capabilites of agents configured in that environment.


Question: Acquiring the ability to tackle geometry problems using natural language examples from textbooks.

This review delves into the potential of imitation learning as a pivotal pathway to advance the creation of autonomous humanoid robots, intersecting recent advancements in artificial intelligence and neural computation. The hypothesis is that by examining imitation learning, we can uncover novel insights into perceptual motor control mechanisms, crucial for the development of humanoid robots. The review centers on three key areas: the efficient acquisition of motor skills, the interplay between perception and action, and the modular control of movements through primitives. It explores how research into the representation and functional links between action and perception has deepened our understanding of imitating the motor actions of others. The discovery that specific regions in the primate brain are activated during both the perception and execution of movements provides a potential neural foundation for imitation. The review also discusses computational approaches to imitation learning, beginning with a traditional AI and robotics perspective, then shifting to neural network models and statistical learning research. It highlights the similarities and differences between biological and computational approaches to imitation, offering an overview of ongoing projects that utilize imitation learning in the design of humanoid robots.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A novel algorithm for economic dispatch with valve-point effect using a direct search method
In this paper, a novel algorithm is presented for solving the economic dispatch problem with valve-point effect using a direct search method. The valve-point effect, a nonlinear function that models the non-smoothness in the cost function of power plants, is a common issue in practical power systems. The proposed algorithm, which is based on the direct search method, is capable of efficiently handling this nonlinear characteristic. The direct search method is a derivative-free optimization technique that does not require the gradient of the objective function. It is particularly suitable for problems with complex and non-differentiable cost functions, such as the economic dispatch problem with valve-point effect. The algorithm is designed to find the optimal power generation schedule that minimizes the total cost while satisfying the operational constraints of the power system. The effectiveness of the proposed algorithm is demonstrated through its application to various test systems, including those with multiple generators and valve-point effects. The results obtained are compared with those from other established optimization algorithms, highlighting the superior performance of the proposed method in terms of solution quality and computational efficiency. The algorithm is shown to be robust and reliable, making it a promising approach for solving the economic dispatch problem in practical power systems.

**Similar Documents**: This paper introduces a groundbreaking algorithm for addressing the economic dispatch problem, specifically dealing with the valve-point effect, through the

This paper introduces a fully automatic method for generating video game tutorials. The AtDELFI system (Automatically DEsigning Legible, Full Instructions for games) was created to investigate procedural generation of instructions that teach players how to play video games. We present a representation of game rules and mechanics using a graph system as well as a tutorial generation method that uses said graph representation. We demonstrate the concept by testing it on games within the General Video Game Artificial Intelligence (GVG-AI) framework; the paper discusses tutorials generated for eight different games. Our findings suggest that a graph representation scheme works well for simple arcade style games such as Space Invaders and Pacman, but it appears that tutorials for more complex games might require higher-level understanding of the game than just single mechanics.

We introduce a new dataset1 for the evaluation of models addressing reasoning tasks. For a position in a chess game, we provide question and answer pairs, an image of the board, and the sequence of moves up to that position. We hope this synthetic task will improve our understanding in memory based Deep Learning with posed challenges.

Many comprehension studies of grammatical development have focused on the ultimate interpretation that children assign to sentences and phrases, yielding somewhat static snapshots of children's emerging grammatical knowledge. Studies of the dynamic processes underlying children's language comprehension have to date been rare, owing in part to the lack of online sentence processing techniques suitable for use with children. In this chapter, we describe recent work from our research group, which examines the moment-by-moment interpretation decisions of children (age 4 to 6 years) while they listen to spoken sentences. These real-time measures were obtained by recording the children's eye movements as they visually interrogated and manipulated objects in response to spoken instructions. The first of these studies established some striking developmental differences in processing ability, with the youngest children showing an inability to use relevant properties of the referential scene to resolve temporary grammatical ambiguities (Trueswell, Sekerina, Hill, & Logrip, 1999). This finding could be interpreted as support for an early encapsulated syntactic processor that has difficulty using non-syntactic information to revise parsing commitments. However, we will review evidence from a series of follow-up experiments which suggest that this pattern arises from a developing interactive parsing system. Under this account, adult and child sentence comprehension is a " perceptual guessing game " in which multiple statistical cues are used to recover detailed linguistic structure. These cues, which include lexical-distribution evidence, verb semantic biases, and referential scene information, come " online " (become automated) at different points in the course of development. The developmental timing of these effects is related to their differential reliability and ease of detection in the input.

In this thesis, we present a novel approach for modeling textual entailment using lexicalsemantic information on the level of predicate-argument structure. To this end, we adopt information provided by the Berkeley FrameNet repository and embed it into an implemented end-to-end system. The two main goals of this thesis are the following: (i) to provide an analysis of the potential contribution of frame semantic information to the recognition textual entailment and (ii) to present a robust system architecture that can serve as basis for future experiments, research, and improvement. Our work was carried out in the context of the textual entailment initiative, which since 2005 has set the stage for the broad investigation of inference in natural-language processing tasks, including empirical evaluation of its coverage and reliability. In short, textual entailment describes inferential relations between (entailing) texts and (entailed) hypotheses as interpreted by typical language users. This pre-theoretic notion captures a natural range of inferences as compared to logical entailment, which has traditionally been used within theoretical approaches to natural language semantics. Various methods for modeling textual entailment have been proposed in the literature, ranging from shallow techniques like lexical overlap to shallow syntactic parsing and the exploitation of WordNet relations. Recently, there has been a move towards more structured meaning representations. In particular, the level of predicate-argument structure has gained much attention, which seems to be a natural and straightforward choice. Predicate-argument structure allows annotating sentences or texts with nuclear meaning representations (“who did what to whom”), which are of obvious relevance for this task. For example, it can account for paraphrases like “Ghosts scare John” vs. “John is scared by ghosts”. In this thesis, we present an approach to textual entailment that is centered around the analysis of predicate-argument structure. It combines LFG grammatical analysis, predicate-argument structure in the FrameNet paradigm, and taxonomic information from WordNet into tripartite graph structures. By way of a declarative graph matching algorithm, the “structural and semantic” similarity of hypotheses and texts is computed and the result is represented as feature vectors. A supervised machine learning architecture trained on entailment corpora is used to check textual entailment for new text/hypothesis pairs. The approach is implemented in the SALSA RTE system, which successfully participated in the second and third RTE challenge. While system performance is on a par with that of comparable systems, the intuitively expected strong positive effect of using FrameNet information has not yet been confirmed. In order to evaluate different system components and to assess the potential contribution of FrameNet information for checking textual entailment, we conducted a number of experiments. For example, with the help of a gold-standard corpus, we

Current research in natural language generation is situated in a computational linguistics tradition that was founded several decades ago. We critically analyse some of the architectural assumptions underlying existing systems and point out some problems in the domains of text planning and lexicalization. Guided by the identification of major generation challenges viewed from the angles of knowledge-based systems and cognitive psychology, we sketch some new directions for future research.

Over the last years, the robotics community has made substantial progress in detection and 3D pose estimation of known and unknown objects. However, the question of how to identify objects based on language descriptions has not been investigated in detail. While the computer vision community recently started to investigate the use of attributes for object recognition, these approaches do not consider the task settings typically observed in robotics, where a combination of appearance attributes and object names might be used in referral language to identify specific objects in a scene. In this paper, we introduce an approach for identifying objects based on natural language containing appearance and name attributes. To learn rich RGB-D features needed for attribute classification, we extend recently introduced sparse coding techniques so as to automatically learn attribute-dependent features. We introduce a large data set of attribute descriptions of objects in the RGB-D object dataset. Experiments on this data set demonstrate the strong performance of our approach to language based object identification. We also show that our attribute-dependent features provide significantly better generalization to previously unseen attribute values, thereby enabling more rapid learning of new attribute values.

We study automatic question generation for sentences from text passages in reading comprehension. We introduce an attention-based sequence learning model for the task and investigate the effect of encoding sentencevs. paragraph-level information. In contrast to all previous work, our model does not rely on hand-crafted rules or a sophisticated NLP pipeline; it is instead trainable end-to-end via sequenceto-sequence learning. Automatic evaluation results show that our system significantly outperforms the state-of-the-art rule-based system. In human evaluations, questions generated by our system are also rated as being more natural (i.e., grammaticality, fluency) and as more difficult to answer (in terms of syntactic and lexical divergence from the original text and reasoning needed to answer).

Generating an article automatically with computer program is a challenging task in artificial intelligence and natural language processing. In this paper, we target at essay generation, which takes as input a topic word in mind and generates an organized article under the theme of the topic. We follow the idea of text planning (Reiter and Dale, 1997) and develop an essay generation framework. The framework consists of three components, including topic understanding, sentence extraction and sentence reordering. For each component, we studied several statistical algorithms and empirically compared between them in terms of qualitative or quantitative analysis. Although we run experiments on Chinese corpus, the method is language independent and can be easily adapted to other language. We lay out the remaining challenges and suggest avenues for future research.

The Tactical Language and Culture Training System (TLCTS) helps people quickly acquire communicative skills in foreign languages and cultures. More than 20,00 0 learners worldwide have used TLCTS courses. TLCTS utili zes artificial intelligence technologies in multiple wa ys: during the authoring process, and at run time to process l arner speech, interpret learner actions, control the resp on e of non-player characters, and evaluate and assess lear ner performance and proficiency. This paper describes the architecture of TLCTS and the artificial intelligence techn ologies that it employs, and presents results from multiple evaluation studies that demonstrate the benefits of learn ing foreign language and culture using this approach.

It is often debated whether a problem presented in a straightforward minimalist fashion is better, or worse, for learning than the same problem presented with a "real-life" or "concrete" context. The presentation, contextualization, or "problem description" has been well studied over several decades in disciplines such as mathematics education and psychology; however, little has been published in the field of computing education. In psychology it has been found that not only the presence of context, but the type of context can have dramatic results on problem success. In mathematics education it has been demonstrated that there are non-mathematical factors in problem presentation that can affect success in solving the problem and learning. The contextual background of a problem can also impact cognitive load, which should be considered when evaluating the effects of context. Further, it has been found that regarding cognitive load, computer science has unique characteristics compared to other disciplines, with the consequence that results from other disciplines may not apply to computer science, thus requiring investigation within computer science.
 This paper presents a multi-national, multi-institutional study of the effects of problem contextualization on novice programmer success in a typical CS1 exercise.


Question: Acquiring the ability to resolve geometry problems from textbook demonstrations presented in natural language.

This comprehensive review delves into the field of robot Learning from Demonstration (LfD), a methodology that enables robots to learn policies through example mappings of state to action. The paper offers a structured framework for understanding the critical design choices in LfD, encompassing the nature of the demonstrator, the problem space, the process of policy derivation, and the evaluation of performance. It meticulously examines the diverse methods employed to collect examples, spanning from teleoperation to direct imitation, and explores the various strategies for deriving policies, including matching functions, dynamic models, and planning algorithms. The review concludes with an insightful discussion on the current limitations of LfD and highlights promising avenues for future research in related areas.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.

**Similar Documents**: This paper introduces a novel evolutionary learning algorithm, Hybrid Genetic Algorithm-Particle Swarm Optimization (HGAPSO), designed to automate the creation of recurrent neural/fuzzy networks. HGAPSO integrates the strengths of genetic algorithms (GA

We present a novel methodology involving mappings between different modes of semantic representations. We propose distributional semantic models as a mechanism for representing the kind of world knowledge inherent in the system of abstract symbols characteristic of a sophisticated community of language users. Then, motivated by insight from ecological psychology, we describe a model approximating affordances, by which we mean a language learner’s direct perception of opportunities for action in an environment. We present a preliminary experiment involving mapping between these two representational modalities, and propose that our methodology can become the basis for a cognitively inspired model of grounded language learning.

Language is used to communicate ideas. Ideas are mental tools for coping with a complex and uncertain world. Thus human conceptual structures should be key to language meaning, and probability—the mathematics of uncertainty— should be indispensable for describing both language and thought. Indeed, probabilistic models are enormously useful in modeling human cognition (Tenenbaum et al., 2011) and aspects of natural language (Bod et al., 2003; Chater et al., 2006). With a few early exceptions (e.g. Adams, 1975; Cohen, 1999b), probabilistic tools have only recently been used in natural language semantics and pragmatics. In this chapter we synthesize several of these modeling advances, exploring a formal model of interpretation grounded, via lexical semantics and pragmatic inference, in conceptual structure. Flexible human cognition is derived in large part from our ability to imagine possibilities (or possible worlds). A rich set of concepts, intuitive theories, and other mental representations support imagining and reasoning about possible worlds—together we will call these the conceptual lexicon. We posit that this collection of concepts also forms the set of primitive elements available for lexical semantics: word meanings can be built from the pieces of conceptual structure. Larger semantic structures are then built from word meanings by composition, ultimately resulting in a sentence meaning which is a phrase in the “language of thought” provided by the conceptual lexicon. This expression is truth-functional in that it takes on a Boolean value for each imagined world, and it can thus be used as the basis for belief updating. However, the connection between cognition, semantics, and belief is not direct: because language must flexibly adapt to the context of communication, the connection between lexical representation and interpreted meaning is mediated by pragmatic inference.

We present a system that produces sentential descriptions of video: who did what to whom, and where and how they did it. Action class is rendered as a verb, participant objects as noun phrases, properties of those objects as adjectival modifiers in those noun phrases, spatial relations between those participants as prepositional phrases, and characteristics of the event as prepositional-phrase adjuncts and adverbial modifiers. Extracting the information needed to render these linguistic entities requires an approach to event recognition that recovers object tracks, the trackto-role assignments, and changing body posture.

For a machine to be able to ‘see’, it must know something about the object it is ‘looking’ at, A common method in machine vision is to provide the machine with general rather than specific knowledge about the object. An alternative technique, and the one used in thti paper, b a model-based approach in which particulars aboat the object are given and this drives the analysis. The computer program described here, the WALKER model, maps images into a description in which a person is represented by the series of hierarchical levels, i.e. a person has an arm which has a lower-arm which has a hand. l7te performance of the program ti illustrated by superimposing the machinegenerated picture over the original photographic images.

Three laboratory experiments involving students' behavior and brain imaging and one randomized field experiment in a college physics class explored the importance of physical experience in science learning. We reasoned that students' understanding of science concepts such as torque and angular momentum is aided by activation of sensorimotor brain systems that add kinetic detail and meaning to students' thinking. We tested whether physical experience with angular momentum increases involvement of sensorimotor brain systems during students' subsequent reasoning and whether this involvement aids their understanding. The physical experience, a brief exposure to forces associated with angular momentum, significantly improved quiz scores. Moreover, improved performance was explained by activation of sensorimotor brain regions when students later reasoned about angular momentum. This finding specifies a mechanism underlying the value of physical experience in science education and leads the way for classroom practices in which experience with the physical world is an integral part of learning.

321 Abstract— A recent report demonstrated that 8-month-olds can segment a continuous stream of speech syllables, containing no acoustic or prosodic cues to word boundaries, into wordlike units after only 2 min of listening experience (Saffran, Aslin, & Newport, 1996). Thus, a powerful learning mechanism capable of extracting statistical information from fluent speech is available early in development. The present study extends these results by documenting the particular type of statistical computation—transitional (conditional) probability—used by infants to solve this word-segmentation task. An artificial language corpus, consisting of a continuous stream of trisyllabic nonsense words, was presented to 8-month-olds for 3 min. A postfamiliarization test compared the infants' responses to words versus part-words (tri-syllabic sequences spanning word boundaries). The corpus was constructed so that test words and part-words were matched in frequency, but differed in their transitional probabilities. Infants showed reliable discrimination of words from part-words, thereby demonstrating rapid segmentation of continuous speech into words on the basis of transitional probabilities of syllable pairs. Many aspects of the patterns of human languages are signaled in the speech stream by what is called distributional evidence, that is, regularities in the relative positions and order of elements over a corpus of utterances (Bloomfield, 1933; Maratsos & Chalkley, 1980). This type of evidence, along with linguistic theories about the characteristics of human languages, is what comparative linguists use to discover the structure of exotic languages (Harris, 1951). Similarly, this type of evidence , along with tendencies to perform certain kinds of analyses on language input (Chomsky, 1957), could be used by human language learners to acquire their native languages. However, using such evidence would require rather complex distributional and statistical computations , and surprisingly little is known about the abilities of human infants and young children to perform these computations. By using the term computation, we do not mean, of course, that infants are consciously performing a mathematical calculation, but rather that they might be sensitive to and able to store quantitative aspects of distribu-tional information about a language corpus. Recently, we have begun studying this problem by investigating the abilities of human learners to use statistical information to dis-Words are known to vary dramatically from one language to another, so finding the words of a language is clearly a problem that must involve learning from the linguistic environment. Moreover, the beginnings and ends of the sequences of sounds that form words in a …

ion Information that has been selected because it is important and/or relevant to the schema is further reduced during the encoding process by abstraction. This process codes the meaning but not the format of a message (e.g., Bobrow, 1970; Bransford, Barclay, & Franks, 1972). Thus, details such as the lexical form of an individual word (e.g., Schank, 1972, 1976) and the syntactic form of a sentence (e.g., Sachs, 1967) will not be preserved in memory. Because memory for syntax appears to be particularly sparce as well as brief (e.g., J. R. Anderson, 1974; Begg & Wickelgren, 1974; Jarvella, 1971; Sachs, 1967, 1974), the abstraction process is thought to operate during encoding. Additional support for the notion that what is stored is an abstracted representation of the original stimulus comes from studies that demonstrate that after a passage is read, it takes subjects the same amount of time to verify information originally presented in a complex linguistic format as it does to verify that same information presented in a simpler format (e.g., King & Greeno, 1974; Kintsch &Monk, 1972). There are a considerable number of theories that assume that memory consists of sets of propositions and their relations (e.g., J. R. Anderson, 1976; Bransford, et al., 1972; Brewer, 1975; Frederiksen, 1975a; Kintsch, 1974; Norman & Rumelhart, 1975; Schank, 1972, 1976). One formalized presentation of this idea is Schank's conceptual dependency theory (1972). The theory asserts that all propositions can be expressed by a small set of primitive concepts. All lexical expressions that share an identical meaning will be represented in one way (and so stored economically) regardless of their presentation format. As a result people should often incorrectly recall or misrecognize synonyms of originally presented words, and they do (e.g., Anderson & Bower, 1973; R. C. Anderson, 1974; Anisfeld & Knapp, 1968; Brewer, 1975; Graesser, 1978b; Sachs, 1974). Abstraction and memory theories. Since considerable detail is lost via the abstraction process, this process can easily account for the incompleteness that is characteristic of people's recall of complex events. In light of the abstraction process, the problem for schema theories becomes one of accounting for accurate recall. Schema theories do this by borrowing a finding from psycholinguistic research, to wit, that speakers of a language share preferred ways of expressing information. If both the creator and perceiver of a message are operating with the same preferences or under the same biases, the perceiver's reproduction of the input may appear to be accurate. The accuracy, however, is the product of recalling the semantic content of the message and imposing the preferred structure onto it. Thus, biases operate in a manner that is similar to the "probable detail" reconstruction process. Biases have been documented for both syntactic information (J. R. Anderson, 1974; Bock, 1977; Bock & Brewer, 1974; Clark & Clark, 1968; James, Thompson, & Baldwin, 1973) and lexical information (Brewer, 1975; Brewer & Lichtenstein, 1974). Distortions may result from the abstraction process if biases are not shared by the person who creates the message and the one who receives it. More importantly, the ab-

While natural languages are compositional, how state-of-the-art neural models achieve compositionality is still unclear. We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior. That is, while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words. In contrast, the higher layers compose meaningful phrases and clauses, whose lengths increase as the networks get deeper until fully composing the sentence.

This paper describes the results of an implemented computational model that cashes out the belief that metaphor interpretation is grounded in embodied primitives. The speci c task addressed is the interpretation of simple causal narratives in the domains of Politics and Economics. The stories are taken from newspaper articles in these domains. When presented with a preparsed version of these narratives as input, the system described is able to generate commonsense inferences consistent with the input.

As part of the information fusion task we wish to automatically fuse information derived from the text extraction process with data from a structured knowledge base. This process will involve resolving, aggregating, integrating and abstracting information via the methodologies of Knowledge Representation and Reasoning into a single comprehensive description of an individual or event. This report surveys the key principles underlying research in the field of Knowledge Representation and Reasoning. It represents an initial step in deciding upon a Knowledge Representation and Reasoning system for our information fusion task. APPROVED FOR PUBLIC RELEASE