Question: An agent-centric indoor navigation system utilizing digital signage.

This document outlines a novel methodology for integrating human spatial cognition into the design of spaces to enhance wayfinding, with airports serving as a case study. Wayfinding, a fundamental human activity, involves navigating through spaces. To create environments that support intuitive wayfinding, it's essential to incorporate an understanding of human spatial cognition into design. The proposed methodology, based on image schemata, combines cognitive science and engineering principles in a three-step process. First, it involves conducting interviews to gather insights into individuals' spatial experiences while performing wayfinding tasks in the specific setting. Second, the data from these interviews is analyzed to identify recurring image schemata, which are then used to create a sequence of subtasks. Finally, the application space, in this case, an airport, is structured according to these image schemata to improve wayfinding. The study reveals that image schemata typically appear in clusters, or 'image-schematic blocks,' rather than in isolation. These blocks function as a cognitive framework for wayfinding tasks, facilitating navigation and enhancing the user experience in complex spaces like airports. This approach, by focusing on experiential patterns, aims to create more intuitive and user-friendly spatial designs.

Applications that provide location related services need to understand the environment in which humans live such that verbal references and human interaction are possible. We formulate this semantic labelling task as the problem of learning the semantic labels from the perceived 3D structure. In this contribution we propose a batch approach and a novel multi-view frame fusion technique to exploit multiple views for improving the semantic labelling results. The batch approach works offline and is the direct application of an existing single-view method to scene reconstructions with multiple views. The multi-view frame fusion works in an incremental fashion accumulating the single-view results, hence allowing the online multi-view semantic segmentation of single frames and the offline reconstruction of semantic maps. Our experiments show the superiority of the approaches based on our fusion scheme, which leads to a more accurate semantic labelling.

This paper is concerned with constructing a prototype smart home environment which has been built in the research building of Korea Institute of Industrial Technology (KITECH) to demonstrate the practicability of a robot-assisted future home environment. Key functionalities that a home service robot must provide are localization, navigation, object recognition and object handling. A considerable amount of research has been conducted to make the service robot perform these operations with its own sensors, actuators and a knowledge database. With all heavy sensors, actuators and a database, the robot could have performed the given tasks in a limited environment or showed the limited capabilities in a natural environment. We initiated a smart home environment project for light-weight service robots to provide reliable services by interacting with the environment through the wireless sensor networks. This environment consists of the following three main components: smart objects with an radio frequency identification (RFID) tag and smart appliances with sensor network functionality; the home server that connects smart devices as well as maintains information for reliable services; and the service robots that perform tasks in collaboration with the environment. In this paper, we introduce various types of smart devices which are developed for assisting the robot by providing sensing and actuation, and present our approach on the integration of these devices to construct the smart home environment. Finally, we discuss the future directions of our project.

Towards bridging the gap between machine and human intelligence, it is of utmost importance to introduce environments that are visually realistic and rich in content. In such environments, one can evaluate and improve a crucial property of practical intelligent systems, namely generalization. In this work, we build House3D, a rich, extensible and efficient environment that contains 45,622 human-designed 3D scenes of houses, ranging from single-room studios to multistoreyed houses, equipped with a diverse set of fully labeled 3D objects, textures and scene layouts, based on the SUNCG dataset (Song et al., 2017). With an emphasis on semantic-level generalization, we study the task of concept-driven navigation, RoomNav, using a subset of houses in House3D. In RoomNav, an agent navigates towards a target specified by a semantic concept. To succeed, the agent learns to comprehend the scene it lives in by developing perception, understand the concept by mapping it to the correct semantics, and navigate to the target by obeying the underlying physical rules. We train RL agents with both continuous and discrete action spaces and show their ability to generalize in new unseen environments. In particular, we observe that (1) training is substantially harder on large house sets but results in better generalization, (2) using semantic signals (e.g. segmentation mask) boosts the generalization performance, and (3) gated networks on semantic input signal lead to improved training performance and generalization. We hope House3D1, including the analysis of the RoomNav task, serves as a building block towards designing practical intelligent systems and we wish it to be broadly adopted by the community.

This paper surveys the developments of the last 20 years in the area of vision for mobile robot navigation. Two major components of the paper deal with indoor navigation and outdoor navigation. For each component, we have further subdivided our treatment of the subject on the basis of structured and unstructured environments. For indoor robots in structured environments, we have dealt separately with the cases of geometrical and topological models of space. For unstructured environments, we have discussed the cases of navigation using optical flows, using methods from the appearance-based paradigm, and by recognition of specific objects in the environment.

This paper presents SAIL, a Single Access Point Based Indoor Localization system. Although there have been advances in WiFi-based positioning techniques, we find that existing solutions either require a dense deployment of access points (APs), manual fingerprinting, energy hungry WiFi scanning, or sophisticated AP hardware. We design SAIL using a single commodity WiFi AP to avoid these restrictions. SAIL computes the distance between the client and an AP using the propagation delay of the signal traversing between the two, combines the distance with smartphone dead-reckoning techniques, and employs geometric methods to ultimately yield the client's location using a single AP. SAIL combines physical layer (PHY) information and human motion to compute the propagation delay of the direct path by itself, eliminating the adverse effect of multipath and yielding sub-meter distance estimation accuracy. Furthermore, SAIL systematically addresses some of the common challenges towards dead-reckoning using smartphone sensors and achieves 2-5x accuracy improvements over existing techniques. We have implemented SAIL on commodity wireless APs and smartphones. Evaluation in a large-scale enterprise environment with 10 mobile users demonstrates that SAIL can capture the user's location with a mean error of 2.3m using just a single AP.

1. ABSTRACT Animated agents based either on real video, cartoon-style drawings or even model-based 3D graphics offer great promise for computer-based presentations as they make presentations more lively and appealing and allow for the emulation of conversation styles known from human-human communication. In this paper, we describe a life-like interface agent which presents multimedia material to the user following the directives of a script. The overall behavior of the presentation agent is partly determined by such a script, and partly by the agent's self-behavior. In our approach, the agent's behavior is defined in a declarative specification language. Behavior specifications are used to automatically generate a control module for an agent display system. The first part of the paper describes the generation process which involves AI planning and a two-step compilation. Since the manual creation of presentation scripts is tedious and error-prone, we also address the automated generation of presentation scripts which may be forwarded to the interface agent. The second part of the paper presents an approach for multimedia presentation design which combines hierarchical planning with temporal reasoning.

A driver assistance system (DAS) should support the driver by monitoring road and vehicle events and presenting relevant and timely information to the driver. It is impossible to know what a driver is thinking, but we can monitor the driver’s gaze direction and compare it with the position of information in the driver’s viewfield to make inferences. In this way, not only do we monitor the driver’s actions, we monitor the driver’s observations as well. In this paper we present the automated detection and recognition of road signs, combined with the monitoring of the driver’s response. We present a complete system that reads speed signs in real-time, compares the driver’s gaze, and provides immediate feedback if it appears the sign has been missed by t ©

Locating an indoor mobile station based on a wireless communication infrastructure, has practical applications. The most widely employed methods today use an RF propagation loss (PL) model or location fingerprinting (LF). The PL method is known to perform poorly compared to LF. But LF requires an extensive training dataset and cannot adapt well to configuration changes or receiver breakdown. In this paper, we develop a hybrid method that combines the strength of these two methods. It first formulates the RF PL in a nonlinear, censored regression model and adjusts the regression function to the observed signal strength in the fingerprint dataset. In the absence of a training dataset, the hybrid method coincides with the PL method, and, as the spatial granularity of the training dataset increases, the result of the algorithm approaches the result of the LF method. It balances the flexibility and accuracy of the two traditional methods, makes intelligent use of missing values, produces error bounds, and can be made dynamic. We evaluate the performance of the algorithm by applying it to a real site and observe satisfactory positioning accuracy.

Several cooking support systems have been studied to give users instructions based on the recipes stepby-step, using multi-media contents. These systems usually disturb users’ cooking process forcing them to provide information to the system in order to give them beneficial information. In this sense, these systems are considered to be ”system centric”. We propose a system called ”Smart Kitchen” considered to be in ”user centric”, in which a user can cook normally without being concerned about the system. Smart Kitchen can understand cooking processes, in other words, what the user is doing. In this paper, we discuss the design of the Smart Kitchen system and explain three essential modules of tracking food, recognizing food material, and recognizing cooking action.

This thesis presents WiTrack, a system that tracks the 3D motion of a user from the radio signals reflected off her body. It works even if the person is occluded from the WiTrack device or in a different room. WiTrack does not require the user to carry any wireless device, yet its accuracy exceeds current RF localization systems, which require the user to hold a transceiver. Empirical measurements with a WiTrack prototype show that, on average, it localizes the center of a human body to within a median of 10 to 13 cm in the x and y dimensions, and 21 cm in the z dimension. It also provides coarse tracking of body parts, identifying the direction of a pointing hand with a median of 11.20. WiTrack bridges a gap between RF-based localization systems which locate a user through walls and occlusions, and human-computer interaction systems like Kinect, which can track a user without instrumenting her body, but require the user to stay within the direct line of sight of the device. Thesis Supervisor: Dina Katabi Title: Professor of Computer Science and Engineering


Question: An agent-oriented indoor navigation system utilizing a digital signage network.

Integrating Local Landmarks into Wayfinding Instructions for Enhanced Navigation
In urban navigation, route guidance often relies solely on geometric data from the street network to provide instructions. However, this approach may lack the contextual information that can make navigation more intuitive and user-friendly. This paper explores the enhancement of wayfinding instructions by incorporating local landmarks, which can serve as recognizable points of reference for users. We propose a method to formally assess the saliency of potential landmarks within a given dataset. This assessment is based on a set of defined measures that are subjected to hypothesis testing to identify and extract landmarks. Once these landmarks are determined, they are seamlessly integrated into the wayfinding instructions. A practical case study in the city of Vienna illustrates the effectiveness and practicality of this method, showcasing how the inclusion of landmarks can improve the wayfinding experience for users.

This paper describes performance evaluation of a wearable augmented reality system for natural outdoor environments. Applied Research Associates (ARA), as prime integrator on the DARPA ULTRA-Vis (Urban Leader Tactical, Response, Awareness, and Visualization) program, is developing a soldier-worn system to provide intuitive ‘heads-up’ visualization of tactically-relevant geo-registered icons. Our system combines a novel pose estimation capability, a helmet-mounted see-through display, and a wearable processing unit to accurately overlay geo-registered iconography (e.g., navigation waypoints, sensor points of interest, blue forces, aircraft) on the soldier’s view of reality. We achieve accurate pose estimation through fusion of inertial, magnetic, GPS, terrain data, and computer-vision inputs. We leverage a helmet-mounted camera and custom computer vision algorithms to provide terrain-based measurements of absolute orientation (i.e., orientation of the helmet with respect to the earth). These orientation measurements, which leverage mountainous terrain horizon geometry and mission planning landmarks, enable our system to operate robustly in the presence of external and body-worn magnetic disturbances. Current field testing activities across a variety of mountainous environments indicate that we can achieve high icon geo-registration accuracy (<10mrad) using these vision-based methods.

The problem that a robot navigates autonomously through its environment, builds its own map and localizes itself in the map, is still an open problem. It is known as the SLAM (Simultaneous Localization and Map Building) problem. This problem is made even more difficult when we have several robots cooperating to build a common map of an unknown environment, due to the problem of map integration of several submaps built independently by each robot, and with a high degree of error, making the map matching specially difficult. Most of the approaches to solve map building problems are quantitative, resulting in a great computational cost and a low level of abstraction. In order to fulfil these drawbacks qualitative models have been recently used. However, qualitative models are non deterministic. Therefore, the solution recently adopted has been to mix both qualitative and quantitative models to represent the environment and build maps. However, no reasoning process has been used to deal with the information stored in maps up to now, therefore maps are only static storage of landmarks. In this paper we propose a novel method for cooperative map building based on hybrid (qualitative+quantitative) representation which includes also a reasoning process. Distinctive landmarks acquisition for map representation is provided by the cognitive vision and infrared modules which compute differences from the expected data according to the current map and the actual information perceived. We will store in the map the relative orientation information of the landmarks which appear in the environment, after a qualitative reasoning process, therefore the map will be independent of the point of view of the robot. Map integration will then be achieved by localizing each robot in the maps made by the other robots, through a process of pattern matching of the hybrid maps elaborated by each robot, resulting in an integrated map which all robots share, and which is the main objective of this work. This map building method is currently being tested on a team of Sony AIBO four

This paper presents a mobile augmented reality platform that exploits service-orientation for the visualisation of cultural media objects in personalised interactive museum environments. The service orientated architecture is composed of a mobile client, web service framework and service providers utilized to perform augmented reality tasks as well as support for media contents acquisition and consumption via a mobile or wireless network. The mobile client performs augmented reality tasks including multiple object tracking of physical 3D objects concurrently and then visualizes associated media contents (e.g. 3D models, videos, images, text, social media data, etc.) that are associated with the tracked physical (or reference) objects and then overlaid in the correct perspective in the real scene in a personalised museum visualisation scenario. A unique feature includes media content acquisition from open service providers through a web service framework. Typical services include, a photogrammetry service that allows users to obtain virtual 3D models of preferred cultural objects through image-based reconstruction techniques. These acquired 3D reconstruction contents can then be integrated with existing associated contents in a personalized augmented reality museum environment that can be stored and saved for later viewing, such as in the home after visiting a particular augmented reality based museum interactive. This paper focuses on describing the service orientation architectural components that supports personalized mobile augmented reality museum environments.

D.L.D., Dynamic Lighting Design, is a three year, E.U. funded, ongoing research project, concerning ways of designing dynamic luminous environments for pedestrians in open public spaces. This paper presents a set of ideas about how space and the surface of architecture are defined in a dynamic luminous environment, which enhance pedestrian lighting and encourage people to walk and visit public spaces. The D.L.D., parametric interactive lighting software in urban public space, will also be outlined in this paper.

Latest advances in hardware technology and state of the art of computer vision and arti,cial intelligence research can be employed to develop autonomous and distributed monitoring systems. The paper proposes a multi-agent architecture for the understanding of scene dynamics merging the information streamed by multiple cameras. A typical application would be the monitoring of a secure site, or any visual surveillance application deploying a network of cameras. Modular software (the agents) within such architecture controls the di0erent components of the system and incrementally builds a model of the scene by merging the information gathered over extended periods of time. The role of distributed arti,cial intelligence composed of separate and autonomous modules is justi,ed by the need for scalable designs capable of co-operating to infer an optimal interpretation of the scene. Decentralizing intelligence means creating more robust and reliable sources of interpretation, but also allows easy maintenance and updating of the system. Results are presented to support the choice of a distributed architecture, and to prove that scene interpretation can be incrementally and e4ciently built by modular software. ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

The accuracy of indoor positioning systems is commonly computed as a metric based on the Euclidean distance from estimated locations to actual locations. This paper suggests that positioning error distances should be computed as the lengths of the paths that a person may follow when going from wrongly estimated positions to the real positions. The paper proposes a method that calculates the paths from floor plan and obstacles information using the visibility graphs and offsetting techniques, which are commonly used in robotics and CAD/CAM for navigation and manufacturing, respectively. Demonstration of the proposed method was done using a WiFi fingerprinting method based on kNN for pedestrian navigation. Comparisons between our proposed distance and the simple Euclidean distance have shown that the error distances are underestimated and that the differences between the two distances cannot be accurately represented by a fixed quantity in the context of an Indoor Positioning System (IPS) deployed in a library building. We consider that our proposed positioning error distance is more in line with the subjective error perceived by IPS users.

For robots to become more popular for domestic applications, the short comings of current indoor navigation technologies have to be overcome. In this paper, we propose the use of UWB-IR for indoor robot navigation. Various parts of an actual implementation of a UWB-IR based robot navigation system such as system architecture, RF sub-system design, antennas and localization algorithms are discussed. It is shown that by properly addressing the various issues, a localization error of less than 25 cm can be achieved at all points within a realistic indoor localization space.

Tracking, or camera pose determination, is the main technical challenge in creating augmented realities. Constraining the degree to which the environment may be altered to support tracking heightens the challenge. This paper describes several years of work at the USC Computer Graphics and Immersive Technologies (CGIT) laboratory to develop self-contained, minimally intrusive tracking systems for use in both indoor and outdoor settings. These hybrid-technology tracking systems combine vision and inertial sensing with research in fiducial design, feature detection, motion estimation, recursive filters, and pragmatic engineering to satisfy realistic application requirements.

Mobile robots are increasingly populating our human environments. To interact with humans in a socially compliant way, these robots need to understand and comply with mutually accepted rules. In this paper, we present a novel approach to model the cooperative navigation behavior of humans. We model their behavior in terms of a mixture distribution that captures both the discrete navigation decisions, such as going left or going right, as well as the natural variance of human trajectories. Our approach learns the model parameters of this distribution that match, in expectation, the observed behavior in terms of user-defined features. To compute the feature expectations over the resulting high-dimensional continuous distributions, we use Hamiltonian Markov chain Monte Carlo sampling. Furthermore, we rely on a Voronoi graph of the environment to efficiently explore the space of trajectories from the robot’s current position to its target position. Using the proposed model, our method is able to imitate the behavior of pedestrians or, alternatively, to replicate a specific behavior that was taught by tele-operation in the target environment of the robot. We implemented our approach on a real mobile robot and demonstrate that it is able to successfully navigate in an office environment in the presence of humans. An extensive set of experiments suggests that our technique outperforms state-of-the-art methods to model the behavior of pedestrians, which makes it also applicable to fields such as behavioral science or computer graphics.

The proliferation of mobile computing devices and local-area wireless networks has fostered a growing interest in location-aware systems and services. In this paper we present RADAR, a radio-frequency (RF) based system for locating and tracking users inside buildings. RADAR operates by recording and processing signal strength information at multiple base stations positioned to provide overlapping coverage in the area of interest. It combines empirical measurements with signal propagation modeling to determine user location and thereby enable locationaware services and applications. We present experimental results that demonstrate the ability of RADAR to estimate user location with a high degree of accuracy.


Question: An agent-oriented indoor navigation system utilizing digital signage.

This paper introduces a novel model for context-specific route directions, addressing the common challenge of navigating from one location to another, a fundamental daily task for individuals. The model aims to enhance the effectiveness of route directions by considering the unique characteristics of the route and its surrounding environment. This is in contrast to existing systems which often provide generic directions without customization. Our proposed approach is grounded in an information and representation-theoretic analysis of routes, complemented by insights from behavioral research. This systematic framework serves as the foundation for an optimization process designed to tailor route directions to specific contexts. We explore the implications of employing an optimization process for generating route directions and provide an overview of its algorithmic implementation. This approach holds promise in improving the user experience in wayfinding, particularly in complex indoor environments where traditional navigation systems may fall short.

Current outdoor localization techniques fail to provide the required accuracy for estimating the car's lane. In this paper, we present LaneQuest: a system that leverages the ubiquitous and low-energy inertial sensors available in commodity smart-phones to provide an accurate estimate of the car's current lane. LaneQuest leverages hints from the phone sensors about the surrounding environment to detect the car's lane. For example, a car making a right turn most probably will be in the right-most lane, a car passing by a pothole will be in a specific lane, and the car's angular velocity when driving through a curve reflects its lane. Our investigation shows that there are amble opportunities in the environment, i.e. lane “anchors”, that provide cues about the car's lane. To handle the ambiguous location, sensors noise, and fuzzy lane anchors; LaneQuest employs a novel probabilistic lane estimation algorithm. Furthermore, it uses an unsupervised crowd-sourcing approach to learn the position and lane-span distribution of the different lane-level anchors. Our evaluation results from implementation on different android devices and 260Km driving traces by 13 drivers in different cities shows that LaneQuest can detect the different lane-level anchors with an average precision and recall of more than 90%. This leads to an accurate detection of the exact car's lane position 80% of the time, increasing to 89% of the time to within one lane. This comes with a low-energy footprint, allowing LaneQuest to be implemented on the energy-constrained mobile devices.

We explore the development of an experimental augmented reality application that provides benefits to professional mechanics performing maintenance and repair tasks in a field setting. We developed a prototype that supports military mechanics conducting routine maintenance tasks inside an armored vehicle turret, and evaluated it with a user study. Our prototype uses a tracked headworn display to augment a mechanic's natural view with text, labels, arrows, and animated sequences designed to facilitate task comprehension, localization, and execution. A within-subject controlled user study examined professional military mechanics using our system to complete 18 common tasks under field conditions. These tasks included installing and removing fasteners and indicator lights, and connecting cables, all within the cramped interior of an armored personnel carrier turret. An augmented reality condition was tested against two baseline conditions: the same headworn display providing untracked text and graphics and a fixed flat panel display representing an improved version of the laptop-based documentation currently employed in practice. The augmented reality condition allowed mechanics to locate tasks more quickly than when using either baseline, and in some instances, resulted in less overall head movement. A qualitative survey showed that mechanics found the augmented reality condition intuitive and satisfying for the tested sequence of tasks.

We propose an interactive bookshelf surface to augment a human ability for in situ book searching and storing. In book searching support, when a user touches the edge of the bookshelf, the cover image of a stored book located above the touched position is projected directly onto the book spine. As a result, the user can search for a desired book by sliding his (or her) finger across the shelf edge. In book storing support, when a user brings a book close to the bookshelf, the place where the book should be stored is visually highlighted by a projection light. This paper also presents sensing technologies to achieve the above mentioned interactive techniques. In addition, by considering the properties of the human visual system, we propose a simple visual effect to reduce the legibility degradation of the projected image contents by the complex textures and geometric irregularities of the spines. We confirmed the feasibility of the system and the effectiveness of the proposed interaction techniques through user studies.

This paper would present the system integration and overview of the autonomous cleaning robot Roboking. Roboking is self-propelled autonomously navigating vacuum cleaning robot. It uses several sensors in order to protect indoor environments and itself while cleaning. In this paper, we would describe the principle of operation along with system structure, sensors, functions and integrated subsystems.

With rising concerns about obesity and sedentary lifestyles in youth, there has been an increasing interest in understanding how pervasive and ubiquitous computing technologies can catalyze positive health behaviors in children and teens. School-based interventions seem like a natural choice, and ubiquitous computing technologies hold much promise for these interventions. Yet the literature contains little guidance for how to approach school-based ubicomp deployments. Grounded in our analysis of a large-scale US school-based intervention for promoting youth physical activity, we present an approach to the design and evaluation of school-based ubicomp that treats the school as a social institution. We show how the school regulates students' daily lives, drawing from work in the sociology of schools to create a framing for planning, executing and analyzing school-based ubicomp deployments. These insights will assist other researchers and designers engaging in deployments of ubiquitous computing systems in settings with established institutional structures.

Research on mobile robot navigation has produced two major paradigms for mapping indoor environments: grid-based and topological. While grid-based methods produce accurate metric maps, their complexity often prohibits efficient planning and problem solving in large-scale indoor environments. Topological maps, on the other hand, can be used much more efficiently, yet accurate and consistent topological maps are considerably difficult to learn in large-scale environments. This paper describes an approach that integrates both paradigms: grid-based and topological. Grid-based maps are learned using artificial neural networks and Bayesian integration. Topological maps are generated on top of the grid-based maps, by partitioning the latter into coherent regions. By combining both paradigms—grid-based and topological—, the approach presented here gains the best of both worlds: accuracy/consistency and efficiency. The paper gives results for autonomously operating a mobile robot equipped with sonar sensors in populated multi-room envi-

Recent introduction of low-cost 3D sensing and affordable immersive virtual reality have lowered the barriers for creating and maintaining 3D virtual worlds. In this paper, we propose a way to combine these technologies with discrete-event simulation to improve the use of simulation in decision making in manufacturing. This work will describe how feedback is possible from real world systems directly into a simulation model to guide smart behaviors. Technologies included in the research include feedback from RGBD images of shop floor motion and human interaction within full immersive virtual reality that includes the latest headset technologies.

Although location awareness and turn-by-turn instructions are prevalent outdoors due to GPS, we are back into the darkness in uninstrumented indoor environments such as underground parking structures. We get confused, disoriented when driving in these mazes, and frequently forget where we parked, ending up circling back and forth upon return.In this paper, we propose VeTrack, a smartphone-only system that tracks the vehicle's location in real time using the phone's inertial sensors. It does not require any environment instrumentation or cloud backend. It uses a novel "shadow" tracing method to accurately estimate the vehicle's trajectories despite arbitrary phone/vehicle poses and frequent disturbances. We develop algorithms in a Sequential Monte Carlo framework to represent vehicle states probabilistically, and harness constraints by the garage map and detected landmarks to robustly infer the vehicle location. We also find landmark (e.g., speed bumps, turns) recognition methods reliable against noises, disturbances from bumpy rides and even hand-held movements. We implement a highly efficient prototype and conduct extensive experiments in multiple parking structures of different sizes and structures, with multiple vehicles and drivers. We find that VeTrack can estimate the vehicle's real time location with almost negligible latency, with error of 2-4 parking spaces at 80-percentile.

Nowadays, the convergence of Internet of Things (IoT) networking and mobile applications is favoring the deployment of novel and advanced smart parking systems through which users can be informed in real-time about the presence of vacant parking spots close to their destinations. In this paper, we provide an example of such opportunity, by describing Park Here!, a novel mobile application that aims at mitigating the overhead caused by parking spot seeking operations in urban areas. Our solution targets common city environments, where no per-spot sensors are available, and there is no remote service allowing the reservation in-advance of a parking spot. For this scenario, we propose a novel algorithm for the automatic detection of parking actions performed by the user, through the analysis of smartphone embedded sensors' (accelerometer/gyroscope), and of the Bluetooth connectivity. Once a parking event has been detected, an adaptive strategy allows disseminating the information over the target scenario, using a combination of Internet connection to a remote server, and Device-to-Device (D2D) connections over WiFi Direct links. Preliminary experiments demonstrate the accuracy of the proposed algorithm in correctly identifying parking events in an automatic way, and hence in notifying information to other potentially interested users.

The availability of location information has become a key factor in today’s communications systems allowing location based services. In outdoor scenarios, the mobile terminal position is obtained with high accuracy thanks to the global positioning system (GPS) or to the standalone cellular systems. However, the main problem of GPS and cellular systems resides in the indoor environment and in scenarios with deep shadowing effects where the satellite or cellular signals are broken. In this paper, we survey different technologies and methodologies for indoor and outdoor localization with an emphasis on indoor methodologies and concepts. Additionally, we discuss in this review different localization-based applications, where the location information is critical to estimate. Finally, a comprehensive discussion of the challenges in terms of accuracy, cost, complexity, security, scalability, etc. is given. The aim of this survey is to provide a comprehensive overview of existing efforts as well as auspicious and anticipated dimensions for future work in indoor localization techniques and applications.


Question: An agent-centric indoor navigation system utilizing digital signage.

In the realm of Artificial Intelligence (AI), a comprehensive approach to understanding and implementing intelligent systems is presented in the widely acclaimed book "Artificial Intelligence: A Modern Approach." The authors delve into the vast expanse of AI, encompassing various mathematical foundations, perception, reasoning, learning, action, and applications across different scales and domains. This book is notable for its depth and breadth, covering topics from foundational theories to advanced applications in robotics and beyond. The digital repository of Thang Long University houses this resource, making it accessible to students and researchers alike. The file "CS503_TriTueNhanTaoNC_GTStuart Russell, Peter Norvig-Artificial Intelligence. A Modern Approach [Global Edition]Pearson (2016).pdf" provides a detailed insight into the content of the book, offering readers a comprehensive guide to the world of AI.

Shifting focus to a specific application of AI, this document discusses an agent-based indoor wayfinding system that integrates digital sign technology. Unlike traditional wayfinding methods, this innovative system employs intelligent agents to guide individuals through complex indoor environments, utilizing digital signs as a key interface. The agents are designed to interact with users, understand their destination, and provide personalized navigation instructions. This system leverages AI techniques such as machine learning and pathfinding algorithms to dynamically adjust routes based on real-time data, such as crowd density and environmental conditions. The integration of digital signs enhances the user experience by providing visual and auditory cues, ensuring accessibility for all users, including those with visual impairments. This agent-based wayfinding system represents a significant advancement in indoor navigation, offering a more efficient and user-friendly solution for navigating large and complex buildings.

The paper describes experience with applying a user-centric design methodology in developing systems for human-robot teaming in Urban Search & Rescue. A human-robot team consists of several semi-autonomous robots (rovers/UGVs, microcopter/UAVs), several humans at an off-site command post (mission commander, UGV operators) and one on-site human (UAV operator). This system has been developed in close cooperation with several rescue organizations, and has been deployed in a real-life tunnel accident use case. The human-robot team jointly explores an accident site, communicating using a multi-modal team interface, and spoken dialogue. The paper describes the development of this complex socio-technical system per se, as well as recent experience in evaluating the performance of this system.

In this work we examine people's ability to maintain spatial awareness of an HMD--based immersive virtual environment when the terrain of the virtual world does not match the real physical world in which they are locomoting. More specifically, we examine what happens when a subject explores a hilly virtual environment while physically walking in a flat room. In our virtual environments, subjects maintain their height (or distance to the ground plane) at all times. In the experiment described in this work, we directly compare spatial orientation in a flat virtual environment to two variations of a hilly virtual environment. Interestingly, we find that subjects' spatial awareness of the environment is improved with the addition of uneven terrain.

In this paper, a practical real-time path planning and robot navigation algorithm for a non-holonomic indoor mobile robot based on visual servoing is implemented. The proposed algorithm is divided into three parts; the first part uses Multi-Stencils Fast Marching (MSFM) as a path planning method. But the generated path results from fast marching methods when used directly, not guaranteed to be safe and smooth. Subsequently, the robot can touch corners, walls and obstacles. The proposed algorithm uses image processing methods to solve this problem. The second part estimates the position and orientation of the robot, from the visual information, to follow the desired path with avoiding obstacles. The third part proposes a decentralized PD-like Fuzzy Logic Controller (FLC) to keep up the robot on the desired path. Experimental results show that the developed design is valid to estimate shortest-path by avoiding obstacles and able to guide the robot to follow the path in real-time.

In this paper we describe a new technique for the creation of featurebased stochastic maps using standard Polaroid sonar sensors. The fundamental contributions of our proposal are: (1) a perceptual grouping process that permits the robust identification and localization of environmental features, such as straight segments and corners, from the sparse and noisy sonar data; (2) a map joining technique that allows the system to build a sequence of independent limited-size stochastic maps and join them in a globally consistent way; (3) a robust mechanism to determine which features in a stochastic map correspond to the same environment feature, allowing the system to update the stochastic map accordingly, and perform tasks such as revisiting and loop closing. We demonstrate the practicality of this approach by building a geometric map of a medium size, real indoor environment, with several people moving around the robot. Maps built from laser data for the same experiment are provided for comparison. KEY WORDS—map building, local maps, data association, sonar sensors, Hough transform

1 We explore the notion of a tour guide software agent for assisting users browsing the World Wide Web. A Web tour guide agent provides assistance similar to that provided by a human tour guide in a museum { it guides the user along an appropriate path through the collection, based on its knowledge of the user's interests, of the location and relevance of various items in the collection, and of the way in which others have interacted with the collection in the past. This paper describes a simple but operational tour guide, called WebWatcher, which has given over 5000 tours to people browsing CMU's School of Computer Science Web pages. WebWatcher accompanies users from page to page, suggests appropriate hyperlinks, and learns from experience to improve its advice-giving skills. We describe the learning algorithms used by WebWatcher, experimental results showing their e ectiveness, and lessons learned from this case study in Web tour guide agents.

This paper presents our vision of Human Computer Interaction (HCI): "Tangible Bits." Tangible Bits allows users to "grasp & manipulate" bits in the center of users’ attention by coupling the bits with everyday physical objects and architectural surfaces. Tangible Bits also enables users to be aware of background bits at the periphery of human perception using ambient display media such as light, sound, airflow, and water movement in an augmented space. The goal of Tangible Bits is to bridge the gaps between both cyberspace and the physical environment, as well as the foreground and background of human activities. This paper describes three key concepts of Tangible Bits: interactive surfaces; the coupling of bits with graspable physical objects; and ambient media for background awareness. We illustrate these concepts with three prototype systems – the metaDESK, transBOARD and ambientROOM – to identify underlying research issues.

The ability to represent knowledge about space and its position therein is crucial for a mobile robot. To this end, topological and semantic descriptions are gaining popularity for augmenting purely metric space representations. In this paper we present a multi-modal place classification system that allows a mobile robot to identify places and recognize semantic categories in an indoor environment. The system effectively utilizes information from different robotic sensors by fusing multiple visual cues and laser range data. This is achieved using a high-level cue integration scheme based on a Support Vector Machine (SVM) that learns how to optimally combine and weight each cue. Our multi-modal place classification approach can be used to obtain a real-time semantic space labeling system which integrates information over time and space. We perform an extensive experimental evaluation of the method for two different platforms and environments, on a realistic off-line database and in a live experiment on an autonomous robot. The results clearly demonstrate the effecThe International Journal of Robotics Research Vol. 00, No. 00, Xxxxxxxx 2009, pp. 000–000 DOI: 10.1177/0278364909356483 c The Author(s), 2009. Reprints and permissions: http://www.sagepub.co.uk/journalsPermissions.nav Figures 1–15, 17, 18 appear in color online: http://ijr.sagepub.com tiveness of our cue integration scheme and its value for robust place classification under varying conditions. KEY WORDS—recognition, sensor fusion, localization, multi-modal place classification, sensor and cue integration, semantic annotation of space

Build accurate DNN models requires training on large labeled, context specific datasets, especially those matching the target scenario. We believe advances in wireless localization, working in unison with cameras, can produce automated annotation of targets on images and videos captured in the wild. Using pedestrian and vehicle detection as examples, we demonstrate the feasibility, benefits, and challenges of an automatic image annotation system. Our work calls for new technical development on passive localization, mobile data analytics, and error-resilient ML models, as well as design issues in user privacy policies.

Gesture recognition is a well-establish topic of research that is widely adopted for a broad range of applications. For instance, it can be exploited for the command of a smart environment without any remote control unit or even for the recognition of human activities from a set of video cameras deployed in strategic position. Many researchers working on assistive smart home, such as our team, believe that the intrusiveness of that technology will prevent the future adoption and commercialization of smart homes. In this paper, we propose a novel gesture recognition algorithm that is solely based on passive RFID technology. This technology enables the localization of small tags that can be embedded in everyday life objects (a cup or a book, for instance) while remaining non intrusive. However, until now, this technology has been largely ignored by researchers on gesture recognition, mostly because it is easily disturbed by noise (metal, human, etc.) and offer limited precision. Despite these issues, the localization algorithms have improved over the years, and our recent efforts resulted in a real-time tracking algorithm with a precision approaching 14cm. With this, we developed a gesture recognition algorithm able to perform segmentation of gestures and prediction on a spatio-temporal data series. Our new model, exploiting works on qualitative spatial reasoning, achieves recognition of 91%. Our goal is to ultimately use that knowledge for both human activity recognition and errors detection.

Reliable autonomous navigation is an active research topic that has drawn the attention for decades, however, human factors such as navigational comfort has not received the same level of attention. This work proposes the concept of “comfortable map” and presents a navigation approach for autonomous passenger vehicles which in top of being safe and reliable is comfortable. In our approach we first extract information from users preference related to comfort while sitting on a robotic wheelchair under different conditions in an indoor corridor environment. Human-comfort factors are integrated to a geometric map generated by SLAM framework. Then a global planner computes a safe and comfortable path which is followed by the robotic wheelchair. Finally, an evaluation with 29 participants using a fully autonomous robotic wheelchair, showed that more than 90% of them found the proposed approach more comfortable than a shortest-path state of the art approach.


Question: An agent-centric indoor navigation system utilizing a digital signage framework.

The enigmatic number seven, give or take a couple: exploring constraints on our information processing abilities.

For the past seven years, I've found myself persistently haunted by a particular digit. This number, with an uncanny knack for disguise, has manifested in my most confidential records and has boldly confronted me within the pages of our most prominent publications. Its persistence, never straying too far from its base form, is far from coincidental. There appears to be a systematic design orchestrating its appearances, a pattern that governs its presence. The question is: does this number truly hold some peculiar significance, or am I merely succumbing to an illusion of persecution?

Let me present the evidence by recounting a series of experiments designed to assess human accuracy in assigning numerical values to the magnitude of various stimulus attributes. In psychological parlance, these would be categorized as absolute judgment experiments.

In the realm of indoor navigation, a novel approach has been developed that integrates digital signage systems with agent-based modeling. This method, unlike traditional wayfinding techniques, leverages the power of digital signage to guide individuals through complex indoor environments. The system is designed to dynamically adjust based on real-time data, optimizing routes and providing personalized guidance to users. Central to this approach is the use of agent-based models that simulate individual behaviors and interactions within the space. By integrating these models with a network of digital signs, the system can predict and influence pedestrian flow, enhancing safety and efficiency in crowded areas. This innovative solution not only improves the wayfinding experience for individuals but also provides facility managers with valuable insights into pedestrian movement patterns, aiding in the optimization of space usage and emergency response planning.

This paper proposes an indoor positioning system (IPS) based on WLAN using a single fifth-generation (5G) Wi-Fi access point. The proposed method uses fingerprint and classification models based on KNN (K-nearest neighbor) and Bayes rule. The fingerprint is formed by beam RSS (Received Signal Strength) samples, collected in some 2D locations of the indoor environment. Numerical simulations shown that using the best beam samples, it is possible to locate the stationary user's mobile device with average error less than 2.5 m.

JackIn is a new human-human communication framework for connecting two or more people. With first-person view video streaming from a person (called Body) wearing a transparent head-mounted display and a head-mounted camera, the other person (called Ghost) participates in the shared first-person view. With JackIn, people's activities can be shared and assistance or guidance can be given through other peoples expertise. This can be applied to daily activities such as cooking lessons, shopping navigation, education in craft-work or electrical work, and sharing experiences of sporting and live events. For a better viewing experience with frist-person view, we developed the out-of-body view in which first-person images are integrated to construct a scene around a Body, and a Ghost can virtually control the viewpoint to look around the space surrounding the Body. We also developed a tele-pointing gesture interface. We conducted an experiment to evaluate how effective this framework is and found that Ghosts can understand the spatial situation of the Body.

Concerning the phenomenon that common parking service could not satisfy the increasing demand of the private vehicle owners, an intelligent parking guidance system based on Zig Bee network and geomagnetic sensors was designed. Real-time vehicle position and related traffic information were collected by geomagnetic sensors around parking lots and updated to center server via Zig Bee network. On the other hand, out-door Liquid Crystal Display screens controlled by center server can display information of available parking places. In this paper, guidance strategy was divided into four levels, which could provide clear and effective information to drivers. The experimental results prove that the distance detection accuracy of geomagnetic sensors was within 0.4m, and the lowest package loss rate of the wireless network in the range of 150m is 0%. This system can provide solution for better parking service in intelligent cities.

Robot vision refers to the capability of a robot to visually perceive the environment and use this information for execution of different tasks. Visual feedback has been used extensively for robot navigation and obstacle avoidance. In the recent years, there are also examples that include interaction with people and manipulation of objects. In this paper, we review some of the work that goes beyond of using artificial landmarks and fiducial markers for the purpose of implementing vision based control in robots. We discuss different application areas, both from the systems perspective and individual problems such as object tracking and recognition.

This paper presents our vision of Human Computer Interaction (HCI): "Tangible Bits." Tangible Bits allows users to "grasp & manipulate" bits in the center of users’ attention by coupling the bits with everyday physical objects and architectural surfaces. Tangible Bits also enables users to be aware of background bits at the periphery of human perception using ambient display media such as light, sound, airflow, and water movement in an augmented space. The goal of Tangible Bits is to bridge the gaps between both cyberspace and the physical environment, as well as the foreground and background of human activities. This paper describes three key concepts of Tangible Bits: interactive surfaces; the coupling of bits with graspable physical objects; and ambient media for background awareness. We illustrate these concepts with three prototype systems – the metaDESK, transBOARD and ambientROOM – to identify underlying research issues.

With the proliferation of Internet of Things (IoT) devices such as smartphones, sensors, cameras, and RFIDs, it is possible to collect massive amount of data for localization and tracking of people within commercial buildings. Enabled by such occupancy monitoring capabilities, there are extensive opportunities for improving the energy consumption of buildings via smart HVAC control. In this respect, the major challenges we envision are 1) to achieve occupancy monitoring in a minimally intrusive way, e.g., using the existing infrastructure in the buildings and not requiring installation of any apps in the users' smart devices, and 2) to develop effective data fusion techniques for improving occupancy monitoring accuracy using a multitude of sources. This paper surveys the existing works on occupancy monitoring and multi-modal data fusion techniques for smart commercial buildings. The goal is to lay down a framework for future research to exploit the spatio-temporal data obtained from one or more of various IoT devices such as temperature sensors, surveillance cameras, and RFID tags that may be already in use in the buildings. A comparative analysis of existing approaches and future predictions for research challenges are also provided.

This paper describes a method for allowing people to virtually move around a CAVETM without ever having to turn to face the missing back wall. We describe the method, and report a pilot study of 28 participants, half of whom moved through the virtual world using a hand-held controller, and the other half used the new technique called ‘Redirected Walking in Place’ (RWP). The results show that the current instantiation of the RWP technique does not result in a lower frequency of looking towards the missing wall. However, the results also show that the sense of presence in the virtual environment is significantly and negatively correlated with the amount that the back wall is seen. There is evidence that RWP does reduce the chance of seeing the blank wall for some participants. The increased sense of presence through never having to face the blank wall, and the results of this pilot study show the RWP has promise and merits further development.

This paper presents a novel algorithm that utilizes a 2D floorplan to align panorama RGBD scans. While effective panorama RGBD alignment techniques exist, such a system requires extremely dense RGBD image sampling. Our approach can significantly reduce the number of necessary scans with the aid of a floorplan image. We formulate a novel Markov Random Field inference problem as a scan placement over the floorplan, as opposed to the conventional scan-to-scan alignment. The technical contributions lie in multi-modal image correspondence cues (between scans and schematic floorplan) as well as a novel coverage potential avoiding an inherent stacking bias. The proposed approach has been evaluated on five challenging large indoor spaces. To the best of our knowledge, we present the first effective system that utilizes a 2D floorplan image for building-scale 3D pointcloud alignment. The source code and the data are shared with the community to further enhance indoor mapping research.

Inspired by the emerging WiFi-based applications, in this paper, we leverage ubiquitous WiFi signals and propose a device-free step counting system, called WiStep. Based on the multipath propagation model, when a person is walking, her torso and limbs move at different speeds, which modulates wireless signals to the propagation paths with different lengths and thus introduces different frequency components into the received Channel State Information (CSI). To count walking steps, we first utilize time-frequency analysis techniques to segment and recognize the walking movement, and then dynamically select the sensitive subcarriers with largest amplitude variances from multiple CSI streams. Wavelet decomposition is applied to extract the detail coefficients corresponding to the frequencies induced by feet or legs, and compress the data so as to improve computing speed. Short-time energy of the coefficients is then calculated as the metric for step counting. Finally, we combine the results derived from the selected subcarriers to produce a reliable step count estimation. In contrast to counting steps based on the torso frequency analysis, WiStep can count the steps of in-place walking even when the person's torso speed is null. We implement WiStep on commodity WiFi devices in two different indoor scenarios, and various influence factors are taken into consideration when evaluating the performance of WiStep. The experimental results demonstrate that WiStep can realize overall step counting accuracies of 90.2% and 87.59% respectively in these two scenarios, and it is resilient to the change of scenarios.

In this paper, we introduce the DEEP-SEE FACE framework, an assistive device designed to improve cognition, interaction, and communication of visually impaired (VI) people in social encounters. The proposed approach jointly exploits computer vision algorithms (region proposal networks, ATLAS tracking and global, and low-level image descriptors) and deep convolutional neural networks in order to detect, track, and recognize, in real-time, various persons existent in the video streams. The major contribution of the paper concerns a global, fixed-size face representation that takes into the account of various video frames while remaining independent of the length of the image sequence. To this purpose, we introduce an effective weight adaptation scheme that is able to determine the relevance assigned to each face instance, depending on the frame degree of motion/camera blur, scale variation, and compression artifacts. Another relevant contribution involves a hard negative mining stage that helps us differentiating between known and unknown face identities. The experimental results, carried out on a large-scale data set, validate the proposed methodology with an average accuracy and recognition rates superior to 92%. When tested in real life, indoor/outdoor scenarios, the DEEP-SEE FACE prototype proves to be effective and easy to use, allowing the VI people to access visual information during social events.


Question: A rapid learning technique for deep belief networks.

This paper introduces a novel framework, coined Fields of Experts (FoE), designed to learn comprehensive and versatile image priors that encapsulate the statistical characteristics of natural scenes. These priors can be leveraged across a range of machine vision tasks. The methodology extends the conventional Markov random field (MRF) models by training potential functions over expansive pixel neighborhoods. The potential functions are modeled within a Products-of-Experts framework, incorporating nonlinear transformations of multiple linear filter responses. A significant departure from traditional MRF approaches is the comprehensive learning of all parameters, inclusive of the linear filters, from a dataset. The efficacy of the FoE model is showcased through two illustrative applications: image denoising and image inpainting. These tasks are executed using a straightforward, albeit approximate, inference procedure. Despite being trained on a generic image repository without task-specific optimization, the FoE model yields competitive, and at times superior, results compared to specialized techniques.

In the paper, several modifications to the conventional learning algorithms of the Hierarchical Temporal Memory (HTM) – a biologically inspired largescale model of the neocortex by Numenta – have been proposed. Firstly, an alternative spatial pooling method has been introduced, which makes use of a random pattern generator exploiting the Metropolis-Hastings algorithm. The original inference algorithm by Numenta has been reformulated, in order to reduce a number of tunable parameters and to optimize its computational efficiency. The main contribution of the paper consists in the proposal of a novel temporal pooling method – the pair-wise explorer – which allows faster and more reliable training of the HTM networks using data without inherent temporal information (e.g., static images). While the conventional temporal pooler trains the HTM network on a finite segment of the smooth Brownian-like random walk across the training images, the 902 S. Štolc, I. Bajla, K. Valent́ın, R. Škoviera proposed method performs training by means of the pairs of patterns randomly sampled (in a special manner) from a virtually infinite smooth random walk. We have conducted a set of experiments with the single-layer HTM network applied to the position, scale, and rotation-invariant recognition of geometric objects. The obtained results provide a clear evidence that the pair-wise method yields significantly faster convergence to the theoretical maximum of the classification accuracy with respect to both the length of the training sequence (defined by the maximum allowed number of updates of the time adjacency matrix – TAM) and the number of training patterns. The advantage of the proposed explorer manifested itself mostly in the lower range of TAM updates where it caused up to 10% relative accuracy improvement over the conventional method. Therefore we suggest to use the pair-wise explorer, instead of the smooth explorer, always when the HTM network is trained on a set of static images, especially when the exhaustive training is impossible due to the complexity of the given task.

We consider the problem of learning deep representation when target labels are available. In this paper, we show that there exists intrinsic relationship between target coding and feature representation learning in deep networks. Specifically, we found that distributed binary code with error correcting capability is more capable of encouraging discriminative features, in comparison to the 1-of-K coding that is typically used in supervised deep learning. This new finding reveals additional benefit of using error-correcting code for deep model learning, apart from its well-known error correcting property. Extensive experiments are conducted on popular visual benchmark datasets.

Learning algorithms related to artificial neural networks and in particular for Deep Learning may seem to involve many bells and whistles, called hyperparameters. This chapter is meant as a practical guide with recommendations for some of the most commonly used hyper-parameters, in particular in the context of learning algorithms based on backpropagated gradient and gradient-based optimization. It also discusses how to deal with the fact that more interesting results can be obtained when allowing one to adjust many hyper-parameters. Overall, it describes elements of the practice used to successfully and efficiently train and debug large-scale and often deep multi-layer neural networks. It closes with open questions about the training difficulties observed with deeper architectures.

Stacking-based deep neural network (S-DNN), in general, denotes a deep neural network (DNN) resemblance in terms of its very deep, feedforward network architecture. The typical S-DNN aggregates a variable number of individually learnable modules in series to assemble a DNN-alike alternative to the targeted object recognition tasks. This work likewise devises an S-DNN instantiation, dubbed deep analytic network (DAN), on top of the spectral histogram (SH) features. The DAN learning principle relies on ridge regression, and some key DNN constituents, specifically, rectified linear unit, fine-tuning, and normalization. The DAN aptitude is scrutinized on three repositories of varying domains, including FERET (faces), MNIST (handwritten digits), and CIFAR10 (natural objects). The empirical results unveil that DAN escalates the SH baseline performance over a sufficiently deep layer.

Extreme learning machine (ELM) has become a popular topic in machine learning in recent years. ELM is a new kind of single-hidden layer feedforward neural network with an extremely low computational cost. ELM, however, has two evident drawbacks: 1) the output weights solved by Moore-Penrose generalized inverse is a least squares minimization issue, which easily suffers from overfitting and 2) the accuracy of ELM is drastically sensitive to the number of hidden neurons so that a large model is usually generated. This brief presents a sparse Bayesian approach for learning the output weights of ELM in classification. The new model, called Sparse Bayesian ELM (SBELM), can resolve these two drawbacks by estimating the marginal likelihood of network outputs and automatically pruning most of the redundant hidden neurons during learning phase, which results in an accurate and compact model. The proposed SBELM is evaluated on wide types of benchmark classification problems, which verifies that the accuracy of SBELM model is relatively insensitive to the number of hidden neurons; and hence a much more compact model is always produced as compared with other state-of-the-art neural network classifiers.

The premise of this article is that learning procedures used to train artificial neural networks are inherently statistical techniques. It follows that statistical theory can provide considerable insight into the properties, advantages, and disadvantages of different network learning methods. We review concepts and analytical results from the literatures of mathematical statistics, econometrics, systems identification, and optimization theory relevant to the analysis of learning in artificial neural networks. Because of the considerable variety of available learning procedures and necessary limitations of space, we cannot provide a comprehensive treatment. Our focus is primarily on learning procedures for feedforward networks. However, many of the concepts and issues arising in this framework are also quite broadly relevant to other network learning paradigms. In addition to providing useful insights, the material reviewed here suggests some potentially useful new training methods for artificial neural networks.

We consider the problem of learning from examples in layered linear feed-forward neural networks using optimization methods, such as back propagation, with respect to the usual quadratic error function E of the connection weights. Our main result is a complete description of the landscape attached to E in terms of principal component analysis. We show that E has a unique minimum corresponding to the projection onto the subspace generated by the first principal vectors of a covariance matrix associated with the training patterns. All the additional critical points of E are saddle points (corresponding to projections onto subspaces generated by higher order vectors). The auto-associative case is examined in detail. Extensions and implications for the learning algorithms are discussed.

We discsus Hinton’s (1989) relative payoff procedure (RPP), a static reinforcement learning algorithm whose foundation is not stochastic gradient ascent. We show circumstances under which applying the RPP is guaranteed to increase the mean return, even though it can make large changes in the values of the parameters. The proof is based on a mapping between the RPP and a form of the expectation-maximisation procedure of Dempster, Laird & Rubin (1976).

Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.

The applicability of machine learning methods is often limi ted by the amount of available labeled data, and by the ability (or inability) of the de signer to produce good internal representations and good similarity measures for the input data vectors. The aim of this thesis is to alleviate these two limitations by proposing al gorithms tolearngood internal representations, and invariant feature hierarchies from u nlabeled data. These methods go beyond traditional supervised learning algorithms, and rely on unsupervised, and semi-supervised learning. In particular, this work focuses on “deep learning” methods , a et of techniques and principles to train hierarchical models. Hierarchical mod els produce feature hierarchies that can capture complex non-linear dependencies among the observed data variables in a concise and efficient manner. After training, these mode ls can be employed in real-time systems because they compute the representation by a very fast forward propagation of the input through a sequence of non-linear transf ormations. When the paucity of labeled data does not allow the use of traditional supervi s d algorithms, each layer of the hierarchy can be trained in sequence starting at the bott om by using unsupervised or semi-supervised algorithms. Once each layer has been train ed, the whole system can be fine-tuned in an end-to-end fashion. We propose several unsu pervised algorithms that can be used as building block to train such feature hierarchi es. We investigate algorithms that produce sparse overcomplete representations a nd fe tures that are invariant to known and learned transformations. These algorithms are designed using the Energy-


Question: A rapid learning technique for deep belief networks.

This paper explores a novel method for enhancing the efficiency of learning in Deep Belief Networks (DBNs), which are complex hierarchical models capable of capturing intricate patterns in data. Unlike traditional training algorithms that may suffer from slow convergence and local optima, the proposed Fast Learning Algorithm for DBNs introduces a series of innovations designed to accelerate the learning process while ensuring robust model performance. The method leverages a greedy layer-wise pre-training phase, followed by a fine-tuning stage that employs a more sophisticated optimization technique. This approach allows the network to rapidly learn useful representations of the input data, making it particularly well-suited for large-scale datasets and complex feature extraction tasks. By optimizing the learning process, the Fast Learning Algorithm for DBNs not only speeds up model training but also improves the overall accuracy and generalization capabilities of the network, thus offering a significant advantage over conventional learning algorithms in the realm of deep learning. This advancement is particularly crucial in applications where computational resources are limited and rapid model deployment is essential.

Deep neural networks (DNNs) are widely used in machine translation (MT). This article gives an overview of DNN applications in various aspects of MT.

Large knowledge bases (KBs) are useful in many tasks, but it is unclear how to integrate this sort of knowledge into “deep” gradient-based learning systems. To address this problem, we describe a probabilistic deductive database, called TensorLog, in which reasoning uses a differentiable process. In TensorLog, each clause in a logical theory is first converted into certain type of factor graph. Then, for each type of query to the factor graph, the message-passing steps required to perform belief propagation (BP) are “unrolled” into a function, which is differentiable. We show that these functions can be composed recursively to perform inference in non-trivial logical theories containing multiple interrelated clauses and predicates. Both compilation and inference in TensorLog are efficient: compilation is linear in theory size and proof depth, and inference is linear in database size and the number of message-passing steps used in BP. We also present experimental results with TensorLog and discuss its relationship to other first-order probabilistic logics.

Probably the most important, problem in machine learning is the preliminary biasing of a learner’s hypothesis space so that it is small enough to ensure good generalisation from reasonable training sets, yet large enough that it contains a good solution to the problem being learnt, In this paper a mechanism for automailcall,y learning or biasing the learner’s hypothesis space is introduced. It works by first learning an appropriate [nterna! represenlaiton for a learning en~-ironment and then using that representation to hiss the learner’s hypothesis space for the learning of future tasks drawn from the same environment. Jln internal represent at ion must be learnt by sampling from many s2mtiar tasks, not just a single task as occurs in ordinary machine learning. It is proved that the number of examples m per task required to ensure good generalisation from a representation learner obeys }n = O(a + b/n) where n is the number of tasks being learnt, and a and b are constants. If the tasks are learnt independently (/. e. without a common representation ) then m = 0( a + b). It is argued that for learning environments such as speech and character recognition b >> a and hence representation learning in these environments can potentially yield a drastic reduction in the number of examples required per task. It is also proved that if n = O(b) (with m = O(a+b/n ) ) then the representation learnt will be good for learning novel tasks from the same environment, and that the number of examples required to generalise well on a novel task will be reduced to O(a) (as opposed to {)( a + b) if no represent ation is used). It, is shown that gradient descent can be used to train neural network representations and the results of an experiment are reported in which a neural network representation was learnt for an environment consisting of transJaf ion ail,y in uarlanf Boolean functions. The experiment, provides strong qualitative support for the theoretical results.

A central challenge faced by memory systems is the robust retrieval of a stored pattern in the presence of interference due to other stored patterns and noise. A theoretically well-founded solution to robust retrieval is given by attractor dynamics, which iteratively clean up patterns during recall. However, incorporating attractor dynamics into modern deep learning systems poses difficulties: attractor basins are characterised by vanishing gradients, which are known to make training neural networks difficult. In this work, we avoid the vanishing gradient problem by training a generative distributed memory without simulating the attractor dynamics. Based on the idea of memory writing as inference, as proposed in the Kanerva Machine, we show that a likelihood-based Lyapunov function emerges from maximising the variational lower-bound of a generative memory. Experiments shows it converges to correct patterns upon iterative retrieval and achieves competitive performance as both a memory model and a generative model.

Training deep neural networks generally requires massive amounts of data and is very computation intensive. We show here that it may be possible to circumvent the expensive gradient descent procedure and derive the parameters of a neural network directly from properties of the training data. We show that, near convergence, the gradient descent equations for layers close to the input can be linearized and become stochastic equations with noise related to the covariance of data for each class. We derive the distribution of solutions to these equations and discover that it is related to a “supervised principal component analysis.” We implement these results on image datasets MNIST, CIFAR10 and CIFAR100 and find that, indeed, pretrained layers using our findings performs comparable or superior to neural networks of the same size and architecture trained with gradient descent. Moreover, our pretrained layers can often be calculated using a fraction of the training data, owing to the quick convergence of the covariance matrix. Thus, our findings indicate that we can cut the training time both by requiring only a fraction of the data used for gradient descent, and by eliminating layers in the costly backpropagation step of the training. Additionally, these findings partially elucidate the inner workings of deep neural networks and allow us to mathematically calculate optimal solutions for some stages of classification problems, thus significantly boosting our ability to solve such problems efficiently.

People learning new concepts can often generalize successfully from just a single example, yet machine learning algorithms typically require tens or hundreds of examples to perform with similar accuracy. People can also use learned concepts in richer ways than conventional algorithms—for action, imagination, and explanation. We present a computational model that captures these human learning abilities for a large class of simple visual concepts: handwritten characters from the world’s alphabets. The model represents concepts as simple programs that best explain observed examples under a Bayesian criterion. On a challenging one-shot classification task, the model achieves human-level performance while outperforming recent deep learning approaches. We also present several “visual Turing tests” probing the model’s creative generalization abilities, which in many cases are indistinguishable from human behavior.

Machine-learning technology powers many aspects of modern society: from web searches to content filtering on social networks to recommendations on e-commerce websites, and it is increasingly present in consumer products such as cameras and smartphones. Machine-learning systems are used to identify objects in images, transcribe speech into text, match news items, posts or products with users’ interests, and select relevant results of search. Increasingly, these applications make use of a class of techniques called deep learning. Conventional machine-learning techniques were limited in their ability to process natural data in their raw form. For decades, constructing a pattern-recognition or machine-learning system required careful engineering and considerable domain expertise to design a feature extractor that transformed the raw data (such as the pixel values of an image) into a suitable internal representation or feature vector from which the learning subsystem, often a classifier, could detect or classify patterns in the input. Representation learning is a set of methods that allows a machine to be fed with raw data and to automatically discover the representations needed for detection or classification. Deep-learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but non-linear modules that each transform the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. With the composition of enough such transformations, very complex functions can be learned. For classification tasks, higher layers of representation amplify aspects of the input that are important for discrimination and suppress irrelevant variations. An image, for example, comes in the form of an array of pixel values, and the learned features in the first layer of representation typically represent the presence or absence of edges at particular orientations and locations in the image. The second layer typically detects motifs by spotting particular arrangements of edges, regardless of small variations in the edge positions. The third layer may assemble motifs into larger combinations that correspond to parts of familiar objects, and subsequent layers would detect objects as combinations of these parts. The key aspect of deep learning is that these layers of features are not designed by human engineers: they are learned from data using a general-purpose learning procedure. Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years. It has turned out to be very good at discovering intricate structures in high-dimensional data and is therefore applicable to many domains of science, business and government. In addition to beating records in image recognition and speech recognition, it has beaten other machine-learning techniques at predicting the activity of potential drug molecules, analysing particle accelerator data, reconstructing brain circuits, and predicting the effects of mutations in non-coding DNA on gene expression and disease. Perhaps more surprisingly, deep learning has produced extremely promising results for various tasks in natural language understanding, particularly topic classification, sentiment analysis, question answering and language translation. We think that deep learning will have many more successes in the near future because it requires very little engineering by hand, so it can easily take advantage of increases in the amount of available computation and data. New learning algorithms and architectures that are currently being developed for deep neural networks will only accelerate this progress.

The report introduces a constructive learning algorithm for recurrent neural networks, which modifies only the weights to output units in order to achieve the learning task. key words: recurrent neural networks, supervised learning Zusammenfassung. Der Report führt ein konstruktives Lernverfahren für rekurrente neuronale Netze ein, welches zum Erreichen des Lernzieles lediglich die Gewichte der zu den Ausgabeneuronen führenden Verbindungen modifiziert. Stichwörter: rekurrente neuronale Netze, überwachtes Lernen

Learning algorithms related to artificial neural networks and in particular for Deep Learning may seem to involve many bells and whistles, called hyperparameters. This chapter is meant as a practical guide with recommendations for some of the most commonly used hyper-parameters, in particular in the context of learning algorithms based on backpropagated gradient and gradient-based optimization. It also discusses how to deal with the fact that more interesting results can be obtained when allowing one to adjust many hyper-parameters. Overall, it describes elements of the practice used to successfully and efficiently train and debug large-scale and often deep multi-layer neural networks. It closes with open questions about the training difficulties observed with deeper architectures.

Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning. One aspect of the field receiving considerable attention is efficiently executing deep models in resource-constrained environments, such as mobile or embedded devices. This paper focuses on this problem, and proposes two new compression methods, which jointly leverage weight quantization and distillation of larger networks, called “teachers,” into compressed “student” networks. The first method we propose is called quantized distillation and leverages distillation during the training process, by incorporating distillation loss, expressed with respect to the teacher network, into the training of a smaller student network whose weights are quantized to a limited set of levels. The second method, differentiable quantization, optimizes the location of quantization points through stochastic gradient descent, to better fit the behavior of the teacher model. We validate both methods through experiments on convolutional and recurrent architectures. We show that quantized shallow students can reach similar accuracy levels to state-of-the-art full-precision teacher models, while providing up to order of magnitude compression, and inference speedup that is almost linear in the depth reduction. In sum, our results enable DNNs for resource-constrained environments to leverage architecture and accuracy advances developed on more powerful devices.