Question: Recognition of facial action units using sparse representation.

This paper introduces a real-time facial tracking system capable of maintaining a frame rate of 30 frames per second on an HP-9000 workstation, equipped with a framegrabber and a Canon VC-CI camera. The system's robustness allows it to track a person's face effectively, even as the individual moves freely within a room, engaging in activities such as walking, jumping, sitting, and standing. The development of this system incorporates three distinct modeling approaches. Firstly, a probabilistic model is employed to accurately describe the skin-color distributions of human faces, providing the necessary information for reliable tracking across diverse poses and viewing angles. This model is designed to be adaptable in real-time to accommodate variations in lighting and different individuals. Secondly, a motion model is utilized to estimate image motion and to define search windows, enhancing the system's tracking precision. Lastly, a camera model is integrated to predict and compensate for camera motion, ensuring stable tracking under dynamic conditions. The system's applications are wide-ranging, including tele-conferencing and human-computer interaction (HCI) scenarios, such as lip-reading and gaze tracking. The foundational principles underlying this system's development can be readily extended to other tracking challenges, such as the tracking of human hands.

**Question**: A Multi-Objective Evolutionary Algorithm for Scheduling in Cloud Computing

**Documents**: A Multi-Objective Evolutionary Algorithm for Task Scheduling in Cloud Computing
Cloud computing is a new paradigm in computing that promises pay-per-use and on-demand services to users. It provides computing resources and services over the Internet to computers and other devices on demand. The key challenge in cloud computing is task scheduling which plays a vital role in the efficient utilization of resources. In this paper, we propose a multi-objective evolutionary algorithm for task scheduling in cloud computing. The proposed algorithm aims to minimize the makespan, cost, and energy consumption while maximizing resource utilization. The algorithm is based on the Non-dominated Sorting Genetic Algorithm II (NSGA-II) and uses a tournament selection strategy, one-point crossover, and uniform mutation. The performance of the proposed algorithm is evaluated using CloudSim, a popular cloud simulation toolkit. The simulation results show that the proposed algorithm outperforms existing scheduling algorithms in terms of makespan, cost, and energy consumption.

**Similar Documents**: This paper presents a novel multi-objective evolutionary algorithm designed for task scheduling in the cloud computing environment. Cloud computing, as an emerging computing model, offers scalable, pay-as-you-go services accessible via the Internet to a broad range of devices. A

In this paper we introduce a template-based method for recognizing human actions called action MACH. Our approach is based on a maximum average correlation height (MACH) filter. A common limitation of template-based methods is their inability to generate a single template using a collection of examples. MACH is capable of capturing intra-class variability by synthesizing a single Action MACH filter for a given action class. We generalize the traditional MACH filter to video (3D spatiotemporal volume), and vector valued data. By analyzing the response of the filter in the frequency domain, we avoid the high computational cost commonly incurred in template-based approaches. Vector valued data is analyzed using the Clifford Fourier transform, a generalization of the Fourier transform intended for both scalar and vector-valued data. Finally, we perform an extensive set of experiments and compare our method with some of the most recent approaches in the field by using publicly available datasets, and two new annotated human action datasets which include actions performed in classic feature films and sports broadcast television.

Automatic emotion recognition is an interdisciplinary research field which deals with the algorithmic detection of human affect, e.g. anger or sadness, from a variety of sources, such as speech or facial gestures. Apart from the obvious usage for industry applications in human-robot interaction, acquiring the emotional state of a person automatically also is of great potential for the health domain, especially in psychology and psychiatry. Here, evaluation of human emotion is often done using oral feedback or questionnaires during doctor-patient sessions. However, this can be perceived as intrusive by the patient. Furthermore, the evaluation can only be done in a noncontinuous manner, e.g. once a week during therapy sessions. In contrast, using automatic emotion detection, the affect state of a person can be evaluated in a continuous non-intrusive manner, for example to detect early on-sets of depression. An additional benefit of automatic emotion recognition is the objectivity of such an approach, which is not influenced by the perception of the patient and the doctor. To reach the goal of objectivity, it is important, that the source of the emotion is not easily manipulable, e.g. as in the speech modality. To circumvent this caveat, novel approaches in emotion detection research the potential of using physiological measures, such as galvanic skin sensors or pulse meters. In this paper we outline a way of detecting emotion from brain waves, i.e., EEG data. While EEG allows for a continuous, real-time automatic emotion recognition, it furthermore has the charm of measuring the affect close to the point of emergence: the brain. Using EEG data for emotion detection is nevertheless a challenging task: Which features, EEG channel locations and frequency bands are best suited for is an issue of ongoing research. In this paper we evaluate the use of state of the art feature extraction, feature selection and classification algorithms for EEG emotion classification using data from the de facto standard dataset, DEAP. Moreover, we present results that help choose methods to enhance classification performance while simultaneously reducing computational complexity.

Humans modify facial expressions in order to mislead observers regarding their true emotional states. Being able to recognize the authenticity of emotional displays is notoriously difficult for human observers. Evidence in experimental psychology shows that discriminative facial responses are short and subtle. This suggests that such behavior would be easier to distinguish when captured in high resolution at an increased frame rate. We are proposing SASE-FE, the first dataset of genuine and deceptive facial expressions of emotions for automatic recognition1 . We show that overall the problem of recognizing deceptive facial expressions can be successfully addressed by learning spatio-temporal representations of the data. For this purpose, we propose a method that aggregates features along fiducial trajectories in a deeply learnt feature space. Interesting additional results show that on average it is easier to distinguish among genuine expressions than deceptive ones and that certain emotion pairs are more difficult to distinguish than others.

Face recognition algorithms commonly assume that face images are well aligned and have a similar pose -- yet in many practical applications it is impossible to meet these conditions. Therefore extending face recognition to unconstrained face images has become an active area of research. To this end, histograms of Local Binary Patterns (LBP) have proven to be highly discriminative descriptors for face recognition. Nonetheless, most LBP-based algorithms use a rigid descriptor matching strategy that is not robust against pose variation and misalignment. We propose two algorithms for face recognition that are designed to deal with pose variations and misalignment. We also incorporate an illumination normalization step that increases robustness against lighting variations. The proposed algorithms use descriptors based on histograms of LBP and perform descriptor matching with spatial pyramid matching (SPM) and Naive Bayes Nearest Neighbor (NBNN), respectively. Our contribution is the inclusion of flexible spatial matching schemes that use an image-to-class relation to provide an improved robustness with respect to intra-class variations. We compare the accuracy of the proposed algorithms against Ahonen's original LBP-based face recognition system and two baseline holistic classifiers on four standard datasets. Our results indicate that the algorithm based on NBNN outperforms the other solutions, and does so more markedly in presence of pose variations.

We have advanced an effort to develop vision based human understanding technologies for realizing human-friendly machine interfaces. Visual information, such as gender, age ethnicity, and facial expression play an important role in face-to-face communication. This paper addresses a novel approach for ethnicity classification with facial images. In this approach, the Gabor wavelets transformation and retina sampling are combined to extract key facial features, and support vector machines that are used for ethnicity classification. Our system, based on this approach, has achieved approximately 94% for ethnicity estimation under various lighting conditions.

Keywords: Eigenface, face Recognition, k-means clustering, principle component analysis (PCA)

Faces represent complex, multidimensional, meaningful visual stimuli an d developing a computational model for face recognition is difficult [42]. We present a hybrid neural network solution which compares favorably with other methods. The system combines local image sam pling, a self-organizing map neural network, and a convolutional neural network. The self-organizi ng map provides a quantization of the image samples into a topological space where inputs that are n e rby in the original space are also nearby in the output space, thereby providing dimensionality red uction and invariance to minor changes in the image sample, and the convolutional neural network prov ides for partial invariance to translation, rotation, scale, and deformation. The convolutional net work extracts successively larger features in a hierarchical set of layers. We present results using the Karhunen -Loève transform in place of the self-organizing map, and a multi-layer perceptron in place of the con volutional network. The Karhunen-Loève transform performs almost as well (5.3% error versus 3 .8%). The multi-layer perceptron performs very poorly (40% error versus 3.8%). The method is cap able of rapid classification, requires only fast, approximate normalization and preprocessing, and cons istently exhibits better classification performance than the eigenfaces approach [42] on the database consider ed as the number of images per person in the training database is varied from 1 to 5. With 5 imag es per person the proposed method and eigenfaces result in 3.8% and 10.5% error respectively. The recogni zer provides a measure of confidence in its output and classification error approaches zero when rejecting as few as 10% of the examples. We use a database of 400 images of 40 individuals which cont ains quite a high degree of variability in expression, pose, and facial details. We analyze computati onal complexity and discuss how new classes could be added to the trained recognizer.

Predicting face attributes in the wild is challenging due to complex face variations. We propose a novel deep learning framework for attribute prediction in the wild. It cascades two CNNs, LNet and ANet, which are fine-tuned jointly with attribute tags, but pre-trained differently. LNet is pre-trained by massive general object categories for face localization, while ANet is pre-trained by massive face identities for attribute prediction. This framework not only outperforms the state-of-the-art with a large margin, but also reveals valuable facts on learning face representation. (1) It shows how the performances of face localization (LNet) and attribute prediction (ANet) can be improved by different pre-training strategies. (2) It reveals that although the filters of LNet are fine-tuned only with image-level attribute tags, their response maps over entire images have strong indication of face locations. This fact enables training LNet for face localization with only image-level annotations, but without face bounding boxes or landmarks, which are required by all attribute recognition works. (3) It also demonstrates that the high-level hidden neurons of ANet automatically discover semantic concepts after pre-training with massive face identities, and such concepts are significantly enriched after fine-tuning with attribute tags. Each attribute can be well explained with a sparse linear combination of these concepts.

We analyze the usage of Speeded Up Robust Features (SURF) as local descriptors for face recognition. The effect of different feature extraction and viewpoint consistency constrained matching approaches are analyzed. Furthermore, a RANSAC based outlier removal for system combination is proposed. The proposed approach allows to match faces under partial occlusions, and even if they are not perfectly aligned or illuminated. Current approaches are sensitive to registration errors and usually rely on a very good initial alignment and illumination of the faces to be recognized. A grid-based and dense extraction of local features in combination with a block-based matching accounting for different viewpoint constraints is proposed, as interest-point based feature extraction approaches for face recognition often fail. The proposed SURF descriptors are compared to SIFT descriptors. Experimental results on the AR-Face and CMU-PIE database using manually aligned faces, unaligned faces, and partially occluded faces show that the proposed approach is robust and can outperform current generic approaches.

The Face Expression Based Emoticon Identification (FEBEI) system is an open source extension to the Tracker.js framework which converts a human facial expression to the best matching emoticon. The contribution of this project was to build this robust classifier which can identify facial expression in real time without any reliance on an external server or computation node. An entirely client-side JavaScript implementation has clear privacy benefits as well as the avoidance of any lag inherent in uploading and downloading images. We accomplished this by utilizing several computationally efficient methods. Tracking.js provided a Viola Jones based face detector which we used to pass facial images to our own implementation of an eigenemotion detection system which was trained to distinguish between happy and angry faces. We have implemented a similar eigenface classifier in python and have trained a Convoluted Neural Network (CNN) to classify emotions to provide a comparative view of its advantages. We aim to make FEBEI easily extendable so that the developer community will be able to add classifiers for more emoticon.


Question: Recognition of facial action units using sparse representation.

This paper offers an extensive review of face detection techniques, emphasizing the critical role these methods play in intelligent vision-based human-computer interaction systems. Face detection is a foundational step for various face processing tasks, including recognition, tracking, pose estimation, and expression analysis. Despite the advancements, most existing algorithms presuppose that faces have already been located within images or image sequences. To create comprehensive automated systems capable of analyzing face images, robust and efficient face detection mechanisms are indispensable. The core challenge of face detection lies in accurately identifying all face regions in an image, irrespective of their orientation, position, lighting, size, shape, color, or texture. The paper meticulously categorizes and evaluates a wide array of face detection algorithms developed to address these challenges in single images. It also delves into ancillary topics such as data acquisition, performance evaluation metrics, and benchmarking practices. After a thorough analysis of these methods and their limitations, the paper concludes by highlighting several promising avenues for future research and development in face detection technology.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An efficient evolutionary algorithm for economic dispatch with valve-point effects
This paper presents a new evolutionary algorithm for solving the economic dispatch problem with valve-point effects. The proposed algorithm, referred to as the Enhanced Genetic Algorithm (EGA), incorporates several improvements over the standard genetic algorithm. These enhancements include adaptive mutation, elitism, and a tournament selection mechanism. The algorithm is designed to efficiently handle the non-convex characteristics of the economic dispatch problem, which are typically caused by valve-point effects in thermal power plants. The EGA is tested on several benchmark systems, including systems with valve-point effects, and its performance is compared to other state-of-the-art methods. The results demonstrate the superior performance of the EGA in terms of solution quality and computational efficiency.

**Similar Documents**: This article introduces an innovative evolutionary algorithm, termed the Enhanced Genetic Algorithm (EGA), tailored specifically for addressing the economic dispatch problem in the presence of valve-point effects. The EGA introduces several modifications to the traditional genetic algorithm framework, such as adaptive mutation, an elitism strategy, and a tournament selection process. These enhancements are aimed at improving the algorithm's ability to navigate the non-convex landscape often encountered in economic dispatch problems due to valve-point effects in thermal power plants. The EGA is rigorously evaluated on a variety of benchmark systems, including those characterized by valve-point effects, and its performance is contrasted against leading-edge methods. The experimental outcomes underscore the EGA's

Facial expressions are an important way through which humans interact socially. Building a system capable of automatically recognizing facial expressions from images and video has been an intense field of study in recent years. Interpreting such expressions remains challenging and much research is needed about the way they relate to human affect. This paper presents a general overview of automatic RGB, 3D, thermal and multimodal facial expression analysis. We define a new taxonomy for the field, encompassing all steps from face detection to facial expression recognition, and describe and classify the state of the art methods accordingly. We also present the important datasets and the bench-marking of most influential methods. We conclude with a general discussion about trends, important questions and future lines of research.

We propose a modeland exemplar-based approach for face recognition. This problem has been previously tackled using either models or exemplars, with limited success. Our idea uses models to synthesize many more exemplars, which are then used in the learning stage of a face recognition system. To demonstrate this, we develop a statistical shape-fromshading model to recover face shape from a single image, and to synthesize the same face under new illumination. We then use this to build a simple and fast classifier that was not possible before because of a lack of training data.

The Face Expression Based Emoticon Identification (FEBEI) system is an open source extension to the Tracker.js framework which converts a human facial expression to the best matching emoticon. The contribution of this project was to build this robust classifier which can identify facial expression in real time without any reliance on an external server or computation node. An entirely client-side JavaScript implementation has clear privacy benefits as well as the avoidance of any lag inherent in uploading and downloading images. We accomplished this by utilizing several computationally efficient methods. Tracking.js provided a Viola Jones based face detector which we used to pass facial images to our own implementation of an eigenemotion detection system which was trained to distinguish between happy and angry faces. We have implemented a similar eigenface classifier in python and have trained a Convoluted Neural Network (CNN) to classify emotions to provide a comparative view of its advantages. We aim to make FEBEI easily extendable so that the developer community will be able to add classifiers for more emoticon.

In this paper we present a comprehensive and critical survey of face detection algorithms. Face detection is a necessary first-step in face recognition systems, with the purpose of localizing and extracting the face region from the background. It also has several applications in areas such as content-based image retrieval, video coding, video conferencing, crowd surveillance, and intelligent human–computer interfaces. However, it was not until recently that the face detection problem received considerable attention among researchers. The human face is a dynamic object and has a high degree of variability in its apperance, which makes face detection a difficult problem in computer vision. A wide variety of techniques have been proposed, ranging from simple edge-based algorithms to composite high-level approaches utilizing advanced pattern recognition methods. The algorithms presented in this paper are classified as either feature-based or image-based and are discussed in terms of their technical approach and performance. Due to the lack of standardized tests, we do not provide a comprehensive comparative evaluation, but in cases where results are reported on common datasets, comparisons are presented. We also give a presentation of some proposed applications and possible application areas. c © 2001 Academic Press

In this paper, we describe a Convolutional Neural Network (CNN) approach to realtime emotion detection. We utilize data from the Extended Cohn-Kanade dataset, Japanese Female Facial Expression data set, and our own custom images in order to train this model, and apply preprocessing steps to improve performance. We re-train a LeNet and AlexNet implementation, both of which perform with above 97% accuracy. Qualitative analysis of real-time images shows that the above models perform reasonably well at classifying facial expressions, but not as well as the quantitative results would indicate.

Stochastic modelling of non-stationary vector timeseries based on HMMs has been very successful for speech applications [5]. Recently it has been applied to a range of image recognition problems [7, 9]. Previously reported work [6] has investigated the use of HMMs to model human faces for identi cation purposes. Faces can be intuitively divided into regions such as the mouth, eyes, nose, etc., and these regions can be associated with the states of an HMM. The identi cation performance of a top-bottom HMM compares favourably with some of the well-known algorithms, for example eigenfaces as detailed in [8]. However, the HMM parameterisation in the work presented so far was arrived at by subjective intuition. This paper presents experimental results which show how identi cation rates vary with HMM parameters, and which indicate the most sensible choice of parameters. The paper is organised as follows: section 2 gives an overview of the HMM-based approach; section 3 details the training and recognition processes; section 4 describes the experimental setup; section 5 presents the identi cation results; section 6 concludes the paper.

In this paper, weinvestigaterecognition of humanfaces in a meetingroom. The major challenges of identifying humanfacesin this environmentincludelow quality of input images,poor illumination,unrestrictedheadposesand continuouslychangingfacial expressionsandocclusion.In order to addresstheseproblemswe proposea novel algorithm, DynamicSpaceWarping (DSW).Thebasic idea of the algorithm is to combinelocal features under certain spatial constraints. We compare DSWwith the eigenface approachondatacollectedfromvariousmeetings.Wehave testedboth front and profile face imagesand imageswith two stagesof occlusion.Theexperimentalresultsindicate thattheDSWapproachoutperformstheeigenfaceapproach in bothcases.

Gender Recognition is important in different commercial and law enforcement applications. In this paper we have proposed a gender recognition system through facial images. We have used a different technique that involves Bandlet Transform instead of previously used Wavelet Transform, which is a multi-resolution technique and more efficiently provides the edges of images, and then mean is combined to create the feature vectors of the images. To classify the images for gender, we have used fuzzy c mean clustering. Experimental results have shown that average 97.1% accuracy have been achieved using this technique when SUMS database was used and 93.3% was achieved when FERET database was used. Keywords----Bandlet, Gender Recognition, Fuzzy C-mean, Multi Resolution.

In this paper, we describe the acquisition and contents of a large-scale Chinese face database: the CAS-PEAL face database. The goals of creating the CAS-PEAL face database include the following: 1) providing the worldwide researchers of face recognition with different sources of variations, particularly pose, expression, accessories, and lighting (PEAL), and exhaustive ground-truth information in one uniform database; 2) advancing the state-of-the-art face recognition technologies aiming at practical applications by using off-the-shelf imaging equipment and by designing normal face variations in the database; and 3) providing a large-scale face database of Mongolian. Currently, the CAS-PEAL face database contains 99 594 images of 1040 individuals (595 males and 445 females). A total of nine cameras are mounted horizontally on an arc arm to simultaneously capture images across different poses. Each subject is asked to look straight ahead, up, and down to obtain 27 images in three shots. Five facial expressions, six accessories, and 15 lighting changes are also included in the database. A selected subset of the database (CAS-PEAL-R1, containing 30 863 images of the 1040 subjects) is available to other researchers now. We discuss the evaluation protocol based on the CAS-PEAL-R1 database and present the performance of four algorithms as a baseline to do the following: 1) elementarily assess the difficulty of the database for face recognition algorithms; 2) preference evaluation results for researchers using the database; and 3) identify the strengths and weaknesses of the commonly used algorithms.

We develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face, under varying illumination but fixed pose, lie in a 3D linear subspace of the high dimensional image space—if the face is a Lambertian surface without shadowing. However, since faces are not truly Lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. Rather than explicitly modeling this deviation, we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation. Our projection method is based on Fisher’s Linear Discriminant and produces well separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expressions. The Eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed “Fisherface” method has error rates that are lower than those of the Eigenface technique for tests on the Harvard and Yale Face Databases. Index Terms —Appearance-based vision, face recognition, illumination invariance, Fisher’s linear discriminant. —————————— ✦ ——————————


Question: A model-based path planning technique for autonomous vehicles in a dynamic environment.

This paper introduces an innovative technique for constrained path planning, utilizing a specialized search space that optimally represents viable routes. The methodology encodes paths through connections between states, focusing exclusively on feasible and proximate links. With the development of this search space, we are able to methodically produce a nearly minimal collection of spatially unique path elements. This collection not only delineates the local interconnectivity of constrained movements but also removes unnecessary duplications. These path elements serve as the foundation for a heuristic search algorithm, leading to a highly efficient path planner at the selected resolution. Furthermore, we explore the broad spectrum of space and terrestrial robotics applications where this motion planning strategy can be particularly advantageous, including scenarios for self-driving cars navigating dynamic environments.

Path planning has important applications in many areas, for example industrial robotics, autonomous systems, virtual prototyping, and computer-aided drug design. This thesis presents a new framework for developing and evaluating path planning algorithms. The framework is named CoPP (Components for Path Planning). It consists of loosely coupled and reusable components that are useful for building path planning applications. The framework is especially designed to make it easy to do fair comparisons between different path planning algorithms. CoPP is also designed to allow almost any user-defined moving system. The default type of moving system is a robot class, which is capable of describing tree-like kinematic chains. Additional features of this robot class are: joint couplings, numerical or closed-form inverse kinematics, and hierarchical robot representations. The last feature is useful when planning for complex systems like a mobile platform equipped with an arm and a hand. During the last six years, Rapidly-exploring Random Trees (RRTs) have become a popular framework for developing randomized path planning algorithms. This thesis presents a method for augmenting bidirectional RRT-planners with local trees. For problems where the solution trajectory has to pass through several narrow passages, local trees help to reduce the required planning time. To reduce the work needed for programming of industrial robots, it is desirable to allow task specifications at a very high level, leaving it up to the robot system to figure out what to do. Here we present a fast and flexible pick-and-place planner. Given an object that has to be moved to another position, the planner chooses a suitable grasp of the object and finds motions that bring the object to the desired position. The planner can also handle constraints on, e.g., the orientation of the manipulated object. For planning of pick-and-place tasks it is necessary to choose a grasp suitable to the task. Unless the grasp is given, some sort of grasp planning has to be performed. This thesis presents a fast grasp planner for a threefingered robot hand. The grasp planner could be used in an industrial setting, where a robot is to pick up irregularly shaped objects from a conveyor belt. In conjunction with grasp planning, a new method for evaluating grasp stability is presented.

The paper describes how a high-level abstract world model can be used to support the decision-making process of an autonomous driving system. The approach uses a hierarchical world model and distinguishes between a low-level model for the trajectory planning and a high-level model for solving the traffic coordination problem. The abstract world model used in the CyberCars-2 project is presented. It is based on a topological lane segmentation and introduces relations to represent the semantic context of the traffic scenario. This makes it much easier to realize a consistent and complete driving control system, and to analyze, evaluate and simulate such a system.

Model-based dual heuristic dynamic programming (MB-DHP) is a popular approach in approximating optimal solutions in control problems. Yet, it usually requires offline training for the model network, and thus resulting in extra computational cost. In this brief, we propose a model-free DHP (MF-DHP) design based on finite-difference technique. In particular, we adopt multilayer perceptron with one hidden layer for both the action and the critic networks design, and use delayed objective functions to train both the action and the critic networks online over time. We test both the MF-DHP and MB-DHP approaches with a discrete time example and a continuous time example under the same parameter settings. Our simulation results demonstrate that the MF-DHP approach can obtain a control performance competitive with that of the traditional MB-DHP approach while requiring less computational resources.

We cast the partially observable control problem as a fully observable underactuated stochastic control problem in belief space and apply standard planning and control techniques. One of the difficulties of belief space planning is modeling the stochastic dynamics resulting from unknown future observations . The core of our proposal is to define deterministic beliefsystem dynamics based on an assumption that the maximum likelihood observation (calculated just prior to the observation) is always obtained. The stochastic effects of future observation s are modeled as Gaussian noise. Given this model of the dynamics, two planning and control methods are applied. In the first, linear quadratic regulation (LQR) is applied to generate policies in the belief space. This approach is shown to be optimal for linearGaussian systems. In the second, a planner is used to find locally optimal plans in the belief space. We propose a replanning approach that is shown to converge to the belief space goal in a finite number of replanning steps. These approaches are characterized in the context of a simple nonlinear manipulation problem where a planar robot simultaneously locates and grasps an object.

In this paper we present a model predictive control algorithm designed for optimizing non-linear systems subject to complex cost criteria. The algorithm is based on a stochastic optimal control framework using a fundamental relationship between the information theoretic notions of free energy and relative entropy. The optimal controls in this setting take the form of a path integral, which we approximate using an efficient importance sampling scheme. We experimentally verify the algorithm by implementing it on a Graphics Processing Unit (GPU) and apply it to the problem of controlling a fifth-scale Auto-Rally vehicle in an aggressive driving task.

For the Oxford Aerial Tracking System (OATS) we are developing a robot helicopter that can track moving ground objects. Here we describe algorithms for the device to perform path planning and trajectory prediction. The path planner uses superquadratic potential fields and incorporates a height change mechanism that is triggered where necessary and in order to avoid local minima traps. The goal of the trajectory prediction system is to reliably predict the target trajectory during occlusion caused by ground obstacles. For this we use two artificial neural networks in parallel whose retraining is automatically triggered if major changes in the target behaviour pattern are detected. Simulation results for both systems are presented.

In this paper, a practical real-time path planning and robot navigation algorithm for a non-holonomic indoor mobile robot based on visual servoing is implemented. The proposed algorithm is divided into three parts; the first part uses Multi-Stencils Fast Marching (MSFM) as a path planning method. But the generated path results from fast marching methods when used directly, not guaranteed to be safe and smooth. Subsequently, the robot can touch corners, walls and obstacles. The proposed algorithm uses image processing methods to solve this problem. The second part estimates the position and orientation of the robot, from the visual information, to follow the desired path with avoiding obstacles. The third part proposes a decentralized PD-like Fuzzy Logic Controller (FLC) to keep up the robot on the desired path. Experimental results show that the developed design is valid to estimate shortest-path by avoiding obstacles and able to guide the robot to follow the path in real-time.

This paper introduces a new integrated algorithm to achieve the complete coverage path planning (CCPP) task for the mobile robot in a given obstacles-included terrain. The algorithm combines the cellular decomposition approach and the chaotic Standard map together to design the coverage procedure. The cellular decomposition approach decompose the target region into several rectangular feasible sub-regions. Then the chaotic Standard map in the full mapping state produces the complete coverage trajectories inside the feasible sub-regions, and the connection trajectories between two adjacent feasible sub-regions. Compared with the general cellular decomposition method, the proposed integrated algorithm needs no designated start point and goal point to link two adjacent sub-regions. The planned trajectories demonstrate a good distribution characteristics with regard to completeness and evenness. No obstacles-avoidance method and boundaries detection are needed in the coverage procedure.

We study an algorithm that allows a vehicle to autonomously change lanes in a safe but personalized fashion without the driver's explicit initiation (e.g. activating the turn signals). Lane change initiation in autonomous driving is typically based on subjective rules, functions of the positions and relative velocities of surrounding vehicles. This approach is often arbitrary, and not easily adapted to the driving style preferences of an individual driver. Here we propose a data-driven modeling approach to capture the lane change decision behavior of human drivers. We collect data with a test vehicle in typical lane change situations and train classifiers to predict the instant of lane change initiation with respect to the preferences of a particular driver. We integrate this decision logic into a model predictive control (MPC) framework to create a more personalized autonomous lane change experience that satisfies safety and comfort constraints. We show the ability of the decision logic to reproduce and differentiate between two lane changing styles, and demonstrate the safety and effectiveness of the control framework through simulations.

Mobile robots are increasingly populating our human environments. To interact with humans in a socially compliant way, these robots need to understand and comply with mutually accepted rules. In this paper, we present a novel approach to model the cooperative navigation behavior of humans. We model their behavior in terms of a mixture distribution that captures both the discrete navigation decisions, such as going left or going right, as well as the natural variance of human trajectories. Our approach learns the model parameters of this distribution that match, in expectation, the observed behavior in terms of user-defined features. To compute the feature expectations over the resulting high-dimensional continuous distributions, we use Hamiltonian Markov chain Monte Carlo sampling. Furthermore, we rely on a Voronoi graph of the environment to efficiently explore the space of trajectories from the robot’s current position to its target position. Using the proposed model, our method is able to imitate the behavior of pedestrians or, alternatively, to replicate a specific behavior that was taught by tele-operation in the target environment of the robot. We implemented our approach on a real mobile robot and demonstrate that it is able to successfully navigate in an office environment in the presence of humans. An extensive set of experiments suggests that our technique outperforms state-of-the-art methods to model the behavior of pedestrians, which makes it also applicable to fields such as behavioral science or computer graphics.


Question: A model-driven path planning method for autonomous vehicles in a dynamic setting.

This document explores a path planning technique that incorporates a dynamic vehicle model, aiming to enhance the realism and feasibility of generated paths for self-driving cars at higher speeds. Previous research, which primarily relied on basic kinematic models, has been shown to produce paths that are viable only at very low speeds. When navigating at higher speeds, the vehicle's deviation from the planned trajectory becomes significant, rendering the path unsuitable for mission accomplishment. To address this issue and create a seamless integration between planning and navigation phases, we introduce a path planning algorithm that utilizes a dynamic vehicle model. This approach ensures that the planned paths remain realistic and effective even at higher speeds, bridging the gap between theoretical planning and practical navigation scenarios.

One of the most important areas of research on mobile robots is that of their moving from one point in a space to the other and that too keeping aloof from the different objects in their path i. e. real time obstacle detection. This is the basic problem of path planning through a continuous domain. For this a large number of algorithms have been proposed of which only a few are really good as far as local and global path planning are concerned as to some extent a trade of has to be made between the efficiency of the algorithm and its accuracy. In this project an integrated approach for both local as well as global path planning of a robot has been proposed. The primary algorithm that has been used for path planning is the artificial Potential field approach [1] and a* search algorithm has been used for finding the most optimum path for the robot. Obstacle detection for collision avoidance (a high level planning problem) can be effectively done by doing complex robot operations in real time and distributing the whole problem between different control levels. This project proposes the artificial potential field algorithm not only for static but also for moving obstacles using real time potential field values by creating sub-goals which eventually lead to the main goal of the most optimal complete path found by the A* search algorithm. Apart from these scan line and convex hull techniques have been used to improve the robustness of the algorithm. To some extent the shape and size of a robot has also been taken into consideration. The effectiveness of these algorithms has been

Robot navigation has multi-spectral applications across a plethora of industries. Being one of the fastest evolving field of Artificial intelligence, it has garnered a lot of speculation and the problem of efficient Robot navigation has become a talking point in the “Artificial Intelligence research community”. Robot Navigation can be thought of as a collection of sub-problems that when solved efficiently will (most likely) give the complete and the most efficient solution. While the main Navigation problem can be divided into numerous smaller ones some of the prevalent (sub)-tasks associated with Robot Navigation are Path Planning, Collision Prevention, Search Algorithm and an appropriate pictorial (graphical) representation of the given environment. Path Planning gives us the most optimal path finding approach while keeping the dynamism of the environment and the objects in mind. Collision Prevention ensures the Robot reaches its goal without colliding with any of the objects present in the environment. Collision Prevention techniques' vitality increases considerably when the number of objects in the input space is large (>10, approximately). Search Algorithms are the functional units of this problem as they determine the path the Robot follows while traveling to its destination. Since the environment greatly affects the choice of technique for all the other tasks it is vital to have a clear, concise and a non-ambiguous representation of the same. The main aim of the paper is to summarise the development of various techniques in the multiple domains of the Robot Navigation problem in the last fifteen years (2000-2015) while stating the scope and the restriction of each of them.

Elastic bands are proposed as the basis for a new framework to close the gap between global path planning and real-time sensor-based robot control. An elastic band is a deformable collision-free path. The initial shape of the elastic is the free path generated by a planner. Subjected to artificial forces, the elastic band deforms in real time to a short and smooth path that maintains clearance from the obstacles. The elastic continues to deform as changes in the environment are detected by sensors, enabling the robot to accommodate uncertainties and react to unexpected and moving obstacles. While providing a tight connection between the robot and its environment, the elastic band preserves the global nature of the planned path. This paper outlines the framework and discusses an efficient implementation based on bubbles.

Future driver assistance systems are likely to use a multisensor approach with heterogeneous sensors for tracking dynamic objects around the vehicle. The quality and type of data available for a data fusion algorithm depends heavily on the sensors detecting an object. This article presents a general framework which allows the use sensor specific advantages while abstracting the specific details of a sensor. Different tracking models are used depending on the current set of sensors detecting the object. A sensor independent algorithm for classifying objects regarding their current and past movement state is presented. The described architecture and algorithms have been successfully implemented in Tartan racingpsilas autonomous vehicle for the urban grand challenge. Results are presented and discussed.

This is the first installment of a two-part tutorial. The goal of the first part is to give the reader a basic understanding of the technical issues and types of approaches to solving the basic path planning or obstacle avoidance problem. The second installment will cover more advanced issues, including feedback, differential constraints, and uncert ainty. Note that is a brieftutorial, rather than a comprehensive surveyof methods. For the latter, consult recent textbooks [4], [9].

Self-driving cars do not sound like a part of science fiction movies anymore. With most of the automobile giants committed to launching their autonomous cars as soon as possible, it is expected that self-driving cars will hit the road in a very short time. Increasing the penetration factor of autonomous vehicles will need algorithms to control them in an efficient way in different scenarios. One such scenario is intersection management (IM). We propose an intuitive heuristic to resolve space–time conflicts in vehicle’s trip, enabling vehicles to cross the intersection safely, and with minimum delay. The intersection is modeled as a group of conflict points where intersecting internal links cross each other. For safe and efficient scheduling of vehicles, heuristics are proposed separately for: 1) entering in approach lane; 2) safe traversal in the approach lane; 3) safe and efficient crossing of intersection; and (4) safe departure along depart lane. Vehicle scheduling is done using a three-leveled heuristic in which each level serves a distinct purpose and together guarantee collision-free passing of vehicles. Performance evaluation of the proposed scheme is done in Simulation of Urban MObility simulator and a comparative analysis is done with three other IM schemes. Experiments show that the proposed scheme gives a smaller average trip delay of vehicles compared with other IM schemes.

This paper proposes a novel path prediction algorithm for progressing one step further than the existing works focusing on single target path prediction. In this paper, we consider moving dynamics of co-occurring objects for path prediction in a scene that includes crowded moving objects. To solve this problem, we first suggest a two-layered probabilistic model to find major movement patterns and their cooccurrence tendency. By utilizing the unsupervised learning results from the model, we present an algorithm to find the future location of any target object. Through extensive qualitative/quantitative experiments, we show that our algorithm can find a plausible future path in complex scenes with a large number of moving objects.

We propose a novel approach to constrained path planning that is based on a special search space which efficiently encodes feasible paths. The paths are encoded implicitly as connections between states, but only feasible and local connections are included. Once this search space is developed, we systematically generate a near-minimal set of spatially distinct path primitives. This set expresses the local connectivity of constrained motions and also eliminates redundancies. The set of primitives is used to define heuristic search, and thereby create a very efficient path planner at the chosen resolution. We also discuss a wide variety of space and terrestrial robotics applications where this motion planner can be especially useful.

The Teaching-Learning-Based Optimization (TLBO) algorithm has been proposed in recent years. It is a new swarm intelligence optimization algorithm simulating the teaching-learning phenomenon of a classroom. In this paper, a novel global path planning method for mobile robots is presented, which is based on an improved TLBO algorithm called Nonlinear Inertia Weighted Teaching-Learning-Based Optimization (NIWTLBO) algorithm in our previous work. Firstly, the NIWTLBO algorithm is introduced. Then, a new map model of the path between start-point and goal-point is built by coordinate system transformation. Lastly, utilizing the NIWTLBO algorithm, the objective function of the path is optimized; thus, a global optimal path is obtained. The simulation experiment results show that the proposed method has a faster convergence rate and higher accuracy in searching for the path than the basic TLBO and some other algorithms as well, and it can effectively solve the optimization problem for mobile robot global path planning.

Wright Laboratory, at Tyndall AFB, Florida, has contracted the University ofFlorida to develop autonomous navigation systems for a variety ofrobotic vehicles, capable ofperforming tasks associated with the location and removal of bombs and mines. One ofthe tasks involves surveying closed target ranges for unexploded buried munitions. Accuracy in path following is critical to the task. There are hundreds of acres that currently require surveying. The sites are typically divided into regions, where each mission can take up to 4.5 hours. These sites are usually surveyed along parallel rows. By improving the accuracy ofpath following, the distance betweenthe rows can be increased to nearly the detection width ofthe ground penetrating sensors, resulting in increased acreage surveyed per mission. This paper evaluates a high-level PID and a pure pursuit steering controller. The controllers were combined into a weighted solution so that the desirable features of each controller is preserved. This strategy was demonstrated in simulation and implemented on a Navigation Test Vehicle (NTV). For a test path ofvarying curvature, the average lateral control error was 2 cm at a vehicle speed of 1.34 mIs.


Question: A model-based path planning technique for autonomous vehicles in dynamic surroundings.

This document outlines a novel approach to autonomous local path planning for self-driving cars operating in dynamic environments, utilizing a genetic algorithm for optimization. The focus is on devising a path-planning algorithm that not only ensures a feasible route but also optimizes it in terms of efficiency. Key objectives include minimizing the distance of the path and the number of turns required for navigation. The proposed methodology is designed to accommodate the unrestricted movement of self-driving vehicles in all directions, thereby enabling effective path planning in complex and ever-changing environments. This approach is particularly suited for scenarios where the search space is intricate and dynamic, requiring a robust and adaptable path-planning solution.

Conflict-free routing is an important problem to be addressed in AGV Systems, in particular when they are bi-directional. This paper considers the problem of routing AGVs in the presence of contingencies. The importance of this work stems from the realization that path-planning approaches are ill suited to AGVSs subject to contingencies. In this paper, a two stage closed-loop control strategy is proposed. In the first control stage, a pre-planning method proposed by Kim and Tanchoco (1991) is used to establish the fastest conflict-free routes for AGVs. The objective of the second stage is the avoidance of deadlocks in the presence of interruptions while maintaining the planned routes. Copyright © 2005 IFAC

Grids with blocked and unblocked cells are often used to represent continuous 2D and 3D environments in robotics and video games. The shortest paths formed by the edges of 8neighbor 2D grids can be up to ≈ 8% longer than the shortest paths in the continuous environment. Theta* typically finds much shorter paths than that by propagating information along graph edges (to achieve short runtimes) without constraining paths to be formed by graph edges (to find short “any-angle” paths). We show in this paper that the shortest paths formed by the edges of 26-neighbor 3D grids can be ≈ 13% longer than the shortest paths in the continuous environment, which highlights the need for smart path planning algorithms in 3D. Theta* can be applied to 3D grids in a straight-forward manner, but it performs a line-of-sight check for each unexpanded visible neighbor of each expanded vertex and thus it performs many more line-of-sight checks per expanded vertex on a 26-neighbor 3D grid than on an 8-neighbor 2D grid. We therefore introduce Lazy Theta*, a variant of Theta* which uses lazy evaluation to perform only one line-of-sight check per expanded vertex (but with slightly more expanded vertices). We show experimentally that Lazy Theta* finds paths faster than Theta* on 26-neighbor 3D grids, with one order of magnitude fewer line-of-sight checks and without an increase in path length.

In this paper, we propose an efficient vehicle trajectory prediction framework based on recurrent neural network. Basically, the characteristic of the vehicle's trajectory is different from that of regular moving objects since it is affected by various latent factors including road structure, traffic rules, and driver's intention. Previous state of the art approaches use sophisticated vehicle behavior model describing these factors and derive the complex trajectory prediction algorithm, which requires a system designer to conduct intensive model optimization for practical use. Our approach is data-driven and simple to use in that it learns complex behavior of the vehicles from the massive amount of trajectory data through deep neural network model. The proposed trajectory prediction method employs the recurrent neural network called long short-term memory (LSTM) to analyze the temporal behavior and predict the future coordinate of the surrounding vehicles. The proposed scheme feeds the sequence of vehicles' coordinates obtained from sensor measurements to the LSTM and produces the probabilistic information on the future location of the vehicles over occupancy grid map. The experiments conducted using the data collected from highway driving show that the proposed method can produce reasonably good estimate of future trajectory.

The last few decades witnessed the birth and growth of a new sensibility to transportation efficiency. In particular, the need for efficient and improved people and goods mobility pushed researchers to address the problem of intelligent transportation systems. This paper surveys the most advanced approaches to the (partial) customization of road following task, using on-board systems based on artificial vision. The functionalities of lane detection, obstacle detection and pedestrian detection are described and classified, and their possible application on future road vehicles is discussed.

In this paper, we present a heuristic planning approach for guarding a valuable asset by a team of autonomous unmanned surface vehicles (USVs) operating in a continuous state-action space. The team's objective is to maximize the amount of time it takes an intruder boat to reach the asset. The team must cooperatively deal with uncertainty about which boats are actual intruders, employ active blocking to slow down intruders' movement towards the asset, and intelligently distribute themselves around the target to optimize future guarding opportunities. Our planner incorporates a market-based algorithm for allocating tasks to individual USVs by forward-simulating the mission and assigning estimated utilities to candidate task-allocation plans. The planner can be automatically adapted to a specific mission by optimizing the behaviors used to fulfil individual tasks. We present detailed simulation results that demonstrate the effectiveness of our approach.

In this paper, we propose an autonomous navigation for Unmanned Ground Vehicles (UGVs) by a path selection method based on local features of terrain in unknown outdoor rough terrain environment. The correlation between a local terrain feature obtained from a path and a value of the path obtained from the global path planning is extracted in advance. When UGV comes to a branch while it is running, the value of path is estimated using the correlation with local terrain feature. Thereby, UGV navigation is performed by path selection under the criterion of shortest time in unknown outdoor environment. We experimented on a simulator and confirmed that the proposed method can select more effective paths in comparison with a simple path selection method.

In this work, we examine the problem of constructing and maintaining a map of an autonomous vehicle’s environment for the purpose of navigation, using evidential reasoning. The inherent uncertainty in the origin of measurements of sensors demands a probabilistic approach to processing, or fusing, the new sensory information to build an accurate map. In this paper, the map is based on a two-dimensional (2-D) occupancy grid. The sensor readings are “fused” into the map using the Dempster–Shafer inference rule. This evidential approach with its multivalued hypotheses allows quantitative analysis of the quality of the data. The map building system is experimentally evaluated using sonar data from real environments.

The autonomous vehicle, as an emerging and rapidly growing field, has received extensive attention for its futuristic driving experiences. Although the fast developing depth sensors and machine learning methods have given a huge boost to self-driving research, existing autonomous driving vehicles do meet with several avoidable accidents during their road testings. The major cause is the misunderstanding between self-driving systems and human drivers. To solve this problem, we propose a humanlike driving system in this paper to give autonomous vehicles the ability to make decisions like a human. In our method, a convolutional neural network model is used to detect, recognize, and abstract the information in the input road scene, which is captured by the on-board sensors. And then a decision-making system calculates the specific commands to control the vehicles based on the abstractions. The biggest advantage of our work is that we implement a decision-making system which can well adapt to real-life road conditions, in which a massive number of human drivers exist. In addition, we build our perception system with only the depth information, rather than the unstable RGB data. The experimental results give a good demonstration of the efficiency and robustness of the proposed method.

Autonomous vehicles promise numerous improvements to vehicular traffic: an increase in both highway capacity and traffic flow because of faster response times, less fuel consumption and pollution thanks to more foresighted driving, and hopefully fewer accidents thanks to collision avoidance systems. In addition, drivers can save time for more useful activities. In order for these vehicles to safely operate in everyday traffic or in harsh off-road environments, a multitude of problems in perception, navigation, and control have to be solved. This paper gives an overview of the most current trends in autonomous vehicles, highlighting the concepts common to most successful systems as well as their differences. It concludes with an outlook into the promising future of autonomous vehicles.

In this paper we explore the theoretical boundaries of planning in a setting where no model of the agent’s actions is given. Instead of an action model, a set of successfully executed plans are given and the task is to generate a plan that is safe, i.e., guaranteed to achieve the goal without failing. To this end, we show how to learn a conservative model of the world in which actions are guaranteed to be applicable. This conservative model is then given to an off-the-shelf classical planner, resulting in a plan that is guaranteed to achieve the goal. However, this reduction from a model-free planning to a model-based planning is not complete: in some cases a plan will not be found even when such exists. We analyze the relation between the number of observed plans and the likelihood that our conservative approach will indeed fail to solve a solvable problem. Our analysis show that the number of trajectories needed scales gracefully.


Question: A modeling approach for path planning in self-driving vehicles navigating dynamic environments.

This document outlines a sophisticated real-time obstacle avoidance technique designed for manipulators and autonomous vehicles navigating dynamic environments. The proposed algorithm is tailored to ensure swift and safe path adjustments in response to unexpected obstacles. By continuously monitoring the surroundings, the system can identify and predict the movement of dynamic objects, enabling the vehicle or robot to proactively plan alternative routes. This approach integrates advanced sensor data processing and machine learning algorithms to interpret the environment accurately, making it particularly suitable for self-driving cars operating in urban settings with high levels of unpredictability. The real-time capability ensures that the vehicle can make split-second decisions, avoiding collisions while maintaining a smooth and efficient journey. Through extensive simulations and field tests, the effectiveness of this real-time obstacle avoidance method has been validated, demonstrating its potential to enhance the safety and reliability of autonomous navigation in complex, dynamic scenarios.

Algorithms for route planning in transportation networks have recently undergone a rapid development, leading to methods that are up to one million times faster than Dijkstra’s algorithm. We outline ideas, algorithms, implementations, and experimental methods behind this development. We also explain why the story is not over yet because dynamically changing networks, flexible objective functions, and new applications pose a lot of interesting challenges.

This paper presents model predictive control of a vehicle in a varying road-traffic environment for ecological (eco) driving. The vehicle control input is derived by rigorous reasoning approach of model based anticipation of road, traffic and fuel consumption in a crowded road network regulated by traffic signals. Model predictive control with Continuation and generalized minimum residual method for optimization is used to calculate the sequence of control inputs aiming at long run fuel economy maintaining a safe driving. Performance of the proposed eco-driving system is evaluated through simulations in AIMSUN microscopic transport simulator. In spite of nonlinearity and discontinuous movement of other traffic and signals, the proposed system is robust enough to control the vehicle safely. The driving behavior with fuel saving aspects is graphically illustrated, compared and analyzed to signify the prospect of the proposed eco-driving of a vehicle.

In this paper, we present an implementation of 3D reciprocal collision avoidance on real quadrotor helicopters where each quadrotor senses the relative position and veloc ity of other quadrotors using an on-board camera. We show that using our approach, quadrotors are able to successfully avo id pairwise collisions GPS and motion-capture denied environ ments, without communication between the quadrotors, and even when human operators deliberately attempt to induce collision. To our knowledge, this is the first time that reciprocal collision avoidance has been successfully implemented on r eal robots where each agent independently observes the others using on-board sensors. We theoretically analyze the respo nse of the collision-avoidance algorithm to the violated assumptions by the use of real robots. We quantitatively analyze our experimental results. A particularly striking observation is that at times the quadrotors exhibit “reciprocal dance” behavior, which is also observed when humans move past each other in constrained environments. This seems to be the result of sensing uncertainty, which causes both robots involved to h ave a different belief about the relative positions and velocit ies and, as a result, choose the same side on which to pass.

This technical report describes Team MIT's approach to the DARPA Urban Challenge. We have developed a novel strategy for using many inexpensive sensors, mounted on the vehicle periphery, and calibrated with a new crossmodal calibration technique. Lidar, camera, and radar data streams are processed using an innovative, locally smooth state representation that provides robust perception for real time autonomous control. A resilient planning and control architecture has been developed for driving in traffic, comprised of an innovative combination of wellproven algorithms for mission planning, situational planning, situational interpretation, and trajectory control. These innovations are being incorporated in two new robotic vehicles equipped for autonomous driving in urban environments, with extensive testing on a DARPA site visit course. Experimental results demonstrate all basic navigation and some basic traffic behaviors, including unoccupied autonomous driving, lane following using purepursuit control and our local frame perception strategy, obstacle avoidance using kinodynamic RRT path planning, Uturns, and precedence evaluation amongst other cars at intersections using our situational interpreter. We are working to extend these approaches to advanced navigation and traffic scenarios. † Executive Summary This technical report describes Team MIT's approach to the DARPA Urban Challenge. We have developed a novel strategy for using many inexpensive sensors, mounted on the vehicle periphery, and calibrated with a new cross-modal calibration technique. Lidar, camera, and radar data streams are processed using an innovative, locally smooth state representation that provides robust perception for real-time autonomous control. A resilient planning and control architecture has been developed for driving in traffic, comprised of an innovative combination of well-proven algorithms for mission planning, situational planning, situational interpretation, and trajectory control. These innovations are being incorporated in two new robotic vehicles equipped for autonomous driving in urban environments, with extensive testing on a DARPA site visit course. Experimental results demonstrate all basic navigation and some basic traffic behaviors, including unoccupied autonomous driving, lane following using pure-pursuit control and our local frame perception strategy, obstacle avoidance using kino-dynamic RRT path planning, U-turns, and precedence evaluation amongst other cars at intersections using our situational interpreter. We are working to extend these approaches to advanced navigation and traffic scenarios. DISCLAIMER: The information contained in this paper does not represent the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency (DARPA) or the Department of Defense. DARPA does not guarantee the accuracy or reliability of the information in this paper. Additional support …

Motion and path-planning algorithms often use complex cost functions for both global navigation and local smoothing of trajectories. Obtaining good results typically requires carefully hand-engineering the trade-offs between different terms in the cost function. In practice, it is often much easier to demonstrate a few good trajectories. In this paper, we describe an efficient algorithm which - when given access to a few trajectory demonstrations - can automatically infer good trade-offs between the different costs. In our experiments, we apply our algorithm to the problem of navigating a robotic car in a parking lot.

The task of building a map of an unknown environment and concurrently using that map to navigate is a central problem in mobile robotics research. This paper addresses the problem of how to perform concurrent mapping and localization (CML) adaptively using sonar. Stochastic mapping is a feature-based approach to CML that generalizes the extended Kalman lter to incorporate vehicle localization and environmental mapping. We describe an implementation of stochastic mapping that uses a delayed nearest neighbor data association strategy to initialize new features into the map, match measurements to map features, and delete out-of-date features. We introduce a metric for adaptive sensing which is de ned in terms of Fisher information and represents the sum of the areas of the error ellipses of the vehicle and feature estimates in the map. Predicted sensor readings and expected dead-reckoning errors are used to estimate the metric for each potential action of the robot, and the action which yields the lowest cost (i.e., the maximum information) is selected. This technique is demonstrated via simulations, in-air sonar experiments, and underwater sonar experiments. Results are shown for 1) adaptive control of motion and 2) adaptive control of motion and scanning. The vehicle tends to explore selectively di erent objects in the environment. The performance of this adaptive algorithm is shown to be superior to straight-line motion and random motion.

We cast the partially observable control problem as a fully observable underactuated stochastic control problem in belief space and apply standard planning and control techniques. One of the difficulties of belief space planning is modeling the stochastic dynamics resulting from unknown future observations . The core of our proposal is to define deterministic beliefsystem dynamics based on an assumption that the maximum likelihood observation (calculated just prior to the observation) is always obtained. The stochastic effects of future observation s are modeled as Gaussian noise. Given this model of the dynamics, two planning and control methods are applied. In the first, linear quadratic regulation (LQR) is applied to generate policies in the belief space. This approach is shown to be optimal for linearGaussian systems. In the second, a planner is used to find locally optimal plans in the belief space. We propose a replanning approach that is shown to converge to the belief space goal in a finite number of replanning steps. These approaches are characterized in the context of a simple nonlinear manipulation problem where a planar robot simultaneously locates and grasps an object.

This paper describes the software architecture of Stanley, an autonomous land vehicle developed for high-speed desert driving without human intervention. The vehicle recently won the DARPA Grand Challenge, a major robotics competition. The article describes the software architecture of the robot, which relied pervasively on state-of-the-art AI technologies, such as machine learning and probabilistic reasoning.

Classical object tracking approaches use a Kalman-filter with a single dynamic model which is therefore optimised to a single driving maneuver. In contrast the interacting multiple model (IMM) filter allows for several parallel models which are combined to a weighted estimate. Choosing models for different driving modes, such as constant speed, acceleration and strong acceleration changes, the object state estimation can be optimised for highly dynamic driving maneuvers. The paper describes the analysis of Stop&Go situations and the systematic parametrisation of the IMM method based on these statistics. The evaluation of the IMM approach is presented based on real sensor measurements of laser scanners, a radar and a video image processing unit.

This paper presents a stereo vision-based autonomous navigation system using a GPS and a modified version of the VFH algorithm. In order to obtain a high-accuracy disparity map and meet the time constraints of the real time navigation system, this work proposes the use of a semi-global stereo method. By not suffering the same issues of the regularly used local stereo methods, the employed stereo technique enables the generation of a highly dense, efficient, and accurate disparity map. Obstacles are detected using a method that checks for relative slopes and heights differences. Experimental tests using an electric vehicle in an urban environment were performed to validate the proposed approach.


Question: A modeling approach for path planning in dynamic environments for autonomous vehicles.

This document outlines an innovative method for reactive nonholonomic trajectory generation using parametric optimal control, particularly relevant for self-driving cars in dynamic environments. In scenarios where real-time perceptual data necessitates the immediate creation of a viable nonholonomic motion plan, parametric trajectory models offer a streamlined solution by narrowing the search space, albeit potentially introducing suboptimality. By adopting curvature polynomials of arbitrary order as the assumed solution form, the optimal control problem is transformed into a nonlinear programming challenge. This polynomial-based approach ensures a broad coverage of feasible control options while enabling the system dynamics' general solution to be expressed through decoupled quadratures. These quadratures can then be easily linearized to articulate the optimality conditions, leading to trajectories that are not only user-friendly for vehicle controllers but also amenable to real-time computation through a simple numerical method. This technique's key advantages include its adaptability to real-world conditions, efficiency in computation, and seamless integration into autonomous vehicle control systems. Keywords: mobile robots, car-like robots, trajectory generation, curve generation, nonholonomic, clothoid, cornu spiral, optimal control.

We introduce a dynamic technique for fast route planning in large road networks. For the first time, it is possible to handle the practically relevant scenarios that arise in present-day navigation systems: When an edge weight changes (e.g., due to a traffic jam), we can update the preprocessed information in 2–40 ms allowing subsequent fast queries in about one millisecond on average. When we want to perform only a single query, we can skip the comparatively expensive update step and directly perform a prudent query that automatically takes the changed situation into account. If the overall cost function changes (e.g., due to a different vehicle type), recomputing the preprocessed information takes typically less than two minutes. The foundation of our dynamic method is a new static approach that generalises and combines several previous speedup techniques. It has outstandingly low memory requirements of only a few bytes per node.

Full autonomy of ground vehicles is a major goal of the ITS (Intelligent Transportation Systems) community. However, reaching such highest autonomy level in all situations (weather, traffic, ...) may seem difficult in practice, despite recent results regarding driverless cars (e.g., Google Cars). In addition, an automated vehicle should also self-assess its own perception abilities, and not only perceive its environment. In this paper, we propose an intermediate approach towards full automation, by defining a spectrum of automation layers, from fully manual (the car is driven by a driver) to fully automated (the car is driven by a computer), based on an ontological model for representing knowledge. We also propose a second ontology for situation assessment (what does the automated car perceive?), including the sensors/actuators state, environmental conditions and driver's state. Finally, we also define inference rules to link the situation assessment ontology to the automation level one. Both ontological models have been built and first results are presented.

In Vehicular Ad Hoc Networks (VANETs), moving vehicles are considered as mobile nodes in the network and they are connected to each other via wireless links when they are within the communication radius of each other. Efficient message delivery in VANETs is still a very challenging research issue. In this paper, a Q-learning based routing protocol (i.e., QGrid) is introduced to help to improve the message delivery from mobile vehicles to a specific location. QGrid considers both macroscopic and microscopic aspects when making the routing decision, while the traditional routing methods focus on computing meeting information between different vehicles. QGrid divides the region into different grids. The macroscopic aspect determines the optimal next-hop grid and the microscopic aspect determines the specific vehicle in the optimal next-hop grid to be selected as next-hop vehicle. QGrid computes the Q-values of different movements between neighboring grids for a given destination via Q-learning. Each vehicle stores Q-value table learned offline, then selects optimal next-hop grid by querying Q-value table. Inside the selected next-hop grid, we either greedily select the nearest neighboring vehicle to the destination or select the neighboring vehicle with highest probability of moving to the optimal next-hop grid predicted by the two-order Markov chain. The performance of QGrid is evaluated by using real life trajectory GPS data of Shanghai taxies. Simulation comparison among QGrid and other existing position-based routing protocols confirms the advantages of proposed QGrid routing protocol for VANETs.

In the near future, humans will be relieved from parking. Major improvements in autonomous driving allow the realization of automated valet parking (AVP). It enables the vehicle to drive to a parking spot and park itself. This paper presents a review of the intelligent vehicles literature on AVP. An overview and analysis of the core components of AVP such as the platforms, sensor setups, maps, localization, perception, environment model, and motion planning is provided. Leveraging the potential of AVP, high density parking (HDP) is reviewed as a future research direction with the capability to either reduce the necessary space for parking by up to 50 % or increase the capacity of future parking facilities. Finally, a synthesized view discussing the remaining challenges in automated valet parking and the technological requirements for high density parking is given.

This paper addresses the real-time control of autonomous vehicles under a minimum traveling time objective. Control inputs for the vehicle are computed from a nonlinear model predictive control (MPC) scheme. The time-optimal objective is reformulated such that it can be tackled by existing efficient algorithms for real-time nonlinear MPC that build on the generalized Gauss-Newton method. We numerically validate our approach in simulations and present a real-world hardware setup of miniature race cars that is used for an experimental comparison of different approaches.

We study an algorithm that allows a vehicle to autonomously change lanes in a safe but personalized fashion without the driver's explicit initiation (e.g. activating the turn signals). Lane change initiation in autonomous driving is typically based on subjective rules, functions of the positions and relative velocities of surrounding vehicles. This approach is often arbitrary, and not easily adapted to the driving style preferences of an individual driver. Here we propose a data-driven modeling approach to capture the lane change decision behavior of human drivers. We collect data with a test vehicle in typical lane change situations and train classifiers to predict the instant of lane change initiation with respect to the preferences of a particular driver. We integrate this decision logic into a model predictive control (MPC) framework to create a more personalized autonomous lane change experience that satisfies safety and comfort constraints. We show the ability of the decision logic to reproduce and differentiate between two lane changing styles, and demonstrate the safety and effectiveness of the control framework through simulations.

Micro aerial vehicles, such as multirotors, are particular well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance in industrial plants. Key prerequisites for the fully autonomous operation of micro aerial vehicles in restricted environments are 3D mapping, real-time pose tracking, obstacle detection, and planning of collision-free trajectories. In this article, we propose a complete navigation system with a multimodal sensor setup for omnidirectional environment perception. Measurements of a 3D laser scanner are aggregated in egocentric local multiresolution grid maps. Local maps are registered and merged to allocentric maps in which the MAV localizes. For autonomous navigation, we generate trajectories in a multi-layered approach: from mission planning over global and local trajectory planning to reactive obstacle avoidance. We evaluate our approach in a GNSS-denied indoor environment where multiple collision hazards require reliable omnidirectional perception and quick navigation reactions.

The autonomous vehicle, as an emerging and rapidly growing field, has received extensive attention for its futuristic driving experiences. Although the fast developing depth sensors and machine learning methods have given a huge boost to self-driving research, existing autonomous driving vehicles do meet with several avoidable accidents during their road testings. The major cause is the misunderstanding between self-driving systems and human drivers. To solve this problem, we propose a humanlike driving system in this paper to give autonomous vehicles the ability to make decisions like a human. In our method, a convolutional neural network model is used to detect, recognize, and abstract the information in the input road scene, which is captured by the on-board sensors. And then a decision-making system calculates the specific commands to control the vehicles based on the abstractions. The biggest advantage of our work is that we implement a decision-making system which can well adapt to real-life road conditions, in which a massive number of human drivers exist. In addition, we build our perception system with only the depth information, rather than the unstable RGB data. The experimental results give a good demonstration of the efficiency and robustness of the proposed method.

We illustrate the use of the techniques of modern geometric optimal control theory by studying the shortest paths for a model of a car that can move forwards and backwards. This problem was discussed in recent work by Reeds and Shepp who showed, by special methods, (a) that shortest path motion could always be achieved by means of trajectories of a special kind, namely, concatenations of at most five pieces, each of which is either a straight line or a circle, and (b) that these concatenations can be classified into 48 three-parameter families. We show how these results fit in a much more general framework, and can be discovered and proved by applying in a systematic way the techniques of Optimal Control Theory. It turns out that the “classical” optimal control tools developed in the 1960’s, such as the Pontryagin Maximum Principle and theorems on the existence of optimal trajectories, are helpful to go part of the way and get some information on the shortest paths, but do not suffice to get the full result. On the other hand, when these classical techniques are combined with the use of a more recently developed body of theory, namely, geometric methods based on the Lie algebraic analysis of trajectories, then one can recover the full power of the Reeds-Shepp results, and in fact slightly improve upon them by lowering their 48 to a 46.

A simple and efficient randomized algorithm is presented for solving single-query path planning problems in high-dimensional configuration spaces. The method works by incrementally building two Rapidly-exploring Random Trees (RRTs) rooted at the start and the goal configurations. The trees each explore space around them and also advance towards each other through the use of a simple greedy heuristic. Although originally designed to plan motions for a human arm (modeled as a 7-DOF kinematic chain) for the automatic graphic animation of collision-free grasping and manipulation tasks, the algorithm has been successfully applied to a variety of path planning problems. Computed examples include generating collision-free motions for rigid objects in 2D and 3D, and collision-free manipulation motions for a 6-DOF PUMA arm in a 3D workspace. Some basic theoretical analysis is also presented.


Question: Optimization-based method for colorization.

This paper presents an innovative scale-space technique and edge detection method that overcomes the limitations of the traditional Witkin approach. In Witkin's method, coarser resolution images are generated by convolving the original image with a Gaussian kernel, which, however, struggles to accurately determine the locations of "semantically meaningful" edges at lower resolutions. Our proposal introduces a novel scale-space definition alongside a set of algorithms that implement this definition through an anisotropic diffusion process. The diffusion coefficient is spatially variable, designed to prioritize smoothing within regions over smoothing across region boundaries. This ensures that the fundamental property of conventional scale-space, which states that no new maxima should be generated at coarser scales, is maintained. Importantly, our method retains sharp region boundaries, leading to a high-quality edge detector that effectively utilizes global information. Demonstrative results are provided for various images, showcasing the effectiveness of the technique. The algorithm's reliance on simple, local operations makes it well-suited for parallel hardware implementation, enhancing its computational efficiency.

Fig. 1. Given a natural image, the proposed approach can hallucinate different versions of the same scene in a wide range of conditions, e.g.night, sunset, winter, spring, rain, fog or even a combination of those. First, our method utilizes a single generator network to imagine the scene with respect to its semantic layout and the desired set of attributes. Then, it directly transfers the look from the hallucinated output to the input image, without a need to have access to a reference style image.

We propose a novel integral imaging system using holographic optical element (HOE) which can provide multi-color 3-dimensional (3D) images. The optical properties of a lens array are recorded on a photopolymer using color multiplexing method with two different wavelength lasers (473 nm and 532 nm). The lens array HOE recorded on the photopolymer functions as 3D imaging HOE for integral imaging instead of the conventional lens array or concave mirror array. Recording scheme for color multiplexed HOE is presented and the proposed system is experimentally verified.

The applicability of computer vision to real paintings and artworks has been rarely investigated, even though a vast heritage would greatly benefit from techniques which can understand and process data from the artistic domain. This is partially due to the small amount of annotated artistic data, which is not even comparable to that of natural images captured by cameras. In this paper, we propose a semantic-aware architecture which can translate artworks to photo-realistic visualizations, thus reducing the gap between visual features of artistic and realistic data. Our architecture can generate natural images by retrieving and learning details from real photos through a similarity matching strategy which leverages a weakly-supervised semantic understanding of the scene. Experimental results show that the proposed technique leads to increased realism and to a reduction in domain shift, which improves the performance of pre-trained architectures for classification, detection, and segmentation. Code will be made publicly available.

In this paper, we propose a novel explicit image filter called guided filter. Derived from a local linear model, the guided filter computes the filtering output by considering the content of a guidance image, which can be the input image itself or another different image. The guided filter can be used as an edge-preserving smoothing operator like the popular bilateral filter [1], but it has better behaviors near edges. The guided filter is also a more generic concept beyond smoothing: It can transfer the structures of the guidance image to the filtering output, enabling new filtering applications like dehazing and guided feathering. Moreover, the guided filter naturally has a fast and nonapproximate linear time algorithm, regardless of the kernel size and the intensity range. Currently, it is one of the fastest edge-preserving filters. Experiments show that the guided filter is both effective and efficient in a great variety of computer vision and computer graphics applications, including edge-aware smoothing, detail enhancement, HDR compression, image matting/feathering, dehazing, joint upsampling, etc.

Local binary pattern (LBP) is widely adopted for efficient image feature description and simplicity. To describe the color images, it is required to combine the LBPs from each channel of the image. The traditional way of binary combination is to simply concatenate the LBPs from each channel, but it increases the dimensionality of the pattern. In order to cope with this problem, this paper proposes a novel method for image description with multichannel decoded LBPs. We introduce adder- and decoder-based two schemas for the combination of the LBPs from more than one channel. Image retrieval experiments are performed to observe the effectiveness of the proposed approaches and compared with the existing ways of multichannel techniques. The experiments are performed over 12 benchmark natural scene and color texture image databases, such as Corel-1k, MIT-VisTex, USPTex, Colored Brodatz, and so on. It is observed that the introduced multichannel adder- and decoder-based LBPs significantly improve the retrieval performance over each database and outperform the other multichannel-based approaches in terms of the average retrieval precision and average retrieval rate.

Over the past years there has been considerable interest in statistically optimal reconstruction of cross-sectional images from tomographic data. In particular, a variety of such algorithms have been proposed for maximum a posteriori (MAP) reconstruction from emission tomographic data. While MAP estimation requires the solution of an optimization problem, most existing reconstruction algorithms take an indirect approach based on the expectation maximization (EM) algorithm. We propose a new approach to statistically optimal image reconstruction based on direct optimization of the MAP criterion. The key to this direct optimization approach is greedy pixel-wise computations known as iterative coordinate decent (ICD). We propose a novel method for computing the ICD updates, which we call ICD/Newton-Raphson. We show that ICD/Newton-Raphson requires approximately the same amount of computation per iteration as EM-based approaches, but the new method converges much more rapidly (in our experiments, typically five to ten iterations). Other advantages of the ICD/Newton-Raphson method are that it is easily applied to MAP estimation of transmission tomograms, and typical convex constraints, such as positivity, are easily incorporated.

We generalize the alternating minimization algorithm recently proposed in [32] to efficiently solve a general, edge-preserving, variational model for recovering multichannel images degraded by withinand cross-channel blurs, as well as additive Gaussian noise. This general model allows the use of localized weights and higher-order derivatives in regularization, and includes a multichannel extension of total variation (MTV) regularization as a special case. In the MTV case, we show that the model can be derived from an extended half-quadratic transform of Geman and Yang [14]. For color images with three channels and when applied to the MTV model (either locally weighted or not), the per-iteration computational complexity of this algorithm is dominated by nine fast Fourier transforms. We establish strong convergence results for the algorithm including finite convergence for some variables and fast q-linear convergence for the others. Numerical results on various types of blurs are presented to demonstrate the performance of our algorithm compared to that of the MATLAB deblurring functions. We also present experimental results on regularization models using weighted MTV and higher-order derivatives to demonstrate improvements in image quality provided by these models over the plain MTV model.

We propose a new fast algorithm for solving one of the standard formulations of image restoration and reconstruction which consists of an unconstrained optimization problem where the objective includes an l2 data-fidelity term and a nonsmooth regularizer. This formulation allows both wavelet-based (with orthogonal or frame-based representations) regularization or total-variation regularization. Our approach is based on a variable splitting to obtain an equivalent constrained optimization formulation, which is then addressed with an augmented Lagrangian method. The proposed algorithm is an instance of the so-called alternating direction method of multipliers, for which convergence has been proved. Experiments on a set of image restoration and reconstruction benchmark problems show that the proposed algorithm is faster than the current state of the art methods.

In this paper, we present an active volumetric model (AVM) for 3D reconstruction from multiple calibrated images of a scene. The AVM is a physically motivated 3D deformable model which shrinks actively under the influence of multiple simulated forces towards the real scene by throwing away some of its voxels. It provides a computational framework to integrate several constraints in an intelligible way. In the current work, we use three forces derived respectively from the smooth constraint, the compulsory silhouette constraint, and the color consistency constraint. Based on the composition of the 3 forces, our algorithm can significantly restrain holes and floating voxels, which plague voxel coloring algorithms, and produce precise and smooth models. We test our algorithm by experiments based on both synthetic and real data.

We introduce our solution of the problem of clothing recognition and identification. The cause of taking convolutional neural network as a mathematic model for the further research was the result of an experimental survey of methods of keypoints detection and computing their descriptors. The convolutional neural network model was developed and trained on the Apparel Classification with Style clothing image dataset. The training image dataset was expanded with the help of augmentation methods. The computational experiment aimed at determination of neural network efficiency and stability proved its productivity. Moreover, we developed clothing color recognition, which let us perform clothing identification by 2 parameters: type and color.