Question: Categorization of human actions utilizing a Stacked Autoencoder model.

This paper presents a comprehensive overview of deep learning, a subset of machine learning technology that is transforming numerous facets of contemporary society. Deep learning is instrumental in web searches, content filtering on social networks, recommendations on e-commerce platforms, and is integrated into consumer gadgets like cameras and smartphones for features such as object recognition in images, speech-to-text conversion, news or product matching with user interests, and search result relevancy. Traditional machine learning was constrained in processing raw natural data, necessitating extensive domain expertise to design feature extractors that transformed raw input into a format suitable for pattern detection or classification. Representation learning, of which deep learning is a part, enables machines to automatically discover the representations needed for detection or classification directly from raw data. Deep learning methods involve multiple levels of representation, achieved through the composition of simple yet non-linear modules that transform representations from one level to the next, starting with the raw input. This transformation facilitates the learning of complex functions. For classification tasks, higher levels of representation emphasize discriminative features of the input while suppressing irrelevant variations. For instance, in image processing, the first layer of representation might detect edges, the second layer might identify motifs from these edges, the third layer might assemble motifs into recognizable object parts, and subsequent layers could detect entire objects from these parts. A distinctive feature of deep learning is that these layers of features are learned from data using a general-purpose learning procedure, rather than being manually designed by human engineers. Deep learning has made significant strides in solving problems that have challenged the artificial intelligence community for decades. Its proficiency in discovering complex structures in high-dimensional data makes it applicable across various scientific, business, and governmental domains. Deep learning has set new benchmarks in image and speech recognition, drug molecule activity prediction, particle accelerator data analysis, brain circuit reconstruction, and the prediction of non-coding DNA mutation effects on gene expression and disease. Surprisingly, it has also shown remarkable potential in natural language understanding tasks such as topic classification, sentiment analysis, question answering, and language translation. The future of deep learning appears promising due to its minimal reliance on manual engineering, allowing it to capitalize on increased computational resources and data availability. Advancements in learning algorithms and architectures for deep neural networks will further expedite progress in this field.

The mathematical properties of high-dimensional spaces seem remarkably suited for describing behaviors produces by brains. Brain-inspired hyperdimensional computing (HDC) explores the emulation of cognition by computing with hypervectors as an alternative to computing with numbers. Hypervectors are high-dimensional, holographic, and (pseudo)random with independent and identically distributed (i.i.d.) components. These features provide an opportunity for energy-efficient computing applied to cyberbiological and cybernetic systems. We describe the use of HDC in a smart prosthetic application, namely hand gesture recognition from a stream of Electromyography (EMG) signals. Our algorithm encodes a stream of analog EMG signals that are simultaneously generated from four channels to a single hypervector. The proposed encoding effectively captures spatial and temporal relations across and within the channels to represent a gesture. This HDC encoder achieves a high level of classification accuracy (97.8%) with only 1/3 the training data required by state-of-the-art SVM on the same task. HDC exhibits fast and accurate learning explicitly allowing online and continuous learning. We further enhance the encoder to adaptively mitigate the effect of gesture-timing uncertainties across different subjects endogenously; further, the encoder inherently maintains the same accuracy when there is up to 30% overlapping between two consecutive gestures in a classification window.

Gaming systems like Kinect and XBox always have to tackle the problem of extracting features from video data sets and classifying the body movement. In this study, reasonable features like human joints positions, joints velocities, joint angles and joint angular velocities are extracted. We used several machine learning methods including Naive Bayes, Support Vector Machine and Random Forest to learn and classify the human gestures. Simulation results and the final confusion matrix show that by combining delicately preprocessed data sets and Random Forest methods, the F-scores of the correct predictions can be maximized. Such methods can also be applied in real-time scenarios.

This paper presents a human posture recognition method from a single image. We rst segment an image into homogeneous regions and extract curve segments corresponding to human body parts. Each body part is considered as a 2D ribbon. From the smooth curve segments in skin regions, 2D ribbons are extracted and a human body model is constructed. We assign a predeened posture type to the image according to the constructed body model. For the user input query to retrieve images containing human of speciic posture, the system convert the query to a body model. The body model is compared to other body models saved in the local storage of target images and images of good matches are retrieved. When a face detection result is available for the given image, it is also used to increase the reliability of body model. For the query human posture, our system retrieves images of the corresponding posture. As another application, the proposed method provides an initial location of a human body to track in a video sequence.

We address the task of inferring the future actions of people from noisy visual input. We denote this task activity forecasting. To achieve accurate activity forecasting, our approach models the effect of the physical environment on the choice of human actions. This is accomplished by the use of state-of-the-art semantic scene understanding combined with ideas from optimal control theory. Our unified model also integrates several other key elements of activity analysis, namely, destination forecasting, sequence smoothing and transfer learning. As proof-of-concept, we focus on the domain of trajectory-based activity analysis from visual input. Experimental results demonstrate that our model accurately predicts distributions over future actions of individuals. We show how the same techniques can improve the results of tracking algorithms by leveraging information about likely goals and trajectories.

Research in human action recognition has accelerated significantly since the introduction of powerful machine learning tools such as Convolutional Neural Networks (CNNs). However, effective and efficient methods for incorporation of temporal information into CNNs are still being actively explored in the recent literature. Motivated by the popular recurrent attention models in the research area of natural language processing, we propose the Attention-aware Temporal Weighted CNN (ATW CNN) for action recognition in videos, which embeds a visual attention model into a temporal weighted multi-stream CNN. This attention model is simply implemented as temporal weighting yet it effectively boosts the recognition performance of video representations. Besides, each stream in the proposed ATW CNN framework is capable of end-to-end training, with both network parameters and temporal weights optimized by stochastic gradient descent (SGD) with back-propagation. Our experimental results on the UCF-101 and HMDB-51 datasets showed that the proposed attention mechanism contributes substantially to the performance gains with the more discriminative snippets by focusing on more relevant video segments.

We propose DeepHand to estimate the 3D pose of a hand using depth data from commercial 3D sensors. We discriminatively train convolutional neural networks to output a low dimensional activation feature given a depth map. This activation feature vector is representative of the global or local joint angle parameters of a hand pose. We efficiently identify 'spatial' nearest neighbors to the activation feature, from a database of features corresponding to synthetic depth maps, and store some 'temporal' neighbors from previous frames. Our matrix completion algorithm uses these 'spatio-temporal' activation features and the corresponding known pose parameter values to estimate the unknown pose parameters of the input feature vector. Our database of activation features supplements large viewpoint coverage and our hierarchical estimation of pose parameters is robust to occlusions. We show that our approach compares favorably to state-of-the-art methods while achieving real time performance (≈ 32 FPS) on a standard computer.

We propose a generalized approach to human gesture recognition based on multiple data modalities such as depth video, articulated pose and speech. In our system, each gesture is decomposed into large-scale body motion and local subtle movements such as hand articulation. The idea of learning at multiple scales is also applied to the temporal dimension, such that a gesture is considered as a set of characteristic motion impulses, or dynamic poses. Each modality is first processed separately in short spatio-temporal blocks, where discriminative data-specific features are either manually extracted or learned. Finally, we employ a Recurrent Neural Network for modeling large-scale temporal dependencies, data fusion and ultimately gesture classification. Our experiments on the 2013 Challenge on Multimodal Gesture Recognition dataset have demonstrated that using multiple modalities at several spatial and temporal scales leads to a significant increase in performance allowing the model to compensate for errors of individual classifiers as well as noise in the separate channels.

Many object classes are primarily defined by their functions. However, this fact has been left largely unexploited by visual object categorization or detection systems. We propose a method to learn an affordance detector. It identifies locations in the 3d space which “support” the particular function. Our novel approach “imagines” an actor performing an action typical for the target object class, instead of relying purely on the visual object appearance. So, function is handled as a cue complementary to appearance, rather than being a consideration after appearance-based detection. Experimental results are given for the functional category “sitting”. Such affordance is tested on a 3d representation of the scene, as can be realistically obtained through SfM or depth cameras. In contrast to appearance-based object detectors, affordance detection requires only very few training examples and generalizes very well to other sittable objects like benches or sofas when trained on a few chairs.

A category of stimuli of great importance for primates, humans in particular, is that formed by actions done by other individuals. If we want to survive, we must understand the actions of others. Furthermore, without action understanding, social organization is impossible. In the case of humans, there is another faculty that depends on the observation of others' actions: imitation learning. Unlike most species, we are able to learn by imitation, and this faculty is at the basis of human culture. In this review we present data on a neurophysiological mechanism--the mirror-neuron mechanism--that appears to play a fundamental role in both action understanding and imitation. We describe first the functional properties of mirror neurons in monkeys. We review next the characteristics of the mirror-neuron system in humans. We stress, in particular, those properties specific to the human mirror-neuron system that might explain the human capacity to learn by imitation. We conclude by discussing the relationship between the mirror-neuron system and language.

We propose a method for human pose estimation based on Deep Neural Networks (DNNs). The pose estimation is formulated as a DNN-based regression problem towards body joints. We present a cascade of such DNN regres- sors which results in high precision pose estimates. The approach has the advantage of reasoning about pose in a holistic fashion and has a simple but yet powerful formula- tion which capitalizes on recent advances in Deep Learn- ing. We present a detailed empirical analysis with state-of- art or better performance on four academic benchmarks of diverse real-world images.


Question: Using a stacked autoencoder for the classification of human activities.

This paper explores the application of a Stacked Autoencoder for the classification of human activity from accelerometer data, contributing to the broader field of context awareness. The focus is on recognizing user activities by interpreting the patterns in accelerometer readings, which is approached as a classification task. The study evaluates the efficacy of various base-level classifiers and meta-level classifiers, with Plurality Voting emerging as a robust choice across diverse experimental conditions. This highlights the potential of ensemble methods in enhancing the accuracy and reliability of activity recognition systems based on accelerometer data.

Analysis and recognition of motion patterns from data acquired by body-worn inertial sensors is an emerging technology in sports. In this paper we propose an effective method for recognition of fencing footwork using a single body-worn accelerometer. We present a challenging dataset consisting of six actions, which were performed by ten persons and repeated ten times by each of them. We propose a segment-based SVM for time-series classification together with a set of informative features. We demonstrate that the method is competitive with 1-NN DTW in terms of classification accuracy. The proposed method achieves classification accuracy slightly better than 70% on the fencing footwork dataset.

Intelligent agents, such as robots, have to serve a multitude of autonomous functions. Examples are, e.g., collision avoidance, navigation and route planning, active sensing of its environment, or the interaction and non-verbal communication with people in the extended reach space. Here, we focus on the recognition of the action of a human agent based on a biologically inspired visual architecture of analyzing articulated movements. The proposed processing architecture builds upon coarsely segregated streams of sensory processing along different pathways which separately process form and motion information (Layher et al., 2014). Action recognition is performed in an event-based scheme by identifying representations of characteristic pose configurations (key poses) in an image sequence. In line with perceptual studies, key poses are selected unsupervised utilizing a feature-driven criterion which combines extrema in the motion energy with the horizontal and the vertical extendedness of a body shape. Per class representations of key pose frames are learned using a deep convolutional neural network consisting of 15 convolutional layers. The network is trained using the energy-efficient deep neuromorphic networks (Eedn) framework (Esser et al., 2016), which realizes the mapping of the trained synaptic weights onto the IBM Neurosynaptic System platform (Merolla et al., 2014). After the mapping, the trained network achieves real-time capabilities for processing input streams and classify input images at about 1,000 frames per second while the computational stages only consume about 70 mW of energy (without spike transduction). Particularly regarding mobile robotic systems, a low energy profile might be crucial in a variety of application scenarios. Cross-validation results are reported for two different datasets and compared to state-of-the-art action recognition approaches. The results demonstrate, that (I) the presented approach is on par with other key pose based methods described in the literature, which select key pose frames by optimizing classification accuracy, (II) compared to the training on the full set of frames, representations trained on key pose frames result in a higher confidence in class assignments, and (III) key pose representations show promising generalization capabilities in a cross-dataset evaluation.

The capability of processing and digesting raw data is one of the key features of a humanlike artificial intelligence system. For instance, real-time machine translation should be able to process and understand spoken natural language, and autonomous driving relies on the comprehension of visual inputs. Representation learning is a class of machine learning techniques that autonomously learn to derive latent features from raw data. These new features are expected to represent the data instances in a vector space that facilitates the machine learning task. This thesis studies two specific data situations that require efficient representation learning: knowledge graph data and high dimensional sequences. In the first part of this thesis, we first review multiple relational learning models based on tensor decomposition for knowledge graphs. We point out that relational learning is in fact a means of learning representations through one-hot mapping of entities. Furthermore, we generalize this mapping function to consume a feature vector that encodes all known facts about each entity. It enables the relational model to derive the latent representation instantly for a new entity, without having to re-train the tensor decomposition. In the second part, we focus on learning representations from high dimensional sequential data. Sequential data often pose the challenge that they are of variable lengths. Electronic health records, for instance, could consist of clinical event data that have been collected at subsequent time steps. But each patient may have a medical history of variable length. We apply recurrent neural networks to produce fixed-size latent representations from the raw feature sequences of various lengths. By exposing a prediction model to these learned representations instead of the raw features, we can predict the therapy prescriptions more accurately as a means of clinical decision support. We further propose Tensor-Train recurrent neural networks. We give a detailed introduction to the technique of tensorizing and decomposing large weight matrices into a few smaller tensors. We demonstrate the specific algorithms to perform the forward-pass and the back-propagation in this setting. Then we apply this approach to the input-to-hidden weight matrix in recurrent neural networks. This novel architecture can process extremely high dimensional sequential features such as video data. The model also provides a promising solution to processing sequential features with high sparsity. This is, for instance, the case with electronic health records, since they are often of categorical nature and have to be binary-coded. We incorporate a statistical survival model with this representation learning model, which shows superior prediction quality.

Gesture recognition is a well-establish topic of research that is widely adopted for a broad range of applications. For instance, it can be exploited for the command of a smart environment without any remote control unit or even for the recognition of human activities from a set of video cameras deployed in strategic position. Many researchers working on assistive smart home, such as our team, believe that the intrusiveness of that technology will prevent the future adoption and commercialization of smart homes. In this paper, we propose a novel gesture recognition algorithm that is solely based on passive RFID technology. This technology enables the localization of small tags that can be embedded in everyday life objects (a cup or a book, for instance) while remaining non intrusive. However, until now, this technology has been largely ignored by researchers on gesture recognition, mostly because it is easily disturbed by noise (metal, human, etc.) and offer limited precision. Despite these issues, the localization algorithms have improved over the years, and our recent efforts resulted in a real-time tracking algorithm with a precision approaching 14cm. With this, we developed a gesture recognition algorithm able to perform segmentation of gestures and prediction on a spatio-temporal data series. Our new model, exploiting works on qualitative spatial reasoning, achieves recognition of 91%. Our goal is to ultimately use that knowledge for both human activity recognition and errors detection.

In this paper, we propose an automated computer platform for the purpose of classifying Electroencephalography (EEG) signals associated with left and right hand movements using a hybrid system that uses advanced feature extraction techniques and machine learning algorithms. It is known that EEG represents the brain activity by the electrical voltage fluctuations along the scalp, and Brain-Computer Interface (BCI) is a device that enables the use of the brain’s neural activity to communicate with others or to control machines, artificial limbs, or robots without direct physical movements. In our research work, we aspired to find the best feature extraction method that enables the differentiation between left and right executed fist movements through various classification algorithms. The EEG dataset used in this research was created and contributed to PhysioNet by the developers of the BCI2000 instrumentation system. Data was preprocessed using the EEGLAB MATLAB toolbox and artifacts removal was done using AAR. Data was epoched on the basis of Event-Related (De) Synchronization (ERD/ERS) and movement-related cortical potentials (MRCP) features. Mu/beta rhythms were isolated for the ERD/ERS analysis and delta rhythms were isolated for the MRCP analysis. The Independent Component Analysis (ICA) spatial filter was applied on related channels for noise reduction and isolation of both artifactually and neutrally generated EEG sources. The final feature vector included the ERD, ERS, and MRCP features in addition to the mean, power and energy of the activations of the resulting Independent Components (ICs) of the epoched feature datasets. The datasets were inputted into two machinelearning algorithms: Neural Networks (NNs) and Support Vector Machines (SVMs). Intensive experiments were carried out and optimum classification performances of 89.8 and 97.1 were obtained using NN and SVM, respectively. This research shows that this method of feature extraction holds some promise for the classification of various pairs of motor movements, which can be used in a BCI context to mentally control a computer or machine. Keywords—EEG; BCI; ICA; MRCP; ERD/ERS; machine learning; NN; SVM

In this work, a novel training scheme for generating bottleneck features from deep neural networks is proposed. A stack of denoising auto-encoders is first trained in a layer-wise, unsupervised manner. Afterwards, the bottleneck layer and an additional layer are added and the whole network is fine-tuned to predict target phoneme states. We perform experiments on a Cantonese conversational telephone speech corpus and find that increasing the number of auto-encoders in the network produces more useful features, but requires pre-training, especially when little training data is available. Using more unlabeled data for pre-training only yields additional gains. Evaluations on larger datasets and on different system setups demonstrate the general applicability of our approach. In terms of word error rate, relative improvements of 9.2% (Cantonese, ML training), 9.3% (Tagalog, BMMI-SAT training), 12% (Tagalog, confusion network combinations with MFCCs), and 8.7% (Switchboard) are achieved.

Traditional approaches for action detection use trimmed data to learn sophisticated action detector models. Although these methods have achieved great success at detecting human actions, we argue that huge information is discarded when ignoring the process, through which this trimmed data is obtained. In this paper, we propose Action Search, a novel approach that mimics the way people annotate activities in video sequences. Using a Recurrent Neural Network, Action Search can efficiently explore a video and determine the time boundaries during which an action occurs. Experiments on the THUMOS14 dataset reveal that our model is not only able to explore the video efficiently but also accurately find human activities, outperforming stateof-the-art methods.

This paper presents a new framework for human activity recognition from video sequences captured by a depth camera. We cluster hypersurface normals in a depth sequence to form the polynormal which is used to jointly characterize the local motion and shape information. In order to globally capture the spatial and temporal orders, an adaptive spatio-temporal pyramid is introduced to subdivide a depth video into a set of space-time grids. We then propose a novel scheme of aggregating the low-level polynormals into the super normal vector (SNV) which can be seen as a simplified version of the Fisher kernel representation. In the extensive experiments, we achieve classification results superior to all previous published results on the four public benchmark datasets, i.e., MSRAction3D, MSRDailyActivity3D, MSRGesture3D, and MSRActionPairs3D.

Understanding the activities of human from videos is demanding task in Computer Vision. Identifying the actions being accomplished by the human in the video sequence automatically and tagging their actions is the prime functionality of intelligent video systems. The goal of activity recognition is to identify the actions and objectives of one or more objects from a series of examination on the action of object and their environmental condition. The major applications of Human Activity Recognition varies from Content-based Video Analytics, Robotics, Human-Computer Interaction, Human fall detection, Ambient Intelligence, Visual Surveillance, Video Indexing etc. This paper collectively summarizes and deciphers the various methodologies, challenges and issues of Human Activity Recognition systems. Variants of Human Activity Recognition systems such as Human Object Interactions and Human-Human Interactions are also explored. Various benchmarking datasets and their properties are being explored. The Experimental Evaluation of various papers are analyzed efficiently with the various performance metrics like Precision, Recall, and Accuracy.

We present a video-based gesture dataset and a methodology for annotating video-based gesture datasets. Our dataset consists of user-defined gestures generated by 18 participants from a previous investigation of gesture memorability. We design and use a crowd-sourced classification task to annotate the videos. The results are made available through a web-based visualization that allows researchers and designers to explore the dataset. Finally, we perform an additional descriptive analysis and quantitative modeling exercise that provide additional insights into the results of the original study. To facilitate the use of the presented methodology by other researchers we share the data, the source of the human intelligence tasks for crowdsourcing, a new taxonomy that integrates previous work, and the source code of the visualization tool.


Question: Using a stacked autoencoder for the classification of human activities.

This guide offers practical advice for the effective training of deep architectures, focusing on hyperparameters crucial for deep learning algorithms, particularly those associated with artificial neural networks. Hyperparameters, often seen as intricate adjustments, are vital for optimizing the learning process in gradient-based training methods, including those that employ backpropagation. The guide highlights recommendations for commonly employed hyperparameters, providing insights into their optimal tuning to enhance model performance. It also addresses the complexity of managing numerous hyperparameters, emphasizing strategies to achieve more intriguing outcomes. The document shares insights into the established practices for efficiently training and debugging large-scale, deep neural networks, ensuring their successful operation. It concludes by posing open questions about the challenges encountered during the training of deeper architectures, inviting further exploration into the intricacies of deep learning.

**Question**: A new method for detecting anomalies in time series data using machine learning

**Documents**: A review of anomaly detection techniques for cyber security

This paper presents a comprehensive review of the various anomaly detection techniques used in cyber security. The review is organized based on the type of anomaly detection techniques, namely, statistical, machine learning, and hybrid techniques. The paper discusses the advantages and limitations of each technique, and provides a comparative analysis of their performance. The review also highlights the challenges faced in anomaly detection, such as the difficulty in defining normal behavior, the high false positive rate, and the need for real-time detection. Finally, the paper identifies the future research directions in the field of anomaly detection for cyber security.

**Similar Documents**: This document offers an exhaustive overview of anomaly detection methodologies pertinent to cyber security, categorizing them under statistical, machine learning, and hybrid approaches. It delves into the merits and drawbacks of each technique, offering a detailed comparison of their effectiveness. The review underscores the hurdles in anomaly detection, notably the ambiguity in defining normalcy, the prevalence of false positives, and the imperative for immediate detection. Additionally, it outlines the evolving research trends in anomaly detection within the cyber security domain, pinpointing areas requiring further investigation.

This thesis studies the new possibilities to gesture interfaces that emerged with a Leap Motion sensor. The Leap Motion is an innovative, 3D motion capturing device designed especially for hands and fingers tracking with precision up to 0.01mm. The outcome of the thesis is the LeapGesture library dedicated to the developers for Leap Motion Controller that contains algorithms allowing to learn and recognize gestures. The authors examined the data provided by the sensor in context of recognition of hand poses (static gestures), hand movements (dynamic gestures) and in task of a finger recognition. The static gestures are recognized using the Support Vector Machine (SVM) with median filtering an input data and using the correspondences between consecutive recognitions. The thesis contains evaluation of different feature sets, which have a significant impact on the recognition rate. The chosen feature set allowed to recognize a set of five gestures with 99% accuracy and a set of ten gestures with 85%. The dynamic gestures (movements of a hand and fingers) are recognized with the Hidden Markov Models (HMM). Recognition with HMMs allowed to achieve accuracy of 80% for a set containing six classes of dynamic gestures. Finger recognition algorithms proposed in this thesis works with 93% accuracy on a recorded dataset. The LeapGesture library contains presented approaches using a C++ interface, that can be easily used in any application for many purposes.

—In the last decades, the researchers of the human arm prosthesis are using different types of machine learning algorithms. This review article firstly gives a brief explanation about type of machine learning methods. Secondly, some recent applications of myoelectric control of human arm prosthesis by using machine learning algorithms are compared. This study presents two different comparisons based on feature extraction methods which are time series modeling and wavelet transform of EMG signal. Finally, of characterization of EMG for of human arm prosthesis have been and discussed.

Demand for interactive wearable devices is rapidly increasing with the development of smart devices. To accurately utilize wearable devices for remote robot controls, limited data should be analyzed and utilized efficiently. For example, the motions by a wearable device, called Myo device, can be estimated by measuring its orientation, and calculating a Bayesian probability based on these orientation data. Given that Myo device can measure various types of data, the accuracy of its motion estimation can be increased by utilizing these additional types of data. This paper proposes a motion estimation method based on weighted Bayesian probability and concurrently measured data, orientations and electromyograms (EMG). The most probable motion among estimated is treated as a final estimated motion. Thus, recognition accuracy can be improved when compared to the traditional methods that employ only a single type of data. In our experiments, seven subjects perform five predefined motions. When orientation is measured by the traditional methods, the sum of the motion estimation errors is 37.3%; likewise, when only EMG data are used, the error in motion estimation by the proposed method was also 37.3%. The proposed combined method has an error of 25%. Therefore, the proposed method reduces motion estimation errors by 12%.

This paper proposes AutoRec, a novel autoencoder framework for collaborative filtering (CF). Empirically, AutoRec's compact and efficiently trainable model outperforms state-of-the-art CF techniques (biased matrix factorization, RBM-CF and LLORMA) on the Movielens and Netflix datasets.

The autoencoder is an artificial neural network that learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar to the eigenvalues in PCA that are paired with eigenvectors. We propose a novel supervised node saliency (SNS) method that ranks the hidden nodes, which contain weight vectors for transformations. SNS is able to indicate the nodes specialized in a learning task. The latent representations of a hidden node can be described using a one-dimensional histogram. We apply normalized entropy difference (NED) to measure the ”interestingness” of the histograms, and conclude a property for NED values to identify a good classifying node. By applying our methods to real datasets, we demonstrate their ability to find valuable nodes and explain the learned tasks in autoencoders.

A new classification approach for human body postures based on a neural fuzzy network is proposed in this paper, and the approach is applied to detect emergencies that are caused by accidental falls. Four main body postures are used for posture classification, including standing, bending, sitting, and lying. After the human body is segmented from the background, the classification features are extracted from the silhouette. The body silhouette is projected onto horizontal and vertical axes, and then, a discrete Fourier transform is applied to each projected histogram. Magnitudes of significant Fourier transform coefficients together with the silhouette length-width ratio are used as features. The classifier is designed by a neural fuzzy network. The four postures can be classified with high accuracy according to experimental results. Classification results are also applicable to home care emergency detection of a person who suddenly falls and remains in the lying posture for a period of time due to experiments that were performed.

Motor Imagery Electroencephalography (MI-EEG) plays an important role in brain computer interface (BCI) based rehabilitation robot, and its recognition is the key problem. The Discrete Wavelet Transform (DWT) has been applied to extract the time-frequency features of MI-EEG. However, the existing EEG classifiers, such as support vector machine (SVM), linear discriminant analysis (LDA) and BP network, did not make full use of the time sequence information in time-frequency features, the resulting recognition performance were not very ideal. In this paper, a Long Short-Term Memory (LSTM) based recurrent Neural Network (RNN) is integrated with Discrete Wavelet Transform (DWT) to yield a novel recognition method, denoted as DWT-LSTM. DWT is applied to analyze the each channel of MI-EEG and extract its effective wavelet coefficients, representing the time-frequency features. Then a LSTM based RNN is used as a classifier for the patten recognition of observed MI-EEG data. Experiments are conducted on a publicly available dataset, and the 5-fold cross validation experimental results show that DWT-LSTM yields relatively higher classification accuracies compared to the existing approaches. This is helpful for the further research and application of RNN in processing of MI-EEG.

Activity recognition is an important subject in different areas of research, like e-health and ubiquitous computing. In this paper we give an overview of recent work in the field of activity recognition from both body mounted sensors and ambient sensors. We discuss some of the differences among approaches and present guidelines for choosing a suitable approach under various circumstances.

Pattern recognition is one of the traditional uses of neural networks. When trained with gradient-based learning methods, these networks can learn the classification of input data by example. An introduction to classifiers and gradient-based learning is given. It is shown how several perceptrons can be combined and trained gradient-based. Furthermore, an overview of convolutional neural networks, as well as a real-world example, are discussed.

We propose DeepHand to estimate the 3D pose of a hand using depth data from commercial 3D sensors. We discriminatively train convolutional neural networks to output a low dimensional activation feature given a depth map. This activation feature vector is representative of the global or local joint angle parameters of a hand pose. We efficiently identify 'spatial' nearest neighbors to the activation feature, from a database of features corresponding to synthetic depth maps, and store some 'temporal' neighbors from previous frames. Our matrix completion algorithm uses these 'spatio-temporal' activation features and the corresponding known pose parameter values to estimate the unknown pose parameters of the input feature vector. Our database of activation features supplements large viewpoint coverage and our hierarchical estimation of pose parameters is robust to occlusions. We show that our approach compares favorably to state-of-the-art methods while achieving real time performance (≈ 32 FPS) on a standard computer.


Question: Using a stacked autoencoder for the classification of human activities.

This paper presents a Public Domain Dataset designed for Human Activity Recognition (HAR) utilizing smartphones. The dataset was compiled to support research in human-centered computing, a field focused on integrating human behavior and social context with computing systems. A significant application in this domain involves using smartphones to sense human body motion for gathering contextual information about daily activities. The described dataset was constructed by recording 30 participants performing Activities of Daily Living (ADL) while wearing a smartphone equipped with inertial sensors at the waist. This comprehensive dataset is now publicly available on a reputable online repository. Additionally, the paper includes the outcomes of applying a multiclass Support Vector Machine (SVM) to the dataset, showcasing the potential for machine learning in classifying human activities based on smartphone sensor data. The results highlight the efficacy of using SVM for HAR, providing a benchmark for future research and development in this area.

An important aspect of human perception is anticipation and anticipating which activities will a human do next (and how to do them) in useful for many applications, for example, anticipation enables an assistive robot to plan ahead for reactive responses in the human environments. In this work, we present a constructive approach for generating various possible future human activities by reasoning about the rich spatial-temporal relations through object affordances. We represent each possible future using an anticipatory temporal conditional random field (ATCRF) where we sample the nodes and edges corresponding to future object trajectories and human poses from a generative model. We then represent the distribution over the potential futures using a set of constructed ATCRF particles. In extensive evaluation on CAD-120 human activity RGB-D dataset, for new subjects (not seen in the training set), we obtain an activity anticipation accuracy (defined as whether one of top three predictions actually happened) of 75.4%, 69.2% and 58.1% for an anticipation time of 1, 3 and 10 seconds respectively. 1

PURPOSE
Large physical activity surveillance projects such as the UK Biobank and NHANES are using wrist-worn accelerometer-based activity monitors that collect raw data. The goal is to increase wear time by asking subjects to wear the monitors on the wrist instead of the hip, and then to use information in the raw signal to improve activity type and intensity estimation. The purposes of this work was to obtain an algorithm to process wrist and ankle raw data and to classify behavior into four broad activity classes: ambulation, cycling, sedentary, and other activities.

METHODS
Participants (N = 33) wearing accelerometers on the wrist and ankle performed 26 daily activities. The accelerometer data were collected, cleaned, and preprocessed to extract features that characterize 2-, 4-, and 12.8-s data windows. Feature vectors encoding information about frequency and intensity of motion extracted from analysis of the raw signal were used with a support vector machine classifier to identify a subject's activity. Results were compared with categories classified by a human observer. Algorithms were validated using a leave-one-subject-out strategy. The computational complexity of each processing step was also evaluated.

RESULTS
With 12.8-s windows, the proposed strategy showed high classification accuracies for ankle data (95.0%) that decreased to 84.7% for wrist data. Shorter (4 s) windows only minimally decreased performances of the algorithm on the wrist to 84.2%.

CONCLUSIONS
A classification algorithm using 13 features shows good classification into the four classes given the complexity of the activities in the original data set. The algorithm is computationally efficient and could be implemented in real time on mobile devices with only 4-s latency.

Group activity recognition plays a fundamental role in a variety of applications, e.g. sports video analysis and intelligent surveillance. How to model the spatio-temporal contextual information in a scene still remains a crucial yet challenging issue. We propose a novel attentive semantic recurrent neural network (RNN), dubbed as stagNet, for understanding group activities in videos, based on the spatio-temporal attention and semantic graph. A semantic graph is explicitly modeled to describe the spatial context of the whole scene, which is further integrated with the temporal factor via structural-RNN. Benefiting from the ‘factor sharing’ and ‘message passing’ mechanisms, our model is capable of extracting discriminative spatio-temporal features and capturing inter-group relationships. Moreover, we adopt a spatio-temporal attention model to attend to key persons/frames for improved performance. Two widely-used datasets are employed for performance evaluation, and the extensive results demonstrate the superiority of our method.

OBJECTIVE
Brain-computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional neural networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible.

APPROACH
In this work we introduce EEGNet, a compact convolutional neural network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR).

MAIN RESULTS
We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, the reference algorithms when only limited training data is available across all tested paradigms. In addition, we demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features.

SIGNIFICANCE
Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks. Our models can be found at: https://github.com/vlawhern/arl-eegmodels.

Interpreting brain image experiments requires analysis of complex, multivariate data. In recent years, one analysis approach that has grown in popularity is the use of machine learning algorithms to train classifiers to decode stimuli, mental states, behaviours and other variables of interest from fMRI data and thereby show the data contain information about them. In this tutorial overview we review some of the key choices faced in using this approach as well as how to derive statistically significant results, illustrating each point from a case study. Furthermore, we show how, in addition to answering the question of 'is there information about a variable of interest' (pattern discrimination), classifiers can be used to tackle other classes of question, namely 'where is the information' (pattern localization) and 'how is that information encoded' (pattern characterization).

Deep networks have achieved excellent performance in learning representation from visual data. However, the supervised deep models like convolutional neural network require large quantities of labeled data, which are very expensive to obtain. To solve this problem, this paper proposes an unsupervised deep network, called the stacked convolutional denoising auto-encoders, which can map images to hierarchical representations without any label information. The network, optimized by layer-wise training, is constructed by stacking layers of denoising auto-encoders in a convolutional way. In each layer, high dimensional feature maps are generated by convolving features of the lower layer with kernels learned by a denoising auto-encoder. The auto-encoder is trained on patches extracted from feature maps in the lower layer to learn robust feature detectors. To better train the large network, a layer-wise whitening technique is introduced into the model. Before each convolutional layer, a whitening layer is embedded to sphere the input data. By layers of mapping, raw images are transformed into high-level feature representations which would boost the performance of the subsequent support vector machine classifier. The proposed algorithm is evaluated by extensive experimentations and demonstrates superior classification performance to state-of-the-art unsupervised networks.

Neural network (NN) bottleneck (BN) features are typically created by training a NN with a middle bottleneck layer. Recently, an alternative structure was proposed which trains a NN with a constant number of hidden units to predict output targets, and then reduces the dimensionality of these output probabilities through an auto-encoder, to create auto-encoder bottleneck (AE-BN) features. The benefit of placing the BN after the posterior estimation network is that it avoids the loss in frame classification accuracy incurred by networks that place the BN before the softmax. In this work, we investigate the use of pre-training when creating AE-BN features. Our experiments indicate that with the AE-BN architecture, pre-trained and deeper NNs produce better AE-BN features. On a 50-hour English Broadcast News task, the AE-BN features provide over a 1% absolute improvement compared to a state-of-the-art GMM/HMM with a WER of 18.8% and pre-trained NN hybrid system with a WER of 18.4%. In addition, on a larger 430-hour Broadcast News task, AE-BN features provide a 0.5% absolute improvement over a strong GMM/HMM baseline with a WER of 16.0%. Finally, system combination with the GMM/HMM baseline and AE-BN systems provides an additional 0.5% absolute on 430 hours over the AE-BN system alone, yielding a final WER of 15.0%.

Classifying electroencephalogram (EEG) signal in Brain Computer Interface (BCI) is a useful methods to analysis different organs of human body and it can be used for communicate with the outside world and controlling external device. Accuracy classification of extracted features from EEG signals is a problem which many researcher try to improve it. Although many methods for extracting feature and classifying EEG signal have been proposed and developed, many of them suffer from extracting less accurate data from EEG signals. In this work, four signal feature extraction and three ensemble learning method have been reviewed and performances of classification techniques are compared for motor imagery task.

In this paper, we use recurrent autoencoder model to predict the time series in single and multiple steps ahead. Previous prediction methods, such as recurrent neural network (RNN) and deep belief network (DBN) models, cannot learn long term dependencies. And conventional long short-term memory (LSTM) model doesn’t remember recent inputs. Combining LSTM and autoencoder (AE), the proposed model can capture long-term dependencies across data points and uses features extracted from recent observations for augmenting LSTM at the same time. Based on comprehensive experiments, we show that the proposed methods significantly improves the state-of-art performance on chaotic time series benchmark and also has better performance on real-world data. Both single-output and multiple-output predictions are investigated.

We present a real-time gesture classification system for skeletal wireframe motion. Its key components include an angular representation of the skeleton designed for recognition robustness under noisy input, a cascaded correlation-based classifier for multivariate time-series data, and a distance metric based on dynamic time-warping to evaluate the difference in motion between an acquired gesture and an oracle for the matching gesture. While the first and last tools are generic in nature and could be applied to any gesture-matching scenario, the classifier is conceived based on the assumption that the input motion adheres to a known, canonical time-base: a musical beat. On a benchmark comprising 28 gesture classes, hundreds of gesture instances recorded using the XBOX Kinect platform and performed by dozens of subjects for each gesture class, our classifier has an average accuracy of 96:9%, for approximately 4-second skeletal motion recordings. This accuracy is remarkable given the input noise from the real-time depth sensor.


Question: RRT* with bidirectional guidance for rapid optimal path planning in cluttered settings.

This paper presents an advanced path planning strategy, Potentially Guided Bidirectionalized RRT* (PG-BiRRT*), designed for rapid and optimal navigation in densely cluttered environments. Unlike traditional methods, PG-BiRRT* integrates potential field guidance into the bidirectional RRT* algorithm, significantly enhancing the efficiency of finding the shortest path while avoiding collisions. The potential field, a virtual force field, directs the tree growth towards the goal while repelling obstacles, ensuring that the tree expansion is always goal-oriented and obstacle-aware. This approach not only accelerates the convergence to the optimal solution but also maintains robustness in complex scenarios. The algorithm is particularly suitable for real-time applications, such as obstacle avoidance for manipulators and mobile robots, where dynamic and unpredictable environments demand quick and adaptive path planning. Through extensive simulations and comparisons with standard RRT* and BiRRT*, PG-BiRRT* demonstrates its superior performance in terms of path optimality and computation time, making it a promising solution for autonomous navigation in cluttered environments.

Coordinating the motion of multiple mobile robots is one of the fundamental problems in robotics. The predominant algorithms for coordinating teams of robots are decoupled and prioritized, thereby avoiding combinatorially hard planning problems typically faced by centralized approaches. While these methods are very efficient, they have two major drawbacks. First, they are incomplete, i.e. they sometimes fail to find a solution even if one exists, and second, the resulting solutions are often not optimal. In this paper we present a method for finding and optimizing priority schemes for such prioritized and decoupled planning techniques. Existing approaches apply a single priority scheme which makes them overly prone to failure in cases where valid solutions exist. By searching in the space of priorization schemes, our approach overcomes this limitation. It performs a randomized search with hill-climbing to find solutions and to minimize the overall path length. To focus the search, our algorithm is guided by constraints generated from the task specification. To illustrate the appropriateness of this approach, this paper discusses experimental results obtained with real robots and through systematic robot simulation. The experimental results illustrate the superior performance of our approach, both in terms of efficiency of robot motion and in the ability to find valid plans.

In this paper the possibilities that Matlab provides to design, implementation and monitoring programs of autonomous navigation for mobile robots, on both simulated and real platforms, through its new toolbox for robotics will be explored. Robotics System Toolbox has established itself as a solid tool of integration between Matlab and robot operating under ROS environment. It will be studied the tools available in this toolbox that allow achieving the connection between the two platforms, in addition to the generation of algorithms of location, path planning, mapping and autonomous navigation.

Markov decision processes (MDPs) have recently been appbed to the problem of modeling decisiontheoretic planning While traditional methods for solving MDPs are often practical for small states spaces, their effectiveness for large AI planning problems is questionable We present an algorithm, called structured policy Iteration (SPI), that con structs optimal policies without explicit enumera tion of the state space The algorithm retains the fundamental computational steps of the commonly used modified policy iteration algorithm, but ex ploits the variable and prepositional independencies reflected in a temporal Bayesian network represen tation of MDPs The principles behind SPI can be applied to any structured representation of stochas tic actions, pobcies and value functions, and the algorithm itself can be used in conjunction with re cent approximation methods

We introduce a new problem of continuous, coverage-aware trajectory optimization under localization and sensing uncertainty. In this problem, the goal is to plan a path from a start state to a goal state that maximizes the coverage of a user-specified region while minimizing the control costs of the robot and the probability of collision with the environment. We present a principled method for quantifying the coverage sensing uncertainty of the robot. We use this sensing uncertainty along with the uncertainty in robot localization to develop C-OPT, a coverage-optimization algorithm which optimizes trajectories over belief-space to find locally optimal coverage paths. We highlight the applicability of our approach in multiple simulated scenarios inspired by surveillance, UAV crop analysis, and search-and-rescue tasks. We also present a case study on a physical, differential-drive robot. We also provide quantitative and qualitative analysis of the paths generated by our approach.

Micro aerial vehicles, such as multirotors, are particular well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance in industrial plants. Key prerequisites for the fully autonomous operation of micro aerial vehicles in restricted environments are 3D mapping, real-time pose tracking, obstacle detection, and planning of collision-free trajectories. In this article, we propose a complete navigation system with a multimodal sensor setup for omnidirectional environment perception. Measurements of a 3D laser scanner are aggregated in egocentric local multiresolution grid maps. Local maps are registered and merged to allocentric maps in which the MAV localizes. For autonomous navigation, we generate trajectories in a multi-layered approach: from mission planning over global and local trajectory planning to reactive obstacle avoidance. We evaluate our approach in a GNSS-denied indoor environment where multiple collision hazards require reliable omnidirectional perception and quick navigation reactions.

This paper describes the Q-routing algorithm for packet routing, in which a reinforcement learning module is embedded into each node of a switching network. Only local communication is used by each node to keep accurate statistics on which routing decisions lead to minimal delivery times. In simple experiments involving a 36-node, irregularly connected network, Q-routing proves superior to a nonadaptive algorithm based on precomputed shortest paths and is able to route e ciently even when critical aspects of the simulation, such as the network load, are allowed to vary dynamically. The paper concludes with a discussion of the tradeo between discovering shortcuts and maintaining stable policies.

In this paper, we propose an autonomous navigation for Unmanned Ground Vehicles (UGVs) by a path selection method based on local features of terrain in unknown outdoor rough terrain environment. The correlation between a local terrain feature obtained from a path and a value of the path obtained from the global path planning is extracted in advance. When UGV comes to a branch while it is running, the value of path is estimated using the correlation with local terrain feature. Thereby, UGV navigation is performed by path selection under the criterion of shortest time in unknown outdoor environment. We experimented on a simulator and confirmed that the proposed method can select more effective paths in comparison with a simple path selection method.

Military operations are turning to more complex and advanced automation technologies for minimum risk and maximum efficiency.A critical piece to this strategy is unmanned aerial vehicles. Unmanned aerial vehicles require the intelligence to safely maneuver along a path to an intended target and avoiding obstacles such as other aircrafts or enemy threats. This paper presents a unique three-dimensional path planning problem formulation and solution approach using particle swarm optimization. The problem formulation was designed with three objectives: 1) minimize risk owing to enemy threats, 2) minimize fuel consumption incurred by deviating from the original path, and 3) fly over defined reconnaissance targets. The initial design point is defined as the original path of the unmanned aerial vehicles. Using particle swarm optimization, alternate paths are generated using B-spline curves, optimized based on the three defined objectives. The resulting paths can be optimized with a preference toward maximum safety, minimum fuel consumption, or target reconnaissance. This method has been implemented in a virtual environment where the generated alternate paths can be visualized interactively to better facilitate the decision-making process. The problem formulation and solution implementation is described along with the results from several simulated scenarios demonstrating the effectiveness of the method.

This paper presents the HDG learning algorithm, which uses a hierarchical decomposition of the state space to make learning to achieve goals more efficient with a small penalty in path quality. Special care must be taken when performing hierarchical planning and learning in stochastic domains, because macro-operators cannot be executed ballistically. The HDG algorithm, which is a descendent of Watkins’ Q-learning algorithm, is described here and preliminary empirical results are presented.

We consider the problem of human-assisted graph search: given a directed acyclic graph with some (unknown) target node(s), we consider the problem of finding the target node(s) by asking an omniscient human questions of the form “Is there a target node that is reachable from the current node?”. This general problem has applications in many domains that can utilize human intelligence, including curation of hierarchies, debugging workflows, image segmentation and categorization, interactive search and filter synthesis. To our knowledge, this work provides the first formal algorithmic study of the optimization of human computation for this problem. We study various dimensions of the problem space, providing algorithms and complexity results. We also compare the performance of our algorithm against other algorithms, for the problem of webpage categorization on a real taxonomy. Our framework and algorithms can be used in the design of an optimizer for crowdsourcing platforms such as Mechanical Turk.


Question: RRT* with bidirectional guidance for rapid optimal path planning in cluttered settings.

Rapidly-Exploring Random Trees (RRTs) have emerged as a powerful technique for path planning in complex environments. This innovative approach facilitates the exploration of a search space by constructing a tree of possible paths from a starting point to a goal. RRTs are particularly adept at navigating through cluttered spaces by randomly sampling the environment and incrementally building a roadmap that avoids obstacles. The method's efficiency and adaptability make it a preferred choice for real-time path planning in robotics and autonomous systems, where quick decision-making is critical. However, to further enhance the speed and optimality of path planning in densely populated areas, researchers have developed variations such as the bidirectionalized RRT*, which optimizes path finding by searching from both the start and goal positions simultaneously. This bidirectional approach significantly reduces the time required to find the shortest path, making it particularly suitable for scenarios with tight constraints on computational resources. By combining the robust exploration capabilities of RRTs with the efficiency of bidirectional search, this method offers a promising solution for fast and optimal path planning in cluttered environments.

Many relevant tasks require an agent to reach a certain state, or to manipulate objects into a desired configuration. For example, we might want a robot to align and assemble a gear onto an axle or insert and turn a key in a lock. These tasks present considerable difficulties for reinforcement learning approaches, since the natural reward function for such goal-oriented tasks is sparse and prohibitive amounts of exploration are required to reach the goal and receive a learning signal. Past approaches tackle these problems by manually designing a task-specific reward shaping function to help guide the learning. Instead, we propose a method to learn these tasks without requiring any prior task knowledge other than obtaining a single state in which the task is achieved. The robot is trained in “reverse”, gradually learning to reach the goal from a set of starting positions increasingly far from the goal. Our method automatically generates a curriculum of starting positions that adapts to the agent’s performance, leading to efficient training on such tasks. We demonstrate our approach on difficult simulated fine-grained manipulation problems, not solvable by state-of-the-art reinforcement learning methods.

We consider the task of driving a remote control car at high speeds through unstructured outdoor environments. We present an approach in which supervised learning is first used to estimate depths from single monocular images. The learning algorithm can be trained either on real camera images labeled with ground-truth distances to the closest obstacles, or on a training set consisting of synthetic graphics images. The resulting algorithm is able to learn monocular vision cues that accurately estimate the relative depths of obstacles in a scene. Reinforcement learning/policy search is then applied within a simulator that renders synthetic scenes. This learns a control policy that selects a steering direction as a function of the vision system's output. We present results evaluating the predictive ability of the algorithm both on held out test data, and in actual autonomous driving experiments.

This paper shows how the recent breakthroughs in reinforcement learning (RL) that have enabled robots to learn to play arcade video games, walk, or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear partial differential equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. Cyberphysical systems (e.g., hydraulic channels, transportation systems, the energy grid, and electromagnetic systems) are commonly modeled by PDEs, which historically have been a reliable way to enable engineering applications in these domains. However, it is known that the control of these PDE models is notoriously difficult. We show how neural network-based RL enables the control of discretized PDEs whose parameters are unknown, random, and time-varying. We introduce an algorithm of mutual weight regularization (MWR), which alleviates the curse of dimensionality of multi-agent control schemes by sharing experience between agents while giving each agent the opportunity to specialize its action policy so as to tailor it to the local parameters of the part of the system it is located in. A discretized PDE, such as the scalar Lighthill–Whitham–Richards PDE can indeed be considered as a macroscopic freeway traffic simulator and which presents the most salient challenges for learning to control large cyberphysical system with multiple agents. We consider two different discretization procedures and show the opportunities offered by applying deep reinforcement for continuous control on both. Using a neural RL PDE controller on a traffic flow simulation based on a Godunov discretization of the San Francisco Bay Bridge, we are able to achieve precise adaptive metering without model calibration thereby beating the state of the art in traffic metering. Furthermore, with the more accurate BeATS simulator, we manage to achieve a control performance on par with ALINEA, a state-of-the-art parametric control scheme, and show how using MWR improves the learning procedure.

Our goal in this work is to make high level decisions for mobile robots. In particular, given a queue of prioritized object delivery tasks, we wish to find a sequence of actions in real time to accomplish these tasks efficiently. We introduce a novel reinforcement learning algorithm called Smoothed Sarsa that learns a good policy for these delivery tasks by delaying the backup reinforcement step until the uncertainty in the state estimate improves. The state space is modeled by a Dynamic Bayesian Network and updated using a Region-based Particle Filter. We take advantage of the fact that only discrete (topological) representations of entity locations are needed for decision-making, to make the tracking and decision making more efficient. Our experiments show that policy search leads to faster task completion times as well as higher total reward compared to a manually crafted policy. Smoothed Sarsa learns a policy orders of magnitude faster than previous policy search algorithms. We demonstrate our results on the Player/Stage simulator and on the Pioneer robot.

Task planning for mobile robots usually relies solely on spatial information and on shallow domain knowledge, like labels attached to objects and places. Although spatial information is necessary for performing basic robot operations (navigation, localization, and obstacle avoidance), the use of deeper domain knowledge is pivotal to endow a robot with higher degrees of autonomy and intelligence. In this paper, we focus on semantic knowlege, and show how this type of knowledge can be profitably used for robot task planning. We start by defining a specific type of semantic maps, which integrate hierarchical spatial information and semantic knowledge. We then proceed to describe how these semantic maps can improve task planning in three ways: enriching the planning domain, relaxing unachievable goals, and improving the efficiency of the planner in large domains. Finally, we show several exepriments that demonstrate the effectiveness of our solutions in a doman involving robot navigation in a domestic environment.

We propose to take a novel approach to robot system design where each building block of a larger system is represented as a differentiable program, i.e. a deep neural network. This representation allows for integrating algorithmic planning and deep learning in a principled manner, and thus combine the benefits of model-free and model-based methods. We apply the proposed approach to a challenging partially observable robot navigation task. The robot must navigate to a goal in a previously unseen 3-D environment without knowing its initial location, and instead relying on a 2-D floor map and visual observations from an onboard camera. We introduce the Navigation Networks (NavNets) that encode state estimation, planning and acting in a single, end-to-end trainable recurrent neural network. In preliminary simulation experiments we successfully trained navigation networks to solve the challenging partially observable navigation task.

This work deals with navigation of an omni-directional mobile robot with active caster wheels. Initially, the posture of the omni-directional mobile robot is calculated by using the odometry information. Next, the position accuracy of the mobile robot is measured through comparison of the odometry information and the external sensor measurement. Finally, for successful navigation of the mobile robot, a motion planning algorithm that employs kinematic redundancy resolution method is proposed. Through experiments for multiple obstacles and multiple moving obstacles, the feasibility of the proposed navigation algorithm was verified.

In this paper we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains We begin by introducing the theory of Markov decision processes mdps and partially observable mdps pomdps We then outline a novel algorithm for solving pomdps o line and show how in some cases a nite memory controller can be extracted from the solution to a pomdp We conclude with a discussion of how our approach relates to previous work the complexity of nding exact solutions to pomdps and of some possibilities for nding approximate solutions

In this paper, a practical real-time path planning and robot navigation algorithm for a non-holonomic indoor mobile robot based on visual servoing is implemented. The proposed algorithm is divided into three parts; the first part uses Multi-Stencils Fast Marching (MSFM) as a path planning method. But the generated path results from fast marching methods when used directly, not guaranteed to be safe and smooth. Subsequently, the robot can touch corners, walls and obstacles. The proposed algorithm uses image processing methods to solve this problem. The second part estimates the position and orientation of the robot, from the visual information, to follow the desired path with avoiding obstacles. The third part proposes a decentralized PD-like Fuzzy Logic Controller (FLC) to keep up the robot on the desired path. Experimental results show that the developed design is valid to estimate shortest-path by avoiding obstacles and able to guide the robot to follow the path in real-time.

Evolution strategies (ES) are a family of black-box optimization algorithms able to train deep neural networks roughly as well as Q-learning and policy gradient methods on challenging deep reinforcement learning (RL) problems, but are much faster (e.g. hours vs. days) because they parallelize better. However, many RL problems require directed exploration because they have reward functions that are sparse or deceptive (i.e. contain local optima), and it is unknown how to encourage such exploration with ES. Here we show that algorithms that have been invented to promote directed exploration in small-scale evolved neural networks via populations of exploring agents, specifically novelty search (NS) and quality diversity (QD) algorithms, can be hybridized with ES to improve its performance on sparse or deceptive deep RL tasks, while retaining scalability. Our experiments confirm that the resultant new algorithms, NS-ES and two QD algorithms, NSR-ES and NSRA-ES, avoid local optima encountered by ES to achieve higher performance on Atari and simulated robots learning to walk around a deceptive trap. This paper thus introduces a family of fast, scalable algorithms for reinforcement learning that are capable of directed exploration. It also adds this new family of exploration algorithms to the RL toolbox and raises the interesting possibility that analogous algorithms with multiple simultaneous paths of exploration might also combine well with existing RL algorithms outside ES.


Question: Rapidly exploring random trees with bidirectional guidance for efficient optimal path planning in cluttered spaces.

This paper explores a novel random sampling strategy for path planning in robotics, aiming to provide a unified perspective on the recent advancements in randomized path planning techniques. These methods have gained popularity due to their versatility, applicable to a wide range of robotic systems, and their demonstrated effectiveness in practical scenarios. The core idea revolves around the stochastic exploration of the robot's configuration space through random sampling. Two specific planners, previously developed, are highlighted as exemplars of this approach, each employing distinct sampling strategies. Notably, these planners exhibit probabilistic completeness, meaning that given sufficient time, they are highly likely to discover a feasible path if one exists. Focusing on one of these planners, a detailed analysis is conducted to elucidate the correlation between the likelihood of failure and the duration of the planning process. Under conditions that define the navigability of the robot's environment, it is shown that the planning time scales logarithmically with the inverse of the acceptable failure probability. Additionally, the analysis reveals that the planning time increases in a manageable manner as the navigational challenges of the environment escalate. The paper concludes by outlining potential avenues for future research, inviting further exploration and innovation in the field of robotic path planning.

In this paper, a simple and efficient method is proposed using one of the basic reinforcement learning tool Q-learning. This method is to achieve the aim of mobile robot maples navigation with collision-free. Thus, an end-to-end navigation model is built, which uses lidar sensor massage as input and moving command of mobile robot as output, so as to simplify the process of environmental perception and decision making on action. Through simulation and experiments, its effectiveness is proved. After trained by this method, mobile robot can safely reach the navigation target in an unknown environment without any prior demonstration. In addition, an extensive quantitative and qualitative evaluation of this method is presented by the comparison with traditional path planning method based on foregone global environment map.

Wayfinding, i.e. getting from some origin to a destination, is one of the prime everyday problems humans encounter. It has received a lot of attention in research and many (commercial) systems propose assistance in this task. We present an approach to route directions based on the idea to adapt route directions to route and environment's characteristics. The lack of such an adaptation is a major drawback of existing systems. Our approach is based on an informationand representation-theoretic analysis of routes and takes into account findings of behavioral research. The resulting systematics is the framework for the optimization process. We discuss the consequences of using an optimization process for generating route directions and outline its algorithmic realization.

This paper presents the design and first test on a simulator of a vehicle trajectory-planning algorithm that adapts to traffic on a lane-structured infrastructure such as highways. The proposed algorithm is designed to run on a fail-safe embedded environment with low computational power, such as an engine control unit, to be implementable in commercial vehicles of the near future. The target platform has a clock frequency of less than 150 MHz, 150 kB RAM of memory, and a 3-MB program memory. The trajectory planning is performed by a two-step algorithm. The first step defines the feasible maneuvers with respect to the environment, aiming at minimizing the risk of a collision. The output of this step is a target group of maneuvers in the longitudinal direction (accelerating or decelerating), in the lateral direction (changing lanes), and in the combination of both directions. The second step is a more detailed evaluation of several possible trajectories within these maneuvers. The trajectories are optimized to additional performance indicators such as travel time, traffic rules, consumption, and comfort. The output of this module is a trajectory in the vehicle frame that represents the recommended vehicle state (position, heading, speed, and acceleration) for the following seconds.

This paper describes an algorithm for deploying the members of a mobile robot team into an unknown environment. The algorithm deploys robots one-at-a-time, with each robot making use of information gathered by the previous robots to determine the next deployment location. The deployment pattern is designed to maximize the area covered by the robots’ sensors, while simultaneously ensuring that the robots maintain line-of-sight contact with one another. This paper describes the basic algorithm and presents results obtained from a series of experiments conducted using both real and simulated

The ability to simultaneously localize a robot and accurately map its surroundings is considered by many to be a key prerequisite of truly autonomous robots. However, few approaches to this problem scale up to handle the very large number of landmarks present in real environments. Kalman filter-based algorithms, for example, require time quadratic in the number of landmarks to incorporate each sensor observation. This paper presents FastSLAM, an algorithm that recursively estimates the full posterior distribution over robot pose and landmark locations, yet scales logarithmically with the number of landmarks in the map. This algorithm is based on an exact factorization of the posterior into a product of conditional landmark distributions and a distribution over robot paths. The algorithm has been run successfully on as many as 50,000 landmarks, environments far beyond the reach of previous approaches. Experimental results demonstrate the advantages and limitations of the FastSLAM algorithm on both simulated and real-world data.

In this paper, we present an efficient 3D object recognition and pose estimation approach for grasping procedures in cluttered and occluded environments. In contrast to common appearance-based approaches, we rely solely on 3D geometry information. Our method is based on a robust geometric descriptor, a hashing technique and an efficient, localized RANSAC-like sampling strategy. We assume that each object is represented by a model consisting of a set of points with corresponding surface normals. Our method simultaneously recognizes multiple model instances and estimates their pose in the scene. A variety of tests shows that the proposed method performs well on noisy, cluttered and unsegmented range scans in which only small parts of the objects are visible. The main procedure of the algorithm has a linear time complexity resulting in a high recognition speed which allows a direct integration of the method into a continuous manipulation task. The experimental validation with a 7-degrees-of-freedom Cartesian impedance controlled robot shows how the method can be used for grasping objects from a complex random stack. This application demonstrates how the integration of computer vision and softrobotics leads to a robotic system capable of acting in unstructured and occluded environments.

This paper presents a novel optimization based path planner that can simultaneously plan multiple contingency paths to account for the uncertain actions of dynamic obstacles. This planner addresses the particular problem of collision avoidance for autonomous road vehicles which are required to safely interact with other vehicles with unknown intentions. The presented path planner utilizes an efficient spline based trajectory representation and fast but accurate collision probability approximations to enable the simultaneous optimization of multiple contingency paths.

Classic approaches to game AI require either a high quality of domain knowledge, or a long time to generate effective AI behaviour. These two characteristics hamper the goal of establishing challenging game AI. In this paper, we put forward Monte-Carlo Tree Search as a novel, unified framework to game AI. In the framework, randomized explorations of the search space are used to predict the most promising game actions. We will demonstrate that Monte-Carlo Tree Search can be applied effectively to (1) classic board-games, (2) modern board-games, and (3) video games.

In the past, there has been a tremendous progress in the area of autonomous robot navigation and a large variety of robots have been developed who demonstrated robust navigation capabilities indoors, in non-urban outdoor environments, or on roads and relatively few approaches focus on navigation in urban environments such as city centers. Urban areas, however, introduce numerous challenges for autonomous robots as they are rather unstructured and dynamic. In this paper, we present a navigation system for mobile robots designed to operate in crowded city environments and pedestrian zones. We describe the different components of this system including a SLAM module for dealing with huge maps of city centers, a planning component for inferring feasible paths taking also into account the traversability and type of terrain, a module for accurate localization in dynamic environments, and means for calibrating and monitoring the platform. Our navigation system has been implemented and tested in several large-scale field tests, in which a real robot autonomously navigated over several kilometers in a complex urban environment. This also included a public demonstration, during which the robot autonomously traveled along a more than three kilometer long route through the city center of Freiburg, Germany.

The subject of this paper is the design, control, and evaluation of a biped–climbing robot featuring a new hybrid hip joint. The hybrid hip provides both prismatic and revolute motion, discretely, to the robot, using a single actuator. This is intended to improve its adaptability in confined environments and its capability to maneuver over and around obstacles. Optimization of the hybrid hip joint relative to robot size, weight, and actuation limits is considered while maximizing range of motion. The mechanical structure of the robot is discussed, as well as forward and inverse kinematics for motion planning. Reachable robot workspace in both revolute and prismatic phases of the hybrid hip joint is analyzed. Different robot locomotion strides are developed and dynamic controller requirements are considered. Experimental evaluation of the robot walking and climbing on and between surfaces with different inclinations is conducted to evaluate performance and repeatability of the system. KEY WORDS—biped robot, climbing robot, robot design, kinematics, locomotion strides