Question: Learning deterministic decision trees in an end-to-end manner.

This paper presents Adam, a novel algorithm for first-order gradient-based optimization of stochastic objective functions. Adam is distinguished by its simplicity in implementation, computational efficiency, minimal memory demands, invariance to diagonal rescaling of gradients, and its suitability for large-scale optimization tasks in terms of both data volume and parameter size. It is particularly effective for non-stationary objectives and scenarios with highly noisy or sparse gradients. The algorithm's hyper-parameters are intuitive, reducing the need for extensive tuning. The paper also explores the theoretical underpinnings of Adam, highlighting its connections to related algorithms and providing a regret bound on the convergence rate that is on par with the best known results within the online convex optimization framework. Empirical evidence confirms Adam's practical effectiveness and its competitive edge over other stochastic optimization techniques. Additionally, the paper introduces AdaMax, an adaptation of Adam that employs the infinity norm.

In this thesis proposal, we address the limitations of conventional pipeline design of taskoriented dialog systems and propose end-toend learning solutions. We design neural network based dialog system that is able to robustly track dialog state, interface with knowledge bases, and incorporate structured query results into system responses to successfully complete task-oriented dialog. In learning such neural network based dialog systems, we propose hybrid offline training and online interactive learning methods. We introduce a multi-task learning method in pre-training the dialog agent in a supervised manner using task-oriented dialog corpora. The supervised training agent can further be improved via interacting with users and learning online from user demonstration and feedback with imitation and reinforcement learning. In addressing the sample efficiency issue with online policy learning, we further propose a method by combining the learning-from-user and learningfrom-simulation approaches to improve the online interactive learning efficiency.

Behavior of Machine Learning Algorithms in Adversarial Environments

Within the framework of ADABOOST.MH, we propose to train vector-valued decision trees to optimize the multi-class edge without reducing the multi-class problem toK binary one-againstall classifications. The key element of the method is a vector-valued decision stump, factorized into an input-independent vector of length K and label-independent scalar classifier. At inner tree nodes, the label-dependent vector is discarded and the binary classifier can be used for partitioning the input space into two regions. The algorithm retains the conceptual elegance, power, and computational efficiency of binary ADABOOST. In experiments it is on par with support vector machines and with the best existing multi-class boosting algorithm AOSOLOGITBOOST, and it is significantly better than other known implementations of ADABOOST.MH.

We introduce the first temporal-difference learning algorithms that converge with smooth value function approximators, such as neural networks. Conventional temporal-difference (TD) methods, such as TD(λ), Q-learning and Sarsa have been used successfully with function approximation in many applications. However, it is well known that off-policy sampling, as well as nonlinear function approximation, can cause these algorithms to become unstable (i.e., the parameters of the approximator may diverge). Sutton et al. (2009a, 2009b) solved the problem of off-policy learning with linear TD algorithms by introducing a new objective function, related to the Bellman error, and algorithms that perform stochastic gradient-descent on this function. These methods can be viewed as natural generalizations to previous TD methods, as they converge to the same limit points when used with linear function approximation methods. We generalize this work to nonlinear function approximation. We present a Bellman error objective function and two gradient-descent TD algorithms that optimize it. We prove the asymptotic almost-sure convergence of both algorithms, for any finite Markov decision process and any smooth value function approximator, to a locally optimal solution. The algorithms are incremental and the computational complexity per time step scales linearly with the number of parameters of the approximator. Empirical results obtained in the game of Go demonstrate the algorithms’ effectiveness.

Sample-based tree search (SBTS) is an approach to solving Markov decision problems based on constructing a lookahead search tree using random samples from a generative model of the MDP. It encompasses Monte Carlo tree search (MCTS) algorithms like UCT as well as algorithms such as sparse sampling. SBTS is well-suited to solving MDPs with large state spaces due to the relative insensitivity of SBTS algorithms to the size of the state space. The limiting factor in the performance of SBTS tends to be the exponential dependence of sample complexity on the depth of the search tree. The number of samples required to build a search tree is O((|A|B)), where |A| is the number of available actions, B is the number of possible random outcomes of taking an action, and d is the depth of the tree. State abstraction can be used to reduce B by aggregating random outcomes together into abstract states. Recent work has shown that abstract tree search often performs substantially better than tree search conducted in the ground state space. This paper presents a theoretical and empirical evaluation of tree search with both fixed and adaptive state abstractions. We derive a bound on regret due to state abstraction in tree search that decomposes abstraction error into three components arising from properties of the abstraction and the search algorithm. We describe versions of popular SBTS algorithms that use fixed state abstractions, and we introduce the Progressive Abstraction Refinement in Sparse Sampling (PARSS) algorithm, which adapts its abstraction during search. We evaluate PARSS as well as sparse sampling with fixed abstractions on 12 experimental problems, and find that PARSS outperforms search with a fixed abstraction and that search with even highly inaccurate fixed abstractions outperforms search without abstraction. These results establish progressive abstraction refinement as a promising basis for new tree search algorithms, and we propose directions for future work within the progressive refinement framework.

In this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction. NTP takes as input a task specification (e.g., video demonstration of a task) and recursively decomposes it into finer sub-task specifications. These specifications are fed to a hierarchical neural program, where bottom-level programs are callable subroutines that interact with the environment. We validate our method in three robot manipulation tasks. NTP achieves strong generalization across sequential tasks that exhibit hierarchal and compositional structures. The experimental results show that NTP learns to generalize well towards unseen tasks with increasing lengths, variable topologies, and changing objectives.stanfordvl.github.io/ntp/.

This paper presents a general framework for employing deep architectures in particular neural networks to solve the inverse reinforcement learning (IRL) problem. Specifically, we propose to exploit the representational capacity and favourable computational complexity of deep networks to approximate complex, nonlinear reward functions. We show that the Maximum Entropy paradigm for IRL lends itself naturally to the efficient training of deep architectures. At test time, the approach leads to a computational complexity independent of the number of demonstrations. This makes it especially well-suited for applications in life-long learning scenarios commonly encountered in robotics. We demonstrate that our approach achieves performance commensurate to the state-of-the-art on existing benchmarks already with simple, comparatively shallow network architectures while significantly outperforming the stateof-the-art on an alternative benchmark based on more complex, highly varying reward structures representing strong interactions between features. Furthermore, we extend the approach to include convolutional layers in order to eliminate the dependency on precomputed features of current algorithms and to underline the substantial gain in flexibility in framing IRL in the context of deep learning.

Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. This leads to poor performance in theory and often in practice. Some recent approaches (Daumé III et al., 2009; Ross and Bagnell, 2010) provide stronger guarantees in this setting, but remain somewhat unsatisfactory as they train either non-stationary or stochastic policies and require a large number of iterations. In this paper, we propose a new iterative algorithm, which trains a stationary deterministic policy, that can be seen as a no regret algorithm in an online learning setting. We show that any such no regret algorithm, combined with additional reduction assumptions, must find a policy with good performance under the distribution of observations it induces in such sequential settings. We demonstrate that this new approach outperforms previous approaches on two challenging imitation learning problems and a benchmark sequence labeling problem.

Deep learning algorithms have recently appeared that pretrain hidden layers of neural networks in unsupervised ways, leading to state-of-the-art performance on large classification problems. These methods can also pretrain networks used for reinforcement learning. However, this ignores the additional information that exists in a reinforcement learning paradigm via the ongoing sequence of state, action, new state tuples. This paper demonstrates that learning a predictive model of state dynamics can result in a pretrained hidden layer structure that reduces the time needed to solve reinforcement learning problems.

Implementing E cient, Portable Computations for Machine Learning


Question: Learning deterministic decision trees in an end-to-end manner.

Innovative Development of Deterministic Decision Trees for Machine Learning. Segment A

This segment introduces an advanced methodology for the end-to-end learning of deterministic decision trees, a critical component in the field of machine learning. Unlike conventional approaches, the proposed method integrates learning and decision-making into a unified framework, enhancing the efficiency and effectiveness of decision tree construction. The core of this methodology lies in its ability to directly learn from data, eliminating the need for manual feature engineering and enabling the model to capture intricate patterns and relationships. The end-to-end learning process ensures that the decision tree evolves in a way that maximizes predictive accuracy while maintaining interpretability, a significant advantage over black-box models. By focusing on deterministic decision trees, this approach offers a clear path from input data to output decisions, facilitating easier understanding and validation of model predictions. The methodology is designed to be scalable, accommodating large datasets and complex decision-making scenarios, thereby broadening its applicability across various domains. Through rigorous testing and comparative analysis with traditional machine learning algorithms, the superior performance of this end-to-end learning approach for deterministic decision trees is convincingly demonstrated.

Transfer and multi-task learning have traditionally focused on either a single source-target pair or very few, similar tasks. Ideally, the linguistic levels of morphology, syntax and semantics would benefit each other by being trained in a single model. We introduce a joint many-task model together with a strategy for successively growing its depth to solve increasingly complex tasks. Higher layers include shortcut connections to lower-level task predictions to reflect linguistic hierarchies. We use a simple regularization term to allow for optimizing all model weights to improve one task’s loss without exhibiting catastrophic interference of the other tasks. Our single end-to-end model obtains state-of-the-art or competitive results on five different tasks from tagging, parsing, relatedness, and entailment tasks.

We introduce Embed to Control (E2C), a method for model learning and control of non-linear dynamical systems from raw pixel images. E2C consists of a deep generative model, belonging to the family of variational autoencoders, that learns to generate image trajectories from a latent space in which the dynamics is constrained to be locally linear. Our model is derived directly from an optimal control formulation in latent space, supports long-term prediction of image sequences and exhibits strong performance on a variety of complex control problems.

We present an approach for the verification of feed-forward neural networks in which all nodes have a piece-wise linear activation function. Such networks are often used in deep learning and have been shown to be hard to verify for modern satisfiability modulo theory (SMT) and integer linear programming (ILP) solvers. The starting point of our approach is the addition of a global linear approximation of the overall network behavior to the verification problem that helps with SMT-like reasoning over the network behavior. We present a specialized verification algorithm that employs this approximation in a search process in which it infers additional node phases for the non-linear nodes in the network from partial node phase assignments, similar to unit propagation in classical SAT solving. We also show how to infer additional conflict clauses and safe node fixtures from the results of the analysis steps performed during the search. The resulting approach is evaluated on collision avoidance and handwritten digit recognition case studies.

One-shot learning is usually tackled by using generative models or discriminative embeddings. Discriminative methods based on deep learning, which are very effective in other learning scenarios, are ill-suited for one-shot learning as they need large amounts of training data. In this paper, we propose a method to learn the parameters of a deep model in one shot. We construct the learner as a second deep network, called a learnet, which predicts the parameters of a pupil network from a single exemplar. In this manner we obtain an efficient feed-forward one-shot learner, trained end-to-end by minimizing a one-shot classification objective in a learning to learn formulation. In order to make the construction feasible, we propose a number of factorizations of the parameters of the pupil network. We demonstrate encouraging results by learning characters from single exemplars in Omniglot, and by tracking visual objects from a single initial exemplar in the Visual Object Tracking benchmark.

In this thesis proposal, we address the limitations of conventional pipeline design of taskoriented dialog systems and propose end-toend learning solutions. We design neural network based dialog system that is able to robustly track dialog state, interface with knowledge bases, and incorporate structured query results into system responses to successfully complete task-oriented dialog. In learning such neural network based dialog systems, we propose hybrid offline training and online interactive learning methods. We introduce a multi-task learning method in pre-training the dialog agent in a supervised manner using task-oriented dialog corpora. The supervised training agent can further be improved via interacting with users and learning online from user demonstration and feedback with imitation and reinforcement learning. In addressing the sample efficiency issue with online policy learning, we further propose a method by combining the learning-from-user and learningfrom-simulation approaches to improve the online interactive learning efficiency.

We present an extension of Monte Carlo Tree Search (MCTS) that strongly increases its efficiency for trees with asymmetry and/or loops. Asymmetric termination of search trees introduces a type of uncertainty for which the standard upper confidence bound (UCB) formula does not account. Our first algorithm (MCTS-T), which assumes a non-stochastic environment, backs-up tree structure uncertainty and leverages it for exploration in a modified UCB formula. Results show vastly improved efficiency in a well-known asymmetric domain in which MCTS performs arbitrarily bad. Next, we connect the ideas about asymmetric termination to the presence of loops in the tree, where the same state appears multiple times in a single trace. An extension to our algorithm (MCTS-T+), which in addition to nonstochasticity assumes full state observability, further increases search efficiency for domains with loops as well. Benchmark testing on a set of OpenAI Gym and Atari 2600 games indicates that our algorithms always perform better than or at least equivalent to standard MCTS, and could be first-choice tree search algorithms for non-stochastic, fully-observable environments.

Neural networks have achieved state-ofthe-art performance on several structuredoutput prediction tasks, trained in a fully supervised fashion. However, annotated examples in structured domains are often costly to obtain, which thus limits the applications of neural networks. In this work, we propose Maximum Margin Reward Networks, a neural networkbased framework that aims to learn from both explicit (full structures) and implicit supervision signals (delayed feedback on the correctness of the predicted structure). On named entity recognition and semantic parsing, our model outperforms previous systems on the benchmark datasets, CoNLL-2003 and WebQuestionsSP.

We present an algorithmic framework for supervised classification learning where the set of labels is organized in a predefined hierarchical structure. This structure is encoded by a rooted tree which induces a metric over the label set. Our approach combines ideas from large margin kernel methods and Bayesian analysis. Following the large margin principle, we associate a prototype with each label in the tree and formulate the learning task as an optimization problem with varying margin constraints. In the spirit of Bayesian methods, we impose similarity requirements between the prototypes corresponding to adjacent labels in the hierarchy. We describe new online and batch algorithms for solving the constrained optimization problem. We derive a worst case loss-bound for the online algorithm and provide generalization analysis for its batch counterpart. We demonstrate the merits of our approach with a series of experiments on synthetic, text and speech data.

This paper presents the HDG learning algorithm, which uses a hierarchical decomposition of the state space to make learning to achieve goals more efficient with a small penalty in path quality. Special care must be taken when performing hierarchical planning and learning in stochastic domains, because macro-operators cannot be executed ballistically. The HDG algorithm, which is a descendent of Watkins’ Q-learning algorithm, is described here and preliminary empirical results are presented.

In this paper, we propose a deep reinforcement learning(DRL) algorithm which combines Deep Deterministic Policy Gradient (DDPG) with expert demonstrations and supervised loss for decision making for autonomous driving. Training DRL agent with supervised learning is adopted to accelerate the exploration process and increase the stability. A supervised loss function is introduced in the algorithm to update the actor networks. In addition, reward construction is combined to make the training process more stable and efficient. The proposed algorithm is applied to a popular autonomous driving simulator called TORCS. The experimental results show that the training efficiency and stability are improved by utilizing our algorithm in autonomous driving.


Question: Learning deterministic decision trees in an end-to-end manner.

This document delves into the application of statistical mechanics principles to understand and analyze clustering phenomena in various data settings. Unlike conventional approaches to clustering, which often rely on heuristic methods, this paper adopts a theoretical framework grounded in statistical physics. By treating data points as particles in a thermodynamic system, it becomes possible to identify phase transitions—critical points where the system's behavior changes abruptly. These phase transitions correspond to distinct clustering regimes, offering insights into the underlying structure of the data and the optimal number of clusters. The paper also explores the relationship between clustering and energy landscapes, showing how the minimization of an energy function can lead to the emergence of stable clusters. This perspective not only enriches the theoretical foundation of clustering but also opens up new avenues for algorithm design, potentially leading to more efficient and robust clustering methods. The application of statistical mechanics to clustering not only enhances our understanding of data organization but also paves the way for developing novel clustering algorithms that are inspired by the principles of physics.

Skilled robot task learning is best implemented by predictive action policies due to the inherent latency of sensorimotor processes. However, training such predictive policies is challenging as it involves finding a trajectory of motor activations for the full duration of the action. We propose a data-efficient deep predictive policy training (DPPT) framework with a deep neural network policy architecture which maps an image observation to a sequence of motor activations. The architecture consists of three sub-networks referred to as the perception, policy and behavior super-layers. The perception and behavior super-layers force an abstraction of visual and motor data trained with synthetic and simulated training samples, respectively. The policy super-layer is a small subnetwork with fewer parameters that maps data in-between the abstracted manifolds. It is trained for each task using methods for policy search reinforcement learning. We demonstrate the suitability of the proposed architecture and learning framework by training predictive policies for skilled object grasping and ball throwing on a PR2 robot. The effectiveness of the method is illustrated by the fact that these tasks are trained using only about 180 real robot attempts with qualitative terminal rewards.

We consider learning in a Markov decision process where we are not explicitly given a reward function, but where instead we can observe an expert demonstrating the task that we want to learn to perform. This setting is useful in applications (such as the task of driving) where it may be difficult to write down an explicit reward function specifying exactly how different desiderata should be traded off. We think of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and give an algorithm for learning the task demonstrated by the expert. Our algorithm is based on using "inverse reinforcement learning" to try to recover the unknown reward function. We show that our algorithm terminates in a small number of iterations, and that even though we may never recover the expert's reward function, the policy output by the algorithm will attain performance close to that of the expert, where here performance is measured with respect to the expert's unknown reward function.

The problem of selecting determinant features generating appropriate model structure is a challenge in epidemiological modelling. Disease spread is highly complex, and experts develop their understanding of its dynamic over years. There is an increasing variety and volume of epidemiological data which adds to the potential confusion. The authors propose here to make use of that data to better understand disease systems. Decision tree techniques have been extensively used to extract pertinent information and improve decision making. In this paper, the authors propose an innovative structured approach combining decision tree induction with Bio-PEPA computational modelling, and illustrate the approach through application to tuberculosis. By using decision tree induction, the enhanced Bio-PEPA model shows considerable improvement over the initial model with regard to the simulated results matching observed data. The key finding is that the developer expresses a realistic predictive model using relevant features, thus considering this approach as decision support, empowers the epidemiologist in his policy decision making. KEywoRDS Bio-PEPA Modelling, Data Mining, Decision Support, Decision Tree Induction, Epidemiology, Modelling and Simulation, Optimisation, Refinement, Tuberculosis

In machine learning, there is a fundamental trade-off between ease of optimization and expressive power. Neural Networks, in particular, have enormous expressive power and yet are notoriously challenging to train. The nature of that optimization challenge changes over the course of learning. Traditionally in deep learning, one makes a static trade-off between the needs of early and late optimization. In this paper, we investigate a novel framework, GradNets, for dynamically adapting architectures during training to get the benefits of both. For example, we can gradually transition from linear to non-linear networks, deterministic to stochastic computation, shallow to deep architectures, or even simple downsampling to fully differentiable attention mechanisms. Benefits include increased accuracy, easier convergence with more complex architectures, solutions to test-time execution of batch normalization, and the ability to train networks of up to 200 layers.

We propose a new fast purely discriminative algorithm for natural language parsing, based on a “deep” recurrent convolutional graph transformer network (GTN). Assuming a decomposition of a parse tree into a stack of “levels”, the network predicts a level of the tree taking into account predictions of previous levels. Using only few basic text features which leverage word representations from Collobert and Weston (2008), we show similar performance (in F1 score) to existing pure discriminative parsers and existing “benchmark” parsers (like Collins parser, probabilistic context-free grammars based), with a huge speed advantage.

Reinforcement learning is considered to be a strong AI paradigm which can be used to teach machines through interaction with the environment and learning from their mistakes. Despite its perceived utility, it has not yet been successfully applied in automotive applications. Motivated by the successful demonstrations of learning of Atari games and Go by Google DeepMind, we propose a framework for autonomous driving using deep reinforcement learning. This is of particular relevance as it is difficult to pose autonomous driving as a supervised learning problem due to strong interactions with the environment including other vehicles, pedestrians and roadworks. As it is a relatively new area of research for autonomous driving, we provide a short overview of deep reinforcement learning and then describe our proposed framework. It incorporates Recurrent Neural Networks for information integration, enabling the car to handle partially observable scenarios. It also integrates the recent work on attention models to focus on relevant information, thereby reducing the computational complexity for deployment on embedded hardware. The framework was tested in an open source 3D car racing simulator called TORCS. Our simulation results demonstrate learning of autonomous maneuvering in a scenario of complex road curvatures and simple interaction of other vehicles. INTRODUCTION A robot car that drives autonomously is a long-standing goal of Artificial Intelligence. Driving a vehicle is a task that requires high level of skill, attention and experience from a human driver. Although computers are more capable of sustained attention and focus than humans, fully autonomous driving requires a level of intelligence that surpasses that achieved so far by AI agents. The tasks involved in creating an autonomous driving agent can be divided into 3 categories, as shown in Figure 1: 1) Recognition: Identifying components of the surrounding environment. Examples of this are pedestrian detection, traffic sign recognition, etc. Although far from trivial, recognition is a relatively easy task nowadays thanks to advances in Deep Learning (DL) algorithms, which have reached human level recognition or above at several object detection and classification problems [8] [2]. Deep learning models are able to learn complex feature representations from raw input data, omitting the need for handcrafted features [15] [2] [7]. In this regard, Convolutional Neural Networks (CNNs) are probably the most successful deep learning model, and have formed the basis of every winning entry on the ImageNet challenge since AlexNet [8]. This success has been replicated in lane & vehicle detection for autonomous driving [6]. 2) Prediction: It is not enough for an autonomous driving agent to recognize its environment; it must also be able to build internal models that predict the future states of the environment. Examples of this class of problem include building a map of the environment or tracking an object. To be able to predict the future, it is important to integrate past information. As such, Recurrent Neural Networks (RNNs) are essential to this class of problem. Long-Short Term Memory (LSTM) networks [5] are one such category of RNN that have been used in end-to-end scene labeling systems [14]. More recently, RNNs have also been used to improve object tracking performance in the DeepTracking model [13]. 3) Planning: The generation of an efficient model that incorporates recognition and prediction to plan the future sequence of driving actions that will enable the vehicle to navigate successfully. Planning is the hardest task of the three. The difficulty lies in integrating the ability of the model to understand the environment (recognition) and its dynamics (prediction) in a way that enables it to plan the future actions such that it avoids unwanted situations (penalties) and drives safely to its destination (rewards). Figure 1: High level autonomous driving tasks The Reinforcement Learning (RL) framework [17] [20] has been used for a long time in control tasks. The mixture of RL with DL was pointed out to be one of the most promising approaches to achieve human-level control in [9]. In [12] and [11] this humanlevel control was demonstrated on Atari games using the Deep Q Networks (DQN) model, in which RL is responsible for the planning part while DL is responsible for the representation learning part. Later, RNNs were integrated in the mixture to account for partial observable scenarios [4]. Autonomous driving requires the integration of information ar X iv :1 70 4. 02 53 2v 1 [ st at .M L ] 8 A pr 2 01 7 from multiple sensors. Some of them are low dimensional, like LIDAR, while others are high dimensional, like cameras. It is noteworthy in this particular example, however, that although raw camera images are high dimensional, the useful information needed to achieve the autonomous driving task is of much lower dimension. For example, the important parts of the scene that affect driving decisions are limited to the moving vehicle, free space on the road ahead, the position of kerbs, etc. Even the fine details of vehicles are not important, as only their spatial location is truly necessary for the problem. Hence the memory bandwidth for relevant information is much lower. If this relevant information can be extracted, while the other non-relevant parts are filtered out, it would improve both the accuracy and efficiency of autonomous driving systems. Moreover, this would reduce the computation and memory requirements of the system, which are critical constraints on embedded systems that will contain the autonomous driving control unit. Attention models are a natural fit for such an information filtering process. Recently, these models were successfully deployed for image recognition in [23] and [10], wherein RL was mixed with RNNs to obtain the parts of the image to attend to. Such models are easily extended and integrated to the DQN [11] and Deep Recurrent Q Networks (DRQN) [4] models. This integration was performed in [16]. The success of attention models drives us to propose them for the extraction of low level information from the raw sensory information to perform autonomous driving. In this paper, we propose a framework for an end-end autonomous driving model that takes in raw sensor inputs and outputs driving actions. The model is able to handle partially observable scenarios. Moreover, we propose to integrate the recent advances in attention models in order to extract only relevant information from the received sensors data, thereby making it suitable for real-time embedded systems. The main contributions of this paper: 1) presenting a survey of the recent advances of deep reinforcement learning and 2) introducing a framework for endend autonomous driving using deep reinforcement learning to the automotive community. The rest of the paper is divided into two parts. The first part provides a survey of deep reinforcement learning algorithms, starting with the traditional MDP framework and Q-learning, followed by the the DQN, DRQN and Deep Attention Recurrent Q Networks (DARQN). The second part of the paper describes the proposed framework that integrates the recent advances in deep reinforcement learning. Finally, we conclude and suggest directions for future work. REVIEW OF REINFORCEMENT LEARNING For a comprehensive overview of reinforcement learning, please refer to the second edition of Rich Sutton’s textbook [18]. We provide a short review of important topics in this section. The Reinforcement Learning framework was formulated in [17] as a model to provide the best policy an agent can follow (best action to take in a given state), such that the total accumulated rewards are maximized when the agent follows that policy from the current and until a terminal state is reached. Motivation for RL Paradigm Driving is a multi-agent interaction problem. As a human driver, it is much easier to keep within a lane without any interaction with other cars than to change lanes in heavy traffic. The latter is more difficult because of the inherent uncertainty in behavior of other drivers. The number of interacting vehicles, their geometric configuration and the behavior of the drivers could have large variability and it is challenging to design a supervised learning dataset with exhaustive coverage of all scenarios. Human drivers employ some sort of online reinforcement learning to understand the behavior of other drivers such as whether they are defensive or aggressive, experienced or in-experienced, etc. This is particularly useful in scenarios which need negotiation, namely entering a roundabout, navigating junctions without traffic lights, lane changes during heavy traffic, etc. The main challenge in autonomous driving is to deal with corner cases which are unexpected even for a human driver, like recovering from being lost in an unknown area without GPS or dealing with disaster situations like flooding or appearance of a sinkhole on the ground. The RL paradigm models uncharted territory and learns from its own experience by taking actions. Additionally, RL may be able to handle non-differentiable cost functions which can create challenges for supervised learning problems. Currently, the standard approach for autonomous driving is to decouple the system into isolated sub-problems, typically supervised-learning-like object detection, visual odometry, etc and then having a post processing layer to combine all the results of the previous steps. There are two main issues with this approach: Firstly, the sub-problems which are solved may be more difficult than autonomous driving. For example, one might be solving object detection by semantic segmentation which is both challenging and unnecessary. Human drivers don’t detect and classify all visible objects while driving, only the most relevant ones. Secondly, the isolated sub-problems may not combine coherently to achieve

Most statistical and machine-learning algorithms assume that the data is a random sample drawn from a stationary distribution. Unfortunately, most of the large databases available for mining today violate this assumption. They were gathered over months or years, and the underlying processes generating them changed during this time, sometimes radically. Although a number of algorithms have been proposed for learning time-changing concepts, they generally do not scale well to very large databases. In this paper we propose an efficient algorithm for mining decision trees from continuously-changing data streams, based on the ultra-fast VFDT decision tree learner. This algorithm, called CVFDT, stays current while making the most of old data by growing an alternative subtree whenever an old one becomes questionable, and replacing the old with the new when the new becomes more accurate. CVFDT learns a model which is similar in accuracy to the one that would be learned by reapplying VFDT to a moving window of examples every time a new example arrives, but with O(1) complexity per example, as opposed to O(w), where w is the size of the window. Experiments on a set of large time-changing data streams demonstrate the utility of this approach.

Randomized experiments have been used to assist decisionmaking in many areas. They help people select the optimal treatment for the test population with certain statistical guarantee. However, subjects can show significant heterogeneity in response to treatments. The problem of customizing treatment assignment based on subject characteristics is known as uplift modeling, differential response analysis, or personalized treatment learning in literature. A key feature for uplift modeling is that the data is unlabeled. It is impossible to know whether the chosen treatment is optimal for an individual subject because response under alternative treatments is unobserved. This presents a challenge to both the training and the evaluation of uplift models. In this paper we describe how to obtain an unbiased estimate of the key performance metric of an uplift model, the expected response. We present a new uplift algorithm which creates a forest of randomized trees. The trees are built with a splitting criterion designed to directly optimize their uplift performance based on the proposed evaluation method. Both the evaluation method and the algorithm apply to arbitrary number of treatments and general response types. Experimental results on synthetic data and industry-provided data show that our algorithm leads to significant performance improvement over other applicable methods. Accepted: 2017 SIAM International Conference on Data

We describe a generalization of the PAC learning model that is based on statistical decision theory. In this model the learner receives randomly drawn examples, each example consisting of an instance x E X and an outcome J E Y, and tries to find a decision rule h: X + A, where h E X, that specifies the appropriate action a E A to take for each instance x in order to minimize the expectation of a loss I( y, a). Here X, Y, and A are arbitrary sets. I is a real-valued function, and examples are generated according to an arbitrary joint distribution on Xx Y. Special cases include the problem of learning a function from X into Y, the problem of learning the conditional probability distribution on Y given X (regression), and the problem of learning a distribution on X (density estimation). We give theorems on the uniform convergence of empirical loss estimates to true expected loss rates for certain decision rule spaces 2, and show how this implies learnability with bounded sample size, disregarding computational complexity. As an application, we give distribution-independent upper bounds on the sample size needed for learning with feedforward neural networks, Our theorems use a generalized notion of VC dimension that applies to classes of real-valued functions, adapted from Vapnik and Pollard’s work, and a notion of capacity and metric dimension for classes of functions that map into a bounded metric space. ‘(‘1 1992 Academic Press, Inc.

Within the framework of ADABOOST.MH, we propose to train vector-valued decision trees to optimize the multi-class edge without reducing the multi-class problem toK binary one-againstall classifications. The key element of the method is a vector-valued decision stump, factorized into an input-independent vector of length K and label-independent scalar classifier. At inner tree nodes, the label-dependent vector is discarded and the binary classifier can be used for partitioning the input space into two regions. The algorithm retains the conceptual elegance, power, and computational efficiency of binary ADABOOST. In experiments it is on par with support vector machines and with the best existing multi-class boosting algorithm AOSOLOGITBOOST, and it is significantly better than other known implementations of ADABOOST.MH.


Question: Using random space partitioning to identify significant locations from raw GPS data.

This paper introduces a robust framework, MapReduce, designed to facilitate simplified data processing on extensive clusters. MapReduce serves as a programming model and a practical implementation, particularly suitable for handling vast datasets across a wide spectrum of real-world applications. The model's flexibility lies in its user-defined computation through a map and a reduce function. The underlying runtime system automates the parallelization of these computations across large-scale clusters, ensuring efficient network and disk utilization by managing machine failures and inter-machine communication. Its ease of use is highlighted by the fact that over ten thousand unique MapReduce programs have been developed internally at Google over the past four years. Furthermore, an average of one hundred thousand MapReduce jobs are executed daily on Google's clusters, processing an astonishing twenty petabytes of data per day, demonstrating its scalability and efficiency in data processing tasks.

However, when addressing the specific challenge of detecting significant locations from raw GPS data using random space partitioning, the MapReduce framework can be adapted to efficiently process large volumes of GPS data. By utilizing the map function to preprocess and categorize raw GPS data into smaller, manageable chunks, and the reduce function to aggregate and analyze these chunks, significant locations can be accurately identified. This approach leverages the inherent parallelism of MapReduce to handle the computational complexity involved in analyzing GPS data, making it a viable solution for real-world applications in location-based services and spatial data analysis.

Localization is one important part of Internet of Things(IoT) where the Location of Everything (LoE) system plays a important role to improve most services in IoT area. On the other hand, data mining techniques are essential analyses when we have big data from IoT platforms. Indeed, integration of location-based methods and data mining analysis process can make a smart system service for IoT scenarios and applications. For this purpose, we design a smart shopping platform including four components, location of everything component, data collection component, data filtering/analysing component and data mining component. Then a novel accurate localization scheme named “location orbital” is developed that estimates the current location of mobile objects (users or everything) based on both current and the previous locations. Finally, an implementation of the experiment in a shopping mall is conducted to practically examine performance evaluation of the location-based scheme. The experimental results show that the proposed scheme could achieve significant higher precision than other localization techniques.

The widespread use of vision-based video surveillance systems has inspired many research efforts on people localization. One of the current main trends in this field is based on probabilistic occupancy map (POM) obtained from multiple camera views. Although the POM-based approaches are robust against noisy foregrounds and can achieve great localization accuracy, they require high computation complexity. In this paper, two enhancement schemes are proposed to improve the efficiency of the POM-based people localization: (i) quick screening of potential people locations, and (ii) timely termination of iterations for occupancy probability estimation. Experimental results show that the proposed approach achieves up to 7.25 times speed-up compared to the standard POM-based approach, while delivering comparable people localization accuracy.

Location-based social networks (LBSNs) have become a popular form of social media in recent years. They provide location related services that allow users to “check-in” at geographical locations and share such experiences with their friends. Millions of “check-in” records in LBSNs contain rich information of social and geographical context and provide a unique opportunity for researchers to study user’s social behavior from a spatial-temporal aspect, which in turn enables a variety of services including place advertisement, traffic forecasting, and disaster relief. In this paper, we propose a social-historical model to explore user’s check-in behavior on LBSNs. Our model integrates the social and historical effects and assesses the role of social correlation in user’s check-in behavior. In particular, our model captures the property of user’s check-in history in forms of power-law distribution and short-term effect, and helps in explaining user’s check-in behavior. The experimental results on a real world LBSN demonstrate that our approach properly models user’s checkins and shows how social and historical ties can help location prediction.

Random sampling is used for several new geometric algorithms. The algorithms are “Las Vegas,” and their expected bounds are with respect to the random behavior of the algorithms. One algorithm reports all the intersecting pairs of a set of line segments in the plane, and requires <italic>&Ogr;</italic>(<italic>A</italic> + <italic>n</italic> log <italic>n</italic>) expected time, where <italic>A</italic> is the size of the answer, the number of intersecting pairs reported. The algorithm requires <italic>&Ogr;</italic>(<italic>n</italic>) space in the worst case. Another algorithm computes the convex hull of a point set in <italic>E</italic><supscrpt>3</supscrpt> in <italic>&Ogr;</italic>(<italic>n</italic> log <italic>A</italic>) expected time, where <italic>n</italic> is the number of points and <italic>A</italic> is the number of points on the surface of the hull. A simple Las Vegas algorithm triangulates simple polygons in <italic>&Ogr;</italic>(<italic>n</italic> log log <italic>n</italic>) expected time. Algorithms for half-space range reporting are also given. In addition, this paper gives asymptotically tight bounds for a combinatorial quantity of interest in discrete and computational geometry, related to halfspace partitions of point sets.

In this paper we present an application framework that leverages geospatial content on the World Wide Web by enabling innovative modes of interaction and novel types of user interfaces on advanced mobile phones and PDAs. We discuss the current development steps involved in building mobile geospatial Web applications and derive three technological pre-requisites for our framework: spatial query operations based on visibility and field of view, a 2.5D environment model, and a presentationindependent data exchange format for geospatial query results. We propose the Local Visibility Model as a suitable XML-based candidate and present a prototype implementation.

Urban land use information plays an essential role in a wide variety of urban planning and environmental monitoring processes. During the past few decades, with the rapid technological development of remote sensing (RS), geographic information systems (GIS) and geospatial big data, numerous methods have been developed to identify urban land use at a fine scale. Points-of-interest (POIs) have been widely used to extract information pertaining to urban land use types and functional zones. However, it is difficult to quantify the relationship between spatial distributions of POIs and regional land use types due to a lack of reliable models. Previous methods may ignore abundant spatial features that can be extracted from POIs. In this study, we establish an innovative framework that detects urban land use distributions at the scale of traffic analysis zones (TAZs) by integrating Baidu POIs and a Word2Vec model. This framework was implemented using a Google open-source model of a deep-learning language in 2013. First, data for the Pearl River Delta (PRD) are transformed into a TAZ-POI corpus using a greedy algorithmby considering the spatial distributions of TAZs and inner POIs. Then, high-dimensional characteristic vectors of POIs and TAZs are extracted using the Word2Vec model. Finally, to validate the reliability of the POI/TAZ vectors, we implement a K-Means-based clustering model to analyze correlations between the POI/TAZ vectors and deploy TAZ vectors to identify urban land use types using a random forest algorithm (RFA) model. Compared with some state-of-the-art probabilistic topic models (PTMs), the proposed method can efficiently obtain the highest accuracy (OA = 0.8728, kappa = 0.8399). Moreover, the results can be used to help urban planners to monitor dynamic urban land use and evaluate the impact of urban planning schemes. ARTICLE HISTORY Received 21 March 2016 Accepted 28 September 2016

Ubisense has developed a platform for building Smart Space applications. The platform addresses the key requirements for building Smart Spaces: accurate 3D positioning, scalable realtime performance, development and deployment tools. This paper deepens the key requirements and describes how the Ubisense platform components meets them. The demonstration exemplifies the smart space platform by tracking players in a game application. The Ubisense Smart Space Platform Ubisense has developed a platform for building Smart Space applications. Our platform addresses the key requirements for building Smart Spaces: Accurate 3D positioning supports applications that can perceive the physical relationships between people and objects in the environment Scalable real-time performance enables arbitrary numbers of applications, used by arbitrary numbers of people over an arbitrarily-large area Development and deployment tools make it easy to design, implement, and manage Smart Space applications. The demonstration shows a Smart Space containing applications that integrate with external software (a simple game that users control by moving around), and devices (a PTZ camera that keeps users in shot while they are playing the game) Fig. 1 Smart space demonstration setup 1 Ubisense, St Andrews House, 90 St Andrews Road, Chesterton, CB4 1DL, United Kingdom, http://www.ubisense.net

A continuous spatial model can be constructed by convolving a very simple, perhaps independent, process with a kernel or point spread function. This approach for constructing a spatial process offers a number of advantages over specification through a spatial covariogram. In particular, this process convolution specification leads to compuational simplifications and easily extends beyond simple stationary models. This paper uses process convolution models to build space and space-time models that are flexible and able to accomodate large amounts of data. Data from environmental monitoring is considered.

A tutorial survey is presented of the quadtree and related hierarchical data structures. They are based on the principle of recursive decomposition. The emphasis is on the representation of data used in applications in image processing, computer graphics, geographic information systems, and robotics. There is a greater emphasis on region data (i.e., two-dimensional shapes) and to a lesser extent on point, curvilinear, and threedimensional data. A number of operations in which such data structures find use are examined in greater detail.

Locations, e.g., countries, states, cities, and point-of-interests, are central to news, emergency events, and people's daily lives. Automatic identification of locations associated with or mentioned in documents has been explored for decades. As one of the most popular online social network platforms, Twitter has attracted a large number of users who send millions of tweets on daily basis. Due to the world-wide coverage of its users and real-time freshness of tweets, location prediction on Twitter has gained significant attention in recent years. Research efforts are spent on dealing with new challenges and opportunities brought by the noisy, short, and context-rich nature of tweets. In this survey, we aim at offering an overall picture of location prediction on Twitter. Specifically, we concentrate on the prediction of user home locations, tweet locations, and mentioned locations. We first define the three tasks and review the evaluation metrics. By summarizing Twitter network, tweet content, and tweet context as potential inputs, we then structurally highlight how the problems depend on these inputs. Each dependency is illustrated by a comprehensive review of the corresponding strategies adopted in state-of-the-art approaches. In addition, we also briefly review two related problems, i.e., semantic location prediction and point-of-interest recommendation. Finally, we make a conclusion of the survey and list future research directions.


Question: Using random space partitioning for identifying significant locations from raw GPS data.

This paper explores a method for extracting significant locations and popular travel patterns from GPS trajectory data collected from numerous users. With the proliferation of GPS-enabled devices, vast quantities of GPS trajectories are now available, detailing people's movement histories. Our objective is to analyze this data to uncover culturally significant sites, such as Tiananmen Square in Beijing, as well as frequently visited public spaces like shopping centers and eateries. This information is crucial for enhancing user understanding of their surroundings and for facilitating travel recommendations. To achieve this, we introduce a hierarchical graph representation, the Tree-Based Hierarchical Graph (TBHG), to model individuals' location histories. Subsequently, we propose a HITS (Hypertext Induced Topic Search)-inspired inference mechanism, treating an individual's visit to a location as a directed link from the user to that location. This model evaluates a location's significance by considering three key aspects: 1) A location's appeal is determined not only by the number of visitors but also by the travel experiences of these visitors. 2) There exists a reciprocal relationship between users' travel experiences and location popularity. 3) The value of a location's interest and a user's travel experience are relative and specific to the region. Furthermore, we identify common travel sequences among locations, taking into account both location popularity and users' travel histories. Our system's effectiveness was evaluated using a comprehensive GPS dataset compiled from 107 users over the course of a year. The results indicated that our HITS-inspired inference model outperformed basic methods like rank-by-count and rank-by-frequency. Moreover, when incorporating users' travel histories and location popularity, our approach demonstrated superior performance compared to baselines such as rank-by-count and rank-by-interest.

A method is described which quantifies the speed and direction of several moving objects in a sequence of digital images. A relationship between the time variation of intensity, the spatial gradient, and velocity has been developed which allows the determination of motion using clustering techniques. This paper describes these relationships, the clustering technique, and provides examples of the technique on real images containing several moving objects.

With the advances of information communication technologies, it is critical to improve the efficiency and accuracy of emergency management systems through modern data processing techniques. The past decade has witnessed the tremendous technical advances in Sensor Networks, Internet/Web of Things, Cloud Computing, Mobile/Embedded Computing, Spatial/Temporal Data Processing, and Big Data, and these technologies have provided new opportunities and solutions to emergency management. GIS models and simulation capabilities are used to exercise response and recovery plans during non-disaster times. They help the decision-makers understand near real-time possibilities during an event. In this paper, a crowdsourcing based model for mining spatial information of urban emergency events is introduced. Firstly, basic definitions of the proposed method are given. Secondly, positive samples are selected to mine the spatial information of urban emergency events. Thirdly, location and GIS information are extracted from positive samples. At last, the real spatial information is determined based on address and GIS information. At last, a case study on an urban emergency event is given.

This paper presents the design, implementation, and evaluation of Cricket, a location-support system for in-building, mobile, location-dependent applications. It allows applications running on mobile and static nodes to learn their physical location by using listeners that hear and analyze information from beacons spread throughout the building. Cricket is the result of several design goals, including user privacy, decentralized administration, network heterogeneity, and low cost. Rather than explicitly tracking user location, Cricket helps devices learn where they are and lets them decide whom to advertise this information to; it does not rely on any centralized management or control and there is no explicit coordination between beacons; it provides information to devices regardless of their type of network connectivity; and each Cricket device is made from off-the-shelf components and costs less than U.S. $10. We describe the randomized algorithm used by beacons to transmit information, the use of concurrent radio and ultrasonic signals to infer distance, the listener inference algorithms to overcome multipath and interference, and practical beacon configuration and positioning techniques that improve accuracy. Our experience with Cricket shows that several location-dependent applications such as in-building active maps and device control can be developed with little effort or manual configuration.

Ubisense has developed a platform for building Smart Space applications. The platform addresses the key requirements for building Smart Spaces: accurate 3D positioning, scalable realtime performance, development and deployment tools. This paper deepens the key requirements and describes how the Ubisense platform components meets them. The demonstration exemplifies the smart space platform by tracking players in a game application. The Ubisense Smart Space Platform Ubisense has developed a platform for building Smart Space applications. Our platform addresses the key requirements for building Smart Spaces: Accurate 3D positioning supports applications that can perceive the physical relationships between people and objects in the environment Scalable real-time performance enables arbitrary numbers of applications, used by arbitrary numbers of people over an arbitrarily-large area Development and deployment tools make it easy to design, implement, and manage Smart Space applications. The demonstration shows a Smart Space containing applications that integrate with external software (a simple game that users control by moving around), and devices (a PTZ camera that keeps users in shot while they are playing the game) Fig. 1 Smart space demonstration setup 1 Ubisense, St Andrews House, 90 St Andrews Road, Chesterton, CB4 1DL, United Kingdom, http://www.ubisense.net

This paper presents a new approach to hierarchical clustering of very large data sets, named GridClustering. The method organizes unlike the conventional methods the space surrounding the patterns and not the patterns. It uses a multidimensional grid data structure. The resulting block partitioning of the value space is clustered via a topological neighbor search. The Grid-Clustering method is able to deliver structural pattern distribution information for very large data sets. It superceeds all conventional hierarchical algorithms in runtime behavior and memory space requirements. The algorithm was analyzed within a testbed and suitable values for the tunable parameters of the algorithm are proposed. A comparison of the executions times to other commonly used clustering algorithms and a heuristic runtime analysis is given.

The proliferation of mobile computing devices and local-area wireless networks has fostered a growing interest in location-aware systems and services. In this paper we present RADAR, a radio-frequency (RF) based system for locating and tracking users inside buildings. RADAR operates by recording and processing signal strength information at multiple base stations positioned to provide overlapping coverage in the area of interest. It combines empirical measurements with signal propagation modeling to determine user location and thereby enable locationaware services and applications. We present experimental results that demonstrate the ability of RADAR to estimate user location with a high degree of accuracy.

In library, the management of books is very complication and timing costing. The location of books could be altered by librarian, students, teachers and any one around the library. Therefore, allocating a book is not an easy task in big library. Indoor positioning is an important technology to help storage management and customer services providing. RFID provides a good wireless platform to facilitate indoor positioning. However, duo to the small width of each book spine, adopting positioning based RFID alone is not enough to locate books in a library. In this work, we combine image matching with L-GEM based RBFNN to enhance the accuracy and robustness of the book locating system. We apply this new method in a library environment to position the certain books. Experimental results show that the proposed method is highly accurate and robust to white noise of RFID signals.

This paper proposes a shrinkage-based framework for unsupervised trajectory clustering. Facing to the challenges of trajectory clustering, e.g., large variations within a cluster and ambiguities across clusters, we first introduce an adaptive multi-kernel-based estimation process to estimate the 'shrunk' positions and speeds of trajectories' points. This kernel-based estimation effectively leverages both multiple structural information within a trajectory and the local motion patterns across multiple trajectories, such that the discrimination of the shrunk point can be properly increased. We further introduce a speed-regularized optimization process, which utilizes the estimated speeds to regularize the optimal shrunk points, so as to guarantee the smoothness and the discriminative pattern of the final shrunk trajectory. Using our approach, the variations among similar trajectories can be reduced while the boundaries between different clusters are enlarged. Experimental results demonstrate that our approach is superior to the state-of-art approaches on both clustering accuracy and robustness. Besides, additional experiments further reveal the effectiveness of our approach when applied to trajectory analysis applications such as anomaly detection and route analysis.

Geography and social relationships are inextricably intertwined; the people we interact with on a daily basis almost always live near us. As people spend more time online, data regarding these two dimensions -- geography and social relationships -- are becoming increasingly precise, allowing us to build reliable models to describe their interaction. These models have important implications in the design of location-based services, security intrusion detection, and social media supporting local communities.
 Using user-supplied address data and the network of associations between members of the Facebook social network, we can directly observe and measure the relationship between geography and friendship. Using these measurements, we introduce an algorithm that predicts the location of an individual from a sparse set of located users with performance that exceeds IP-based geolocation. This algorithm is efficient and scalable, and could be run on a network containing hundreds of millions of users.

We propose UnLoc, an unsupervised indoor localization scheme that bypasses the need for war-driving. Our key observation is that certain locations in an indoor environment present identifiable signatures on one or more sensing dimensions. An elevator, for instance, imposes a distinct pattern on a smartphone's accelerometer; a corridor-corner may overhear a unique set of WiFi access points; a specific spot may experience an unusual magnetic fluctuation. We hypothesize that these kind of signatures naturally exist in the environment, and can be envisioned as internal landmarks of a building. Mobile devices that "sense" these landmarks can recalibrate their locations, while dead-reckoning schemes can track them between landmarks. Results from 3 different indoor settings, including a shopping mall, demonstrate median location errors of 1:69m. War-driving is not necessary, neither are floorplans the system simultaneously computes the locations of users and landmarks, in a manner that they converge reasonably quickly. We believe this is an unconventional approach to indoor localization, holding promise for real-world deployment.


Question: Identifying important locations from unprocessed GPS data through random space partitioning.

This paper reveals a novel perspective on rounding algorithms for Linear Programming (LP) and Semidefinite Programming (SDP) typically employed in approximation algorithms, reinterpreting them as locality sensitive hashing schemes for various sets of objects. A locality sensitive hashing scheme, characterized by a distribution on a family $\F$ of hash functions, facilitates the estimation of similarity between objects from their compact representations. This approach leads to efficient algorithms for approximate nearest neighbor search and clustering. Traditionally, min-wise independent permutations have provided a locality sensitive hashing scheme for sets with the similarity measure defined as $sim(A,B) = \frac{|A \cap B|}{|A \cup B|}$. Building on this insight, we propose new locality sensitive hashing schemes for:

1. A set of vectors where the distance between vectors $\vec{u}$ and $\vec{v}$ is measured by $\theta(\vec{u}, \vec{v})/\pi$, with $\theta(\vec{u}, \vec{v})$ denoting the angle between $\vec{u}$ and $\vec{v}$. This results in a sketching scheme for estimating the cosine similarity between two vectors, providing a simple alternative to min-wise independent permutations for estimating set similarity.

2. A set of distributions over $n$ points in a metric space, with the distance between distributions assessed by the Earth Mover's Distance (EMD), a widely used measure in graphics and vision. Our hash functions map distributions to points in the metric space such that, for distributions $P$ and $Q$, $EMD(P, Q) \leq E_{h\in\F} [d(h(P),h(Q))] \leq O(\log n \log\log n) \cdot EMD(P, Q)$. This scheme effectively captures the EMD between distributions through hashing, offering a compact representation and efficient computation of similarity.

In the context of detecting significant locations from raw GPS data using random space partitioning, this reinterpretation of rounding algorithms as locality sensitive hashing schemes presents a promising approach to efficiently process and analyze large datasets. By applying these hashing schemes to GPS data, we can develop compact representations of locations, enabling faster computation of similarities between locations and potentially improving the detection of significant points of interest. The use of EMD as a similarity measure, in particular, aligns well with the spatial context of GPS data, making this framework particularly relevant for location-based data analysis tasks.

Indexing and query processing is an emerging research field in spatio temporal data. Most of the real-time applications such as location based services, fleet management, traffic prediction and radio frequency identification and sensor networks are based on spatiotemporal indexing and query processing. All the indexing and query processing applications is any one of the forms, such as spatio index access and supporting queries or spatio-temporal indexing method and support query or temporal dimension, while in spatial data it is considered as the second priority. In this paper, give the survey of the various uncertain indexing and query processing techniques. Most of the existing survey works on spatio-temporal are based on indexing methods and query processing, but presented separately. Both the indexing and querying are related, hence state of art of both the indexing and query processing techniques are considered together. This paper gives the details of spatio-temporal data classification, various types of indexing methods, query processing, application areas and research direction of spatio-temporal indexing and query

Crowd density estimating is a crucial service in many applications (e.g., smart guide, crowd control, etc.), which is often conducted using pattern recognition technologies based on video surveillance. However, these kinds of methods are high cost, and cannot work well in low-light environments. Radio frequency based technologies are adopted more and more in indoor application, since radio signal strength (RSS) can be easily obtained by various wireless devices without additional cost. In this paper, we introduce a low cost crowd density estimating method using wireless sensor networks. The proposed approach is a device-free crowd counting approach without objects carrying any assistive device. It is hard to count objects based on RSS measurement, since different number of mobile people at different positions often generates different RSS due to the multipath phenomenon. This paper utilizes the space-time relativity of crowd distribution to reduce the estimation errors. The proposed approach is an iterative process, which contains three phases: the training phase, the monitoring phase, and the calibrating phase. Our experiments are implemented based on TelosB sensor platform. We also do some large-scale simulations to verify the feasibility and the effectiveness of our crowd density estimating approach.

We seek to recognize the place depicted in a query image using a database of “street side” images annotated with geolocation information. This is a challenging task due to changes in scale, viewpoint and lighting between the query and the images in the database. One of the key problems in place recognition is the presence of objects such as trees or road markings, which frequently occur in the database and hence cause significant confusion between different places. As the main contribution, we show how to avoid features leading to confusion of particular places by using geotags attached to database images as a form of supervision. We develop a method for automatic detection of image-specific and spatially-localized groups of confusing features, and demonstrate that suppressing them significantly improves place recognition performance while reducing the database size. We show the method combines well with the state of the art bag-of-features model including query expansion, and demonstrate place recognition that generalizes over wide range of viewpoints and lighting conditions. Results are shown on a geotagged database of over 17K images of Paris downloaded from Google Street View.

Recent years have witnessed the wide proliferation of geo-sensory applications wherein a bundle of sensors are deployed at different locations to cooperatively monitor the target condition. Given massive geo-sensory data, we study the problem of mining spatial co-evolving patterns (SCPs), i.e., groups of sensors that are spatially correlated and co-evolve frequently in their readings. SCP mining is of great importance to various real-world applications, yet it is challenging because (1) the truly interesting evolutions are often flooded by numerous trivial fluctuations in the geo-sensory time series; and (2) the pattern search space is extremely large due to the spatiotemporal combinatorial nature of SCP. In this paper, we propose a two-stage method called Assember. In the first stage, Assember filters trivial fluctuations using wavelet transform and detects frequent evolutions for individual sensors via a segment-and-group approach. In the second stage, Assember generates SCPs by assembling the frequent evolutions of individual sensors. Leveraging the spatial constraint, it conceptually organizes all the SCPs into a novel structure called the SCP search tree, which facilitates the effective pruning of the search space to generate SCPs efficiently. Our experiments on both real and synthetic data sets show that Assember is effective, efficient, and scalable.

This paper presents a set of segmentation methods for various types of 3D point clouds. Segmentation of dense 3D data (e.g. Riegl scans) is optimised via a simple yet efficient voxelisation of the space. Prior ground extraction is empirically shown to significantly improve segmentation performance. Segmentation of sparse 3D data (e.g. Velodyne scans) is addressed using ground models of non-constant resolution either providing a continuous probabilistic surface or a terrain mesh built from the structure of a range image, both representations providing close to real-time performance. All the algorithms are tested on several hand labeled data sets using two novel metrics for segmentation evaluation.

Point-of-Interest (POI) recommendation has received increasing attention in Location-based Social Networks (LBSNs). It involves user behavior analysis, movement pattern model and trajectory sequence prediction, in order to recommend personalized services to target user. Existing POI recommendation methods are confronted with three problems: (1) they only consider the location information of users' check-ins, which causes data sparsity; (2) they fail to consider the order of users' visited locations, which is valuable to reflect the interest or preference of users; (3) users cannot be recommended the suitable services when they move into the new place. To address the above issues, we propose a semantical pattern and preference-aware service mining method called SEM-PPA to make full use of the semantic information of locations for personalized POI recommendation. In SEM-PPA, we firstly propose a novel algorithm to classify the locations into different types for location identification; then we construct the user model for each user from four aspects, which are location trajectory, semantic trajectory, location popularity and user familiarity; in addition, a potential friends discovery algorithm based on movement pattern is proposed. Finally, we conduct extensive experiments to evaluate the recommendation accuracy and recommendation effectiveness on two real-life datasets from GeoLife and Beijing POI. Experimental results show that SEM-PPA can achieve better recommendation performance in particular for sparse data and recommendation accuracy in comparison with other methods.

It is challenging to map large spatial flow data due to the problem of occlusion and cluttered display, where hundreds of thousands of flows overlap and intersect each other. Existing flow mapping approaches often aggregate flows using predetermined high-level geographic units (e.g. states) or bundling partial flow lines that are close in space, both of which cause a significant loss or distortion of information and may miss major patterns. In this research, we developed a flow clustering method that extracts clusters of similar flows to avoid the cluttering problem, reveal abstracted flow patterns, and meanwhile preserves data resolution as much as possible. Specifically, our method extends the traditional hierarchical clustering method to aggregate and map large flow data. The new method considers both origins and destinations in determining the similarity of two flows, which ensures that a flow cluster represents flows from similar origins to similar destinations and thus minimizes information loss during aggregation. With the spatial index and search algorithm, the new method is scalable to large flow data sets. As a hierarchical method, it generalizes flows to different hierarchical levels and has the potential to support multi-resolution flow mapping. Different distance definitions can be incorporated to adapt to uneven spatial distribution of flows and detect flow clusters of different densities. To assess the quality and fidelity of flow clusters and flow maps, we carry out a case study to analyze a data set of 243,850 taxi trips within an urban area.

Localization is crucial to many applications in wireless sensor networks. In this article, we propose a range-free anchor-based localization algorithm for mobile wireless sensor networks that builds upon the Monte Carlo Localization algorithm. We concentrate on improving the localization accuracy and efficiency by making better use of the information a sensor node gathers and by drawing the necessary location samples faster. To do so, we constrain the area from which samples are drawn by building a box that covers the region where anchors’ radio ranges overlap. This box is the region of the deployment area where the sensor node is localized. Simulation results show that localization accuracy is improved by a minimum of 4% and by a maximum of 73%, on average 30%, for varying node speeds when considering nodes with knowledge of at least three anchors. The coverage is also strongly affected by speed and its improvement ranges from 3% to 55%, on average 22%. Finally, the processing time is reduced by 93% for a similar localization accuracy.

Build accurate DNN models requires training on large labeled, context specific datasets, especially those matching the target scenario. We believe advances in wireless localization, working in unison with cameras, can produce automated annotation of targets on images and videos captured in the wild. Using pedestrian and vehicle detection as examples, we demonstrate the feasibility, benefits, and challenges of an automatic image annotation system. Our work calls for new technical development on passive localization, mobile data analytics, and error-resilient ML models, as well as design issues in user privacy policies.

In this study we analyze the travel patterns of 500,000 individuals in Cote d'Ivoire using mobile phone call data records. By measuring the uncertainties of movements using entropy, considering both the frequencies and temporal correlations of individual trajectories, we find that the theoretical maximum predictability is as high as 88%. To verify whether such a theoretical limit can be approached, we implement a series of Markov chain (MC) based models to predict the actual locations visited by each user. Results show that MC models can produce a prediction accuracy of 87% for stationary trajectories and 95% for non-stationary trajectories. Our findings indicate that human mobility is highly dependent on historical behaviors, and that the maximum predictability is not only a fundamental theoretical limit for potential predictive power, but also an approachable target for actual prediction accuracy.


Question: Utilizing random space partitioning for identifying important locations from unprocessed GPS data.

This article presents an innovative assisted cognition information technology framework designed to construct personalized maps tailored to individual users, while simultaneously deducing their daily routines and movements from raw GPS data. The system employs a dual modeling strategy, combining discriminative and generative models to address distinct aspects of the task. A relational Markov network, a discriminative model, is utilized to pinpoint and categorize significant locations. Conversely, a generative dynamic Bayesian network is employed to grasp transportation patterns, anticipate destinations, and promptly identify potential user errors. The focus of the discussion revolves around the foundational architectures of these models, alongside a cursory overview of the employed inference and learning methodologies. Empirical evaluations affirm the system's proficiency in precisely identifying and labeling places, forecasting an individual's objectives, and discerning instances where the user encounters errors, such as boarding the incorrect bus.

Availability of large volume of community contributed location data enables a lot of location providing services and these services have attracted many industries and academic researchers by its importance. In this paper we propose the new recommender system that recommends the new POI for next hours. First we find the users with similar check-in sequences and depict their check-in sequences as a directed graph, then find the users current location. To recommend the new POI recommendation for next hour we refer to the directed graph we have created. Our algorithm considers both the temporal factor i.e., recommendation time, and the spatial(distance) at the same time. We conduct an experiment on random data collected from Foursquare and Gowalla. Experiment results show that our proposed model outperforms the collaborative-filtering based state-of-the-art recommender techniques.

The availability of devices that can be used to track moving objects has increased dramatically leading to a great growth in movement data from almost every application domain. Therefore, there has been an increasing interest in proposing new methodologies for indexing, classifying, clustering, querying and measuring similarity between moving objects' data. One of the main functions for a wide range of application domains is to measure the similarity between two moving objects' trajectories. In this paper, we present a comparative study between widely used trajectory similarity measures observing the advantages and disadvantages of these measures.

In this paper we present an application framework that leverages geospatial content on the World Wide Web by enabling innovative modes of interaction and novel types of user interfaces on advanced mobile phones and PDAs. We discuss the current development steps involved in building mobile geospatial Web applications and derive three technological pre-requisites for our framework: spatial query operations based on visibility and field of view, a 2.5D environment model, and a presentationindependent data exchange format for geospatial query results. We propose the Local Visibility Model as a suitable XML-based candidate and present a prototype implementation.

A great quantity of information is required to support urban planning. Usually there are many (not integrated) data sources, originating from different government bodies, in distinct formats and variable properties (e.g. reliability, completeness). The effort to handle these data, integrate and analyze them is high, taking to much time for the information to be available to help decision making. We argue that data from location-based social networks (LBSN) could be used to provide useful information in reasonable time, despite several limitations they have. To asses this, as a case study, we used data from different LBSN to calculate the Local Availability Index (IOL) for a Brazilian city. This index is part of a methodology to estimate quality of urban life inside cities and is used to support urban planning. The results suggest that data from LBSN are useful and could be used to provide insights for local governments.

Road traffic accidents are among the principal causes of traffic congestion, causing human losses, damages to health and the environment, economic losses and material damages. Studies about traditional road traffic accidents in urban zones represents very high inversion of time and money, additionally, the result are not current. However, nowadays in many countries, the crowdsourced GPS based traffic and navigation apps have emerged as an important source of information to low cost to studies of road traffic accidents and urban congestion caused by them. In this article we identified the zones, roads and specific time in the CDMX in which the largest number of road traffic accidents are concentrated during 2016. We built a database compiling information obtained from the social network known as Waze. The methodology employed was Discovery of knowledge in the database (KDD) for the discovery of patterns in the accidents reports. Furthermore, using data mining techniques with the help of Weka. The selected algorithms was the Maximization of Expectations (EM) to obtain the number ideal of clusters for the data and k-means as a grouping method. Finally, the results were visualized with the Geographic Information System QGIS. Keywords—Data mining, K-means, road traffic accidents, Waze, Weka.

This paper describes a personalized k-anonymity model for protecting location privacy against various privacy threats through location information sharing. Our model has two unique features. First, we provide a unified privacy personalization framework to support location k-anonymity for a wide range of users with context-sensitive personalized privacy requirements. This framework enables each mobile node to specify the minimum level of anonymity it desires as well as the maximum temporal and spatial resolutions it is willing to tolerate when requesting for k-anonymity preserving location-based services (LBSs). Second, we devise an efficient message perturbation engine which runs by the location protection broker on a trusted server and performs location anonymization on mobile users' LBS request messages, such as identity removal and spatio-temporal cloaking of location information. We develop a suite of scalable and yet efficient spatio-temporal cloaking algorithms, called CliqueCloak algorithms, to provide high quality personalized location k-anonymity, aiming at avoiding or reducing known location privacy threats before forwarding requests to LBS provider(s). The effectiveness of our CliqueCloak algorithms is studied under various conditions using realistic location data synthetically generated using real road maps and traffic volume data

Resolving location expressions in text to the correct physical location, also known as geocoding or grounding, is complicated by the fact that so many places around the world share the same name. Correct resolution is made even more difficult when there is little context to determine which place is intended, as in a 140-character Twitter message, or when location cues from different sources conflict, as may be the case among different metadata fields of a Twitter message. We used supervised machine learning to weigh the different fields of the Twitter message and the features of a world gazetteer to create a model that will prefer the correct gazetteer candidate to resolve the extracted expression. We evaluated our model using the F1 measure and compared it to similar algorithms. Our method achieved results higher than state-of-the-art competitors.

Many sensor network applications require location awareness, but it is often too expensive to include a GPS receiver in a sensor network node. Hence, localization schemes for sensor networks typically use a small number of seed nodes that know their location and protocols whereby other nodes estimate their location from the messages they receive. Several such localization techniques have been proposed, but none of them consider mobile nodes and seeds. Although mobility would appear to make localization more difficult, in this paper we introduce the sequential Monte Carlo Localization method and argue that it can exploit mobility to improve the accuracy and precision of localization. Our approach does not require additional hardware on the nodes and works even when the movement of seeds and nodes is uncontrollable. We analyze the properties of our technique and report experimental results from simulations. Our scheme outperforms the best known static localization schemes under a wide range of conditions.

The paper discussu some antral irrw in the cooperation between disuibuted agcnu using lhe following case study: The objective is to explore a distant planet. more concretely to collect samples of a parricular typ of precious ruck. The location of the ruck .iamplu is unknown in advance but they arc rypiwlly clustered in cenain spm There i S a vehicle that can drive around on the planet and later reenter the s p x d (0 g0 back to ewlh There is no deuiled map of the t e m h although it h known hat the terrain is fuU of obstacles. hills. valleys. etc.

In this paper we present an algorithm which automatically extracts geopositional coordinate index terms from text to support georeferenced document indexing and retrieval. Under this algorithm, words and phrases containing geographic place names or characteristics are extracted from a text document and used as input to database functions which use spatial reasoning to approximate statistically the geoposition being referenced in the text. We conclude with a discussion of preliminary results and future work.