Question: Deep generative deconvolutional learning approach.

This paper introduces an enhanced approach for scene category recognition that moves beyond simple bag-of-features representations by incorporating global geometric correspondence. The methodology involves progressively subdividing the image into finer sub-regions and compiling histograms of local features within each segment. This spatial pyramid representation is a straightforward and efficient augmentation of the unordered bag-of-features model, leading to markedly improved outcomes on complex scene categorization tasks. Notably, the proposed method outperforms current benchmarks on the Caltech-101 dataset and attains high accuracy on a substantial database comprising fifteen categories of natural scenes. Additionally, the spatial pyramid framework provides a deeper understanding of the effectiveness of recent image descriptors, such as Torralba's "gist" and Lowe's SIFT descriptors.

Generative Adversarial Net is a frontier method of generative models for images, audios and videos. In this paper, we focus on conditional image generation and introduce conditional Feature-Matching Generative Adversarial Net to generate images from category labels. By visualizing state-of-art discriminative conditional generative models, we find these networks do not gain clear semantic concepts. Thus we design the loss function in the light of metric learning to measure semantic distance. The proposed model is evaluated on several well-known datasets. It is shown to be of higher perceptual quality and better diversity then existing generative models.

Generative Adversarial Networks (GANs) have recently demonstrated to successfully approximate complex data distributions. A relevant extension of this model is conditional GANs (cGANs), where the introduction of external information allows to determine specific representations of the generated images. In this work, we evaluate encoders to inverse the mapping of a cGAN, i.e., mapping a real image into a latent space and a conditional representation. This allows, for example, to reconstruct and modify real images of faces conditioning on arbitrary attributes. Additionally, we evaluate the design of cGANs. The combination of an encoder with a cGAN, which we call Invertible cGAN (IcGAN), enables to re-generate real images with deterministic complex modifications.

We propose a novel multiple-input multiple-output convolution neural network (MIMO-Net) for cell segmentation in fluorescence microscopy images. The proposed network trains the network parameters using multiple resolutions of the input image, connects the intermediate layers for better localization and context and generates the output using multi-resolution deconvolution filters. The MIMO-Net allows us to deal with variable intensity cell boundaries and highly variable cell size in the mouse pancreatic tissue by adding extra convolutional layers which bypass the max-pooling operation. The results show that our method outperforms state-of-the-art deep learning based approaches for segmentation.

Convolutional neural networks (CNN) for medical imaging are constrained by the number of annotated data required for the training stage. Usually, manual annotation is considered to be the “gold-standard’. However, medical imaging datasets with expert manual segmentation are scarce as this step is time-consuming and expensive. Moreover, the single-rater manual annotation is often used in data-driven approaches making the algorithm optimal for one guideline only. In this work, we propose a convolutional neural network (CNN) for brain magnetic resonance (MR) imaging extraction, fully trained with what we refer to as silver-standard masks. To the best of our knowledge, our approach is the first deep learning approach that is fully trained using silver-standards masks, having an optimal generalization. Furthermore, regarding the Dice coefficient, we outperform the current skull-stripping (SS) state-of-the-art method in three datasets. Our method consists of 1) a dataset with silver-standard masks as input, 2) a tri-planar method using parallel 2D U-Net-based CNNs, which we refer to as CONSNet, and 3) an auto-context implementation of CONSNet. CONSNet refers to our integrated approach, i.e., training with silver-standard masks and us∗Corresponding author Email address: oeslle@dca.fee.unicamp.br (Oeslle Lucena) URL: miclab.fee.unicamp.br (Oeslle Lucena) Preprint submitted to Medical Image Analysis April 13, 2018 ar X iv :1 80 4. 04 98 8v 1 [ cs .C V ] 1 3 A pr 2 01 8 ing a CNN architecture. Masks are generated by forming a consensus from a set of publicly available automatic segmentation methods using the simultaneous truth and performance level estimation (STAPLE) algorithm. The CNN architecture is robust, with dropout and batch normalization as regularizers. It was inspired by the 2D U-Net model published by the RECOD group. We conducted our analysis using three publicly available datasets: the Calgary-Campinas-359 (CC-359 ), the LONI Probabilistic Brain Atlas (LPBA40), and the Open Access Series of Imaging Studies (OASIS). Five performance metrics were used in our experiments: Dice coefficient, sensitivity, specificity, Hausdorff distance, and symmetric surface-to-surface mean distance. Our results also demonstrate that we have comparable overall performance versus current methods, including recently proposed deep learningbased strategies that were trained using manually annotated masks. Our usage of silver-standard masks, however, reduced the cost of manual annotation, diminished inter-/intra-rater variability, and avoided CNN segmentation super-specialization towards one specific SS method found when mask generation is done through an agreement among many automatic methods. Moreover, usage of silver-standard masks enlarges the volume of input annotated data since we can generate annotation from unlabelled data. Also, our method has the advantage that, once trained, it takes only a few seconds to process a typical brain image volume using a modern graphics processing unit (GPU). In contrast, many of the other competitive consensus-building methods have processing times on the order of minutes.

Motivated in part by the hierarchical organization of the cortex, a number of algorithms have recently been proposed that try to learn hierarchical, or “deep,” structure from unlabeled data. While several authors have formally or informally compared their algorithms to computations performed in visual area V1 (and the cochlea), little attempt has been made thus far to evaluate these algorithms in terms of their fidelity for mimicking computations at deeper levels in the cortical hierarchy. This paper presents an unsupervised learning model that faithfully mimics certain properties of visual area V2. Specifically, we develop a sparse variant of the deep belief networks of Hinton et al. (2006). We learn two layers of nodes in the network, and demonstrate that the first layer, similar to prior work on sparse coding and ICA, results in localized, oriented, edge filters, similar to the Gabor functions known to model V1 cell receptive fields. Further, the second layer in our model encodes correlations of the first layer responses in the data. Specifically, it picks up both colinear (“contour”) features as well as corners and junctions. More interestingly, in a quantitative comparison, the encoding of these more complex “corner” features matches well with the results from the Ito & Komatsu’s study of biological V2 responses. This suggests that our sparse variant of deep belief networks holds promise for modeling more higher-order features.

Extremely low-quality images, on the order of 20 pixels in width, appear with frustrating frequency in many forensic investigations. Even advanced de-noising and super-resolution technologies are unable to extract useful information from such lowquality images. We show, however, that useful information is present in such highly degraded images. We also show that convolutional neural networks can be trained to decipher the contents of highly degraded images of license plates, and that these networks significantly outperform human observers. Introduction Recognizing text in images is a well-studied problem [1]. Text recognition can either be done by recognizing individual characters or by recognizing the full word. For degraded images, however, it is difficult to localize and recognize individual characters in an image. Word recognition, therefore, has become central to text recognition in degraded images. Deep convolutional neural networks [2] have been used for text recognition in natural images. Goodfellow et al. [3] used a deep neural network to localize, segment and recognize multiple digits on street view images. Jaderberg et al. [4] also proposed an end to end text recognition system for natural images using a deep neural network. Jaderberg et al. used a large word dictionary and formulated the text recognition task as a large scale classification problem. Recently, Svoboda et al. [5] used CNN to remove motion blur from images of license plate blurred with a blur kernel of various directions and lengths. Although this approach is able to deblur highly blurred images, it does not contend with extremely low resolution and noisy images. Unlike much of this previous work we focus on extracting text from highly degraded images on the scale of only a few pixels per character. Hsieh et al. [6] were the first to show that information can be extracted from highly degraded license plates. In this work the authors assume a known font type, font size, and character layout, and assume that the degraded image is blurry and perspectively distorted, but does not necessarily contain additive noise. Although the authors only show results on a small set of images, they do show that information is present in license plates as small as 20 pixels in width. Building on these ideas, in this paper we propose to train a CNN for recognizing highly degraded license plates with an unknown background template, font type, size and character location. We also explicitly work in the presence of high amounts of additive noise – a common occurrence in real-world imagery. Recognition by Human Observers We begin by performing a perceptual study to determine how well human observers can decipher degraded license plates. This study provides a baseline against which to compare our computational approaches. Observers were shown images of synthetically Figure 1. An example of the type of degraded license plate that we seek to

Recently, deep learning has demonstrated great success in computer vision with the capability to learn powerful image features from a large training set. However, most of the published work has been confined to solving 2D problems, with a few limited exceptions that treated the 3D space as a composition of 2D orthogonal planes. The challenge of 3D deep learning is due to a much larger input vector, compared to 2D, which dramatically increases the computation time and the chance of over-fitting, especially when combined with limited training samples (hundreds to thousands), typical for medical imaging applications. To address this challenge, we propose an efficient and robust deep learning algorithm capable of full 3D detection in volumetric data. A two-step approach is exploited for efficient detection. A shallow network (with one hidden layer) is used for the initial testing of all voxels to obtain a small number of promising candidates, followed by more accurate classification with a deep network. In addition, we propose two approaches, i.e., separable filter decomposition and network sparsification, to speed up the evaluation of a network. To mitigate the over-fitting issue, thereby increasing detection robustness, we extract small 3D patches from a multi-resolution image pyramid. The deeply learned image features are further combined with Haar wavelet features to increase the detection accuracy. The proposed method has been quantitatively evaluated for carotid artery bifurcation detection on a head-neck CT dataset from 455 patients. Compared to the state-ofthe-art, the mean error is reduced by more than half, from 5.97 mm to 2.64 mm, with a detection speed of less than 1 s/volume.

Generative Adversarial Networks (GAN) have shown great promise in tasks like synthetic image generation, image inpainting, style transfer, and anomaly detection. However, generating discrete data is a challenge. This work presents an adversarial training based correlated discrete data (CDD) generation model. It also details an approach for conditional CDD generation. The results of our approach are presented over two datasets; job-seeking candidates skill set (private dataset) and MNIST (public dataset). From quantitative and qualitative analysis of these results, we show that our model performs better as it leverages inherent correlation in the data, than an existing model that overlooks correlation.

Previous studies have shown that spike-timing-dependent plasticity (STDP) can be used in spiking neural networks (SNN) to extract visual features of low or intermediate complexity in an unsupervised manner. These studies, however, used relatively shallow architectures, and only one layer was trainable. Another line of research has demonstrated - using rate-based neural networks trained with back-propagation - that having many layers increases the recognition robustness, an approach known as deep learning. We thus designed a deep SNN, comprising several convolutional (trainable with STDP) and pooling layers. We used a temporal coding scheme where the most strongly activated neurons fire first, and less activated neurons fire later or not at all. The network was exposed to natural images. Thanks to STDP, neurons progressively learned features corresponding to prototypical patterns that were both salient and frequent. Only a few tens of examples per category were required and no label was needed. After learning, the complexity of the extracted features increased along the hierarchy, from edge detectors in the first layer to object prototypes in the last layer. Coding was very sparse, with only a few thousands spikes per image, and in some cases the object category could be reasonably well inferred from the activity of a single higher-order neuron. More generally, the activity of a few hundreds of such neurons contained robust category information, as demonstrated using a classifier on Caltech 101, ETH-80, and MNIST databases. We also demonstrate the superiority of STDP over other unsupervised techniques such as random crops (HMAX) or auto-encoders. Taken together, our results suggest that the combination of STDP with latency coding may be a key to understanding the way that the primate visual system learns, its remarkable processing speed and its low energy consumption. These mechanisms are also interesting for artificial vision systems, particularly for hardware solutions.

In this work, we show the intrinsic relations between optimal transportation and convex geometry, especially the variational approach to solve Alexandrov problem: constructing a convex polytope with prescribed face normals and volumes. This leads to a geometric interpretation to generative models, and leads to a novel framework for generative models. By using the optimal transportation view of GAN model, we show that the discriminator computes the Kantorovich potential, the generator calculates the transportation map. For a large class of transportation costs, the Kantorovich potential can give the optimal transportation map by a close-form formula. Therefore, it is sufficient to solely optimize the discriminator. This shows the adversarial competition can be avoided, and the computational architecture can be simplified. Preliminary experimental results show the geometric method outperforms WGAN for approximating probability measures with multiple clusters in low dimensional space.


Question: DeepTransport: Forecasting and Modeling of Human Mobility and Transportation Mode across an Entire City

This paper introduces a novel sparse deep belief network model designed to accurately mimic the computational properties of visual area V2 in the human brain. Inspired by the hierarchical structure of the cortex, the model builds upon recent efforts to learn deep hierarchical representations from unlabeled data. Unlike previous studies that have primarily focused on comparing their models to computations in visual area V1 or the cochlea, this research extends the comparison to deeper levels within the cortical hierarchy, specifically focusing on V2. The model presented is a sparse variant of the deep belief networks proposed by Hinton et al. (2006), and it comprises two layers of nodes. The first layer, similar to previous work on sparse coding and independent component analysis (ICA), generates localized, oriented edge filters that closely resemble Gabor functions, which are known to model V1 cell receptive fields. The second layer of the model, however, takes this a step further by encoding correlations between the first layer responses in the data. This layer captures more complex features, including colinear ("contour") features and corners or junctions. Remarkably, when quantitatively compared to biological V2 responses as documented by Ito & Komatsu, the model's encoding of these complex "corner" features shows a strong match. This finding suggests that the sparse variant of deep belief networks holds significant potential for modeling higher-order visual features, demonstrating its fidelity in mimicking computations found at deeper levels within the cortical hierarchy.

The simulation of urban growth is an important part of urban planning and development. Due to large data and computational challenges, urban growth simulation models demand efficient data analytic frameworks for scaling them to large geographic regions. Agent-based models are widely used to observe and analyze the urban growth simulation at various scales. The incorporation of the agent-based model makes the scaling task even harder due to communication and coordination among agents. Many existing agent-based model frameworks were implemented using traditional shared and distributed memory programming models. On the other hand, Apache Spark is becoming a popular platform for distributed big data in-memory analytics. This paper presents an implementation of agent-based sub-model in Apache Spark framework. With the in-memory computation, Spark implementation outperforms the traditional distributed memory implementation using MPI. This paper provides (i) an overview of our framework capable of running urban growth simulations at a fine resolution of 30 meter grid cells, (ii) a scalable approach using Apache Spark to implement an agent-based model for simulating human decisions, and (iii) the comparative analysis of performance of Apache Spark and MPI based implementations.

Algorithms for route planning in transportation networks have recently undergone a rapid development, leading to methods that are up to one million times faster than Dijkstra’s algorithm. We outline ideas, algorithms, implementations, and experimental methods behind this development. We also explain why the story is not over yet because dynamically changing networks, flexible objective functions, and new applications pose a lot of interesting challenges.

A datatype with increasing importance in GIS is what we call the location history–a record of an entity’s location in geographical space over an interval of time. This paper proposes a number of rigorously defined data structures and algorithms for analyzing and generating location histories. Stays are instances where a subject has spent some time at a single location, and destinations are clusters of stays. Using stays and destinations, we then propose two methods for modeling location histories probabilistically. Experiments show the value of these data structures, as well as the possible applications of probabilistic models of location histories.

In recent years, research on location predictions by mining trajectories of users has attracted a lot of attention. Existing studies on this topic mostly treat such predictions as just a type of location recommendation, that is, they predict the next location of a user using location recommenders. However, an user usually visits somewhere for reasons other than interestingness. In this article, we propose a novel mining-based location prediction approach called Geographic-Temporal-Semantic-based Location Prediction (GTS-LP), which takes into account a user's geographic-triggered intentions, temporal-triggered intentions, and semantic-triggered intentions, to estimate the probability of the user in visiting a location. The core idea underlying our proposal is the discovery of trajectory patterns of users, namely GTS patterns, to capture frequent movements triggered by the three kinds of intentions. To achieve this goal, we define a new trajectory pattern to capture the key properties of the behaviors that are motivated by the three kinds of intentions from trajectories of users. In our GTS-LP approach, we propose a series of novel matching strategies to calculate the similarity between the current movement of a user and discovered GTS patterns based on various moving intentions. On the basis of similitude, we make an online prediction as to the location the user intends to visit. To the best of our knowledge, this is the first work on location prediction based on trajectory pattern mining that explores the geographic, temporal, and semantic properties simultaneously. By means of a comprehensive evaluation using various real trajectory datasets, we show that our proposed GTS-LP approach delivers excellent performance and significantly outperforms existing state-of-the-art location prediction methods.

Micro-video sharing social services, as a new phenomenon in social media, enable users to share micro-videos and thus gain increasing enthusiasm among people. One distinct characteristic of micro-videos is the multi-modality, as these videos always have visual signals, audio tracks, textual descriptions as well as social clues. Such multi-modality data makes it possible to obtain a comprehensive understanding of videos and hence provides new opportunities for researchers. However, limited efforts thus far have been dedicated to this new emerging user-generated contents (UGCs) due to the lack of large-scale benchmark dataset. Towards this end, in this paper, we construct a large-scale micro-video dataset, which can support many research domains, such as popularity prediction and venue estimation. Based upon this dataset, we conduct an initial study in popularity prediction of micro-videos. Finally, we identify our future work.

In this paper, customer restaurant preference is predicted based on social media location check-ins. Historical preferences of the customer and the influence of the customer’s social network are used in combination with the customer’s mobility characteristics as inputs to the model. As the popularity of social media increases, more and more customer comments and feedback about products and services are available online. It not only becomes a way of sharing information among friends in the social network but also forms a new type of survey which can be utilized by business companies to improve their existing products, services, and market analysis. Approximately 121,000 foursquare restaurant check-ins in the Greater New York City area are used in this research. Artificial neural networks (ANN) and support vector machine (SVM) are developed to predict the customers’ behavior regarding restaurant preferences. ANN provides 93.13% average accuracy across investigated customers, compared to only 54.00% for SVM with a sigmoid kernel function.

Using detailed traces from 255 users, we conduct a comprehensive study of smartphone use. We characterize intentional user activities -- interactions with the device and the applications used -- and the impact of those activities on network and energy usage. We find immense diversity among users. Along all aspects that we study, users differ by one or more orders of magnitude. For instance, the average number of interactions per day varies from 10 to 200, and the average amount of data received per day varies from 1 to 1000 MB. This level of diversity suggests that mechanisms to improve user experience or energy consumption will be more effective if they learn and adapt to user behavior. We find that qualitative similarities exist among users that facilitate the task of learning user behavior. For instance, the relative application popularity for can be modeled using an exponential distribution, with different distribution parameters for different users. We demonstrate the value of adapting to user behavior in the context of a mechanism to predict future energy drain. The 90th percentile error with adaptation is less than half compared to predictions based on average behavior across users.

Taxi-booking apps have been very popular all over the world as they provide convenience such as fast response time to the users. The key component of a taxi-booking app is the dispatch system which aims to provide optimal matches between drivers and riders. Traditional dispatch systems sequentially dispatch taxis to riders and aim to maximize the driver acceptance rate for each individual order. However, the traditional systems may lead to a low global success rate, which degrades the rider experience when using the app. In this paper, we propose a novel system that attempts to optimally dispatch taxis to serve multiple bookings. The proposed system aims to maximize the global success rate, thus it optimizes the overall travel efficiency, leading to enhanced user experience. To further enhance users' experience, we also propose a method to predict destinations of a user once the taxi-booking APP is started. The proposed method employs the Bayesian framework to model the distribution of a user's destination based on his/her travel histories.
 We use rigorous A/B tests to compare our new taxi dispatch method with state-of-the-art models using data collected in Beijing. Experimental results show that the proposed method is significantly better than other state-of-the art models in terms of global success rate (increased from 80% to 84%). Moreover, we have also achieved significant improvement on other metrics such as user's waiting-time and pick-up distance. For our destination prediction algorithm, we show that our proposed model is superior to the baseline model by improving the top-3 accuracy from 89% to 93%. The proposed taxi dispatch and destination prediction algorithms are both deployed in our online systems and serve tens of millions of users everyday.

Individual human trajectories are characterized by fat-tailed distributions of jump sizes and waiting times, suggesting the relevance of continuous-time random-walk (CTRW) models for human mobility. However, human traces are barely random. Given the importance of human mobility, from epidemic modelling to traffic prediction and urban planning, we need quantitative models that can account for the statistical characteristics of individual human trajectories. Here we use empirical data on human mobility, captured by mobile-phone traces, to show that the predictions of the CTRW models are in systematic conflict with the empirical results. We introduce two principles that govern human trajectories, allowing us to build a statistically self-consistent microscopic model for individual human mobility. The model accounts for the empirically observed scaling laws, but also allows us to analytically predict most of the pertinent scaling exponents.

Modern transportation system suffers from increasing passenger demand, limited vehicle supply and inefficient mobility service. Towards building an intelligent transportation system to address these issues, we propose a hierarchical framework to implement strategies that is capable of allocating vehicles to serve passengers in different locations based on limited supply. In the higher hierarchy, we optimize idle mileage induced by rebalancing vehicles across regions using receding horizon control towards current and predicted future requests. In addition, we design a dispatch strategy that is robust against passenger demand and vehicle mobility pattern uncertainties. In the lower hierarchy, within each region, pick-up and drop-off schedules for real-time requests are obtained for each vehicle by solving mixed-integer linear programs (MILP). The objective of the MILP is to minimize total mileage delay due to ride-sharing while serving as many requests as possible. We illustrate the validity of our framework via numerical simulations on taxi trip data from New York City.


Question: DeepTransport: Forecasting and modeling of human movement and transportation method across an entire city.

This paper details the development of a deep convolutional neural network (CNN) designed to classify 1.2 million high-resolution images from the ImageNet LSVRC-2010 contest into 1000 distinct categories. Our model achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, on the test dataset, significantly surpassing the previous best results. The CNN architecture is comprised of 60 million parameters and 650,000 neurons, featuring five convolutional layers, several of which are succeeded by max-pooling layers, and three fully connected layers culminating in a 1000-way softmax. To expedite the training process, we utilized non-saturating neurons and an efficient GPU implementation of the convolution operation. In an effort to mitigate overfitting in the fully connected layers, we implemented a novel regularization technique known as "dropout," which proved highly effective. We also submitted a variation of this model to the ILSVRC-2012 competition, where it secured a winning top-5 test error rate of 15.3%, significantly outperforming the second-best entry's rate of 26.2%.

For a similar context, consider DeepTransport, a sophisticated model designed to predict and simulate human mobility and transportation mode across entire cities. Drawing inspiration from the advancements in deep learning, DeepTransport employs a deep neural network to analyze vast urban data sets. This model is trained to understand complex patterns in human movement and transportation preferences, enabling it to predict traffic flow, travel modes, and congestion with remarkable accuracy. By leveraging the power of deep learning, DeepTransport can process and interpret large volumes of data from various sources, including GPS, mobile phone records, and public transportation logs. This capability is crucial for urban planners and policymakers in optimizing city infrastructure, reducing congestion, and improving the overall efficiency of urban transportation systems. The model's architecture is designed to handle the unique challenges of urban mobility data, such as temporal and spatial variations, and it integrates various regularization techniques to ensure robustness against overfitting. Through extensive testing and real-world application, DeepTransport has demonstrated its ability to provide actionable insights into human mobility patterns, making it a valuable tool in the development of smart cities.

High-resolution satellite imagery has been increasingly used on remote sensing classification problems. One of the main factors is the availability of this kind of data. Despite the high availability, very little effort has been placed on the zebra crossing classification problem. In this letter, crowdsourcing systems are exploited in order to enable the automatic acquisition and annotation of a large-scale satellite imagery database for crosswalks related tasks. Then, this data set is used to train deep-learning-based models in order to accurately classify satellite images that contain or not contain zebra crossings. A novel data set with more than 240000 images from 3 continents, 9 countries, and more than 20 cities was used in the experiments. The experimental results showed that freely available crowdsourcing data can be used to accurately (97.11%) train robust models to perform crosswalk classification on a global scale.

An important aspect of human perception is anticipation and anticipating which activities will a human do next (and how to do them) in useful for many applications, for example, anticipation enables an assistive robot to plan ahead for reactive responses in the human environments. In this work, we present a constructive approach for generating various possible future human activities by reasoning about the rich spatial-temporal relations through object affordances. We represent each possible future using an anticipatory temporal conditional random field (ATCRF) where we sample the nodes and edges corresponding to future object trajectories and human poses from a generative model. We then represent the distribution over the potential futures using a set of constructed ATCRF particles. In extensive evaluation on CAD-120 human activity RGB-D dataset, for new subjects (not seen in the training set), we obtain an activity anticipation accuracy (defined as whether one of top three predictions actually happened) of 75.4%, 69.2% and 58.1% for an anticipation time of 1, 3 and 10 seconds respectively. 1

Our daily lives increasingly involve interactions with other individuals via different communication channels, such as email, text messaging, and social media. In this paper we focus on the problem of modeling and predicting how long it takes an individual to respond to an incoming communication event, such as receiving an email or a text. In particular, we explore the effect on response times of an individual’s temporal pattern of activity, such as circadian and weekly patterns which are typically present in individual data. A probabilistic time-warping approach is used, considering linear time to be a transformation of “effective time,” where the transformation is a function of an individual’s activity rate. We apply this transformation of time to two different types of temporal event models, the first for modeling response times directly, and the second for modeling event times via a Hawkes process. We apply our approach to two different sets of real-world email histories. The experimental results clearly indicate that the transformation-based approach produces systematically better models and predictions compared to simpler methods that ignore circadian and weekly patterns. Current technology allows us to collect large quantities of time-stamped individual-level event data characterizing our “digital behavior” in contexts such as texting, email activity, microblogging, social media interactions, and more — and the volume and variety of this type of data is continually increasing. The resulting time-series of events are rich in behavioral information about our daily lives. Tools for obtaining and visualizing such information are becoming increasingly popular, such as the ability to download your entire email history for mail applications such as Gmail, and various software packages for tracking personal fitness using data from devices such as Fitbit. This paper is focused on modeling the temporal aspects of how an individual (also referred to as the “ego”) responds to others, given a sequence of timestamped events (e.g. communication messages via email or text). What can we learn from the way we respond to others? Are there systematic ∗Currently employed at Google. The research described in this paper was conducted while the author was a graduate student at UC Irvine. Copyright c © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Mon Tue Wed Thu Fri Sat Sun 0 1 a (t ) Mon Tue Wed Thu Fri Sat Sun 0 1 a (t ) Figure 1: Example of smoothed daily and weekly patterns over two separate individuals’ email histories. a(t) on the yaxis represents an individual’s typical activity as a function of time. patterns that can be extracted and used predictively? How do our daily sleep and work patterns factor in to our response patterns? These are common issues that arise when modeling communication events, as the ego’s behavior typically changes significantly over the course of a day or week. Examples of such patterns are shown in Figure 1. We propose a novel method for parameterizing these circadian (daily) and weekly patterns, allowing the time dimension to be transformed according to such patterns. This transformation of time will allow models to describe and predict behavioral patterns which are invariant to the ego’s routine patterns. We apply this approach to predicting the ego’s response time to an event. Learning such response patterns is useful not only for identifying relationships between pairs of individuals (Halpin and De Boeck 2013), but also as features for prioritization of events, e.g., the priority inbox implemented in Gmail (Aberdeen, Pacovsky, and Slater 2010). Our experimental results show clear advantages in terms of predictive power when modeling response times in the transformed time domain. τ0 τ1 τ2 t0 t1 t2 ∆0

The potential of the artificial intelligence, data mining and the so-called Big Data to build solutions based on the understanding of human behaviours is being extended with the capabilities of the Future Internet and the Internet of Things (IoT). The described potential of data and networks have been analyzed by the complex network theory, offering a specialized branch, the defined as Human Dynamics. The goal with the Internet of Things in the social area is to describe in real-time the human behaviours and activities. These goals are starting to be feasible through the quantity of data provided by the personal devices such as smart phone, and the smart environments such as Smart cities that makes more intelligent the environment and offer a smart space that sense our movements, actions, and the evolution of the ecosystem. This work analyses the ecosystem defined through the triangle formed by Big Data, Smart Cities and Personal/Wearable Computing to determinate human behaviours and human dynamics. For this purpose, the life-cycle of human dynamics determinations in Smart Cities have been analysed in order to determinate the current status of the technology, challenges, and opportunities.

In modern cities, more and more vehicles, such as taxis, have been equipped with GPS devices for localization and navigation. Gathering and analyzing these large-scale real-world digital traces have provided us an unprecedented opportunity to understand the city dynamics and reveal the hidden social and economic “realities”. One innovative pervasive application is to provide correct driving strategies to taxi drivers according to time and location. In this paper, we aim to discover both efficient and inefficient passenger-finding strategies from a large-scale taxi GPS dataset, which was collected from 5350 taxis for one year in a large city of China. By representing the passenger-finding strategies in a Time-Location-Strategy feature triplet and constructing a train/test dataset containing both top- and ordinary-performance taxi features, we adopt a powerful feature selection tool, L1-Norm SVM, to select the most salient feature patterns determining the taxi performance. We find that the selected patterns can well interpret the empirical study results derived from raw data analysis and even reveal interesting hidden “facts”. Moreover, the taxi performance predictor built on the selected features can achieve a prediction accuracy of 85.3% on a new test dataset, and it also outperforms the one based on all the features, which implies that the selected features are indeed the right indicators of the passenger-finding strategies.

Disseminating a piece of information, or updates for a piece of information, has been shown to benefit greatly from simple randomized procedures, sometimes referred to as gossiping, or epidemic algorithms. Similarly, in a network where mobile nodes occasionally receive updated content from a base station, gossiping using opportunistic contacts allows for recent updates to be efficiently maintained, for a large number of nodes. In this case, however, gossiping depends on node mobility. For this reason, we introduce a new gossip model, with mobile nodes moving between different classes that can represent locations or states, which determine gossiping behavior of the nodes. Here we prove that, when the number of mobile nodes becomes large, the age of the latest updates received by mobile nodes approaches a deterministic mean-field regime. More precisely, we show that the occupancy measure of the process constructed, with the ages defined above, converges to a deterministic limit that can be entirely characterized by differential equations. This major simplification allows us to characterize how mobility, source inputs and gossiping influence the age distribution for low and high ages. It also leads to a scalable numerical evaluation of the performance of mobile update systems, which we validate (using a trace of 500 taxicabs) and use to propose infrastructure deployment.

The interconnectedness of different actors in the global freight transportation industry has rendered such a system as a large complex system where different sub-systems are interrelated. On such a system, policy-related- exploratory analyses which have predictive capacity are difficult to perform. Although there are many global simulation models for various large complex systems, there is unfortunately very little research aimed to develop a global freight transportation model. In this paper, we present a multi-level framework to develop an integrated model of the global freight transportation system. We employ a system view to incorporate different relevant sub-systems and categorize them in different levels. The fourstep model of freight transport is used as the basic foundation of the framework proposed. In addition to that, we also present the computational framework which adheres to the high level modeling framework to provide a conceptualization of the discrete-event simulation model which will be developed.

The pervasiveness and availability of mobile phone data offer the opportunity of discovering usable knowledge about crowd behaviors in urban environments. Cities can leverage such knowledge in order to provide better services (e.g., public transport planning, optimized resource allocation) and safer cities. Call Detail Record (CDR) data represents a practical data source to detect and monitor unusual events considering the high level of mobile phone penetration, compared with GPS equipped and open devices. In this paper, we provide a methodology that is able to detect unusual events from CDR data that typically has low accuracy in terms of space and time resolution. Moreover, we introduce a concept of unusual event that involves a large amount of people who expose an unusual mobility behavior. Our careful consideration of the issues that come from coarse-grained CDR data ultimately leads to a completely general framework that can detect unusual crowd events from CDR data effectively and efficiently. Through extensive experiments on real-world CDR data for a large city in Africa, we demonstrate that our method can detect unusual events with 16% higher recall and over 10 times higher precision, compared to stateof-the-art methods. We implement a visual analytics prototype system to help end users analyze detected unusual crowd events to best suit different application scenarios. To the best of our knowledge, this is the first work on the detection of unusual events from CDR data with considerations of its temporal and spatial sparseness and distinction between user unusual activities and daily routines.

Demographics are widely used in marketing to characterize different types of customers. However, in practice, demographic information such as age, gender, and location is usually unavailable due to privacy and other reasons. In this paper, we aim to harness the power of big data to automatically infer users' demographics based on their daily mobile communication patterns. Our study is based on a real-world large mobile network of more than 7,000,000 users and over 1,000,000,000 communication records (CALL and SMS). We discover several interesting social strategies that mobile users frequently use to maintain their social connections. First, young people are very active in broadening their social circles, while seniors tend to keep close but more stable connections. Second, female users put more attention on cross-generation interactions than male users, though interactions between male and female users are frequent. Third, a persistent same-gender triadic pattern over one's lifetime is discovered for the first time, while more complex opposite-gender triadic patterns are only exhibited among young people.
 We further study to what extent users' demographics can be inferred from their mobile communications. As a special case, we formalize a problem of double dependent-variable prediction-inferring user gender and age simultaneously. We propose the WhoAmI method, a Double Dependent-Variable Factor Graph Model, to address this problem by considering not only the effects of features on gender/age, but also the interrelation between gender and age. Our experiments show that the proposed WhoAmI method significantly improves the prediction accuracy by up to 10% compared with several alternative methods.

We present a novel approach for determining robot movements that efficiently accomplish the robot's tasks while not hindering the movements of people within the environment. Our approach models the goal-directed trajectories of pedestrians using maximum entropy inverse optimal control. The advantage of this modeling approach is the generality of its learned cost function to changes in the environment and to entirely different environments. We employ the predictions of this model of pedestrian trajectories in a novel incremental planner and quantitatively show the improvement in hindrance-sensitive robot trajectory planning provided by our approach.


Question: DeepTransport: Forecasting and modeling of human movement and transportation methods across an entire city.

This paper introduces a groundbreaking application of deep learning to the realm of multimodal feature extraction, expanding upon the success of deep networks in single modality unsupervised feature learning (such as text, images, or audio). We propose a novel approach to train deep networks for feature learning across multiple modalities, presenting a suite of tasks designed to test the effectiveness of this multimodal learning framework. Specifically, we showcase the ability of our network to perform cross-modality feature learning, where the presence of multiple modalities during the learning phase leads to the development of more refined features for a single modality (for instance, improved video features when audio and video are simultaneously considered). Moreover, we demonstrate how to construct a shared representation across modalities, assessing its efficacy through a unique challenge: training a classifier using data from one modality (audio) and testing it on data from another modality (video), and vice versa. Our methodology is rigorously evaluated on two datasets—CUAVE and AVLetters—using an audio-visual speech classification task. The results highlight our model's superior performance in visual speech classification on the AVLetters dataset and its capability for effective multimodal fusion.

Uber is a popular ride-sharing application that matches people who need a ride (or riders) with drivers who are willing to provide it using their personal vehicles. Despite its growing popularity, there exist few studies that examine large-scale Uber data, or in general the factors affecting user participation in the sharing economy. We address this gap through a study of the Uber market that analyzes large-scale data covering 59 million rides which spans a period of 7 months. The data were extracted from email receipts sent by Uber collected on Yahoo servers, allowing us to examine the role of demographics (e.g., age and gender) on participation in the ride-sharing economy. In addition, we evaluate the impact of dynamic pricing (i.e., surge pricing) and income on both rider and driver behavior. We find that the surge pricing does not bias Uber use towards higher income riders. Moreover, we show that more homophilous matches (e.g., riders to drivers of a similar age) can result in higher driver ratings. Finally, we focus on factors that affect retention and use information from earlier rides to accurately predict which riders or drivers will become active Uber users.

The location based social networking services (LBSNSs) are becoming very popular today. In LBSNSs, such as Foursquare, users can explore their places of interests around their current locations, check in at these places to share their locations with their friends, etc. These check-ins contain rich information and imply human mobility patterns; thus, they can greatly facilitate mining and analysis of local geographic topics driven by users' trajectories. The local geographic topics indicate the potential and intrinsic relations among the locations in accordance with users' trajectories. These relations are useful for users in both location and friend recommendations. In this paper, we focus on exploring the local geographic topics through check-ins in Pittsburgh area in Foursquare. We use the Latent Dirichlet Allocation (LDA) model to discover the local geographic topics from the checkins. We also compare the local geographic topics on weekdays with those at weekends. Our results show that LDA works well in finding the related places of interests.

We analyse a large mobile phone activity dataset provided by Telecom Italia for the Telecom Big Data Challenge contest. The dataset reports the international country codes of every call/SMS made and received by mobile phone users in Milan, Italy, between November and December 2013, with a spatial resolution of about 200 meters. We first show that the observed spatial distribution of international codes well matches the distribution of international communities reported by official statistics, confirming the value of mobile phone data for demographic research. Next, we define an entropy function to measure the heterogeneity of the international phone activity in space and time. By comparing the entropy function to empirical data, we show that it can be used to identify the city’s hotspots, defined by the presence of points of interests. Eventually, we use the entropy function to characterize the spatial distribution of international communities in the city. Adopting a topological data analysis approach, we find that international mobile phone users exhibit some robust clustering patterns that correlate with basic socio-economic variables. Our results suggest that mobile phone records can be used in conjunction with topological data analysis tools to study the geography of migrant communities in a global city.

This paper presents a novel online-capable interaction-aware intention and maneuver prediction framework for dynamic environments. The main contribution is the combination of model-based interaction-aware intention estimation with maneuver-based motion prediction based on supervised learning. The advantages of this framework are twofold. On one hand, expert knowledge in the form of heuristics is integrated, which simplifies the modeling of the interaction. On the other hand, the difficulties associated with the scalability and data sparsity of the algorithm due to the so-called curse of dimensionality can be reduced, as a reduced feature space is sufficient for supervised learning. The proposed algorithm can be used for highly automated driving or as a prediction module for advanced driver assistance systems without the need of intervehicle communication. At the start of the algorithm, the motion intention of each driver in a traffic scene is predicted in an iterative manner using the game-theoretic idea of stochastic multiagent simulation. This approach provides an interpretation of what other drivers intend to do and how they interact with surrounding traffic. By incorporating this information into a Bayesian network classifier, the developed framework achieves a significant improvement in terms of reliable prediction time and precision compared with other state-of-the-art approaches. By means of experimental results in real traffic on highways, the validity of the proposed concept and its online capability is demonstrated. Furthermore, its performance is quantitatively evaluated using appropriate statistical measures.

The dynamic spatial redistribution of individuals is a key driving force of various spatiotemporal phenomena on geographical scales. It can synchronize populations of interacting species, stabilize them, and diversify gene pools. Human travel, for example, is responsible for the geographical spread of human infectious disease. In the light of increasing international trade, intensified human mobility and the imminent threat of an influenza A epidemic, the knowledge of dynamical and statistical properties of human travel is of fundamental importance. Despite its crucial role, a quantitative assessment of these properties on geographical scales remains elusive, and the assumption that humans disperse diffusively still prevails in models. Here we report on a solid and quantitative assessment of human travelling statistics by analysing the circulation of bank notes in the United States. Using a comprehensive data set of over a million individual displacements, we find that dispersal is anomalous in two ways. First, the distribution of travelling distances decays as a power law, indicating that trajectories of bank notes are reminiscent of scale-free random walks known as Lévy flights. Second, the probability of remaining in a small, spatially confined region for a time T is dominated by algebraically long tails that attenuate the superdiffusive spread. We show that human travelling behaviour can be described mathematically on many spatiotemporal scales by a two-parameter continuous-time random walk model to a surprising accuracy, and conclude that human travel on geographical scales is an ambivalent and effectively superdiffusive process.

We describe our first-place solution to the ECML/PKDD discovery challenge on taxi destination prediction. The task consisted in predicting the destination of a taxi based on the beginning of its trajectory, represented as a variable-length sequence of GPS points, and diverse associated meta-information, such as the departure time, the driver id and client information. Contrary to most published competitor approaches, we used an almost fully automated approach based on neural networks and we ranked first out of 381 teams. The architectures we tried use multi-layer perceptrons, bidirectional recurrent neural networks and models inspired from recently introduced memory networks. Our approach could easily be adapted to other applications in which the goal is to predict a fixed-length output from a variable-length sequence. Random order, this does not reflect the weights of contributions. ar X iv :1 50 8. 00 02 1v 2 [ cs .L G ] 2 1 Se p 20 15 2 Artificial Neural Networks Applied to Taxi Destination Prediction

Researches on Location-Based Service (LBS) have been emerging in recent years due to a wide range of potential applications. One of the active topics is the mining and prediction of mobile movements and associated transactions. Most of existing studies focus on discovering mobile patterns from the whole logs. However, this kind of patterns may not be precise enough for predictions since the differentiated mobile behaviors among users and temporal periods are not considered. In this paper, we propose a novel algorithm, namely, Cluster-based Temporal Mobile Sequential Pattern Mine (CTMSP-Mine), to discover the Cluster-based Temporal Mobile Sequential Patterns (CTMSPs). Moreover, a prediction strategy is proposed to predict the subsequent mobile behaviors. In CTMSP-Mine, user clusters are constructed by a novel algorithm named Cluster-Object-based Smart Cluster Affinity Search Technique (CO-Smart-CAST) and similarities between users are evaluated by the proposed measure, Location-Based Service Alignment (LBS-Alignment). Meanwhile, a time segmentation approach is presented to find segmenting time intervals where similar mobile characteristics exist. To our best knowledge, this is the first work on mining and prediction of mobile behaviors with considerations of user relations and temporal property simultaneously. Through experimental evaluation under various simulated conditions, the proposed methods are shown to deliver excellent performance.

Among the most important research in Intelligent Transportation Systems (ITS) is the development of systems that automatically monitor traffic flow at intersections. Rather than being based on global flow analysis as is currently done, these automatic monitoring systems should be based on local analysis of the behavior of each vehicle at the intersection. The systems should be able to identify each vehicle and track its behavior, and to recognize situations or events that are likely to result from a chain of such behavior. The most difficult problem associated with vehicle tracking is the occlusion effect among vehicles. In order to solve this problem we have developed an algorithm, referred to as spatio-temporal Markov random field (MRF), for traffic images at intersections. This algorithm models a tracking problem by determining the state of each pixel in an image and its transit, and how such states transit along both the – image axes as well as the time axes. Vehicles, of course, are of various shapes and they move in random fashion, thereby leading to full or partial occlusion at intersections. Despite these complications, our algorithm is sufficiently robust to segment and track occluded vehicles at a high success rate of 93%–96%. This success has led to the development of an extendable robust event recognition system based on the hidden Markov model (HMM). The system learns various event behavior patterns of each vehicle in the HMM chains and then, using the output from the tracking system, identifies current event chains. The current system can recognize bumping, passing, and jamming. However, by including other event patterns in the training set, the system can be extended to recognize those other events, e.g., illegal U-turns or reckless driving. We have implemented this system, evaluated it using the tracking results, and demonstrated its effectiveness.

This paper proposes a shrinkage-based framework for unsupervised trajectory clustering. Facing to the challenges of trajectory clustering, e.g., large variations within a cluster and ambiguities across clusters, we first introduce an adaptive multi-kernel-based estimation process to estimate the 'shrunk' positions and speeds of trajectories' points. This kernel-based estimation effectively leverages both multiple structural information within a trajectory and the local motion patterns across multiple trajectories, such that the discrimination of the shrunk point can be properly increased. We further introduce a speed-regularized optimization process, which utilizes the estimated speeds to regularize the optimal shrunk points, so as to guarantee the smoothness and the discriminative pattern of the final shrunk trajectory. Using our approach, the variations among similar trajectories can be reduced while the boundaries between different clusters are enlarged. Experimental results demonstrate that our approach is superior to the state-of-art approaches on both clustering accuracy and robustness. Besides, additional experiments further reveal the effectiveness of our approach when applied to trajectory analysis applications such as anomaly detection and route analysis.

Human motion modelling is a classical problem at the intersection of graphics and computer vision, with applications spanning human-computer interaction, motion synthesis, and motion prediction for virtual and augmented reality. Following the success of deep learning methods in several computer vision tasks, recent work has focused on using deep recurrent neural networks (RNNs) to model human motion, with the goal of learning time-dependent representations that perform tasks such as short-term motion prediction and long-term human motion synthesis. We examine recent work, with a focus on the evaluation methodologies commonly used in the literature, and show that, surprisingly, state of the art performance can be achieved by a simple baseline that does not attempt to model motion at all. We investigate this result, and analyze recent RNN methods by looking at the architectures, loss functions, and training procedures used in state-of-the-art approaches. We propose three changes to the standard RNN models typically used for human motion, which results in a simple and scalable RNN architecture that obtains state-of-the-art performance on human motion prediction.


Question: DeepTransport: Forecasting and modeling of human movement and transportation method across an entire city.

This study delves into the interplay between human mobility and social relationships, particularly within the framework of location-based social networks. Through the analysis of mobile phone location data and information from two online location-based social platforms, we seek to unravel the fundamental principles that govern human movement and dynamics. Our findings reveal a dual nature in human motion: a regular, geographically confined pattern coexists with sporadic, long-distance jumps that are closely tied to social networks. Short-distance movements, characterized by periodicity in both space and time, appear to be largely independent of social network structure. In contrast, long-distance travel is more significantly influenced by social ties. We estimate that social relationships account for approximately 10% to 30% of human mobility, while periodic behavior explains a larger portion, ranging from 50% to 70%. Inspired by these insights, we propose a model that integrates periodic short-range movements with long-distance travel motivated by social network interactions. This model demonstrates a high degree of accuracy in forecasting future human movements and dynamics, outperforming existing mobility models by an order of magnitude.

**Question**: A Comparative Study of Machine Learning Algorithms for Prediction of Solar Power Generation

**Documents**: A Novel Approach to Solar Power Forecasting Using Ensemble Learning
Accurate forecasting of solar power generation is critical for effective grid management and planning. This paper introduces a novel ensemble learning approach for solar power forecasting that combines the strengths of multiple machine learning algorithms. The ensemble method, termed SolarForecastNet, integrates predictions from a variety of models, including random forests, gradient boosting machines, and recurrent neural networks. The proposed method is evaluated using real-world solar irradiance data and compared against individual models. Results show that SolarForecastNet outperforms each individual model, offering improved forecast accuracy and robustness. The ensemble approach also demonstrates enhanced performance under varying weather conditions, making it a promising solution for solar power forecasting in diverse environments.

**Similar Documents**: This research presents a new ensemble learning framework, SolarForecastNet, designed specifically for the precise prediction of solar power generation. Given the significance of accurate solar power forecasting for efficient grid operations and planning, the development of this framework is crucial. SolarForecastNet amalgamates predictions from an array of machine learning algorithms, encompassing random forests, gradient boosting machines, and recurrent neural networks. Through rigorous testing with real-world solar irradiance datasets, SolarForecastNet is shown to surpass the performance of individual models, delivering enhanced forecast accuracy and reliability. Notably, the ensemble approach exhibits superior performance across a range of weather conditions

We study map-matching, the problem of estimating the route that is traveled by a vehicle, where the points observed with the Global Positioning System are available. A state-of-the-art approach for this problem is a Hidden Markov Model (HMM). We propose a particular transition probability between latent road segments by the use of the number of turns in addition to the travel distance between the latent road segments. We use inverse reinforcement learning to estimate the importance of the number of turns relative to the travel distance. This estimated importance is incorporated in the transition probability of the HMM. We show, through numerical experiments, that the error of map-matching can be reduced substantially with the proposed transition probability.

Taxi drivers always look for strategies to locate passengers quickly and therefore increase their profit margin. In reality, the passenger seeking strategies are mostly empirical and substantially vary among taxi drivers. From the history taxi data, the top performing taxi drivers can earn 25% more than the ones with mediocre seeking strategy in the same period of time. A better strategy not only helps taxi drivers earn more with less effort, but also reduce fuel consumption and carbon emissions. It is interesting to examine the influential factors in passenger seeking strategies and find algorithms to guide taxi drivers to passenger hotspots with the right timing. With the abundant availability of history taxicab traces, the existing methods of doing taxi business have been radically changed. This paper focuses on the problem of mining efficient operation strategies from a large-scale history taxi traces collected over one year. Our approach presents generic insights into the dynamics of taxicab services with the objective of maximizing the profit margins for the concerned parties. We propose important metrics, such as trip frequency, hot spots, and taxi mileage, and provide valuable insights toward more efficient operation strategies. We analyze these metrics using techniques, such as Newton’s polynomial interpolation and Gamma distribution, to understand their dynamics. Our strategies use the real taxicab traces from the city of Changsha (P.R.China), may predict the taxi rides at different times by 90.68% per day, and increase the taxi drivers income levels up to 19.38% by controlling appropriate mileage per trip and following the route across more urban hot spots.

The investigation of pedestrian spatio-temporal behaviour is of particular interest in many different research fields. Disciplines like travel behaviour research and tourism research, social sciences, artificial intelligence, geoinformation and many others have approached this subject from different perspectives. Depending on the particular research questions, various methods of data collection and analysis have been developed and applied in order to gain insight into specific aspects of human motion behaviour and the determinants influencing spatial activities. In this contribution, we provide a general overview about most commonly used methods for monitoring and analysing human spatio-temporal behaviour. After discussing frequently used empirical methods of data collection and emphasising related advantages and limitations, we present seven case studies concerning the collection and analysis of human motion behaviour following different purposes.

To reduce roadway congestion and improve traffic safety, accurate traffic metrics, such as number of vehicles travelling through lane-ways, are required. Unfortunately most existing infrastructure, such as loop-detectors and many video detectors, do not feasibly provide accurate vehicle counts. Consequently, a novel method is proposed which models vehicle motion using hidden Markov models (HMM). The proposed method represents a specified small region of the roadway as 'empty', 'vehicle entering', 'vehicle inside', and 'vehicle exiting', and then applies a modified Viterbi algorithm to the HMM sequential estimation framework to initialize and track vehicles. Vehicle observations are obtained using an Adaboost trained Haar-like feature detector. When tested on 88 hours of video, from three distinct locations, the proposed method proved to be robust to changes in lighting conditions, moving shadows, and camera motion, and consistently out-performed Multiple Target Tracking (MTT) and Virtual Detection Line(VDL) implementations. The median vehicle count error of the proposed method is lower than MTT and VDL by 28%, and 70% respectively. As future work, this algorithm will be implemented to provide the traffic industry with improved automated vehicle counting, with the intent to eventually provide real-time counts.

The success and impact of activity recognition algorithms largely depends on the availability of the labeled training samples and adaptability of activity recognition models across various domains. In a new environment, the pre-trained activity recognition models face challenges in presence of sensing bias- ness, device heterogeneities, and inherent variabilities in human behaviors and activities. Activity Recognition (AR) system built in one environment does not scale well in another environment, if it has to learn new activities and the annotated activity samples are scarce. Indeed building a new activity recognition model and training the model with large annotated samples often help overcome this challenging problem. However, collecting annotated samples is cost-sensitive and learning activity model at wild is computationally expensive. In this work, we propose an activity recognition framework, UnTran that utilizes source domains' pre-trained autoencoder enabled activity model that transfers two layers of this network to generate a common feature space for both source and target domain activities. We postulate a hybrid AR framework that helps fuse the decisions from a trained model in source domain and two activity models (raw and deep-feature based activity model) in target domain reducing the demand of annotated activity samples to help recognize unseen activities. We evaluated our framework with three real-world data traces consisting of 41 users and 26 activities in total. Our proposed UnTran AR framework achieves ≈ 75% F1 score in recognizing unseen new activities using only 10% labeled activity data in the target domain. UnTran attains ≈ 98% F1 score while recognizing seen activities in presence of only 2-3% of labeled activity samples.

This paper presents a multi-output regression model for crowd counting in public scenes. Existing counting by regression methods either learn a single model for global counting, or train a large number of separate regressors for localised density estimation. In contrast, our single regression model based approach is able to estimate people count in spatially localised regions and is more scalable without the need for training a large number of regressors proportional to the number of local regions. In particular, the proposed model automatically learns the functional mapping between interdependent low-level features and multi-dimensional structured outputs. The model is able to discover the inherent importance of different features for people counting at different spatial locations. Extensive evaluations on an existing crowd analysis benchmark dataset and a new more challenging dataset demonstrate the effectiveness of our approach.

Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications. Object detection has benefited enormously from large-scale datasets, especially in the context of deep learning. For semantic urban scene understanding, however, no current dataset adequately captures the complexity of real-world urban scenes. To address this, we introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. Cityscapes is comprised of a large, diverse set of stereo video sequences recorded in streets from 50 different cities. 5000 of these images have high quality pixel-level annotations, 20 000 additional images have coarse annotations to enable methods that leverage large volumes of weakly-labeled data. Crucially, our effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Our accompanying empirical study provides an in-depth analysis of the dataset characteristics, as well as a performance evaluation of several state-of-the-art approaches based on our benchmark.

Mobile robots are increasingly populating our human environments. To interact with humans in a socially compliant way, these robots need to understand and comply with mutually accepted rules. In this paper, we present a novel approach to model the cooperative navigation behavior of humans. We model their behavior in terms of a mixture distribution that captures both the discrete navigation decisions, such as going left or going right, as well as the natural variance of human trajectories. Our approach learns the model parameters of this distribution that match, in expectation, the observed behavior in terms of user-defined features. To compute the feature expectations over the resulting high-dimensional continuous distributions, we use Hamiltonian Markov chain Monte Carlo sampling. Furthermore, we rely on a Voronoi graph of the environment to efficiently explore the space of trajectories from the robot’s current position to its target position. Using the proposed model, our method is able to imitate the behavior of pedestrians or, alternatively, to replicate a specific behavior that was taught by tele-operation in the target environment of the robot. We implemented our approach on a real mobile robot and demonstrate that it is able to successfully navigate in an office environment in the presence of humans. An extensive set of experiments suggests that our technique outperforms state-of-the-art methods to model the behavior of pedestrians, which makes it also applicable to fields such as behavioral science or computer graphics.

BACKGROUND
Twitter has shown some usefulness in predicting influenza cases on a weekly basis in multiple countries and on different geographic scales. Recently, Broniatowski and colleagues suggested Twitter's relevance at the city-level for New York City. Here, we look to dive deeper into the case of New York City by analyzing daily Twitter data from temporal and spatiotemporal perspectives. Also, through manual coding of all tweets, we look to gain qualitative insights that can help direct future automated searches.

OBJECTIVE
The intent of the study was first to validate the temporal predictive strength of daily Twitter data for influenza-like illness emergency department (ILI-ED) visits during the New York City 2012-2013 influenza season against other available and established datasets (Google search query, or GSQ), and second, to examine the spatial distribution and the spread of geocoded tweets as proxies for potential cases.

METHODS
From the Twitter Streaming API, 2972 tweets were collected in the New York City region matching the keywords "flu", "influenza", "gripe", and "high fever". The tweets were categorized according to the scheme developed by Lamb et al. A new fourth category was added as an evaluator guess for the probability of the subject(s) being sick to account for strength of confidence in the validity of the statement. Temporal correlations were made for tweets against daily ILI-ED visits and daily GSQ volume. The best models were used for linear regression for forecasting ILI visits. A weighted, retrospective Poisson model with SaTScan software (n=1484), and vector map were used for spatiotemporal analysis.

RESULTS
Infection-related tweets (R=.763) correlated better than GSQ time series (R=.683) for the same keywords and had a lower mean average percent error (8.4 vs 11.8) for ILI-ED visit prediction in January, the most volatile month of flu. SaTScan identified primary outbreak cluster of high-probability infection tweets with a 2.74 relative risk ratio compared to medium-probability infection tweets at P=.001 in Northern Brooklyn, in a radius that includes Barclay's Center and the Atlantic Avenue Terminal.

CONCLUSIONS
While others have looked at weekly regional tweets, this study is the first to stress test Twitter for daily city-level data for New York City. Extraction of personal testimonies of infection-related tweets suggests Twitter's strength both qualitatively and quantitatively for ILI-ED prediction compared to alternative daily datasets mixed with awareness-based data such as GSQ. Additionally, granular Twitter data provide important spatiotemporal insights. A tweet vector-map may be useful for visualization of city-level spread when local gold standard data are otherwise unavailable.

The rise of GPS-equipped mobile devices has led to the emergence of big trajectory data. In this paper, we study a new path finding query which finds the most frequent path (MFP) during user-specified time periods in large-scale historical trajectory data. We refer to this query as time period-based MFP (TPMFP). Specifically, given a time period T, a source v_s and a destination v_d, TPMFP searches the MFP from v_s to v_d during T. Though there exist several proposals on defining MFP, they only consider a fixed time period. Most importantly, we find that none of them can well reflect people's common sense notion which can be described by three key properties, namely suffix-optimal (i.e., any suffix of an MFP is also an MFP), length-insensitive (i.e., MFP should not favor shorter or longer paths), and bottleneck-free (i.e., MFP should not contain infrequent edges). The TPMFP with the above properties will reveal not only common routing preferences of the past travelers, but also take the time effectiveness into consideration. Therefore, our first task is to give a TPMFP definition that satisfies the above three properties. Then, given the comprehensive TPMFP definition, our next task is to find TPMFP over huge amount of trajectory data efficiently. Particularly, we propose efficient search algorithms together with novel indexes to speed up the processing of TPMFP. To demonstrate both the effectiveness and the efficiency of our approach, we conduct extensive experiments using a real dataset containing over 11 million trajectories.


Question: DeepTransport: Forecasting and Modeling Human Mobility and Transport Mode across an Entire City

In this work, we present a novel methodology for determining a user's transportation mode using data from their mobile device's GPS sensor and Geographic Information System (GIS) data. Our approach aims to discern the user's context by identifying if they are walking, cycling, or using public transport such as trains. The system leverages real-time bus locations, spatial rail data, and information on bus stops to enhance the accuracy of mode detection. Key features related to the transportation network are extracted and utilized to boost classification performance. Our method achieves a remarkable accuracy of over 93.5% in distinguishing between various modes such as car, bus, aboveground train, walking, biking, and stationary states. Compared to GPS-only approaches, our system improves accuracy by 17%, and by 9% over GPS with GIS models. A significant advancement is the ability to accurately differentiate between different motorized modes, including bus, car, and aboveground train, a feat not previously accomplished with such precision. Furthermore, for users traveling by bus, our system can specify the exact bus they are on. We evaluate five distinct inference models—Bayesian Network, Decision Tree, Random Forest, Naïve Bayesian, and Multilayer Perceptron—in our experiments. The resultant classification system has been deployed and is accessible to the public, showcasing its practical utility in understanding human mobility patterns at a city-wide scale.

Smart Cities gained importance as a means of making ICT enabled services and applications available to the citizens, companies and authorities that are part of a city's system. It aims at increasing citizens' quality of life, and improving the efficiency and quality of the services provided by governing entities and businesses. This perspective requires an integrated vision of a city and of its infrastructures, in all its components. A Smart City can be taken according to six characteristics: All these domains raise new challenges in security and privacy, since users implicitly expect systems to be secure and privacy-preserving. One of the critical elements is which role(s) the city will take up as an actor within an increasingly complex value network. New players enter the market, actors shift their business strategies, roles change, different types of platforms emerge and vie for market dominance, technological developments create new threats and opportunities, etc. An element related to the trend of platformisation is cloud computing, which is increasingly helping the private sector to reduce cost, increase efficiency, and work smarter. One particular challenge relates to open data business models. Activities necessary for Public Sector Information provision can be identified. The development of efficient and effective e-government is a prerequisite. Transnational authentication systems for citizens and businesses, agreed frameworks for data privacy, and the sharing and collection of individual and business data, are key. Smart Cities need to be able to integrate themselves into national, regional and international infrastructures. Although the implementation aspects depend strongly on the authorities of these infrastructures, European wide recommendations and directives will definitely contribute to accelerate the deployment of Smart Cities. Health, inclusion and assisted living will play an essential role, since the demand for related services is rising, because ageing is changing disease composition. Requirements address a number of technologies, beyond the ones related to mobile and fixed networks. An integrated perspective on healthcare solutions for the near-to long-term can be foreseen, bridging a direct gap in between the health area and the technological development of communications (radio and network components). The needs for mobility in urban areas result into a number of problems, such as traffic congestion and energy consumption, which can be alleviated by exploiting Intelligent Transportation Systems and further adoption of vehicle-to-vehicle and vehicle-to-infrastructure communication networks. The information being managed in this area can be relevant to other domains, which increases its potential. An effective deployment …

With the development of GPS and the popularity of smart phones and wearable devices, users can easily log their daily trajectories. Prior works have elaborated on mining trajectory patterns from raw trajectories. Trajectory patterns consist of hot regions and the sequential relationships among them, where hot regions refer the spatial regions with a higher density of data points. Note that some hot regions do not have any meaning for users. Moreover, trajectory patterns do not have explicit time information or semantic information. To enrich trajectory patterns, we propose semantic trajectory patterns which are referred to as the moving patterns with spatial, temporal, and semantic attributes. Given a user trajectory, we aim at mining frequent semantic trajectory patterns. Explicitly, we extract the three attributes from a raw trajectory, and convert it into a semantic mobility sequence. Given such a semantic mobility sequence, we propose two algorithms to discover frequent semantic trajectory patterns. The first algorithm, MB (standing for matching-based algorithm), is a naive method to find frequent semantic trajectory patterns. It generates all possible patterns and extracts the occurrence of the patterns from the semantic mobility sequence. The second algorithm, PS (standing for PrefixSpan-based algorithm), is developed to efficiently mine semantic trajectory patterns. Due to the good efficiency of PrefixSpan, algorithm PS will fully utilize the advantage of PrefixSpan. Since the semantic mobility sequence contains three attributes, we need to further transform it into a raw sequence before using algorithm PrefixSpan. Therefore, we propose the SS algorithm (standing for sequence symbolization algorithm) to achieve this purpose. To evaluate our proposed algorithms, we conducted experiments on the real datasets of Google Location History, and the experimental results show the effectiveness and efficiency of our proposed algorithms.

Micro-blogging services, such as Twitter, and location-based social network applications have generated short text messages associated with geographic information, posting time, and user ids. The availability of such data received from users offers a good opportunity to study the user's spatial-temporal behavior and preference. In this paper, we propose a probabilistic model W4 (short for Who+Where+When+What) to exploit such data to discover individual users' mobility behaviors from spatial, temporal and activity aspects. To the best of our knowledge, our work offers the first solution to jointly model individual user's mobility behavior from the three aspects. Our model has a variety of applications, such as user profiling and location prediction; it can be employed to answer questions such as ``Can we infer the location of a user given a tweet posted by the user and the posting time?" Experimental results on two real-world datasets show that the proposed model is effective in discovering users' spatial-temporal topics, and outperforms state-of-the-art baselines significantly for the task of location prediction for tweets.

In this paper we present an approach to enhance existing maps with fine grained segmentation categories such as parking spots and sidewalk, as well as the number and location of road lanes. Towards this goal, we propose an efficient approach that is able to estimate these fine grained categories by doing joint inference over both, monocular aerial imagery, as well as ground images taken from a stereo camera pair mounted on top of a car. Important to this is reasoning about the alignment between the two types of imagery, as even when the measurements are taken with sophisticated GPS+IMU systems, this alignment is not sufficiently accurate. We demonstrate the effectiveness of our approach on a new dataset which enhances KITTI [8] with aerial images taken with a camera mounted on an airplane and flying around the city of Karlsruhe, Germany.

Probabilistic forecasting, i.e. estimating the probability distribution of a time series’ future given its past, is a key enabler for optimizing business processes. In retail businesses, for example, forecasting demand is crucial for having the right inventory available at the right time at the right place. In this paper we propose DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an auto-regressive recurrent network model on a large number of related time series. We demonstrate how by applying deep learning techniques to forecasting, one can overcome many of the challenges faced by widely-used classical approaches to the problem. We show through extensive empirical evaluation on several real-world forecasting data sets that our methodology produces more accurate forecasts than other state-of-the-art methods, while requiring minimal manual work.

In recent years, research on location predictions by mining trajectories of users has attracted a lot of attention. Existing studies on this topic mostly treat such predictions as just a type of location recommendation, that is, they predict the next location of a user using location recommenders. However, an user usually visits somewhere for reasons other than interestingness. In this article, we propose a novel mining-based location prediction approach called Geographic-Temporal-Semantic-based Location Prediction (GTS-LP), which takes into account a user's geographic-triggered intentions, temporal-triggered intentions, and semantic-triggered intentions, to estimate the probability of the user in visiting a location. The core idea underlying our proposal is the discovery of trajectory patterns of users, namely GTS patterns, to capture frequent movements triggered by the three kinds of intentions. To achieve this goal, we define a new trajectory pattern to capture the key properties of the behaviors that are motivated by the three kinds of intentions from trajectories of users. In our GTS-LP approach, we propose a series of novel matching strategies to calculate the similarity between the current movement of a user and discovered GTS patterns based on various moving intentions. On the basis of similitude, we make an online prediction as to the location the user intends to visit. To the best of our knowledge, this is the first work on location prediction based on trajectory pattern mining that explores the geographic, temporal, and semantic properties simultaneously. By means of a comprehensive evaluation using various real trajectory datasets, we show that our proposed GTS-LP approach delivers excellent performance and significantly outperforms existing state-of-the-art location prediction methods.

Detecting and reacting to user behavior and ambient context are core elements of many emerging mobile sensing and Internet-of-Things (IoT) applications. However, extracting accurate inferences from raw sensor data is challenging within the noisy and complex environments where these systems are deployed. Deep Learning -- is one of the most promising approaches for overcoming this challenge, and achieving more robust and reliable inference. Techniques developed within this rapidly evolving area of machine learning are now state-of-the-art for many inference tasks (such as, audio sensing and computer vision) commonly needed by IoT and wearable applications. But currently deep learning algorithms are seldom used in mobile/IoT class hardware because they often impose debilitating levels of system overhead (e.g., memory, computation and energy). Efforts to address this barrier to deep learning adoption are slowed by our lack of a systematic understanding of how these algorithms behave at inference time on resource constrained hardware. In this paper, we present the first -- albeit preliminary -- measurement study of common deep learning models (such as Convolutional Neural Networks and Deep Neural Networks) on representative mobile and embedded platforms. The aim of this investigation is to begin to build knowledge of the performance characteristics, resource requirements and the execution bottlenecks for deep learning models when being used to recognize categories of behavior and context. The results and insights of this study, lay an empirical foundation for the development of optimization methods and execution environments that enable deep learning to be more readily integrated into next-generation IoT, smartphones and wearable systems.

Successful traffic speed prediction is of great importance for the benefits of both road users and traffic management agencies. To solve the problem, traffic scientists have developed a number of time-series speed prediction approaches, including traditional statistical models and machine learning techniques. However, existing methods are still unsatisfying due to the difficulty to reflect the stochastic traffic flow characteristics. Recently, various deep learning models have been introduced to the prediction field. In this paper, a deep learning method, the Deep Belief Network (DBN) model, is proposed for short-term traffic speed information prediction. The DBN model is trained in a greedy unsupervised method and fine-tuned by labeled data. Based on traffic speed data collected from one arterial in Beijing, China, the model is trained and tested for different prediction time horizons. From experiment analysis, it is concluded that the DBN can outperform Back Propagation Neural Network (BPNN) and Auto-Regressive Integrated Moving Average (ARIMA) for all time horizons. The advantages of DBN indicate that deep learning is promising in traffic research area.

Devices with integrated Wi-Fi chips broadcast beacons for network connection management purposes. Such information can be captured with inexpensive monitors and used to extract user behavior. To understand the behavior of visitors, we deployed our passive monitoring system---CrowdProbe, in a multi-floor museum for six months. We used a Hidden Markov Models (HMM) based trajectory inference algorithm to infer crowd movement using more than 1.7 million opportunistically obtained probe request frames.
 However, as more devices adopt schemes to randomize their MAC addresses in the passive probe session to protect user privacy, it becomes more difficult to track crowd and understand their behavior. In this paper, we try to make use of historical transition probability to reason about the movement of those randomized devices with spatial and temporal constraints. With CrowdProbe, we are able to achieve sufficient accuracy to understand the movement of visitors carrying devices with randomized MAC addresses.

Mobile robots are increasingly populating our human environments. To interact with humans in a socially compliant way, these robots need to understand and comply with mutually accepted rules. In this paper, we present a novel approach to model the cooperative navigation behavior of humans. We model their behavior in terms of a mixture distribution that captures both the discrete navigation decisions, such as going left or going right, as well as the natural variance of human trajectories. Our approach learns the model parameters of this distribution that match, in expectation, the observed behavior in terms of user-defined features. To compute the feature expectations over the resulting high-dimensional continuous distributions, we use Hamiltonian Markov chain Monte Carlo sampling. Furthermore, we rely on a Voronoi graph of the environment to efficiently explore the space of trajectories from the robot’s current position to its target position. Using the proposed model, our method is able to imitate the behavior of pedestrians or, alternatively, to replicate a specific behavior that was taught by tele-operation in the target environment of the robot. We implemented our approach on a real mobile robot and demonstrate that it is able to successfully navigate in an office environment in the presence of humans. An extensive set of experiments suggests that our technique outperforms state-of-the-art methods to model the behavior of pedestrians, which makes it also applicable to fields such as behavioral science or computer graphics.