Question: LTENV2Vsim: An LTE-V2V simulator for examining resource allocation in cooperative awareness systems.

This paper presents an enhanced IEEE 802.16 WiMAX module for the ns-3 network simulator, designed to address the growing demand for simulating broadband wireless networks. With the increasing interest from researchers and industry professionals, various WiMAX simulators have surfaced within the networking community, one of which is the ns-3 WiMAX module. This module stands out for its adherence to standards and its design, benefiting from the advanced features of ns-3, which is poised to become the leading network simulator in the near future. Despite its strengths, the WiMAX module has certain limitations that prompted the need for improvements. The paper begins by reviewing existing WiMAX simulators available in the public domain, pointing out their shortcomings. Subsequently, it details the new features and enhancements integrated into the ns-3 WiMAX module, focusing on: a more accurate and scalable physical layer model, an IP packet classifier for the convergence sub-layer, efficient uplink and downlink schedulers, support for multicast traffic, and pcap packet tracing capabilities. The redesigned physical layer significantly reduces simulation time by several orders of magnitude while maintaining a faithful representation of the standard. Additionally, the IP classifier allows for the simulation of an unlimited number of service flows per subscriber station, while the enhanced schedulers better manage QoS requirements for various service flows. These improvements collectively enhance the module's realism and efficiency, making it a valuable tool for researchers and practitioners in the field of broadband wireless network simulation.

This paper deals with a holistic approach to coordinate a fleet of automated guided vehicles (AGVs) in an industrial environment. We propose an ensemble approach based on a two layer control architecture and on an automatic algorithm for the definition of the roadmap. The roadmap is built by considering the path planning algorithm implemented on the hierarchical architecture and vice versa. In this way, we want to manage the coordination of the whole system in order to increase the flexibility and the global efficiency. Furthermore, the roadmap is computed in order to maximize the redundancy, the coverage and the connectivity. The architecture is composed of two layers. The low-level represents the roadmap itself. The high-level describes the topological relationship among different areas of the environment. The path planning algorithm works on both these levels and the subsequent coordination among AGVs is obtained exploiting shared resource (i.e., centralized information) and local negotiation (i.e., decentralized coordination). The proposed approach is validated by means of simulations and comparison using real plants. Note to Practitioners-The motivation of this work grows from the need to increase the flexibility and efficiency of current multi-AGV systems. In particular, in the current state-of-the-art the AGVs are guided through a manually defined roadmap with ad-hoc strategies for coordination. This is translated into high setup time requested for installation and the impossibility to easily respond to dynamic changes of the environment. The proposed method aims at managing all the design, setup and control process in an automatic manner, decreasing the time for setup and installation. The flexibility is also increased by considering a coordination strategy for the fleet of AGVs not based on manual ad-hoc rules. Simulations are performed in order to compare the proposed approach to the current industrial one. In the future, industrial aspects, as the warehouse management system, will be integrated in order to achieve a real and complete industrial functionality.

In this paper, we apply the concept of a software-defined data plane to defining new services for mobile virtual network operators (MVNOs) to empower them with the capability of tailoring fine-grained subscription plans that can meet users' demands. For this purpose, we have recently proposed the concept of application-and/ or device-specific slicing that classifies application-and/or device-specific traffic into slices and introduced various applications of our proposed system [Optical Fiber Communication Conf. (OFC), 2016, paper Th3I.4]. This paper reports the prototype implementation of the proposal into a real MVNO connecting customized smartphones so we can identify applications/devices from the given traffic with 100% accuracy. We first characterize the traffic patterns of popular applications to build a detailed understanding of how network resources are utilized for real users and applications and what their demands on different network resources are. Then, we classify traffic according to the devices and find that the flow characteristics of LTE RAN slices are very similar to those of 3G. We discover that most of the bandwidth is consumed by those flows with the duration of several hundreds of seconds, and the flow size is larger than 5 Mbytes in both LTE and 3G MVNO networks.

This paper describes the environment perception system designed for intelligent vehicle SmartV-II, which won the 2010 Future Challenge. This system utilizes the cooperation of multiple lasers and cameras to realize several necessary functions of autonomous navigation: road curb detection, lane detection and traffic sign recognition. Multiple single scan lasers are integrated to detect the road curb based on Z-variance method. Vision based lane detection is realized by two scans method combining with image model. Haar-like feature based method is applied for traffic sign detection and SURF matching method is used for sign classification. The results of experiments validate the effectiveness of the proposed algorithms and the whole system.

Device-to-Device (D2D) communication will become an important technology in future networks with the increase of the requirements of local communication services. The interference between cellular communication and D2D communication can be coordinated by proper power control and resource allocation. In this paper, we analyze the resource allocation methods for D2D communication underlaying cellular networks. A novel resource allocation method that D2D can reuse the resources of more than one cellular user is proposed. After that, we discuss the selection of the optimal resource allocation method from the proposed method and the conventional methods. Finally, the performance of different methods is evaluated through numerical simulation. The simulation results show that the proposed method is the optimal resource allocation method when the D2D pair locates at the most part of the cell area in both uplink and downlink. The proposed method can improve the sum throughput of cellular communication and D2D communication significantly.

Automatic dependent surveillance-broadcast (ADS-B) is one of the fundamental surveillance technologies to improve the safety, capacity, and efficiency of the national airspace system. ADS-B shares its frequency band with current radar systems that use the same 1,090 MHz band. The coexistence of radar systems and ADS-B systems is a key issue to detect and resolve conflicts in the next generation air transportation system (NextGen). This paper focuses on the performance evaluation of ADS-B with existing radar systems and performance optimization of ADS-B systems to improve the safety and efficiency of conflict detection and resolution in NextGen. We have developed a simulation environment which models the complex interplay among the air traffic load, the radar systems, the ADS-B systems, and the wireless channel. A simple model is used to derive an analytical expression for a performance metric of ADS-B. This model is then used to design an adaptive ADS-B protocol for maximizing the information coverage while guaranteeing reliable and timely communication in air traffic surveillance networks. Simulation results show that the effect of ADS-B interference on the current radar system is negligible. The operational ability of ADS-B meets the performance requirements of conflict detection and resolution in air traffic control. However, upgrades are required in the current radar system for operation within an ADS-B environment since the current radars can significantly degrade the ADS-B performance. Numerical results indicate that the proposed adaptive protocol has the potential to improve the performance of conflict detection and resolution in air traffic control.

Driverless cars and driver-assisted vehicles are becoming prevalent in today's transportation systems. Companies like TESLA, Audi, and Google are among the leaders in this new era of the automotive industry. Modern vehicles, equipped with sophisticated navigation devices, complex driver-assisted systems, and a whole lot of safety features, bring broader impacts to our quality of life, economic development, and environmental sustainability. Instant safety messages such as pre-collision warnings, blind-spot detection, pedestrian and object awareness significantly improve the safety for drivers, passengers, and pedestrians. As a result, vehicles would be able to travel closely yet safely together, forming a platoon, thus resulting in a reduction of traffic congestion and fuel consumption. Driverless cars also have non-safety-related applications: which are used to facilitate traffic management and infotainment dissemination for drivers and passengers. Vehicular Ad hoc Network (VANET), the wireless communication technology enabling driverless cars, features not only dynamic topology but also high mobility. The vehicle nodes move rapidly on streets and highways. Their movements also depend on but not limited to road traffic, speed limits, and behavior of nearby vehicles. With massive amount of messages exchanged among driverless cars that command the cars' movements at high speeds and in close distances, any malicious alternation could lead to disastrous accidents. Message authentication to ensure data integrity is paramount to attack preparation. This paper presents a novel message authentication scheme that protects cars from bogus messages and makes VANET resilient to Denial-of-Service (DoS) attacks. The work includes a simulation framework that integrates vehicle and data traffic models to validate the effectiveness of the proposed message authentication scheme.

Autonomous driving has shown the capability of providing driver convenience and enhancing safety. While introducing autonomous driving into our current traffic system, one significant issue is to make the autonomous vehicle be able to react in the same way as real human drivers. In order to ensure that an autonomous vehicle of the future will perform like human drivers, this paper proposes a vehicle motion planning model, which can represent how drivers control vehicles based on the assessment of traffic environments in the real signalized intersection. The proposed motion planning model comprises functions of pedestrian intention detection, gap detection and vehicle dynamic control. The three functions are constructed based on the analysis of actual data collected from real traffic environments. Finally, this paper demonstrates the performance of the proposed method by comparing the behaviors of our model with the behaviors of real pedestrians and human drivers. The experimental results show that our proposed model can achieve 85% recognition rate for the pedestrian crossing intention. Moreover, the vehicle controlled by the proposed motion planning model and the actual human-driven vehicle are highly similar with respect to the gap acceptance in intersections.

The recent growth of IoT use-cases in a wide array of industrial, utility and environmental applications has necessitated the need for connectivity solutions with diverse requirements. Connectivity through BLE, Zigbee and 6LoPAN are examples of short-range IoT deployments. But to provide connectivity to a high density of devices over larger coverage areas, Low-Power Wide-Area Network (LPWAN) technologies in both licensed as well as unlicensed bands have been considered. In this paper, we consider modelling the traffic from IoT devices connected through LPWAN technologies. Due to diverse applications of IoT, it is not trivial to have a single traffic model to represent all of them, but the traffic can be broadly classified as either periodic, event-triggered, or a combination of both. We evaluate the performance of LoRaWAN, one such LPWAN technology, in the presence of a hybrid of both traffic types, where the event propagates spatially over time. In a practical deployment of sensor based IoT devices, the devices are usually densely deployed to ensure sufficient & reliable measurement. Thereby, when an event occurs, they exhibit spatial & temporal correlation in their traffic rate due to the natural phenomena of the metric they measure. We use the CMMPP model to represent such characteristic traffic from independent IoT devices triggered by an event. The characteristics of LoRa, the physical layer of LoRaWAN, is abstracted based on required signal strength and interference thresholds for different modulation parameters. Through system simulations, we demonstrate that there is a significant performance hit in LoRaWAN based networks, during the occurrence of events. In particular, using the packet delivery rate (PDR) as the metric, we found that while the system was able to handle regular updates from the devices with a PDR > 80%, event-driven traffic nearly impaired the network causing the PDR to drop below 10%.

We present a beacon-based clustering algorithm aimed at prolonging the cluster lifetime in VANETs. We use a new aggregate local mobility criterion to decide upon cluster re-organisation. The scheme incorporates a contention method to avoid triggering frequent re-organisations when two clusterheads encounter each other for a short period of time. Simulation results show a significant improvement of cluster lifetime and reduced node state/role changes compared to previous popular clustering algorithms.

In this letter, we propose the cooperative relaying system using non-orthogonal multiple access (NOMA) to improve the spectral efficiency. The achievable average rate of the proposed system is analyzed for independent Rayleigh fading channels, and also its asymptotic expression is provided. In addition, a suboptimal power allocation scheme for NOMA used at the source is proposed.


Question: DeepSense: A comprehensive deep learning framework for processing time-series data from mobile sensors.

Exploring the Long Short-Term Memory (LSTM) Universe: An In-depth Analysis

Since its introduction in 1995, the Long Short-Term Memory (LSTM) architecture for recurrent neural networks has evolved through numerous modifications, becoming a leading model for a multitude of machine learning challenges. This resurgence has sparked a fresh wave of curiosity regarding the functionality and effectiveness of various components within LSTM variants. In this comprehensive study, we conduct an extensive analysis of eight distinct LSTM configurations across three pivotal tasks: speech recognition, handwriting recognition, and polyphonic music modeling. To ensure fairness and thoroughness, we independently optimized the hyperparameters for each LSTM variant and task using a random search approach. The significance of these hyperparameters was then evaluated through the sophisticated Functional Analysis of Variance (FANOVA) framework. Our analysis encompasses 5400 experimental runs, amounting to approximately 15 years of CPU time, marking this as the most expansive study on LSTM networks to date. The findings reveal that none of the LSTM variants outperform the conventional LSTM architecture substantially, pinpointing the forget gate and the output activation function as its most indispensable elements. Additionally, we observe that the hyperparameters are largely independent of each other, facilitating the formulation of practical guidelines for their effective calibration.

We present a comparative evaluation of a large number of anomaly detection techniques on a variety of publicly available as well as artificially generated data sets. Many of these are existing techniques while some are slight variants and/or adaptations of traditional anomaly detection techniques to sequence data.

This thesis investigates the use of wearable sensors to recognize human activity. The activity of the user is one example of context information – others include the user’s location or the state of his environment – which can help computer applications to adapt to the user depending on the situation. In this thesis we use wearable sensors – mainly accelerometers – to record, model and recognize human activities. Using wearable sensors allows continuous recording of activities across different locations and independent from external infrastructure. There are many possible applications for activity recognition with wearable sensors, for instance in the areas of healthcare, elderly care, personal fitness, entertainment, or performing arts. In this thesis we focus on two particular research challenges in activity recognition, namely the need for less supervision, and the recognition of high-level activities. We make several contributions towards addressing these challenges. Our first contribution is an analysis of features for activity recognition. Using a data set of activities such as walking, standing, sitting, or hopping, we analyze the performance of commonly used features and window lengths over which the features are computed. Our results indicate that different features perform well for different activities, and that in order to achieve best recognition performance, features and window lengths should be chosen specific for each activity. In order to reduce the need for labeled training data, we propose an unsupervised algorithm which can discover structure in unlabeled recordings of activities. The approach identifies correlated subsets in feature space, and represents these subsets with low-dimensional models. We show that the discovered subsets often correspond to distinct activities, and that the resulting models can be used for recognition of activities in unknown data. In a separate study, we show that the approach can be effectively deployed in a semi-supervised learning framework. More specifically, we combine the approach with a discriminant classifier, and show that this scheme allows high recognition rates even when using only a small amount of labeled training data. Recognition of higher-level activities such as shopping, doing housework, or commuting is challenging, as these activities are composed of changing sub-activities and vary strongly across individuals. We present one study in which we recorded 10h of three different high-level activities, investigating to which extent methods for low-level activities can be scaled to the recognition of high-level activities. Our results indicate that for settings as ours, traditional supervised approaches in combination with data from wearable accelerometers can achieve recognition rates of more than 90%. While unsupervised techniques are desirable for short-term activities, they become crucial for long-term activities, for which annotation is often impractical or impossible. To this end we propose an unsupervised approach based on topic models that allows to discover high-level structure in human activity data. The discovered activity patterns correlate with daily routines such as commuting, office work, or lunch routine, and they can be used to recognize such routines in unknown data.

Sensor networks have the potential to greatly impact many aspects of medical care. By outfitting patients with wireless, wearable vital sign sensors, collecting detailed real-time data on physiological status can be greatly simplified. However, there is a significant gap between existing sensor network systems and the needs of medical care. In particular, medical sensor networks must support multicast routing topologies, node mobility, a wide range of data rates and high degrees of reliability, and security. This paper describes our experiences with developing a combined hardware and software platform for medical sensor networks, called CodeBlue. CodeBlue provides protocols for device discovery and publish/subscribe multihop routing, as well as a simple query interface that is tailored for medical monitoring. We have developed several medical sensors based on the popular MicaZ and Telos mote designs, including a pulse oximeter, EKG and motion-activity sensor. We also describe a new, miniaturized sensor mote designed for medical use. We present initial results for the CodeBlue prototype demonstrating the integration of our medical sensors with the publish/subscribe routing substrate. We have experimentally validated the prototype on our 30-node sensor network testbed, demonstrating its scalability and robustness as the number of simultaneous queries, data rates, and transmitting sensors are varied. We also study the effect of node mobility, fairness across multiple simultaneous paths, and patterns of packet loss, confirming the system’s ability to maintain stable routes despite variations in node location and

Deep Learning approaches have gathered a lot of attention lately. In this work, we study their application to the breast cancer field, in particular for mass detection in mammograms. Several experiments were made on a real mammogram benchmark dataset. Deep Learning approaches were compared to other classification methodologies. It was concluded that, although useful, the implementation used does not outperform SVMs. Further study and adjustment of the method for this application is needed.

We present a real-time gesture classification system for skeletal wireframe motion. Its key components include an angular representation of the skeleton designed for recognition robustness under noisy input, a cascaded correlation-based classifier for multivariate time-series data, and a distance metric based on dynamic time-warping to evaluate the difference in motion between an acquired gesture and an oracle for the matching gesture. While the first and last tools are generic in nature and could be applied to any gesture-matching scenario, the classifier is conceived based on the assumption that the input motion adheres to a known, canonical time-base: a musical beat. On a benchmark comprising 28 gesture classes, hundreds of gesture instances recorded using the XBOX Kinect platform and performed by dozens of subjects for each gesture class, our classifier has an average accuracy of 96:9%, for approximately 4-second skeletal motion recordings. This accuracy is remarkable given the input noise from the real-time depth sensor.

Environmental sound detection is a challenging application of machine learning because of the noisy nature of the signal, and the small amount of (labeled) data that is typically available. This work thus presents a comparison of several state-of-the-art Deep Learning models on the IEEE challenge on Detection and Classification of Acoustic Scenes and Events (DCASE) 2016 challenge task and data, classifying sounds into one of fifteen common indoor and outdoor acoustic scenes, such as bus, cafe, car, city center, forest path, library, train, etc. In total, 13 hours of stereo audio recordings are available, making this one of the largest datasets available.

The dynamic mobility and limitations in computational power, battery resources, and memory availability are main bottlenecks in fully harnessing mobile devices as data mining platforms. Therefore, the mobile devices are augmented with cloud resources in mobile edge cloud computing (MECC) environments to seamlessly execute data mining tasks. The MECC infrastructures provide compute, network, and storage services in one-hop wireless distance from mobile devices to minimize the latency in communication as well as provide localized computations to reduce the burden on federated cloud systems. However, when and how to offload the computation is a hard problem. In this paper, we present an opportunistic computation offloading scheme to efficiently execute data mining tasks in MECC environments. The scheme provides the suitable execution mode after analyzing the amount of unprocessed data, privacy configurations, contextual information, and available on-board local resources (memory, CPU, and battery power). We develop a mobile application for online activity recognition and evaluate the proposed scheme using the event data stream of 5 million activities collected from 12 users for 15 days. The experiments show significant improvement in execution time and battery power consumption resulting in 98% data reduction.

In this paper, we present a real-time algorithm for automatic recognition of not only physical activities, but also, in some cases, their intensities, using five triaxial wireless accelerometers and a wireless heart rate monitor. The algorithm has been evaluated using datasets consisting of 30 physical gymnasium activities collected from a total of 21 people at two different labs. On these activities, we have obtained a recognition accuracy performance of 94.6% using subject-dependent training and 56.3% using subject-independent training. The addition of heart rate data improves subject-dependent recognition accuracy only by 1.2% and subject-independent recognition only by 2.1%. When recognizing activity type without differentiating intensity levels, we obtain a subject-independent performance of 80.6%. We discuss why heart rate data has such little discriminatory power.

BACKGROUND
Mobile phone sensors can be used to develop context-aware systems that automatically detect when patients require assistance. Mobile phones can also provide ecological momentary interventions that deliver tailored assistance during problematic situations. However, such approaches have not yet been used to treat major depressive disorder.

OBJECTIVE
The purpose of this study was to investigate the technical feasibility, functional reliability, and patient satisfaction with Mobilyze!, a mobile phone- and Internet-based intervention including ecological momentary intervention and context sensing.

METHODS
We developed a mobile phone application and supporting architecture, in which machine learning models (ie, learners) predicted patients' mood, emotions, cognitive/motivational states, activities, environmental context, and social context based on at least 38 concurrent phone sensor values (eg, global positioning system, ambient light, recent calls). The website included feedback graphs illustrating correlations between patients' self-reported states, as well as didactics and tools teaching patients behavioral activation concepts. Brief telephone calls and emails with a clinician were used to promote adherence. We enrolled 8 adults with major depressive disorder in a single-arm pilot study to receive Mobilyze! and complete clinical assessments for 8 weeks.

RESULTS
Promising accuracy rates (60% to 91%) were achieved by learners predicting categorical contextual states (eg, location). For states rated on scales (eg, mood), predictive capability was poor. Participants were satisfied with the phone application and improved significantly on self-reported depressive symptoms (beta(week) = -.82, P < .001, per-protocol Cohen d = 3.43) and interview measures of depressive symptoms (beta(week) = -.81, P < .001, per-protocol Cohen d = 3.55). Participants also became less likely to meet criteria for major depressive disorder diagnosis (b(week) = -.65, P = .03, per-protocol remission rate = 85.71%). Comorbid anxiety symptoms also decreased (beta(week) = -.71, P < .001, per-protocol Cohen d = 2.58).

CONCLUSIONS
Mobilyze! is a scalable, feasible intervention with preliminary evidence of efficacy. To our knowledge, it is the first ecological momentary intervention for unipolar depression, as well as one of the first attempts to use context sensing to identify mental health-related states. Several lessons learned regarding technical functionality, data mining, and software development process are discussed.

TRIAL REGISTRATION
Clinicaltrials.gov NCT01107041; http://clinicaltrials.gov/ct2/show/NCT01107041 (Archived by WebCite at http://www.webcitation.org/60CVjPH0n).

We propose an algorithm which combines the discriminative information from depth images as well as from 3D joint positions to achieve high action recognition accuracy. To avoid the suppression of subtle discriminative information and also to handle local occlusions, we compute a vector of many independent local features. Each feature encodes spatiotemporal variations of depth and depth gradients at a specific space-time location in the action volume. Moreover, we encode the dominant skeleton movements by computing a local 3D joint position difference histogram. For each joint, we compute a 3D space-time motion volume which we use as an importance indicator and incorporate in the feature vector for improved action discrimination. To retain only the discriminant features, we train a random decision forest (RDF). The proposed algorithm is evaluated on three standard datasets and compared with nine state-of-the-art algorithms. Experimental results show that, on the average, the proposed algorithm outperform all other algorithms in accuracy and have a processing speed of over 112 frames/second.


Question: DeepSense: A comprehensive deep learning architecture for handling time-series mobile sensing data.

This paper delves into the emerging field of Personal Sensing, a methodology that harnesses the power of ubiquitous sensors in everyday devices like smartphones, wearables, and computers to track and analyze human behaviors, emotions, and mental states. By focusing on the intricate relationship between sensor data and mental health, the research provides a comprehensive overview of how these digital traces can be utilized to understand and monitor psychological well-being. The review centers on the application of personal sensing in smartphones but also encompasses studies involving wearables, social media, and computer usage. A layered, hierarchical framework is proposed for converting raw sensor data into meaningful indicators of mental health behaviors and states. The discussion also addresses methodological approaches and the challenges faced in personal sensing research, such as privacy concerns and data dimensionality issues. Despite being a nascent field, personal sensing shows significant potential as a research tool for mental health studies and as a clinical aid for monitoring at-risk individuals, paving the way for advanced mobile health (mHealth) interventions.

In this paper a novel evaluation framework for measuring the performance of real-time action recognition methods is presented. The evaluation framework will extend the time-based event detection metric to model multiple distinct action classes. The proposed metric provides more accurate indications of the performance of action recognition algorithms for games and other similar applications since it takes into consideration restrictions related to time and consecutive repetitions. Furthermore, a new dataset, G3D for real-time action recognition in gaming containing synchronised video, depth and skeleton data is provided. Our results indicate the need of an advanced metric especially designed for games and other similar real-time applications.

Deep generative models can perform dramatically better than traditional graphical models in a number of machine learning tasks. However, training such models remains challenging because their latent variables typically do not have an analytical posterior distribution, largely due to the nonlinear activation nodes. We present a deep convolutional factor analyser (DCFA) for multivariate time series modeling. Our network is constructed in a way that bottom layer nodes are independent. Through a process of up-sampling and convolution, higher layer nodes gain more temporal dependency. Our model can thus give a time series different representations at different depths. DCFA only consists of linear Gaussian nodes. Therefore, the posterior distributions of latent variables are also Gaussian and can be estimated easily using standard variational Bayes algorithm. We show that even without nonlinearity the proposed deep model can achieve state-of-the-art results in anomaly detection, classification and clustering using both synthetic and real-world datasets.

Implementing E cient, Portable Computations for Machine Learning

Mobile crowdsensing serves as a critical building block for emerging Internet of Things (IoT) applications. However, the sensing devices continuously generate a large amount of data, which consumes much resources (e.g., bandwidth, energy, and storage) and may sacrifice the Quality-of-Service (QoS) of applications. Prior work has demonstrated that there is significant redundancy in the content of the sensed data. By judiciously reducing redundant data, data size and load can be significantly reduced, thereby reducing resource cost and facilitating the timely delivery of unique, probably critical information and enhancing QoS. This article presents a survey of existing works on mobile crowdsensing strategies with an emphasis on reducing resource cost and achieving high QoS. We start by introducing the motivation for this survey and present the necessary background of crowdsensing and IoT. We then present various mobile crowdsensing strategies and discuss their strengths and limitations. Finally, we discuss future research directions for mobile crowdsensing for IoT. The survey addresses a broad range of techniques, methods, models, systems, and applications related to mobile crowdsensing and IoT. Our goal is not only to analyze and compare the strategies proposed in prior works, but also to discuss their applicability toward the IoT and provide guidance on future research directions for mobile crowdsensing.

Activity recognition is an important subject in different areas of research, like e-health and ubiquitous computing. In this paper we give an overview of recent work in the field of activity recognition from both body mounted sensors and ambient sensors. We discuss some of the differences among approaches and present guidelines for choosing a suitable approach under various circumstances.

This work deals with the task of human daily activity recognition using miniature inertial sensors. The proposed method reduces sensitivity to the position and orientation of the sensor on the body, which is inherent in traditional methods, by transforming the observed signals to a “virtual” sensor orientation. By means of this computationally low-cost transform, the inputs to the classification algorithm are made invariant to sensor orientation, despite the signals being recorded from arbitrary sensor placements. Classification results show that improved performance, in terms of both precision and recall, is achieved with the transformed signals, relative to classification using raw sensor signals, and the algorithm performs competitively compared to the state-of-the-art. Activity recognition using data from a sensor with completely unknown orientation is shown to perform very well over a long term recording in a real-life setting.

Human action recognition has attracted much attentions because of its great potential applications. With the rapid development of computer performance and Internet, the methods of human action recognition based on deep learning become mainstream and develop at a breathless pace. This paper will give a novel reasonable taxonomy and a review of deep learning human action recognition methods based on color videos, skeleton sequences and depth maps. In addition, some datasets and effective tricks in action recognition deep learning methods will be introduced, and also the development trend is discussed.

We present a scalable system for high-throughput real-time analysis of heterogeneous data streams. Our architecture enables incremental development of models for predictive analytics and anomaly detection as data arrives into the system. In contrast with batch data-processing systems, such as Hadoop, that can have high latency, our architecture allows for ingest and analysis of data on the fly, thereby detecting and responding to anomalous behavior in near real time. This timeliness is important for applications such as insider threat, financial fraud, and network intrusions. We demonstrate an application of this system to the problem of detecting insider threats, namely, the misuse of an organization's resources by users of the system and present results of our experiments on a publicly available insider threat dataset.

Smartphones are increasingly utilized in society and enable scientists to record a wide range of behavioral and environmental information. These information, referred to as Unobtrusive Ecological Momentary Assessment Data, might support prediction procedures regarding the mood level of users and simultaneously contribute to an enhancement of therapy strategies. In this paper, we analyze how the mood level of healthy clients is affected by unobtrusive measures and how this kind of data contributes to the prediction performance of various statistical models (Bayesian methods, Lasso procedures, etc.). We conduct analyses on a non-user and a user level. We then compare the models by utilizing introduced performance measures. Our findings indicate that the prediction performance increases when considering individual users. However, the implemented models only perform slightly better than the introduced mean model. Indicated by feature selection methods, we assume that more meaningful variables regarding the outcome can potentially increase prediction performance.

Smartwatches provide quick and easy access to information. Due to their wearable nature, users can perceive the information while being stationary or on the go. The main drawback of smartwatches, however, is the limited input possibility. They use similar input methods as smartphones but thereby suffer from a smaller form factor. To extend the input space of smartwatches, we present GestureSleeve, a sleeve made out of touch enabled textile. It is capable of detecting different gestures such as stroke based gestures or taps. With these gestures, the user can control various smartwatch applications. Exploring the performance of the GestureSleeve approach, we conducted a user study with a running application as use case. In this study, we show that input using the GestureSleeve outperforms touch input on the smartwatch. In the future the GestureSleeve can be integrated into regular clothing and be used for controlling various smart devices.


Question: DeepSense: A Comprehensive Deep Learning Architecture for Time-Series Analysis of Mobile Sensing Data

This paper introduces PowerBooter, a novel automated technique for constructing power models on smartphones that leverages built-in battery voltage sensors and an understanding of battery discharge characteristics. PowerBooter monitors power consumption while actively controlling the power management and activity states of individual device components, all without the need for external measurement tools. Additionally, the paper details PowerTutor, a tool that utilizes the power model generated by PowerBooter to estimate power consumption in real-time. PowerBooter aims to streamline the process of power model generation for new smartphone models, each with unique power consumption profiles that necessitate tailored power models. PowerTutor, on the other hand, seeks to simplify the design and selection of power-efficient software for embedded systems. Together, PowerBooter and PowerTutor aspire to broaden access to power modeling and analysis for a wider range of smartphone models and their users, facilitating the development and deployment of energy-efficient applications.

As we are moving towards the Internet of Things (IoT), the number of sensors deployed around the world is growing at a rapid pace. Market research has shown a significant growth of sensor deployments over the past decade and has predicted a significant increment of the growth rate in the future. These sensors continuously generate enormous amounts of data. However, in order to add value to raw sensor data we need to understand it. Collection, modelling, reasoning, and distribution of context in relation to sensor data plays critical role in this challenge. Context-aware computing has proven to be successful in understanding sensor data. In this paper, we survey context awareness from an IoT perspective. We present the necessary background by introducing the IoT paradigm and context-aware fundamentals at the beginning. Then we provide an in-depth analysis of context life cycle. We evaluate a subset of projects (50) which represent the majority of research and commercial solutions proposed in the field of context-aware computing conducted over the last decade (2001-2011) based on our own taxonomy. Finally, based on our evaluation, we highlight the lessons to be learnt from the past and some possible directions for future research. The survey addresses a broad range of techniques, methods, models, functionalities, systems, applications, and middleware solutions related to context awareness and IoT. Our goal is not only to analyse, compare and consolidate past research work but also to appreciate their findings and discuss their applicability towards the IoT.

This study introduces OpenHAR, a free Matlab toolbox to combine and unify publicly open data sets. It provides an easy access to accelerometer signals of ten publicly open human activity data sets. Data sets are easy to access as OpenHAR provides all the data sets in the same format. In addition, units, measurement range and labels are unified, as well as, body position IDs. Moreover, data sets with different sampling rates are unified using downsampling. What is more, data sets have been visually inspected to find visible errors, such as sensor in wrong orientation. OpenHAR improves re-usability of data sets by fixing these errors. Altogether OpenHAR contains over 65 million labeled data samples. This is equivalent to over 280 hours of data from 3D accelerometers. This includes data from 211 study subjects performing 17 daily human activities and wearing sensors in 14 different body positions.

Recently, Deep Neural Networks (DNNs) have emerged as the dominant model across various AI applications. In the era of IoT and mobile systems, the efficient deployment of DNNs on embedded platforms is vital to enable the development of intelligent applications. This paper summarises our recent work on the optimised mapping of DNNs on embedded settings. By covering such diverse topics as DNN-to-accelerator toolflows, high-throughput cascaded classifiers and domain-specific model design, the presented set of works aim to enable the deployment of sophisticated deep learning models on cutting-edge mobile and embedded systems.

We propose DeepHand to estimate the 3D pose of a hand using depth data from commercial 3D sensors. We discriminatively train convolutional neural networks to output a low dimensional activation feature given a depth map. This activation feature vector is representative of the global or local joint angle parameters of a hand pose. We efficiently identify 'spatial' nearest neighbors to the activation feature, from a database of features corresponding to synthetic depth maps, and store some 'temporal' neighbors from previous frames. Our matrix completion algorithm uses these 'spatio-temporal' activation features and the corresponding known pose parameter values to estimate the unknown pose parameters of the input feature vector. Our database of activation features supplements large viewpoint coverage and our hierarchical estimation of pose parameters is robust to occlusions. We show that our approach compares favorably to state-of-the-art methods while achieving real time performance (≈ 32 FPS) on a standard computer.

Sleep plays a vital role in human health, both mental and physical. Sleep disorders like sleep apnea are increasing in prevalence, with the rapid increase in factors like obesity. Sleep apnea is most commonly treated with Continuous Positive Air Pressure (CPAP) therapy, which maintains the appropriate pressure to ensure continuous airflow. It is widely accepted that in addition to preventing air passage collapse, increase in deep and REM sleep stages would be good metrics for how well the CPAP therapy is working in improving sleep health. Presently, however, there is no mechanism to easily detect a patient’s sleep stages from CPAP flow data alone. We propose, for the first time, an automated sleep staging model based only on the flow signal. Recently deep neural networks have shown high accuracy on sleep staging by eliminating handcrafted features. However, these methods focus exclusively on extracting informative features from the input signal, without paying much attention to the dynamics of sleep stages in the output sequence. We propose an end-to-end framework that uses a combination of deep convolution and recurrent neural networks to extract high-level features from raw flow signal and then uses a structured output layer based on a conditional random field to model the temporal transition structure of the sleep stages. We improve upon the previous methods by 10% using our model, that can be augmented to the previous sleep staging deep learning methods. We also show that our method can be used to accurately track sleep metrics like sleep efficiency calculated from sleep stages that can be deployed for monitoring the response of CPAP therapy on sleep apnea patients. Apart from the technical contributions, we expect this study to motivate new research questions in sleep science, especially towards the understanding of sleep architecture trajectory among patients under CPAP therapy.

Analysis and recognition of motion patterns from data acquired by body-worn inertial sensors is an emerging technology in sports. In this paper we propose an effective method for recognition of fencing footwork using a single body-worn accelerometer. We present a challenging dataset consisting of six actions, which were performed by ten persons and repeated ten times by each of them. We propose a segment-based SVM for time-series classification together with a set of informative features. We demonstrate that the method is competitive with 1-NN DTW in terms of classification accuracy. The proposed method achieves classification accuracy slightly better than 70% on the fencing footwork dataset.

Deep learning has dramatically improved the performance of sounds recognition. However, learning acoustic models directly from the raw waveform is still challenging. Current waveform-based models generally use time-domain convolutional layers to extract features. The features extracted by single size filters are insufficient for building discriminative representation of audios. In this paper, we propose multi-scale convolution operation, which can get better audio representation by improving the frequency resolution and learning filters cross all frequency area. For leveraging the waveform-based features and spectrogram-based features in a single model, we introduce twophase method to fuse the different features. Finally, we propose a novel end-to-end network called WaveMsNet based on the multi-scale convolution operation and two-phase method. On the environmental sounds classification datasets ESC-10 and ESC-50, the classification accuracies of our WaveMsNet achieve 93.75% and 79.10% respectively, which improve significantly from the previous methods.

The evolution of smartphones together with increasing computational power has empowered developers to create innovative context-aware applications for recognizing user-related social and cognitive activities in any situation and at any location. The existence and awareness of the context provide the capability of being conscious of physical environments or situations around mobile device users. This allows network services to respond proactively and intelligently based on such awareness. The key idea behind context-aware applications is to encourage users to collect, analyze, and share local sensory knowledge in the purpose for a large-scale community use by creating a smart network. The desired network is capable of making autonomous logical decisions to actuate environmental objects and also assist individuals. However, many open challenges remain, which are mostly arisen because the middleware services provided in mobile devices have limited resources in terms of power, memory, and bandwidth. Thus, it becomes critically important to study how the drawbacks can be elaborated and resolved and, at the same time, better understand the opportunities for the research community to contribute to the context-awareness. To this end, this paper surveys the literature over the period of 1991-2014 from the emerging concepts to applications of context-awareness in mobile platforms by providing up-to-date research and future research directions. Moreover, it points out the challenges faced in this regard and enlightens them by proposing possible solutions.

Nowadays, there is an ever-increasing migration of people to urban areas. Health care service is one of the most challenging aspects that is greatly affected by the vast influx of people to city centers. Consequently, cities around the world are investing heavily in digital transformation in an effort to provide healthier ecosystems for people. In such a transformation, millions of homes are being equipped with smart devices (e.g., smart meters, sensors, and so on), which generate massive volumes of fine-grained and indexical data that can be analyzed to support smart city services. In this paper, we propose a model that utilizes smart home big data as a means of learning and discovering human activity patterns for health care applications. We propose the use of frequent pattern mining, cluster analysis, and prediction to measure and analyze energy usage changes sparked by occupants’ behavior. Since people’s habits are mostly identified by everyday routines, discovering these routines allows us to recognize anomalous activities that may indicate people’s difficulties in taking care for themselves, such as not preparing food or not using a shower/bath. This paper addresses the need to analyze temporal energy consumption patterns at the appliance level, which is directly related to human activities. For the evaluation of the proposed mechanism, this paper uses the U.K. Domestic Appliance Level Electricity data set—time series data of power consumption collected from 2012 to 2015 with the time resolution of 6 s for five houses with 109 appliances from Southern England. The data from smart meters are recursively mined in the quantum/data slice of 24 h, and the results are maintained across successive mining exercises. The results of identifying human activity patterns from appliance usage are presented in detail in this paper along with the accuracy of short- and long-term predictions.

Multivariate time series forecasting is extensively studied throughout the years with ubiquitous applications in areas such as finance, traffic, environment, etc. Still, concerns have been raised on traditional methods for incapable of modeling complex patterns or dependencies lying in real word data. To address such concerns, various deep learning models, mainly Recurrent Neural Network (RNN) based methods, are proposed. Nevertheless, capturing extremely long-term patterns while effectively incorporating information from other variables remains a challenge for time-series forecasting. Furthermore, lack-of-explainability remains one serious drawback for deep neural network models. Inspired by Memory Network proposed for solving the question-answering task, we propose a deep learning based model named Memory Timeseries network (MTNet) for time series forecasting. MTNet consists of a large memory component, three separate encoders, and an autoregressive component to train jointly. Addtionally, the attention mechanism designed enable MTNet to be highly interpretable. We can easily tell which part of the historic data is referenced the most.


Question: DeepSense: A Comprehensive Deep Learning Framework for Time-Series Mobile Sensing Data Analysis

This paper explores the use of multiple GPS receivers as a redundant system to ensure the accurate tracking of a drone's 3D orientation, particularly when inertial measurement units (IMUs) fail. IMUs are crucial for drone control, but they are susceptible to correlated errors. To counteract these issues, we propose utilizing multiple GPS receivers on the drone for added reliability. The primary hurdle is to precisely calculate the relative positions between each GPS receiver pair, and to translate these into the drone's orientation. Achieving orientation accuracy akin to that of an IMU necessitates GPS distances to be accurate within a few centimeters, which is a significant challenge given the typical GPS accuracy of 1-4 meters. Furthermore, GPS-based orientation must remain stable under extreme drone maneuvers, signal obstructions, and intermittent data loss. Our solution, SafetyNet, is a system designed to overcome these obstacles using a combination of techniques, including a novel particle filter framework that operates over multiple global navigation satellite systems (GNSS) such as GPS, GLONASS, and SBAS. The system's effectiveness is demonstrated through 11 flight sessions lasting 5-7 minutes each, under various weather conditions, achieving a median orientation accuracy of 2°. While the addition of multiple GPS receivers increases cost, we argue that this trade-off is justifiable when safety is a paramount concern.

Driver behavior affects traffic safety, fuel/energy consumption and gas emissions. The purpose of driver behavior profiling is to understand and have a positive influence on driver behavior. Driver behavior profiling tasks usually involve an automated collection of driving data and the application of computer models to classify what characterizes the aggressiveness of drivers. Different sensors and classification methods have been employed for this task, although low-cost solutions, high performance and collaborative sensing remain open questions for research. This paper makes an investigation with different Recurrent Neural Networks (RNN), aiming to classify driving events employing data collected by smartphone accelerometers. The results show that specific configurations of RNN upon accelerometer data provide high accuracy results, being a step towards the development of safer transportation systems.

Human action recognition has attracted much attentions because of its great potential applications. With the rapid development of computer performance and Internet, the methods of human action recognition based on deep learning become mainstream and develop at a breathless pace. This paper will give a novel reasonable taxonomy and a review of deep learning human action recognition methods based on color videos, skeleton sequences and depth maps. In addition, some datasets and effective tricks in action recognition deep learning methods will be introduced, and also the development trend is discussed.

Compressive sensing (CS) is a promising technology for realizing low-power and cost-effective wireless sensor nodes (WSNs) in pervasive health systems for 24/7 health monitoring. Due to the high computational complexity (CC) of the reconstruction algorithms, software solutions cannot fulfill the energy efficiency needs for real-time processing. In this paper, we present a 12-237 kS/s 12.8 mW sparse-approximation (SA) engine chip that enables the energy-efficient data aggregation of compressively sampled physiological signals on mobile platforms. The SA engine chip integrated in 40 nm CMOS can support the simultaneous reconstruction of over 200 channels of physiological signals while consuming (1% of a smartphone's power budget. Such energyefficient reconstruction enables two-to-three times energy saving at the sensor nodes in a CS-based health monitoring system as compared to traditional Nyquist-based systems, while providing timely feedback and bringing signal intelligence closer to the user.

With the proliferation of Internet of Things (IoT) devices such as smartphones, sensors, cameras, and RFIDs, it is possible to collect massive amount of data for localization and tracking of people within commercial buildings. Enabled by such occupancy monitoring capabilities, there are extensive opportunities for improving the energy consumption of buildings via smart HVAC control. In this respect, the major challenges we envision are 1) to achieve occupancy monitoring in a minimally intrusive way, e.g., using the existing infrastructure in the buildings and not requiring installation of any apps in the users' smart devices, and 2) to develop effective data fusion techniques for improving occupancy monitoring accuracy using a multitude of sources. This paper surveys the existing works on occupancy monitoring and multi-modal data fusion techniques for smart commercial buildings. The goal is to lay down a framework for future research to exploit the spatio-temporal data obtained from one or more of various IoT devices such as temperature sensors, surveillance cameras, and RFID tags that may be already in use in the buildings. A comparative analysis of existing approaches and future predictions for research challenges are also provided.

We address the problem of structure learning of human motion in order to recognize actions from a continuous monocular motion sequence of an arbitrary person from an arbitrary viewpoint. Human motion sequences are represented by multivariate time series in the joint-trajectories space. Under this structured time series framework, we first propose Kernelized Temporal Cut (KTC), an extension of previous works on change-point detection by incorporating Hilbert space embedding of distributions, to handle the nonparametric and high dimensionality issues of human motions. Experimental results demonstrate the effectiveness of our approach, which yields realtime segmentation, and produces high action segmentation accuracy. Second, a spatio-temporal manifold framework is proposed to model the latent structure of time series data. Then an efficient spatio-temporal alignment algorithm Dynamic Manifold Warping (DMW) is proposed for multivariate time series to calculate motion similarity between action sequences (segments). Furthermore, by combining the temporal segmentation algorithm and the alignment algorithm, online human action recognition can be performed by associating a few labeled examples from motion capture data. The results on human motion capture data and 3D depth sensor data demonstrate the effectiveness of the proposed approach in automatically segmenting and recognizing motion sequences, and its ability to handle noisy and partially occluded data, in the transfer learning module.

Implementing E cient, Portable Computations for Machine Learning

This study introduces OpenHAR, a free Matlab toolbox to combine and unify publicly open data sets. It provides an easy access to accelerometer signals of ten publicly open human activity data sets. Data sets are easy to access as OpenHAR provides all the data sets in the same format. In addition, units, measurement range and labels are unified, as well as, body position IDs. Moreover, data sets with different sampling rates are unified using downsampling. What is more, data sets have been visually inspected to find visible errors, such as sensor in wrong orientation. OpenHAR improves re-usability of data sets by fixing these errors. Altogether OpenHAR contains over 65 million labeled data samples. This is equivalent to over 280 hours of data from 3D accelerometers. This includes data from 211 study subjects performing 17 daily human activities and wearing sensors in 14 different body positions.

Multi-character recognition in arbitrary photographs on mobile platform is difficult, in terms of both accuracy and real-time performance. In this paper, we focus on the domain of hand-written multi-digit recognition. Convolutional neural network (CNN) is the state-of-the-art solution for object recognition, and presents a workload that is both compute and data intensive. To reduce the workload, we train a shallow CNN offline, achieving 99.07% top-1 accuracy. And we utilize preprocessing and segmentation to reduce input image size fed into CNN. For CNN implementation on the mobile platform, we adopt and modify DeepBeliefSDK to support batching fully-connected layers. On NVIDIA SHIELD tablet, the application processes a frame and extracts 32 digits in approximately 60ms, and batching the fully-connected layers reduces the CNN runtime by another 12%.

The dynamic mobility and limitations in computational power, battery resources, and memory availability are main bottlenecks in fully harnessing mobile devices as data mining platforms. Therefore, the mobile devices are augmented with cloud resources in mobile edge cloud computing (MECC) environments to seamlessly execute data mining tasks. The MECC infrastructures provide compute, network, and storage services in one-hop wireless distance from mobile devices to minimize the latency in communication as well as provide localized computations to reduce the burden on federated cloud systems. However, when and how to offload the computation is a hard problem. In this paper, we present an opportunistic computation offloading scheme to efficiently execute data mining tasks in MECC environments. The scheme provides the suitable execution mode after analyzing the amount of unprocessed data, privacy configurations, contextual information, and available on-board local resources (memory, CPU, and battery power). We develop a mobile application for online activity recognition and evaluate the proposed scheme using the event data stream of 5 million activities collected from 12 users for 15 days. The experiments show significant improvement in execution time and battery power consumption resulting in 98% data reduction.

In this paper, we present a real-time algorithm for automatic recognition of not only physical activities, but also, in some cases, their intensities, using five triaxial wireless accelerometers and a wireless heart rate monitor. The algorithm has been evaluated using datasets consisting of 30 physical gymnasium activities collected from a total of 21 people at two different labs. On these activities, we have obtained a recognition accuracy performance of 94.6% using subject-dependent training and 56.3% using subject-independent training. The addition of heart rate data improves subject-dependent recognition accuracy only by 1.2% and subject-independent recognition only by 2.1%. When recognizing activity type without differentiating intensity levels, we obtain a subject-independent performance of 80.6%. We discuss why heart rate data has such little discriminatory power.


Question: DeepSense: A cohesive deep learning framework for handling time-series data from mobile sensors.

This paper explores the application of unsupervised feature learning techniques to the challenge of identifying and reading digits in natural images, a complex and pivotal task in the field of computer vision. Despite significant advancements in character recognition for tasks such as reading handwritten digits, which are nearly solved for practical purposes, recognizing characters within more intricate scenes, such as photographs, remains a formidable challenge. Current state-of-the-art methods fall short of human-level performance in these scenarios. To address this issue, we present a novel benchmark dataset specifically designed for research purposes, containing over 600,000 labeled digits extracted from Street View images. This dataset serves as a testbed to highlight the limitations of traditional approaches that rely on manually crafted features for digit recognition. We then proceed to evaluate two contemporary unsupervised feature learning methodologies and illustrate their marked superiority in performance over conventional techniques on our benchmark. This study underscores the potential of unsupervised feature learning in enhancing the accuracy and efficiency of digit recognition from real-world images, particularly in the context of street-level photography.

The sharp increase in the number of smartphones on the market, with the Android platform posed to becoming a market leader makes the need for malware analysis on this platform an urgent issue.
 In this paper we capitalize on earlier approaches for dynamic analysis of application behavior as a means for detecting malware in the Android platform. The detector is embedded in a overall framework for collection of traces from an unlimited number of real users based on crowdsourcing. Our framework has been demonstrated by analyzing the data collected in the central server using two types of data sets: those from artificial malware created for test purposes, and those from real malware found in the wild. The method is shown to be an effective means of isolating the malware and alerting the users of a downloaded malware. This shows the potential for avoiding the spreading of a detected malware to a larger community.

Recent advances in small inexpensive sensors, low-power processing, and activity modeling have enabled applications that use on-body sensing and machine learning to infer people's activities throughout everyday life. To address the growing rate of sedentary lifestyles, we have developed a system, UbiFit Garden, which uses these technologies and a personal, mobile display to encourage physical activity. We conducted a 3-week field trial in which 12 participants used the system and report findings focusing on their experiences with the sensing and activity inference. We discuss key implications for systems that use on-body sensing and activity inference to encourage physical activity.

Build accurate DNN models requires training on large labeled, context specific datasets, especially those matching the target scenario. We believe advances in wireless localization, working in unison with cameras, can produce automated annotation of targets on images and videos captured in the wild. Using pedestrian and vehicle detection as examples, we demonstrate the feasibility, benefits, and challenges of an automatic image annotation system. Our work calls for new technical development on passive localization, mobile data analytics, and error-resilient ML models, as well as design issues in user privacy policies.

We introduce a new dataset, Human3.6M, of 3.6 Million accurate 3D Human poses, acquired by recording the performance of 5 female and 6 male subjects, under 4 different viewpoints, for training realistic human sensing systems and for evaluating the next generation of human pose estimation models and algorithms. Besides increasing the size of the datasets in the current state-of-the-art by several orders of magnitude, we also aim to complement such datasets with a diverse set of motions and poses encountered as part of typical human activities (taking photos, talking on the phone, posing, greeting, eating, etc.), with additional synchronized image, human motion capture, and time of flight (depth) data, and with accurate 3D body scans of all the subject actors involved. We also provide controlled mixed reality evaluation scenarios where 3D human models are animated using motion capture and inserted using correct 3D geometry, in complex real environments, viewed with moving cameras, and under occlusion. Finally, we provide a set of large-scale statistical models and detailed evaluation baselines for the dataset illustrating its diversity and the scope for improvement by future work in the research community. Our experiments show that our best large-scale model can leverage our full training set to obtain a 20% improvement in performance compared to a training set of the scale of the largest existing public dataset for this problem. Yet the potential for improvement by leveraging higher capacity, more complex models with our large dataset, is substantially vaster and should stimulate future research. The dataset together with code for the associated large-scale learning models, features, visualization tools, as well as the evaluation server, is available online at http://vision.imar.ro/human3.6m.

Ubisense has developed a platform for building Smart Space applications. The platform addresses the key requirements for building Smart Spaces: accurate 3D positioning, scalable realtime performance, development and deployment tools. This paper deepens the key requirements and describes how the Ubisense platform components meets them. The demonstration exemplifies the smart space platform by tracking players in a game application. The Ubisense Smart Space Platform Ubisense has developed a platform for building Smart Space applications. Our platform addresses the key requirements for building Smart Spaces: Accurate 3D positioning supports applications that can perceive the physical relationships between people and objects in the environment Scalable real-time performance enables arbitrary numbers of applications, used by arbitrary numbers of people over an arbitrarily-large area Development and deployment tools make it easy to design, implement, and manage Smart Space applications. The demonstration shows a Smart Space containing applications that integrate with external software (a simple game that users control by moving around), and devices (a PTZ camera that keeps users in shot while they are playing the game) Fig. 1 Smart space demonstration setup 1 Ubisense, St Andrews House, 90 St Andrews Road, Chesterton, CB4 1DL, United Kingdom, http://www.ubisense.net

Recent years have witnessed the wide proliferation of geo-sensory applications wherein a bundle of sensors are deployed at different locations to cooperatively monitor the target condition. Given massive geo-sensory data, we study the problem of mining spatial co-evolving patterns (SCPs), i.e., groups of sensors that are spatially correlated and co-evolve frequently in their readings. SCP mining is of great importance to various real-world applications, yet it is challenging because (1) the truly interesting evolutions are often flooded by numerous trivial fluctuations in the geo-sensory time series; and (2) the pattern search space is extremely large due to the spatiotemporal combinatorial nature of SCP. In this paper, we propose a two-stage method called Assember. In the first stage, Assember filters trivial fluctuations using wavelet transform and detects frequent evolutions for individual sensors via a segment-and-group approach. In the second stage, Assember generates SCPs by assembling the frequent evolutions of individual sensors. Leveraging the spatial constraint, it conceptually organizes all the SCPs into a novel structure called the SCP search tree, which facilitates the effective pruning of the search space to generate SCPs efficiently. Our experiments on both real and synthetic data sets show that Assember is effective, efficient, and scalable.

Deep learning can achieve outstanding results in various fields. However, it requires so significant computational power that graphics processing units (GPUs) and/or numerous computers are often required for the practical application. We have developed a new distributed calculation framework called ”Sashimi” that allows any computer to be used as a distribution node only by accessing a website. We have also developed a new JavaScript neural network framework called ”Sukiyaki” that uses general purpose GPUs with web browsers. Sukiyaki performs 30 times faster than a conventional JavaScript library for deep convolutional neural networks (deep CNNs) learning. The combination of Sashimi and Sukiyaki, as well as new distribution algorithms, demonstrates the distributed deep learning of deep CNNs only with web browsers on various devices. The libraries that comprise the proposed methods are available under MIT license athttp://mil-tokyo.github.io/.

The staggering growth in smartphone and wearable device use has led to a massive scale generation of personal (user-specific) data. To explore, analyze, and extract useful information and knowledge from the deluge of personal data, one has to leverage these devices as the data-mining platforms in ubiquitous, pervasive, and big data environments. This study presents the personal ecosystem where all computational resources, communication facilities, storage and knowledge management systems are available in user proximity. An extensive review on recent literature has been conducted and a detailed taxonomy is presented. The performance evaluation metrics and their empirical evidences are sorted out in this paper. Finally, we have highlighted some future research directions and potentially emerging application areas for personal data mining using smartphones and wearable devices.

While accurately predicting mood and wellbeing could have a number of important clinical benefits, traditional machine learning (ML) methods frequently yield low performance in this domain. We posit that this is because a one-size-fits-all machine learning model is inherently ill-suited to predicting outcomes like mood and stress, which vary greatly due to individual differences. Therefore, we employ Multitask Learning (MTL) techniques to train personalized ML models which are customized to the needs of each individual, but still leverage data from across the population. Three formulations of MTL are compared: i) MTL deep neural networks, which share several hidden layers but have final layers unique to each task; ii) Multi-task Multi-Kernel learning, which feeds information across tasks through kernel weights on feature types; and iii) a Hierarchical Bayesian model in which tasks share a common Dirichlet Process prior. We offer the code for this work in open source. These techniques are investigated in the context of predicting future mood, stress, and health using data collected from surveys, wearable sensors, smartphone logs, and the weather. Empirical results demonstrate that using MTL to account for individual differences provides large performance improvements over traditional machine learning methods and provides personalized, actionable insights.

Action recognition in videos remains a challenging problem in the machine learning community. Particularly challenging is the differing degree of intra-class variation between actions: While background information is enough to distinguish certain classes, many others are abstract and require fine-grained knowledge for discrimination. To approach this problem, in this work we evaluate different modalities on the recently published Moments in Time dataset, a collection of one million videos of short length.


Question: Dynamic multi-level, multi-task learning approach for sentence simplification.

This paper outlines an innovative approach for sentence simplification utilizing monolingual machine translation techniques, enhanced with a re-ranking mechanism based on dissimilarity, and trained using a monolingual parallel corpus. The system's effectiveness is evaluated against a word-substitution baseline and two leading-edge simplification systems, all of which are trained and tested on sentence pairs from the English sections of Wikipedia and Simple Wikipedia. To assess the output quality, human evaluators were engaged to compare the results from these different systems. The analysis of the human judgments reveals that, through meticulous phrase-based paraphrasing, our proposed model achieves comparable simplification outcomes to the state-of-the-art systems, while simultaneously producing more grammatically coherent output. Moreover, we caution against the indiscriminate use of text readability metrics like the Flesch-Kincaid grade level for evaluating the output of simplification systems, advocating for a more nuanced approach to assessing the readability and coherence of simplified sentences.

In this paper, we propose a machine-learning solution to problems consisting of many similar prediction tasks. Each of the individual tasks has a high risk of overrtting. We combine two types of knowledge transfer between tasks to reduce this risk: multi-task learning and hierarchical Bayesian modeling. Multi-task learning is based on the assumption that there exist features typical to the task at hand. To nd these features, we train a huge two-layered neural network. Each task has its own output, but shares the weights from the input to the hidden units with all other tasks. In this way a relatively large set of possible explanatory variables (the network inputs) is reduced to a smaller and easier to handle set of features (the hidden units). Given this set of features and after an appropriate scale transformation, we assume that the tasks are exchangeable. This assumption allows for a hierarchical Bayesian analysis in which the hyperparameters can be estimated from the data. EEectively, these hyperpa-rameters act as regularizers and prevent over-tting. We describe how to make the system robust against nonstationarities in the time series and give directions for further improvement. We illustrate our ideas on a database regarding the prediction of newspaper sales.

Machine reading calls for programs that read and understand text, but most current work only attempts to extract facts from redundant web-scale corpora. In this paper, we focus on a new reading comprehension task that requires complex reasoning over a single document. The input is a paragraph describing a biological process, and the goal is to answer questions that require an understanding of the relations between entities and events in the process. To answer the questions, we first predict a rich structure representing the process in the paragraph. Then, we map the question to a formal query, which is executed against the predicted structure. We demonstrate that answering questions via predicted structures substantially improves accuracy over baselines that use shallower representations.

With the huge amount of information available electronically, there is an increasing demand for automatic text summarization systems. The use of machine learning techniques for this task allows one to adapt summaries to the user needs and to the corpus characteristics. These desirable properties have motivated an increasing amount of work in this field over the last few years. Most approaches attempt to generate summaries by extracting sentence segments and adopt the supervised learning paradigm which requires to label documents at the text span level. This is a costly process, which puts strong limitations on the applicability of these methods. We investigate here the use of semi-supervised algorithms for summarization. These techniques make use of few labeled data together with a larger amount of unlabeled data. We propose new semi-supervised algorithms for training classification models for text summarization. We analyze their performances on two data sets - the Reuters news-wire corpus and the Computation and Language (cmp_lg) collection of TIPSTER SUMMAC. We perform comparisons with a baseline - non learning - system, and a reference trainable summarizer system.

Natural language generation of coherent long texts like paragraphs or longer documents is a challenging problem for recurrent networks models. In this paper, we explore an important step toward this generation task: training an LSTM (Longshort term memory) auto-encoder to preserve and reconstruct multi-sentence paragraphs. We introduce an LSTM model that hierarchically builds an embedding for a paragraph from embeddings for sentences and words, then decodes this embedding to reconstruct the original paragraph. We evaluate the reconstructed paragraph using standard metrics like ROUGE and Entity Grid, showing that neural models are able to encode texts in a way that preserve syntactic, semantic, and discourse coherence. While only a first step toward generating coherent text units from neural models, our work has the potential to significantly impact natural language generation and summarization1.

A challenging problem for spoken dialog systems is the design of utterance generation modules that are fast, flexible and general, yet produce high quality output in particular domains. A promising approach is trainable generation, which uses general-purpose linguistic knowledge automatically adapted to the application domain. This paper presents a trainable sentence planner for the MATCH dialog system. We show that trainable sentence planning can produce output comparable to that of MATCH’s template-based generator even for quite complex information presentations.

In this paper, we introduce Iterative Text Summarization (ITS), an iteration-based model for supervised extractive text summarization, inspired by the observation that it is often necessary for a human to read an article multiple times in order to fully understand and summarize its contents. Current summarization approaches read through a document only once to generate a document representation, resulting in a sub-optimal representation. To address this issue we introduce a model which iteratively polishes the document representation on many passes through the document. As part of our model, we also introduce a selective reading mechanism that decides more accurately the extent to which each sentence in the model should be updated. Experimental results on the CNN/DailyMail and DUC2002 datasets demonstrate that our model significantly outperforms state-of-the-art extractive systems when evaluated by machines and by humans.

Existing target-specific sentiment recognition methods consider only a single target per tweet, and have been shown to miss nearly half of the actual targets mentioned. We present a corpus of UK election tweets, with an average of 3.09 entities per tweet and more than one type of sentiment in half of the tweets. This requires a method for multi-target specific sentiment recognition, which we develop by using the context around a target as well as syntactic dependencies involving the target. We present results of our method on both a benchmark corpus of single targets and the multi-target election corpus, showing state-of-the art performance in both corpora and outperforming previous approaches to multi-target sentiment task as well as deep learning models for singletarget sentiment.

People share their opinions about things like products, movies and services using social media channels. The analysis of these textual contents for sentiments is a gold mine for marketing experts, thus automatic sentiment analysis is a popular area of applied artificial intelligence. We propose a latent syntactic structure-based approach for sentiment analysis which requires only sentence-level polarity labels for training. Our experiments on three domains (movie, IT products, restaurant) show that a sentiment analyzer that exploits syntactic parses and has access only to sentence-level polarity annotation for in-domain sentences can outperform state-of-the-art models that were trained on out-domain parse trees with sentiment annotation for each node of the trees. In practice, millions of sentence-level polarity annotations are usually available for a particular domain thus our approach is applicable for training a sentiment analyzer for a new domain while it can exploit the syntactic structure of sentences as well.

Generating an abstract from a collection of documents is a desirable capability for many realworld applications. However, abstractive approaches to multi-document summarization have not been thoroughly investigated. This paper studies the feasibility of using Abstract Meaning Representation (AMR), a semantic representation of natural language grounded in linguistic theory, as a form of content representation. Our approach condenses source documents to a set of summary graphs following the AMR formalism. The summary graphs are then transformed to a set of summary sentences in a surface realization step. The framework is fully data-driven and flexible. Each component can be optimized independently using small-scale, in-domain training data. We perform experiments on benchmark summarization datasets and report promising results. We also describe opportunities and challenges for advancing this line of research.

In this paper, we propose a sequential neural encoder with latent structured description SNELSD for modeling sentences. This model introduces latent chunk-level representations into conventional sequential neural encoders, i.e., recurrent neural networks with long short-term memory LSTM units, to consider the compositionality of languages in semantic modeling. An SNELSD model has a hierarchical structure that includes a detection layer and a description layer. The detection layer predicts the boundaries of latent word chunks in an input sentence and derives a chunk-level vector for each word. The description layer utilizes modified LSTM units to process these chunk-level vectors in a recurrent manner and produces sequential encoding outputs. These output vectors are further concatenated with word vectors or the outputs of a chain LSTM encoder to obtain the final sentence representation. All the model parameters are learned in an end-to-end manner without a dependency on additional text chunking or syntax parsing. A natural language inference task and a sentiment analysis task are adopted to evaluate the performance of our proposed model. The experimental results demonstrate the effectiveness of the proposed SNELSD model on exploring task-dependent chunking patterns during the semantic modeling of sentences. Furthermore, the proposed method achieves better performance than conventional chain LSTMs and tree-structured LSTMs on both tasks.


Question: Dynamic multi-level, multi-task learning for the simplification of sentences.

This paper presents a unified deep learning framework for natural language processing, featuring a single convolutional neural network (CNN) architecture that simultaneously addresses multiple language analysis tasks. The architecture processes a sentence to produce a variety of linguistic predictions, including part-of-speech tagging, chunking, named entity recognition, semantic role labeling, word semantic similarity, and sentence coherence evaluation based on a language model. The network's training employs multitask learning, characterized by weight-sharing across all tasks, except for the language model, which is trained on unlabeled text, offering a novel semi-supervised learning approach. We demonstrate that both multitask and semi-supervised learning strategies enhance the generalization capabilities of the shared tasks, leading to cutting-edge performance in these linguistic analyses.

We propose a hierarchical attention network for document classification. Our model has two distinctive characteristics: (i) it has a hierarchical structure that mirrors the hierarchical structure of documents; (ii) it has two levels of attention mechanisms applied at the wordand sentence-level, enabling it to attend differentially to more and less important content when constructing the document representation. Experiments conducted on six large scale text classification tasks demonstrate that the proposed architecture outperform previous methods by a substantial margin. Visualization of the attention layers illustrates that the model selects qualitatively informative words and sentences.

Most research on multi-document summarization explores methods that generate summaries based on queries regardless of the users’ preferences. We note that, different users can generate somewhat different summaries on the basis of the same source data and query. This paper presents our study on how to exploit the information regards how users summarized their texts. Models of different users can be used either separately, or in an ensemble-like fashion. Machine learning methods are explored in the construction of the individual models. However, we explore yet another hypothesis. We believe that the sentences selected into the summary should be coherent and supplement each other in their meaning. One method to model this relationship between sentences is by detecting actor–object relationship (AOR). The sentences that satisfy this relationship have their importance value enhanced. This paper combines ensemble summarizing system and AOR to generate summaries. We have evaluated this method on DUC 2006 and DUC 2007 using ROUGE measure. Experimental results show the supervised method that exploits the ensemble summarizing system combined with AOR outperforms previous models when considering performance in query-based multidocument summarization tasks. Keyword User-based summarization · Actor–object relationship · Multi-document summarization · Ensemble summarizing system · Training data construction Communicated by V. Loia. M. Valizadeh (B) · P. Brazdil LIAAD INESC Tec, University of Porto, Porto, Portugal e-mail: valizadehmr@gmail.com P. Brazdil FEP, University of Porto, Porto, Portugal e-mail: pbrazdil@inescporto.pt

This paper investigates the effectiveness of knowledge distillation in the context of multilingual models. We show that with knowledge distillation, Long Short-Term Memory(LSTM) models can be used to train standard feed-forward Deep Neural Network (DNN) models for a variety of low-resource languages. We then examine how the agreement between the teacher's best labels and the original labels affects the student model's performance. Next, we show that knowledge distillation can be easily applied to semi-supervised learning to improve model performance. We also propose a promising data selection method to filter un-transcribed data. Then we focus on knowledge transfer among DNN models with multilingual features derived from CNN+DNN, LSTM, VGG, CTC and attention models. We show that a student model equipped with better input features not only learns better from the teacher's labels, but also outperforms the teacher. Further experiments suggest that by learning from each other, the original ensemble of various models is able to evolve into a new ensemble with even better combined performance.

The skip-thought model has been proven to be effective at learning sentence representations and capturing sentence semantics. In this paper, we propose a suite of techniques to trim and improve it. First, we validate a hypothesis that, given a current sentence, inferring the previous and inferring the next sentence provide similar supervision power, therefore only one decoder for predicting the next sentence is preserved in our trimmed skip-thought model. Second, we present a connection layer between encoder and decoder to help the model to generalize better on semantic relatedness tasks. Third, we found that a good word embedding initialization is also essential for learning better sentence representations. We train our model unsupervised on a large corpus with contiguous sentences, and then evaluate the trained model on 7 supervised tasks, which includes semantic relatedness, paraphrase detection, and text classification benchmarks. We empirically show that, our proposed model is a faster, lighter-weight and equally powerful alternative to the original skip-thought model.

Single-word vector space models have been very successful at learning lexical information. However, they cannot capture the compositional meaning of longer phrases, preventing them from a deeper understanding of language. We introduce a recursive neural network (RNN) model that learns compositional vector representations for phrases and sentences of arbitrary syntactic type and length. Our model assigns a vector and a matrix to every node in a parse tree: the vector captures the inherent meaning of the constituent, while the matrix captures how it changes the meaning of neighboring words or phrases. This matrix-vector RNN can learn the meaning of operators in propositional logic and natural language. The model obtains state of the art performance on three different experiments: predicting fine-grained sentiment distributions of adverb-adjective pairs; classifying sentiment labels of movie reviews and classifying semantic relationships such as cause-effect or topic-message between nouns using the syntactic path between them.

A large class of machine-learning problems in natural language require the characterization of linguistic context. Two characteristic properties of such problems are that their feature space is of very high dimensionality, and their target concepts depend on only a small subset of the features in the space. Under such conditions, multiplicative weight-update algorithms such as Winnow have been shown to have exceptionally good theoretical properties. In the work reported here, we present an algorithm combining variants of Winnow and weighted-majority voting, and apply it to a problem in the aforementioned class: context-sensitive spelling correction. This is the task of fixing spelling errors that happen to result in valid words, such as substituting to for too, casual for causal, and so on. We evaluate our algorithm, WinSpell, by comparing it against BaySpell, a statistics-based method representing the state of the art for this task. We find: (1) When run with a full (unpruned) set of features, WinSpell achieves accuracies significantly higher than BaySpell was able to achieve in either the pruned or unpruned condition; (2) When compared with other systems in the literature, WinSpell exhibits the highest performance; (3) While several aspects of WinSpell's architecture contribute to its superiority over BaySpell, the primary factor is that it is able to learn a better linear separator than BaySpell learns; (4) When run on a test set drawn from a different corpus than the training set was drawn from, WinSpell is better able than BaySpell to adapt, using a strategy we will present that combines supervised learning on the training set with unsupervised learning on the (noisy) test set.

Machine translation systems based on deep neural networks are expensive to train. Curriculum learning aims to address this issue by choosing the order in which samples are presented during training to help train better models faster. We adopt a probabilistic view of curriculum learning, which lets us flexibly evaluate the impact of curricula design, and perform an extensive exploration on a German-English translation task. Results show that it is possible to improve convergence time at no loss in translation quality. However, results are highly sensitive to the choice of sample difficulty criteria, curriculum schedule and other hyperparameters.

Discourse segmentation, which segments texts into Elementary Discourse Units, is a fundamental step in discourse analysis. Previous discourse segmenters rely on complicated hand-crafted features and are not practical in actual use. In this paper, we propose an endto-end neural segmenter based on BiLSTMCRF framework. To improve its accuracy, we address the problem of data insufficiency by transferring a word representation model that is trained on a large corpus. We also propose a restricted self-attention mechanism in order to capture useful information within a neighborhood. Experiments on the RST-DT corpus show that our model is significantly faster than previous methods, while achieving new stateof-the-art performance. 1

In this work, we propose a novel method to incorporate corpus-level discourse information into language modelling. We call this larger-context language model. We introduce a late fusion approach to a recurrent language model based on long short-term memory units (LSTM), which helps the LSTM unit keep intra-sentence dependencies and inter-sentence dependencies separate from each other. Through the evaluation on three corpora (IMDB, BBC, and PennTree Bank), we demonstrate that the proposed model improves perplexity significantly. In the experiments, we evaluate the proposed approach while varying the number of context sentences and observe that the proposed late fusion is superior to the usual way of incorporating additional inputs to the LSTM. By analyzing the trained largercontext language model, we discover that content words, including nouns, adjectives and verbs, benefit most from an increasing number of context sentences. This analysis suggests that larger-context language model improves the unconditional language model by capturing the theme of a document better and more easily.

We present a new approach to evaluate computational models for the task of text understanding by the means of out-of-context error detection. Through the novel design of our automated modification process, existing large-scale data sources can be adopted for a vast number of text understanding tasks. The data is thereby altered on a semantic level, allowing models to be tested against a challenging set of modified text passages that require to comprise a broader narrative discourse. Our newly introduced task targets actual real-world problems of transcription and translation systems by inserting authentic out-of-context errors. The automated modification process is applied to the 2016 TEDTalk corpus. Entirely automating the process allows the adoption of complete datasets at low cost, facilitating supervised learning procedures and deeper networks to be trained and tested. To evaluate the quality of the modification algorithm a language model and a supervised binary classification model are trained and tested on the altered dataset. A human baseline evaluation is examined to compare the results with human performance. The outcome of the evaluation task indicates the difficulty to detect semantic errors for machine-learning algorithms and humans, showing that the errors cannot be identified when limited to a single sentence.