Question: Signaling traffic volume reaching cellular networks from mobile devices.

This paper explores the vast variability in smartphone usage patterns among individuals, utilizing extensive data from 255 users. We present a detailed analysis of intentional user engagements with their devices and the applications they utilize, as well as the implications of these activities on network traffic and energy consumption. Our findings reveal a significant disparity in usage behaviors among users. Across all examined metrics, user differences span one or more orders of magnitude. For example, the daily average of device interactions ranges from 10 to 200, while the average daily data consumption fluctuates from 1 to 1000 MB. This pronounced diversity underscores the necessity for mechanisms that can adapt and optimize user experience and energy efficiency by learning individual usage patterns. Despite this diversity, we identify commonalities among users that facilitate the learning of behavior patterns. Application popularity, for instance, can be effectively modeled using an exponential distribution, albeit with distinct parameters for different users. We showcase the benefits of personalized adaptation by examining a predictive mechanism for estimating future energy depletion. With personalized adaptation, the prediction error at the 90th percentile is less than half compared to predictions based on generalized user behavior averages, highlighting the potential for improved energy management strategies that cater to individual usage profiles.

Many wireless sensor network applications will impose a many-to-one traffic flow pattern. This phenomena in communication networks is referred to as reachback. The higher volume of traffic in traffic hotspots, which usually form around the base station, limits the number of sensors that can participate in the formation of a wireless sensor network with a fixed per sensor data rate. To alleviate the effects of reachback, spatial correlation of sensor data has enabled the use of Slepian-Wolf coding as a compression technique to reduce the volume of traffic in sensor networks. Within a cluster, Slepian-Wolf coding is used to compress the sensor readings. Clusterheads then transmit the compressed sensor readings over the overlay network which is formed by all clusterheads. We develop an expression for the end-to-end (sensor-to-base station) channel capacity of such a wireless sensor network and observe the effect of variations in the number of clusterheads on end-to-end capacity.

A novel system for the location of people in an office environment is described. Members of staff wear badges that transmit signals providing information about their location to a centralized location service, through a network of sensors. The paper also examines alternative location techniques, system design issues and applications, particularly relating to telephone call routing. Location systems raise concerns about the privacy of an individual and these issues are also addressed.

Mobile crowdsensing is a new paradigm in which a crowd of mobile users exploit their carried smart devices to conduct complex computation and sensing tasks in mobile social networks (MSNs). In this paper, we focus on the task assignment problem in mobile crowdsensing. Unlike traditional task scheduling problems, the task assignment in mobile crowdsensing must follow the mobility model of users in MSNs. To solve this problem, we propose an oFfline Task Assignment (FTA) algorithm and an oNline Task Assignment (NTA) algorithm. Both FTA and NTA adopt a greedy task assignment strategy. Moreover, we prove that the FTA algorithm is an optimal offline task assignment algorithm, and give a competitive ratio of the NTA algorithm. In addition, we demonstrate the significant performance of our algorithms through extensive simulations, based on four real MSN traces and a synthetic MSN trace.

People counting is key to a diverse set of sensing applications. In this paper, we design a mote-scale event-driven solution that uses a low-power pulsed radar to estimate the number of people within the ~10m radial range of the radar. In contrast to extant solutions, most of which use computer vision, our solution is light-weight and private. It also better tolerates the presence of obstacles that partially or fully impair line of sight; this is achieved by accounting for “small” indirect radio reflections via joint time-frequency domain features. The counter itself is realized using Support Vector Regression; the regression map is learned from a medium sized dataset of 0-~40 people in various indoor room settings. 10-fold cross validation of our counter yields a mean absolute error of 2.17 between the estimated count and the ground truth and a correlation coefficient of 0.97.We compare the performance of our solution with baseline counters.

Vehicle-to-vehicle (V2V) communications aim to enhance driver safety and traffic efficiency by using the recently designated frequency bands in the 5.9 GHz range in Europe. Due to the time-frequency selective fading behavior of the vehicular communication channel, multi-antenna techniques can provide enhanced link conditions by means of diversity processing. This paper highlights the integration of a four-element (N =4) linear array antenna into the roof-top compartment of a vehicle to conduct Multiple-Input Multiple-Output (MIMO) high-resolution mobile-to-mobile channel measurements.

Nowadays, Smart Cities are positioned as one of the most challenging and important research topics, highlighting major changes in people's lifestyle. Technologies such as smart energy, smart transportation or smart health are being designed to improve citizen's quality of life. Smart Cities leverage the deployment of a network of devices - sensors and mobile devices-, all connected through different means and/or technologies, according to their network availability and capacities, setting a novel framework feeding end-users with an innovative set of smart services. Aligned to this objective, a typical Smart City architecture is organized into layers, including a sensing layer (generates data), a network layer (moves the data), a middleware layer (manages all collected data and makes it ready for usage) and an application layer (provides the smart services benefiting from this data). In this paper a real Smart City is analyzed, corresponding to the city of Barcelona, with special emphasis on the layers responsible for collecting the data generated by the deployed sensors. The amount of daily sensors data transmitted through the network has been estimated and a rough projection has been made assuming an exhaustive deployment that fully covers all city. Finally, we discuss some solutions to both reduce the data transmission and improve the data management.

As data becomes dynamic, large, and distributed, there is increasing demand for what have become known as distributed stream algorithms. Since continuously collecting the data to a central server and processing it there is infeasible, a common approach is to define local conditions at the distributed nodes, such that—as long as they are maintained—some desirable global condition holds.
 Previous methods derived local conditions focusing on communication efficiency. While proving very useful for reducing the communication volume, these local conditions often suffer from heavy computational burden at the nodes. The computational complexity of the local conditions affects both the runtime and the energy consumption. These are especially critical for resource-limited devices like smartphones and sensor nodes. Such devices are becoming more ubiquitous due to the recent trend toward smart cities and the Internet of Things. To accommodate for high data rates and limited resources of these devices, it is crucial that the local conditions be quickly and efficiently evaluated.
 Here we propose a novel approach, designated CB (for Convex/Concave Bounds). CB defines local conditions using suitably chosen convex and concave functions. Lightweight and simple, these local conditions can be rapidly checked on the fly. CB’s superiority over the state-of-the-art is demonstrated in its reduced runtime and power consumption, by up to six orders of magnitude in some cases. As an added bonus, CB also reduced communication overhead in all the tested application scenarios.

This paper presents a novel traffic classification scheme to improve classification performance when few training data are available. In the proposed scheme, traffic flows are described using the discretized statistical features and flow correlation information is modeled by bag-of-flow (BoF). We solve the BoF-based traffic classification in a classifier combination framework and theoretically analyze the performance benefit. Furthermore, a new BoF-based traffic classification method is proposed to aggregate the naive Bayes (NB) predictions of the correlated flows. We also present an analysis on prediction error sensitivity of the aggregation strategies. Finally, a large number of experiments are carried out on two large-scale real-world traffic datasets to evaluate the proposed scheme. The experimental results show that the proposed scheme can achieve much better classification performance than existing state-of-the-art traffic classification methods.

Mobile instant messaging (e.g., via SMS or WhatsApp) often goes along with an expectation of high attentiveness, i.e., that the receiver will notice and read the message within a few minutes. Hence, existing instant messaging services for mobile phones share indicators of availability, such as the last time the user has been online. However, in this paper we not only provide evidence that these cues create social pressure, but that they are also weak predictors of attentiveness. As remedy, we propose to share a machine-computed prediction of whether the user will view a message within the next few minutes or not. For two weeks, we collected behavioral data from 24 users of mobile instant messaging services. By the means of machine-learning techniques, we identified that simple features extracted from the phone, such as the user's interaction with the notification center, the screen activity, the proximity sensor, and the ringer mode, are strong predictors of how quickly the user will attend to the messages. With seven automatically selected features our model predicts whether a phone user will view a message within a few minutes with 70.6% accuracy and a precision for fast attendance of 81.2%

Wireless networks such as mobile networks, with their inflexible and expensive network infrastructure, are facing various challenges in efficiently handling the exponentially growing traffic demands of users. Hence, mobile network operators (MNOs) are looking forward to novel networking paradigms which could simplify the task of network management and control and allow faster deployment of newer solutions on top of existing hardware by software updates. Software defined networking (SDN) is a revolutionary technology which makes networks more agile and flexible by separation of data plane and control plane tasks. SDN is playing a key role in deploying mobile network services on network function virtualization (NFV) infrastructure for elastic and flexible deployment of core network services e.g., virtualization of evolved packet core in LTE to efficiently handle huge control signal overhead and traffic demands from machine-to-machine and internet of things devices. Besides, NFV and SDN are offering scalable, flexible and adaptable network service chaining platforms as a replacement for inflexible middleboxes. Mobile edge computing (MEC) is a new evolving platform which brings together IT services and cloud computing for offering end-users network aware services and solutions. In this survey, we enumerate numerous issues and challenges in designing SDN based wireless networks and review various SDN based seminal solutions for 4G/5G. Finally, this survey presents the role of SDN, NFV and MEC in designing 5G networks.


Question: An internet-based PPGI method for monitoring heart rate using camera-based beat-to-beat detection.

This paper provides an in-depth overview of recent advancements in utilizing digital cameras and image processing techniques to non-invasively monitor physiological parameters, particularly focusing on remote optical photoplethysmographic (PPG) imaging. The widespread availability of digital cameras has opened up new avenues for cost-effective vital sign monitoring, offering a range of potential applications. The review highlights studies that go beyond basic pulse rate measurement, exploring the extraction of additional physiological metrics. It also delves into the challenges posed by non-ideal conditions, such as motion artifacts, and showcases real-world use cases in various environments. To facilitate ongoing research in PPG imaging, the authors have compiled a comprehensive resource, including references, code, and datasets, available at a dedicated website. This website is designed to serve as a valuable tool for the PPG imaging community, fostering collaboration and innovation. The website can be accessed at: http://web.mit.edu/~djmcduff/www/remote-physiology.html.

The focus shifts to the advancements in camera-based heart rate monitoring using remote PPG imaging techniques. The paper examines the state-of-the-art methods in this field, particularly those that address the complexities of extracting physiological signals under non-ideal conditions, such as the presence of motion artifacts. It also highlights studies that explore the use of PPG imaging beyond pulse rate measurement, including the extraction of additional physiological parameters. To support the growing community of researchers in this area, the authors have compiled a comprehensive resource, which includes a list of relevant references, code, and datasets. This resource is made available on a dedicated website, intended to serve as a central hub for PPG imaging research and to promote collaboration and progress in the field. The website can be accessed at: http://web.mit.edu/~djmcduff/www/remote-physiology.html.

In the present study we provide the first empirical evidence that viscero-sensory feedback from an internal organ is associated with decision-making processes. Participants with accurate vs. poor perception of their heart activity were compared with regard to their performance in the Iowa Gambling Task. During this task, participants have to choose between four card decks. Decks A and B yield high gains and high losses, and if played continuously, result in net loss. In contrast, decks C and D yield small gains and also small losses, but result in net profit if they are selected continuously. Accordingly, participants have to learn to avoid the net loss options in favor of the net gain options. In our study, participants with good cardiac perception chose significantly more of the net gain and fewer of the net loss options. Our findings document the substantial role of visceral feedback in decision-making processes in complex situations.

This paper investigates the potential of physiological signals as reliable channels for multi-subject emotion recognition. A three-stage decision framework is proposed for recognizing four emotions of multiple subjects. The decision framework consists of three stages: (1) in the initial stage, identifying a subject group that a test subject can be mapped to; (2) in the second stage, identifying an emotion pool that an instance of the test subject can be assigned to; and (3) in the final stage, generating the predicted emotion from the given emotion pool for the test instance. In comparison with a series of alternative methods, the high accuracy of 70.04% achieved by our proposed method clearly demonstrates the potential of the three-stage decision method in multi-subject emotion recognition.

Some pioneer WiFi signal based human activity recognition systems have been proposed. Their key limitation lies in the lack of a model that can quantitatively correlate CSI dynamics and human activities. In this paper, we propose CARM, a CSI based human Activity Recognition and Monitoring system. CARM has two theoretical underpinnings: a CSI-speed model, which quantifies the correlation between CSI value dynamics and human movement speeds, and a CSI-activity model, which quantifies the correlation between the movement speeds of different human body parts and a specific human activity. By these two models, we quantitatively build the correlation between CSI value dynamics and a specific human activity. CARM uses this correlation as the profiling mechanism and recognizes a given activity by matching it to the best-fit profile. We implemented CARM using commercial WiFi devices and evaluated it in several different environments. Our results show that CARM achieves an average accuracy of greater than 96%.

Among various skin detection methods, Skin Probability Map (SPM) method is an effective one. Though SPM method possesses high true acceptance rate (TAR), its false acceptance rate (FAR) is unacceptable in some cases. The reason is that SPM method only use pixel-level color information. This paper proposes an improved skin detection method that integrates color, texture and space information. After color filter, a texture filter is constructed based on texture features extracted form Gabor wavelet transform. Texture filter will further filter non-skin pixels, meanwhile, it may also filter some skin pixel. To compensate the loss, after texture filter, a marker driven watershed transform is then used to grow already obtained skin regions. Experimental results show that the proposed method can achieve better performance than that of SPM.

Recently, Wi-Fi has caught tremendous attention for its ubiquity. Motivated by Wi-Fi’s low cost and privacy preservation, researchers have been putting lots of investigation into its potential on action recognition and even person identification. In this paper, for bringing a new modality for multimedia community, we offer an comprehensive overview on these two topics in Wi-Fi. Also, through looking at these two topics from an unprecedented perspective, we could achieve generality instead of designing specific ad-hoc features for each scenario. Observing the great resemblance of Channel State Information (CSI, a finegrained information captured from the received Wi-Fi signal) to texture, we propose a brand-new framework based on computer vision methods. To minimize the effect of location dependency embedded in CSI, we propose a novel de-noising method based on Singular Value Decomposition (SVD) to eliminate the background energy and effectively extract the channel information of signals reflected by human bodies. From the experiments conducted, we demonstrate the feasibility and efficacy of the proposed methods. Also, we conclude the factors that would affect the performance and highlight a few potential issues that require further deliberations.

To help accelerate progress in multi-target, multi-camera tracking systems, we present (i) a new pair of precision-recall measures of performance that treats errors of all types uniformly and emphasizes correct identification over sources of error; (ii) the largest fully-annotated and calibrated data set to date with more than 2 million frames of 1080 p, 60 fps video taken by 8 cameras observing more than 2,700 identities over 85 min; and (iii) a reference software system as a comparison baseline. We show that (i) our measures properly account for bottom-line identity match performance in the multi-camera setting; (ii) our data set poses realistic challenges to current trackers; and (iii) the performance of our system is comparable to the state of the art.

The newly inaugurated Research Resource for Complex Physiologic Signals, which was created under the auspices of the National Center for Research Resources of the National Institutes of Health, is intended to stimulate current research and new investigations in the study of cardiovascular and other complex biomedical signals. The resource has 3 interdependent components. PhysioBank is a large and growing archive of well-characterized digital recordings of physiological signals and related data for use by the biomedical research community. It currently includes databases of multiparameter cardiopulmonary, neural, and other biomedical signals from healthy subjects and from patients with a variety of conditions with major public health implications, including life-threatening arrhythmias, congestive heart failure, sleep apnea, neurological disorders, and aging. PhysioToolkit is a library of open-source software for physiological signal processing and analysis, the detection of physiologically significant events using both classic techniques and novel methods based on statistical physics and nonlinear dynamics, the interactive display and characterization of signals, the creation of new databases, the simulation of physiological and other signals, the quantitative evaluation and comparison of analysis methods, and the analysis of nonstationary processes. PhysioNet is an on-line forum for the dissemination and exchange of recorded biomedical signals and open-source software for analyzing them. It provides facilities for the cooperative analysis of data and the evaluation of proposed new algorithms. In addition to providing free electronic access to PhysioBank data and PhysioToolkit software via the World Wide Web (http://www.physionet. org), PhysioNet offers services and training via on-line tutorials to assist users with varying levels of expertise.

Oscillometric blood pressure devices are popular and are considered part of the family's medical chest at home. The popularity of these devices for private use is not shared by physicians mainly due to the fact that the blood pressures are computed instead of measured. The classical way to compute the systolic and diastolic blood pressure is based on the envelope of the oscillometric waveform. The algorithm to compute the blood pressures from the waveform is firm dependent, often patented and lacks scientific foundation. In this paper, we propose a totally new approach. Instead of the envelope of the oscillometric waveform, we use a statistical test to pin-point the time instances where the systolic and diastolic blood pressures are measured in the cuff. This technique has the advantage of being mathematically well-posed instead of the ill-posed problem of envelope fitting. Hence, in order to calibrate the oscillometric blood pressure monitor it is sufficient to make the statistical test unbiased.

Emotions are related with learning processes, and physiological signals can be used to detect them. In this work, it is presented a model for the dynamic personalization of a learning environment. In the model, a specific combination of emotions and cognition processes are connected and integrated with the concept of ‘flow’, since it may provide a way to improve learning abilities. Physiological signals can be used to relate temporal emotions and subject's learning processes; the cardiac pulse is a reliable signal that carries useful information about the subject's emotional condition, which is detected using a classroom chair adapted with non invasive Electro-Mechanical Film (EMFi) sensors and an acquisition system that generates a ballistocardiogram (BCG), which is analyzed by a developed algorithm to obtain cardiac pulse statistics. A study including different data obtained from the chair's sensors was carried out and interpreted in terms of emotions, which together with a cognitive model is used for the personalization of content in a learning scenario. Such model establishes a relation between the learner's knowledge, the content of a course or a learning activity and the emotional condition of each student.

In this study, a wireless bio-radar sensor was designed to detect a human heartbeat and respiration signals without direct skin contact. In order to design a wireless bio-radar sensor quantitatively, the signal-to-noise ratio (SNR) in the baseband output of a sensor should be calculated. Therefore, we analyzed the SNR of the wireless bio-radar sensor, considering the signal power attenuation in a human body and all kinds of noise sources. Especially, we measured a residual phase noise of a typical free-running oscillator and used its value for the SNR analysis. Based on these analysis and the measurement results, a compact, low-cost 2.4 GHz direct conversion bio-radar sensor was designed and implemented in a printed circuit board. The demonstrated sensor consists of two printed antennas, a


Question: An internet-based PPGI method for monitoring heart rate using camera-based beat-to-beat detection.

This paper presents a groundbreaking method for non-invasive and automated measurement of cardiac pulse rates using video imaging and blind source separation (BSS), addressing the limitations of previous approaches which were often manual, prone to motion artifacts, and costly. The proposed technique, designed for use with color video recordings of facial regions, integrates advanced face tracking algorithms with BSS to decompose the color channels into independent components, effectively isolating the cardiac pulse signal. Through rigorous Bland-Altman and correlation analysis, the cardiac pulse rates derived from videos captured by a standard webcam were compared to those obtained from an FDA-approved finger blood volume pulse (BVP) sensor, revealing exceptional accuracy and correlation, even under conditions of moderate movement. Moreover, the technique was successfully applied to simultaneously measure heart rates of multiple individuals, marking the first instance of a cost-effective, video-based method for contact-free heart rate monitoring that is fully automated, resistant to motion artifacts, and capable of concurrent measurements on several subjects.

With the adoption of 2G and 3G cellular network technologies, mobile phones now have the bandwidth capability to stream data back to monitoring stations in real-time. Our paper describes the design and evaluation of a Bluetooth electrocardiogram sensor that transmits medical data to a cell phone. This data is displayed and stored on the phone. Future development of the system will relay this data over a cellular GPRS network. The current system provides a low cost and lightweight alternative to existing EKG event monitors. The final GPRS connected system will provide continuous monitoring of a patient 's heart anywhere cellular coverage is available

Over the past years Time-of-Flight (ToF) sensors have become a considerable alternative to conventional distance sensing techniques like laser scanners or image based stereo-vision. Due to the ability to provide full-range distance information at high frame-rates, ToF sensors achieve a significant impact onto current research areas like online object recognition, collision prevention or scene and object reconstruction. Nevertheless, ToF-cameras like the Photonic Mixer Device (PMD) still exhibit a number of error sources that affect the accuracy of measured distance information. For this reason, major error sources for ToFcameras will be discussed, along with a new calibration approach that combines intrinsic, distance as well as a reflectivity related error calibration in an overall, easy to use system and thus significantly reduces the number of necessary reference images. The main contribution, in this context, is a new intensity-based calibration model that requires less input data compared to other models and thus significantly contributes to the reduction of calibration data. 2010 Elsevier Inc. All rights reserved.

One of the most important challenges in chronic disease self-management is medication non-adherence, which has irrevocable outcomes. Although many technologies have been developed for medication adherence monitoring, the reliability and cost-effectiveness of these approaches are not well understood to date. This paper presents a medication adherence monitoring system by user-activity tracking based on wrist-band wearable sensors. We develop machine learning algorithms that track wrist motions in real-time and identify medication intake activities. We propose a novel data analysis pipeline to reliably detect medication adherence by examining single-wrist motions. Our system achieves an accuracy of 78.3% in adherence detection without need for medication pillboxes and with only one sensor worn on either of the wrists. The accuracy of our algorithm is only 7.9% lower than a system with two sensors that track motions of both wrists.

The proliferation of the video surveillance cameras, which accounts for the vast majority of cameras worldwide, has resulted in the need to nd methods and algorithms for dealing with the huge amount of information that is gathered every second. This encompasses processing tasks, such as raising an alarm or detouring moving objects, as well as some semantic tasks like event monitoring, trajectory or ow analysis, counting people, etc.

Studies have presented that the driver vigilance level has serious implication in the causation of road accidents. This paper focuses on integrating both the vehicle-based control behavior and physiological state to predict the driver vigilance index which is evaluated by using a smartwatch. The vehicle control behavior can be observed from the steering wheel movement. Our study utilized the smartwatch motion sensors to study the steering wheel behavior. Meanwhile, physiological state of driver reflects the driver capability of safety alert driving which is estimated by photoplethysmogram (PPG) and respiration signals in this paper. The PPG sensor is integrated in a sport wristband with a Bluetooth low energy module, transmitted the PPG signals to smartwatch in real time. The steering angle is derived by the reading from smartwatch built-in accelerometer and gyroscope sensors. On the other hand, the respiration is derived using the PPG peak baseline method. In order to utterly investigate the sleepiness-induced factors, the time, spectral, and phase space domain features are calculated. Considering the smartwatch processing capability, mutual-information technique is applied to designate the ten most descriptive features. Then, the extracted descriptive features are serve as parameters to a classifier to determine the driver aptitude status. The features are analyzed for their correlation with the subjective Koralinska sleepiness scale and through recorded video observations. The experimental results reveal that our system is capable of estimating driver hypervigilance at average of 96.5% accuracy rate by evaluating on both driving behavior and driver physiological state, provided a novel and low-cost implementation.

We present a system for capturing ink strokes written with ordinary pen and paper using a fast camera with a frame rate comparable to a stylus digitizer. From the video frames, ink strokes are extracted and used as input to an online handwriting recognition engine. A key component in our system is a pen up/down detection model for detecting the contact of the pen-tip with the paper in the video frames. The proposed model consists of feature representation with convolutional neural networks and classification with a recurrent neural network. We also use a high speed tracker with kernelized correlation filters to track the pen-tip. For training and evaluation, we collected labeled video data of users writing English and Japanese phrases from public datasets, and we report on character accuracy scores for different frame rates in the two languages.

In this paper, we introduce an ECG measurement, analysis and transmission system which uses a mobile phone as a base station. The system is based on a small-sized mobile ECG recording device which sends measurement data wirelessly to the mobile phone. In the mobile phone the received data is analyzed and if abnormalities are found part of the measurement data is sent to a server for the use of medical personnel. The prototype of the system was made with a portable ECG monitor and Nokia 6681 mobile phone. The results show that modern smart phones are indeed capable for this kind of tasks. Thus, with their good networking and data processing capabilities, they are a potential part of the future wireless health care systems.

We present an adaptive biofeedback game that aims to maintain the player’s arousal by modifying game difficulty in response to the player’s physiological state, as measured with wearable sensors. Our approach models the interaction between human physiology and game difficulty during gameplay as a control problem, where game difficulty is the system input and player arousal its output. We validate the approach on a car-racing game with real-time adaptive game mechanics. Specifically, we use (1) car speed, road visibility, and steering jitter as three mechanisms to manipulate game difficulty, (2) electrodermal activity as physiological correlate of arousal, and (3) two types of control law: proportional (P) control, and proportional-integral-derivative (PID) control. We also propose quantitative measures to characterize the effectiveness of these game adaptations and controllers in manipulating the player’s arousal. Experimental trials with 25 subjects in both open-loop (no feedback) and closed-loop (negative feedback) conditions show statistically significant differences in effectiveness among the three game mechanics and also between the two control laws. Specifically, manipulating car speed provides higher control of arousal levels than changing road visibility or vehicle steering. Our results also confirm that PID control leads to lower error and reduced oscillations in the closed-loop response compared to proportional-only control. Finally, we discuss the theoretical and practical implications of our approach.

Some pioneer WiFi signal based human activity recognition systems have been proposed. Their key limitation lies in the lack of a model that can quantitatively correlate CSI dynamics and human activities. In this paper, we propose CARM, a CSI based human Activity Recognition and Monitoring system. CARM has two theoretical underpinnings: a CSI-speed model, which quantifies the correlation between CSI value dynamics and human movement speeds, and a CSI-activity model, which quantifies the correlation between the movement speeds of different human body parts and a specific human activity. By these two models, we quantitatively build the correlation between CSI value dynamics and a specific human activity. CARM uses this correlation as the profiling mechanism and recognizes a given activity by matching it to the best-fit profile. We implemented CARM using commercial WiFi devices and evaluated it in several different environments. Our results show that CARM achieves an average accuracy of greater than 96%.

Health monitoring systems based on Internet-of-things (IoT) have been recently introduced to improve the quality of health care services. However, the number of advanced IoT-based continuous glucose monitoring systems is small and the existing systems have several limitations. In this paper we study feasibility of invasive and continuous glucose monitoring (CGM) system utilizing IoT based approach. We designed an IoT-based system architecture from a sensor device to a back-end system for presenting real-time glucose, body temperature and contextual data (i.e. environmental temperature) in graphical and human-readable forms to end-users such as patients and doctors. In addition, nRF communication protocol is customized for suiting to the glucose monitoring system and achieving a high level of energy efficiency. Furthermore, we investigate energy consumption of the sensor device and design energy harvesting units for the device. Finally, the work provides many advanced services at a gateway level such as a push notification service for notifying patient and doctors in case of abnormal situations (i.e. too low or too high glucose level). The results show that our system is able to achieve continuous glucose monitoring remotely in real-time. In addition, the results reveal that a high level of energy efficiency can be achieved by applying the customized nRF component, the power management unit and the energy harvesting unit altogether in the sensor device. c © 2017 The Authors. Published by E sevier B.V. i ilit f t f re ce r ra hairs.


Question: An internet-based PPGI method for monitoring heart rate using camera-based beat-to-beat detection.

The utilization of wearable photoplethysmographic biosensors (PPG-WBS) is poised to revolutionize cardiovascular (CV) monitoring by enabling continuous surveillance in various innovative environments. The potential advantages extend to improved diagnostics and treatment for major diseases, enhanced surveillance for CV emergencies in high-risk individuals, and better management of chronic conditions through precise therapy adjustment or identification of patient non-compliance. In hazardous operations or mass casualty scenarios, PPG-WBS can provide critical wireless monitoring, especially since CV parameters are fundamental "vital signs" in emergency medicine. This technology could also alleviate the strain on emergency departments by monitoring patients in the waiting area. For hospitalized patients needing CV monitoring, wearable sensors offer a more comfortable alternative to traditional tethered systems, reducing the risk of falls and enhancing patient safety. In daily life, these sensors can detect missed medication doses through monitoring blood pressure, triggering automated reminders for patients. Continuous blood pressure monitoring could also aid doctors in fine-tuning treatments for hypertension, potentially lowering mortality rates. Additionally, PPG-WBS can track patients' exercise regimens, ensuring compliance and improving health outcomes. For those with chronic CV diseases, such as heart failure, home monitoring with PPG-WBS can detect early signs of exacerbation, allowing for timely intervention before the condition worsens to a critical state. This article explores the clinical and technical aspects of PPG-WBS, highlighting their transformative potential in CV care.

Arterial stiffness index is one of the biomechanical indices of vascular healthiness. These indexes are based on detailed pulse waveform analysis which is presented here. After photoplethysmographyic (PPG) pulse wave measurement, we decompose the pulse waveform for the estimation and determination of arterial elasticity. Firstly, it is electro-optically measured PPG signal and by electromechanical film (EMFi) measured signal that are analyzed and investigated by dividing each wave into five logarithmic normal function components. For both the PPG and EMFi waveform we can find very easily a good fit between the original and overlapped and summed wave components. Each wave component is assumed to resemble certain phenomenon in the arteries and certain indexes can be calculated for example based on the mutual timing of the components. Several studies have demonstrated that these kinds of indexes calculated based on actual biomechanical processed can predict future cardiovascular events. Many dynamic factors, e.g., arterial stiffness, depend on fixed structural features of the arterial wall. For more accurate description, arterial stiffness is estimated based on pulse wave decomposition analysis in the radial measured by EMFi and PPG method and tibial arterial walls measured by PPG method parallelly. Elucidation of the precise relationship between endothelial function and arterial stiffness can be done through biomechanics. However, arterial wall elasticity awaits still further biomechanical studies with clinical relations and the influence of arterial flexibility, resistance and ageing inside of the radial pulse waveform.

We describe new techniques to detect and analyze periodic mo t on as seen from both a static and moving camera. By tracking objects of interest, we compute a n object’s self-similarity as it evolves in time. For periodic motion, the self-similarity measure is a l o periodic, and we apply Time-Frequency analysis to detect and characterize the periodic motion. Th e periodicity is also analyzed robustly using the 2-D lattice structures inherent in similarity matrices . A real-time system has been implemented to track and classify objects using periodicity. Examples of o bject classification (people, running dogs, vehicles), person counting, and non-stationary periodici ty are provided.

Driver fatigue is one of the leading causes of traffic accidents. This paper presents a real-time non-intrusive fatigue monitoring system which exploits the driver's facial expression to detect and alert fatigued drivers. The presented approach adopts the Viola-Jones classifier to detect the driver's facial features. The correlation coefficient template matching method is then applied to derive the state of each feature on a frame by frame basis. A Support Vector Machine (SVM) is finally integrated within the system to classify the facial appearance as either fatigued or otherwise. Using this simple and cheap implementation, the overall system achieved an accuracy of 95.2%, outperforming other developed systems employing expensive hardware to reach the same objective.

An automatic configuration that can detect the position of R-waves, classify the normal sinus rhythm (NSR) and other four arrhythmic types from the continuous ECG signals obtained from the MIT-BIH arrhythmia database is proposed. In this configuration, a support vector machine (SVM) was used to detect and mark the ECG heartbeats with raw signals and differential signals of a lead ECG. An algorithm based on the extracted markers segments waveforms of Lead II and V1 of the ECG as the pattern classification features. A self-constructing neural fuzzy inference network (SoNFIN) was used to classify NSR and four arrhythmia types, including premature ventricular contraction (PVC), premature atrium contraction (PAC), left bundle branch block (LBBB), and right bundle branch block (RBBB). In a real scenario, the classification results show the accuracy achieved is 96.4%. This performance is suitable for a portable ECG monitor system for home care purposes.

These days mobile devices like phones or tablets are very common among people of all age. They are connected with network and provide seamless communications through internet or cellular services. These devices can be a big help for the people who are not able to communicatie properly and even in emergency conditions. A disabled person who is not able to speak or a person who speak a different language, these devices can be a boon for them as understanding, translating and speaking systems for these people. This chapter discusses a portable android based hand sign recognition system which can be used by disabled people. This chapter shows a part of on-going project. Computer Vision based techniques were used for image analysis and PCA was used after image tokenizer for recognition. This method was tested with webcam results to make system more robust.

Communication mainly is the transfer of whatever thing or exchanging of data, so that the Internet of things is naught but the transferring or exchanging of anything with several other things. The using of internet authorized system or devices roughly calculated as that by 2020 there will be nearly about billions. The purpose of the paper is to define a safekeeping alert device spending little handling power by Internet of things which help out to observer plus alerts when gestures or else motion are there then send images to a cloud server. Besides, internet of things centered use can be used tenuously to observe the action as well as acquire warning when gestures or else indication are there. The images are showed straight to a cloud attendant, when the cloud attendant is not accessible at that time the records are put in storage close by on a Raspberry Pi. A credit card size Raspberry Pi with a advantage of Open Source Computer Vision (Open-CV) software knobs the image processing, control algorithms used for the attentiveness then shows taken images to concern persons email by the use of Wi-Fi module. The system uses ordinary webcam.

In this work, a deep convolutional neural network (CNN) is proposed to detect atrial fibrillation (AF) among the normal, noisy and other categories of cardiac arrhythmias electrocardiogram (ECG) recordings. The proposed CNN is trained by stochastic gradient descent with the categorical cross-entropy loss function. The network performance is evaluated on training (75%) and validation (25%) data sets that are obtained from 2017 Physionet/CinC challenge database. The proposed CNN model respectively achieves the average accuracy and F1 score of 87% and 0.84 on validation data set. One of the main advantages of this work besides high accuracy and reliability, is to simplify the feature extraction process and to remove the need for detecting ECG signal fiducial points and extracting hand-crafted features unlike conventional methods available in the literature. Moreover, it provides an opportunity for ECG screening in a large population, especially for atrial fibrillation screening, using wearable devices such as OM apparel that records high-quality single channel ECG signal.

Recently the use of video surveillance systems is widely increasing. Different places are equipped by camera surveillances such as hospitals, schools, airports, museums and military places in order to ensure the safety and security of the persons and their property. Therefore it becomes significant to guarantee the proper working of these systems. Intelligent video surveillance systems equipped by sophisticated digital camera can analyze video information's and automatically detect doubtful actions. The camera tampering detection algorithms may indicate that accidental or suspicious activities have occurred and that causes the abnormality works of the video surveillance. It uses several techniques based on image processing and computer vision. In this paper, comparative study of performance of four algorithms that can detect abnormal disturbance for video surveillance is presented also a new proposed method is exposed.

For the purpose of long-term, everyday electrocardiogram (ECG) monitoring, we present a convenient method of ECG measurement without direct conductive contact with the skin while subjects sat on a chair wearing normal clothes. Measurements were made using electrodes attached to the back of a chair, high-input-impedance amplifiers mounted on the electrodes, and a large ground-plane placed on the chair seat. ECGs were obtained by the presented method for several types of clothing and compared to ECGs obtained from conventional measurement using Ag-AgCl electrodes. Motion artifacts caused by usual desk works were investigated. This study shows the feasibility of the method for long-term, convenient, everyday use

Computer vision based technologies have seen widespread adoption over the recent years. This use is not limited to the rapid adoption of facial recognition technology but extends to facial expression recognition, scene recognition and more. These developments raise privacy concerns and call for novel solutions to ensure adequate user awareness, and ideally, control over the resulting collection and use of potentially sensitive data. While cameras have become ubiquitous, most of the time users are not even aware of their presence. In this paper we introduce a novel distributed privacy infrastructure for the Internet-of-Things and discuss in particular how it can help enhance user's awareness of and control over the collection and use of video data about them. The infrastructure, which has undergone early deployment and evaluation on two campuses, supports the automated discovery of IoT resources and the selective notification of users. This includes the presence of computer vision applications that collect data about users. In particular, we describe an implementation of functionality that helps users discover nearby cameras and choose whether or not they want their faces to be denatured in the video streams.


Question: An internet-based PPGI method for camera-based heart rate monitoring utilizing beat-to-beat detection.

This paper proposes an enhanced set of Haar-like features for efficient and accurate object detection. Building upon the work of Viola et al., who pioneered a fast object detection method using a cascade of simple features, we introduce a new collection of rotated Haar-like features. These features expand the original set of simple Haar-like features, significantly enhancing detection capabilities without compromising computational efficiency. In our experimental evaluation, the inclusion of these additional rotated features in a face detection application resulted in a 10% reduction in false alarm rate, at a constant hit rate. Furthermore, we propose a post-optimization technique for a given boosted cascade, which further reduces the false alarm rate by an average of 12.5%. By integrating both enhancements, the number of false detections dropped to just 24, while maintaining a hit rate of 82.3%, as tested on the CMU face dataset. This demonstrates the improved performance and reliability of our extended set of Haar-like features in object detection tasks.

In this paper we describe our methodology and results for the Classifying Heart Sounds PASCAL Challenge. We present our algoritm and results for segmentation and classification of S1 and S2 heart sounds.

We describe a GPU-based implementation of motion detection from a moving platform. Motion detection from a moving platform is inherently difficult as the moving camera induces 2D motion field in the entire image. A step compensating for camera motion is required prior to estimating of the background model. Due to inevitable registration errors, the background model is estimated according to a sliding window of frames to avoid the case where erroneous registration influences the quality of the detection for the whole sequence. However, this approach involves several characteristics that put a heavy burden on real-time CPU implementation. We exploit GPU to achieve significant acceleration over standard CPU implementations. Our GPU-based implementation can build the background model and detect motion regions at around 18 fps on 320times240 videos that are captured for a moving camera.

Color is an efficient feature for object detection as it has the advantage of being invariant to changes in scaling, rotation, and partial occlusion. Skin color detection is an essential required step in various applications related to computer vision. The rapidly-growing research in human skin detection is based on the premise that information about individuals, intent, mode, and image contents can be extracted from colored images, and computers can then respond in an appropriate manner. Detecting human skin in complex images has proven to be a challenging problem because skin color can vary dramatically in its appearance due to many factors such as illumination, race, aging, imaging conditions, and complex background. However, many methods have been developed to deal with skin detection problem in color images. The purpose of this study is to provide an up-to-date survey on skin color modeling and detection methods. We also discuss relevant issues such as color spaces, cost and risks, databases, testing, and benchmarking. After investigating these methods and identifying their strengths and limitations, we conclude with several implications for future direction.

With the rising of AR/VR technology and miniaturization of mobile devices, gesture is becoming an increasingly popular means of interacting with smart devices. Some pioneer ultrasound based human gesture recognition systems have been proposed. They mostly rely on low resolution Doppler Effect, and hence focus on whole hand motion and cannot deal with minor finger motions. In this paper, we present UltraGesture, a Channel Impulse Response (CIR) based ultrasonic finger motion perception and recognition system. CIR measurements can provide with 7 mm resolution, rendering it sufficient for minor finger motion recognition. UltraGesture encapsulates CIR measurements into an image, and builds a Convolutional Neural Network model to classify these images into different categories, which corresponding to distinct gestures. Our system runs on commercial speakers and microphones that already exist on most mobile devices without hardware modification. Our results show that UltraGesture achieves an average accuracy of greater than 97% for 12 gestures including finger click and rotation.

The paper presents an early fire-alarm raising method based on video processing. The basic idea of the proposed of fire-detection is to adopt a RGB (red, green, blue) model based chromatic and disorder measurement for extracting fire-pixels and smoke-pixels. The decision function of fire-pixels is mainly deduced by the intensity and saturation of R component. The extracted fire-pixels will be verified if it is a real fire by both dynamics of growth and disorder, and further smoke. Based on iterative checking on the growing ratio of flames, a fire-alarm is given when the alarm-raising condition is met. Experimental results show that the developed technique can achieve fully automatic surveillance of fire accident with a lower false alarm rate and thus is very attractive for the important military, social security, commercial applications, and so on, at a general cost.

This paper proposes a novel method to detect flames in video by processing the data generated by an ordinary camera monitoring a scene. In addition to ordinary motion and color clues, flame flicker process is also detected by using a hidden Markov model. Markov models representing the flame and flame colored ordinary moving objects are used to distinguish flame flicker process from motion of flame colored moving objects. Spatial color variations in flame are also evaluated by the same Markov models, as well. These clues are combined to reach a final decision. False alarms due to ordinary motion of flame colored moving objects are greatly reduced when compared to the existing video based fire detection systems.

Atrial fibrillation (AF) is an arrhythmia that can lead to several patient risks. This kind of arrhythmia affects mostly elderly people, in particular those who suffer from heart failure (one of the main causes of hospitalization). Thus, detection of AF becomes decisive in the prevention of cardiac threats. In this paper an algorithm for AF detection based on a novel algorithm architecture and feature extraction methods is proposed. The aforementioned architecture is based on the analysis of the three main physiological characteristics of AF: i) P wave absence ii) heart rate irregularity and iii) atrial activity (AA). Discriminative features are extracted using model-based statistic and frequency based approaches. Sensitivity and specificity results (respectively, 93.80% and 96.09% using the MIT-BIH AF database) show that the proposed algorithm is able to outperform state-of-the-art methods.

A method for smoke detection in video is proposed. It is assumed the camera monitoring the scene is stationary. Since the smoke is semi-transparent, edges of image frames start loosing their sharpness and this leads to a decrease in the high frequency content of the image. To determine the smoke in the field of view of the camera, the background of the scene is estimated and decrease of high frequency energy of the scene is monitored using the spatial wavelet transforms of the current and the background images. Edges of the scene are especially important because they produce local extrema in the wavelet domain. A decrease in values of local extrema is also an indicator of smoke. In addition, scene becomes grayish when there is smoke and this leads to a decrease in chrominance values of pixels. Periodic behavior in smoke boundaries and convexity of smoke regions are also analyzed. All of these clues are combined to reach a final decision.

The development of noninvasive blood glucose detector is desired to replace invasive method which is costly and uncomfortable. Near Infrared (NIR) spectroscopy is used to detect blood glucose noninvasively since glucose has specific absorbance spectrum in NIR range, 850nm-2500nm. LED is utilized as incoherent infrared source to irradiate body surface in wavelength 1550 nm. Penetrated light is then detected by InGaAs photodiode, sensitive for wavelength 850-1700 nm. As the measurement sites, earlobe is chosen since it is thin. Photodiode's current is converted into voltage using transimpedance amplifier circuit. In order to minimize high frequency noise, low pass filter is applied consecutively. Early testing is conducted by measuring penetrated light in various concentration of glucose solution with wavelength 1300nm, 1450nm, and 1550nm. Measurement indicates correlation between voltage and glucose concentration. In earlobe testing, light can penetrate tissue with detected voltage, 300mV-4V depend on the wavelength. Linear approximation and two point equation is applied to the data. This two approach extracts formula that can estimate blood glucose concentration with maximum 30% error. By using two point equation, a specific formula for each person can be obtained.

Indoor localization has long been a goal of pervasive computing research. In this paper, we explore the possibility of determining user's location based on the camera images received from a smart phone. In our system, the smart phone is worn by the user as a pendant and images are periodically captured and transmitted over GPRS to a web server. The web server returns the location of the user by comparing the received images with images stored in a database. We tested our system inside the Computer Science department building. Preliminary results show that user's location can be determined correctly with more than 80% probability of success. As opposed to earlier solutions for indoor localization, this approach does not have any infrastructure requirements. The only cost is that of building an image database.


Question: An internet-based PPGI method for camera-based heart rate monitoring utilizing beat-to-beat detection.

This study explores the effectiveness of a multi-imager array in recovering accurate pulse rate measurements during motion artifact for non-contact imaging photoplethysmography (PPG). Photoplethysmography is a technique that measures changes in optical absorption of tissue due to arterial blood flow. Typically, PPG sensors require direct contact with the skin, but recent developments in non-contact imaging PPG have shown that vital signs such as pulse rate, pulse rate variability, and respiration rate can be extracted from facial images under conditions of minimal motion. A significant challenge in this technique is the inability to maintain accuracy under head motion. To address this, nine synchronized visible spectrum cameras were arranged in a semicircular array around the subject, capturing data under controlled lighting. The study also collected three-lead electrocardiogram and finger-tip reflectance PPG as reference signals. The results were compared between stationary and motion conditions, with and without a chinrest. It was found that under conditions of little to no head motion, a single camera could recover pulse rate with an average absolute error of less than two beats per minute. However, during severe head motion, the error in pulse rate measurement from a single camera could be as high as twenty-two beats per minute. This error was significantly reduced by increasing the dimensionality of the imager channel space using multiple cameras in the array before applying blind source separation. Unlike single-camera results, the multi-camera channel space provided pulse rate measurements with an absolute error comparable to those obtained via traditional finger-tip PPG. These findings indicate that non-contact imaging PPG can remain accurate in the presence of head motion when a multi-camera array is utilized to enhance the decomposed channel space dimensionality.

A reliable human skin detection method that is adaptable to different human skin colors and illumination conditions is essential for better human skin segmentation. Even though different human skin-color detection solutions have been successfully applied, they are prone to false skin detection and are not able to cope with the variety of human skin colors across different ethnic. Moreover, existing methods require high computational cost. In this paper, we propose a novel human skin detection approach that combines a smoothed 2-D histogram and Gaussian model, for automatic human skin detection in color image(s). In our approach, an eye detector is used to refine the skin model for a specific person. The proposed approach reduces computational costs as no training is required, and it improves the accuracy of skin detection despite wide variation in ethnicity and illumination. To the best of our knowledge, this is the first method to employ fusion strategy for this purpose. Qualitative and quantitative results on three standard public datasets and a comparison with state-of-the-art methods have shown the effectiveness and robustness of the proposed approach.

A newly developed optical method for noninvasively measuring the switching activity of operating CMOS integrated circuit chips is described. The method, denoted as picosecond imaging circuit analysis (PICA) can be used to characterize the gate-level performance of such chips and identify the locations and nature of their operational faults. The principles underlying PICA and examples of its use are discussed.

Processing images to extract useful information in real-time is a complex task, dealing with large amounts of iconic data and requiring intensive computation. Smart cameras use embedded processing to save the host system from the low-level processing load and to reduce communication flows and overheads. Field programmable devices present special interest for smart cameras design: flexibility, reconfigurability and parallel processing skills are some specially important features. In this paper we present a FPGA-based smart camera research platform. The hardware architecture is described, and some design issues are discussed. Our goal is to use the possibility to reconfigure the FPGA device in order to adapt the system architecture to a given application. To that, a design methodology, based on pre-programmed processing elements, is proposed and sketched. Some implementation issues are discussed and a template tracking application is given as example, with its experimental results.

This paper proposed a multimodal fusion between brain and peripheral signals for emotion detection. The input signals were electroencephalogram, galvanic skin resistance, temperature, blood pressure and respiration, which can reflect the influence of emotion on the central nervous system and autonomic nervous system respectively. The acquisition protocol is based on a subset of pictures which correspond to three specific areas of valance-arousal emotional space (positively excited, negatively excited, and calm). The features extracted from input signals, and to improve the results, correlation dimension as a strong nonlinear feature is used for brain signals. The performance of the Quadratic Discriminant Classifier has been evaluated on different feature sets: peripheral signals, EEG's, and both. In comparison among the results of different feature sets, EEG signals seem to perform better than other physiological signals, and the results confirm the interest of using brain signals as peripherals in emotion assessment. According to the improvement in EEG results compare in each raw of the table, it seems that nonlinear features would lead to better understanding of how emotional activities work.

Pfinder is a real-time system for tracking people and interpreting their behavior. It runs at 10Hz on a standard SGI Indy computer, and has performed reliably on thousands of people in many different physical locations. The system uses a multiclass statistical model of color and shape to obtain a 2D representation of head and hands in a wide range of viewing conditions. Pfinder has been successfully used in a wide range of applications including wireless interfaces, video databases, and low-bandwidth coding. Index Terms —Blobs, blob tracking, real-time, person tracking, 3D person tracking, segmentation, gesture recognition, mixture model, MDL.

Preserving a person's privacy in an efficient manner is very important for critical, life-saving infrastructures like body sensor networks (BSN). This paper presents a novel key agreement scheme which allows two sensors in a BSN to agree to a common key generated using electrocardiogram (EKG) signals. This EKG-based key agreement (EKA) scheme aims to bring the "plug-n-play" paradigm to BSN security whereby simply deploying sensors on the subject can enable secure communication, without requiring any form of initialization such as pre-deployment. Analysis of the scheme based on real EKG data (obtained from MIT PhysioBank database) shows that keys resulting from EKA are: random, time variant, can be generated based on short-duration EKG measurements, identical for a given subject and different for separate individuals.

A syringe-implantable electrocardiography (ECG) monitoring system is proposed. The noise optimization and circuit techniques in the analog front-end (AFE) enable 31 nA current consumption while a minimum energy computation approach in the digital back-end reduces digital energy consumption by 40%. The proposed SoC is fabricated in 65 nm CMOS and consumes 64 nW while successfully detecting atrial fibrillation arrhythmia and storing the irregular waveform in memory in experiments using an ECG simulator, a live sheep, and an isolated sheep heart.

In this paper, a noncontact proximity vital sign sensor, using a phase locked loop (PLL) incorporated with voltage controlled oscillator (VCO) built-in planar type circular resonator, is proposed to enhance sensitivity in severe environments. The planar type circular resonator acts as a series feedback element of the VCO as well as a near-field receiving antenna. The frequency deviation of the VCO related to the body proximity effect ranges from 0.07 MHz/mm to 1.8 MHz/mm (6.8 mV/mm to 205 mV/mm in sensitivity) up to a distance of 50 mm, while the amount of VCO drift is about 21 MHz in the condition of 60 °C temperature range and discrete component tolerance of ±5%. Total frequency variation occurs in the capture range of the PLL which is 60 MHz. Thus, its loop control voltage converts the amount of frequency deviation into a difference of direct current (DC) voltage, which is utilized to extract vital signs regardless of the ambient temperature. The experimental results reveal that the proposed sensor placed 50 mm away from a subject can reliably detect respiration and heartbeat signals without the ambiguity of harmonic signals caused by respiration signal at an operating frequency of 2.4 GHz.

A desirable feature in smart cameras is the ability to autonomously detect any tampering event/attack that would prevent a clear view over the monitored scene. No matter whether tampering is due to atmospheric phenomena (e.g., few rain drops over the camera lens) or to malicious attacks (e.g., occlusions or device displacements), these have to be promptly detected to possibly activate countermeasures. Tampering detection is particularly challenging in battery-powered cameras, where it is not possible to acquire images at full-speed frame-rates, nor use sophisticated image-analysis algorithms. We here introduce a tampering-detection algorithm specifically designed for low-power smart cameras. The algorithm leverages very simple indicators that are then monitored by an outlier-detection scheme: any frame yielding an outlier is detected as tampered. Core of the algorithm is the partitioning of the scene into adaptively defined regions, that are preliminarily defined by segmenting the image during the algorithmconfiguration phase, and which shows to improve the detection of camera displacements. Experiments show that the proposed algorithm can successfully operate on sequences acquired at very low-frame rate, such as one frame every minute, with a very small computational complexity.

In this work, we propose for the first time an autonomous system, called WiSpiro, that continuously monitors a person's breathing volume with high resolution during sleep from afar. WiSpiro relies on a phase-motion demodulation algorithm that reconstructs minute chest and abdominal movements by analyzing the subtle phase changes that the movements cause to the continuous wave signal sent by a 2.4 GHz directional radio. These movements are mapped to breathing volume, where the mapping relationship is obtained via a short training process. To cope with body movement, the system tracks the large-scale movements and posture changes of the person, and moves its transmitting antenna accordingly to a proper location in order to maintain its beam to specific areas on the frontal part of the person's body. It also incorporates interpolation mechanisms to account for possible inaccuracy of our posture detection technique and the minor movement of the person's body. We have built WiSpiro prototype, and demonstrated through a user study that it can accurately and continuously monitor user's breathing volume with a median accuracy from 90% to 95.4% (or 0.0581 to 0.111 of error) to even in the presence of body movement. The monitoring granularity and accuracy are sufficiently high to be useful for diagnosis by clinical doctor.


Question: Eyeriss: a spatially-organized architecture for energy-efficient dataflow in convolutional neural networks.

This research delves into the impact of convolutional network depth on its accuracy within the context of large-scale image recognition. Our primary contribution involves a comprehensive analysis of networks with increasing depth, which reveals that a substantial enhancement in performance can be achieved by extending the depth to 16–19 weight layers. This insight was pivotal to our submission for the ImageNet Challenge 2014, where our team successfully secured the top and second positions in the localisation and classification categories, respectively. Moreover, we demonstrate that our representations are highly adaptable to other datasets, achieving cutting-edge outcomes. Significantly, we have made our two highest-performing ConvNet models freely accessible to the public, thereby encouraging further exploration into the application of deep visual representations within the realm of computer vision. This move is aimed at fostering innovation and collaborative research in the field.

Neural networks are a subset of the field of artificial intelligence (AI). The predominant types of neural networks used for multidimensional signal processing are deep convolutional neural networks (CNNs). The term deep refers generically to networks having from a "few" to several dozen or more convolution layers, and deep learning refers to methodologies for training these systems to automatically learn their functional parameters using data representative of a specific problem domain of interest. CNNs are currently being used in a broad spectrum of application areas, all of which share the common objective of being able to automatically learn features from (typically massive) data bases and to generalize their responses to circumstances not encountered during the learning phase. Ultimately, the learned features can be used for tasks such as classifying the types of signals the CNN is expected to process. The purpose of this "Lecture Notes" article is twofold: 1) to introduce the fundamental architecture of CNNs and 2) to illustrate, via a computational example, how CNNs are trained and used in practice to solve a specific class of problems.

Target encoding plays a central role when learning Convolutional Neural Networks. In this realm, One-hot encoding is the most prevalent strategy due to its simplicity. However, this so widespread encoding schema assumes a flat label space, thus ignoring rich relationships existing among labels that can be exploited during training. In large-scale datasets, data does not span the full label space, but instead lies in a low-dimensional output manifold. Following this observation, we embed the targets into a lowdimensional space, drastically improving convergence speed while preserving accuracy. Our contribution is two fold: (i) We show that random projections of the label space are a valid tool to find such lower dimensional embeddings, boosting dramatically convergence rates at zero computational cost; and (ii) we propose a normalized eigenrepresentation of the class manifold that encodes the targets with minimal information loss, improving the accuracy of random projections encoding while enjoying the same convergence rates. Experiments on CIFAR-100, CUB200-2011, Imagenet, and MIT Places demonstrate that the proposed approach drastically improves convergence speed while reaching very competitive accuracy rates.

This paper presents a recurrent neural circuit for solving quadratic programming problems. The objective is tominimize a quadratic cost function subject to linearconstraints. The proposed circuit employs non-linearfeedback, in the form of unipolar comparators, to introducetranscendental terms in the energy function ensuring fastconvergence to the solution. The proof of validity of the energy function is also provided. The hardware complexity of the proposed circuit comparesfavorably with other proposed circuits for the same task. PSPICE simulation results arepresented for a chosen optimization problem and are foundto agree with the algebraic solution.

This study characterizes the NVIDIA Jetson TK1 and TX1 Platforms, both built on a NVIDIA Tegra System on Chip and combining a quad-core ARM CPU and an NVIDIA GPU. Their heterogeneous nature, as well as their wide operating frequency range, make it hard for application developers to reason about performance and determine which optimizations are worth pursuing. This paper attempts to inform developers’ choices by characterizing the platforms’ performance using Roofline models obtained through an empirical measurement-based approach as well as through a case study of a heterogeneous application (matrix multiplication). Our results highlight a difference of more than an order of magnitude in compute performance between the CPU and GPU on both platforms. Given that the CPU and GPU share the same memory bus, their Roofline models’ balance points are also more than an order of magnitude apart. We also explore the impact of frequency scaling: build CPU and GPU Roofline profiles and characterize both platforms’ balance point variation, power consumption, and performance per watt as frequency is scaled. The characterization we provide can be used in two main ways. First, given an application, it can inform the choice and number of processing elements to use (i.e., CPU/GPU and number of cores) as well as the optimizations likely to lead to high performance gains. Secondly, this characterization indicates that developers can use frequency scaling to tune the Jetson Platform to suit the requirements of their applications. Third, given a required power/performance budget, application developers can identify the appropriate parameters to use to tune the Jetson platforms to their specific workload requirements. We expect that this optimization approach can lead to overall gains in performance and/or power efficiency without requiring application changes.

Applications with large amounts of data, real-time constraints, ultra-low power requirements, and heavy computational complexity present significant challenges for modern computing systems, and often fall within the category of high performance computing (HPC). As such, computer architects have looked to high performance single instruction multiple data (SIMD) architectures, such as accelerator-rich platforms, for handling these workloads. However, since the results of these applications do not always require exact precision, approximate computing may also be leveraged. In this work, we introduce BRAINIAC, a heterogeneous platform that combines precise accelerators with neural-network-based approximate accelerators. These reconfigurable accelerators are leveraged in a multi-stage flow that begins with simple approximations and resorts to more complex ones as needed. We employ high-level, application-specific light-weight checks (LWCs) to throttle this multi-stage acceleration flow and reliably ensure user-specified accuracy at runtime. Evaluation of the performance and energy of our heterogeneous platform for error tolerance thresholds of 5%-25% demonstrates an average of 3× gain over computation that only includes precise acceleration, and 15×-35× gain over software-based computation.

We address the problem of serving Deep Neural Networks (DNNs) efficiently from a cluster of GPUs. In order to realize the promise of very low-cost processing made by accelerators such as GPUs, it is essential to run them at sustained high utilization. Doing so requires cluster-scale resource management that performs detailed scheduling of GPUs, reasoning about groups of DNN invocations that need to be co-scheduled, and moving from the conventional whole-DNN execution model to executing fragments of DNNs. Nexus is a fully implemented system that includes these innovations. On large-scale case studies on 16 GPUs, Nexus shows 1.8-41× better throughput than state-of-the-art systems while staying within latency constraints >99% of the time.

Training deep neural networks (DNNs) requires an increasing amount of computation resources, and it becomes typical to use a mixture of GPU and CPU devices. Due to the heterogeneity of these devices, a recent challenge is how each operation in a neural network can be optimally placed on these devices, so that the training process can take the shortest amount of time possible. The current state-of-the-art solution uses reinforcement learning based on the policy gradient method, and it suffers from suboptimal training times. In this paper, we propose Spotlight, a new reinforcement learning algorithm based on proximal policy optimization, designed specifically for finding an optimal device placement for training DNNs. The design of our new algorithm relies upon a new model of the device placement problem: by modeling it as a Markov decision process with multiple stages, we are able to prove that Spotlight achieves a theoretical guarantee on performance improvements. We have implemented Spotlight in the CIFAR-10 benchmark and deployed it on the Google Cloud platform. Extensive experiments have demonstrated that the training time with placements recommended by Spotlight is 60.9% of that recommended by the policy gradient method.

Convolutional networks are one of the most widely employed architectures in computer vision and machine learning. In order to leverage their ability to learn complex functions, large amounts of data are required for training. Training a large convolutional network to produce state-of-the-art results can take weeks, even when using modern GPUs. Producing labels using a trained network can also be costly when dealing with web-scale datasets. In this work, we present a simple algorithm which accelerates training and inference by a significant factor, and can yield improvements of over an order of magnitude compared to existing state-of-the-art implementations. This is done by computing convolutions as pointwise products in the Fourier domain while reusing the same transformed feature map many times. The algorithm is implemented on a GPU architecture and addresses a number of related challenges.

Convolutional neural networks (CNNs) are usually built by stacking convolutional operations layer-by-layer. Although CNN has shown strong capability to extract semantics from raw pixels, its capacity to capture spatial relationships of pixels across rows and columns of an image is not fully explored. These relationships are important to learn semantic objects with strong shape priors but weak appearance coherences, such as traffic lanes, which are often occluded or not even painted on the road surface as shown in Fig. 1 (a). In this paper, we propose Spatial CNN (SCNN), which generalizes traditional deep layer-by-layer convolutions to slice-byslice convolutions within feature maps, thus enabling message passings between pixels across rows and columns in a layer. Such SCNN is particular suitable for long continuous shape structure or large objects, with strong spatial relationship but less appearance clues, such as traffic lanes, poles, and wall. We apply SCNN on a newly released very challenging traffic lane detection dataset and Cityscapse dataset. The results show that SCNN could learn the spatial relationship for structure output and significantly improves the performance. We show that SCNN outperforms the recurrent neural network (RNN) based ReNet and MRF+CNN (MRFNet) in the lane detection dataset by 8.7% and 4.6% respectively. Moreover, our SCNN won the 1st place on the TuSimple Benchmark Lane Detection Challenge, with an accuracy of 96.53%.

Many studies have shown that Deep Convolutional Neural Networks (DCNNs) exhibit great accuracies given large training datasets in image recognition tasks. Optimization technique known as asynchronous mini-batch Stochastic Gradient Descent (SGD) is widely used for deep learning because it gives fast training speed and good recognition accuracies, while it may increases generalization error if training parameters are in inappropriate ranges. We propose a performance model of a distributed DCNN training system called SPRINT that uses asynchronous GPU processing based on mini-batch SGD. The model considers the probability distribution of mini-batch size and gradient staleness that are the core parameters of asynchronous SGD training. Our performance model takes DCNN architecture and machine specifications as input parameters, and predicts time to sweep entire dataset, mini-batch size and staleness with 5%, 9% and 19% error in average respectively on several supercomputers with up to thousands of GPUs. Experimental results on two different supercomputers show that our model can steadily choose the fastest machine configuration that nearly meets a target mini-batch size.


Question: Eyeriss: a spatially-organized architecture for energy-efficient dataflow in convolutional neural networks.

This paper delves into the implications of restricted numerical precision on the training of deep neural networks, a process often hindered by limitations in computational resources. Our investigation centers on the impact of utilizing reduced precision data formats and calculations during the training phase. Specifically, within the framework of low-precision fixed-point arithmetic, we find that the rounding method is a critical determinant of network performance throughout training. Our findings reveal that deep networks are capable of being trained using a 16-bit fixed-point representation when paired with stochastic rounding, achieving comparable classification accuracy with minimal to no loss. Additionally, we present an energy-efficient hardware accelerator designed to execute low-precision fixed-point arithmetic operations with stochastic rounding, showcasing the practicality of our approach in enhancing computational efficiency while maintaining accuracy.

Although there is a huge progress in complementary-metal-oxide-semiconductor (CMOS) technology, construction of an artificial neural network using CMOS technology to realize the functionality comparable with that of human cerebral cortex containing 1010–1011 neurons is still of great challenge. Recently, phase change memristor neuron has been proposed to realize a human-brain level neural network operating at a high speed while consuming a small amount of power and having a high integration density. Although memristor neuron can be scaled down to nanometer, integration of 1010–1011 neurons still faces many problems in circuit complexity, chip area, power consumption, etc. In this work, we propose a CMOS compatible HfO2 memristor neuron that can be well integrated with silicon circuits. A hybrid Convolutional Neural Network (CNN) based on the HfO2 memristor neuron is proposed and constructed. In the hybrid CNN, one memristive neuron can behave as multiple physical neurons based on the Time Division Multiplexing Access (TDMA) technique. Handwritten digit recognition is demonstrated in the hybrid CNN with a memristive neuron acting as 784 physical neurons. This work paves the way towards substantially shrinking the amount of neurons required in hardware and realization of more complex or even human cerebral cortex level memristive neural networks.

Processing-In-Memory (PIM) is an effective technique that reduces data movements by integrating processing units within memory. The recent advance of “big data” and 3D stacking technology make PIM a practical and viable solution for the modern data processing workloads. It is exemplified by the recent research interests on PIM-based acceleration. Among them, TESSERACT is a PIM-enabled parallel graph processing architecture based on Micron’s Hybrid Memory Cube (HMC), one of the most prominent 3D-stacked memory technologies. It implements a Pregel-like vertex-centric programming model, so that users could develop programs in the familiar interface while taking advantage of PIM. Despite the orders of magnitude speedup compared to DRAM-based systems, TESSERACT generates excessive crosscube communications through SerDes links, whose bandwidth is much less than the aggregated local bandwidth of HMCs. Our investigation indicates that this is because of the restricted data organization required by the vertex programming model. In this paper, we argue that a PIM-based graph processing system should take data organization as a first-order design consideration. Following this principle, we propose GraphP, a novel HMC-based software/hardware co-designed graph processing system that drastically reduces communication and energy consumption compared to TESSERACT. GraphP features three key techniques. 1) “Source-cut” partitioning, which fundamentally changes the cross-cube communication from one remote put per cross-cube edge to one update per replica. 2) “Two-phase Vertex Program”, a programming model designed for the “source-cut” partitioning with two operations: GenUpdate and ApplyUpdate. 3) Hierarchical communication and overlapping, which further improves performance with unique opportunities offered by the proposed partitioning and programming model. We evaluate GraphP using a cycle accurate simulator with 5 real-world graphs and 4 algorithms. The results show that it provides on average 1.7 speedup and 89% energy saving compared to TESSERACT.

We evaluated the effectiveness in classifying food images of a deep-learning approach based on the specifications of Google's image recognition architecture Inception. The architecture is a deep convolutional neural network (DCNN) having a depth of 54 layers. In this study, we fine-tuned this architecture for classifying food images from three well-known food image datasets: ETH Food-101, UEC FOOD 100, and UEC FOOD 256. On these datasets we achieved, respectively, 88.28%, 81.45%, and 76.17% as top-1 accuracy and 96.88%, 97.27%, and 92.58% as top-5 accuracy. To the best of our knowledge, these results significantly improve the best published results obtained on the same datasets, while requiring less computation power, since the number of parameters and the computational complexity are much smaller than the competitors?. Because of this, even if it is still rather large, the deep network based on this architecture appears to be at least closer to the requirements for mobile systems.

The problem of quantizing the activations of a deep neural network is considered. An examination of the popular binary quantization approach shows that this consists of approximating a classical non-linearity, the hyperbolic tangent, by two functions: a piecewise constant sign function, which is used in feedforward network computations, and a piecewise linear hard tanh function, used in the backpropagation step during network learning. The problem of approximating the widely used ReLU non-linearity is then considered. An half-wave Gaussian quantizer (HWGQ) is proposed for forward approximation and shown to have efficient implementation, by exploiting the statistics of of network activations and batch normalization operations. To overcome the problem of gradient mismatch, due to the use of different forward and backward approximations, several piece-wise backward approximators are then investigated. The implementation of the resulting quantized network, denoted as HWGQ-Net, is shown to achieve much closer performance to full precision networks, such as AlexNet, ResNet, GoogLeNet and VGG-Net, than previously available low-precision networks, with 1-bit binary weights and 2-bit quantized activations.

The massive parallelism of graphics processing units (GPUs) offers tremendous performance in many high-performance computing applications. While dense linear algebra readily maps to such platforms, harnessing this potential for sparse matrix computations presents additional challenges. Given its role in iterative methods for solving sparse linear systems and eigenvalue problems, sparse matrix-vector multiplication (SpMV) is of singular importance in sparse linear algebra. In this paper we discuss data structures and algorithms for SpMV that are efficiently implemented on the CUDA platform for the fine-grained parallel architecture of the GPU. Given the memory-bound nature of SpMV, we emphasize memory bandwidth efficiency and compact storage formats. We consider a broad spectrum of sparse matrices, from those that are well-structured and regular to highly irregular matrices with large imbalances in the distribution of nonzeros per matrix row. We develop methods to exploit several common forms of matrix structure while offering alternatives which accommodate greater irregularity. On structured, grid-based matrices we achieve performance of 36 GFLOP/s in single precision and 16 GFLOP/s in double precision on a GeForce GTX 280 GPU. For unstructured finite-element matrices, we observe performance in excess of 15 GFLOP/s and 10 GFLOP/s in single and double precision respectively. These results compare favorably to prior state-of-the-art studies of SpMV methods on conventional multicore processors. Our double precision SpMV performance is generally two and a half times that of a Cell BE with 8 SPEs and more than ten times greater than that of a quad-core Intel Clovertown system.

Tiled architectures provide a paradigm for designers to turn silicon resources into processors with burgeoning quantities of programmable functional units and memories. The architecture has a dual responsibility: first, it must expose these resources in a way that is programmable. Second, it needs to manage the power associated with such resources.We present the power management facilities of the 16-tile Raw microprocessor. This design selectively turns on and off 48 SRAM macros, 96 functional unit clusters, 32 fetch units, and over 250 unique processor pipeline stages, all according to the needs of the computation and environment at hand.

Deep Machine Learning (DML) algorithms have proven to be highly successful at challenging, high-dimensional learning problems, but their widespread deployment is limited by their heavy computational requirements and the associated power consumption. Analog computational circuits offer the potential for large improvements in power efficiency, but noise, mismatch, and other effects cause deviations from ideal computations. In this paper we describe circuits useful for DML algorithms, including a tunable-width bump circuit and a configurable distance calculator. We also discuss the impacts of computational errors on learning performance. Finally we will describe a complete deep learning engine implemented using current-mode analog circuits and compare its performance to digital equivalents.

This paper proposes an 8T SRAMs utilizing a column-based data encoding scheme to reduce read and write power when there are similarities between consecutive data. It is useful in image processing applications where nearby pixels tend to have similar values. The proposed design has two modes of operation: normal and sequential modes. In the normal mode, it operates as a normal SRAM. In the sequential mode, bit-wise differences between consecutive data are written instead of the actual data. This leads to a much higher number of zeros in the array. Accordingly, a new data-aware bitline pre-charge scheme is proposed to minimize write power when writing a zero. A PVT-tracking reference voltage generator is also employed to compensate the read-bitline leakage for ultra-low voltage operation. On average, it offers more than 30% WBL switching power reduction. A 32Kb SRAM implemented in 65 nm CMOS process demonstrated successful operation down to 0.36 V. The total power consumption is 0.37 μW, corresponding to the maximum frequency of 0.25 MHz. Its minimum energy is 0.3 pJ/access achieved at 0.5 V.

Existing research on accelerators has emphasized the performance and energy e ciency improvements they can provide, devoting little attention to practical issues such as accelerator invocation and interaction with other on-chip components (e.g. cores, caches). In this paper we present a quantitative study that considers these aspects by implementing seven high-throughput accelerators following three design models: tight coupling behind a CPU, loose out-of-core coupling with Direct Memory Access (DMA) to the LLC, and loose out-of-core coupling with DMA to DRAM. A salient conclusion of our study is that working sets of non-trivial size are best served by loosely-coupled accelerators that integrate private memory blocks tailored to their needs.

Modern deep neural networks have a large number of parameters, making them very hard to train. We propose DSD, a dense-sparse-dense training flow, for regularizing deep neural networks and achieving better optimization performance. In the first D (Dense) step, we train a dense network to learn connection weights and importance. In the S (Sparse) step, we regularize the network by pruning the unimportant connections with small weights and retraining the network given the sparsity constraint. In the final D (re-Dense) step, we increase the model capacity by removing the sparsity constraint, re-initialize the pruned parameters from zero and retrain the whole dense network. Experiments show that DSD training can improve the performance for a wide range of CNNs, RNNs and LSTMs on the tasks of image classification, caption generation and speech recognition. On ImageNet, DSD improved the Top1 accuracy of GoogLeNet by 1.1%, VGG-16 by 4.3%, ResNet-18 by 1.2% and ResNet-50 by 1.1%, respectively. On the WSJ’93 dataset, DSD improved DeepSpeech and DeepSpeech2 WER by 2.0% and 1.1%. On the Flickr-8K dataset, DSD improved the NeuralTalk BLEU score by over 1.7. DSD is easy to use in practice: at training time, DSD incurs only one extra hyper-parameter: the sparsity ratio in the S step. At testing time, DSD doesn’t change the network architecture or incur any inference overhead. The consistent and significant performance gain of DSD experiments shows the inadequacy of the current training methods for finding the best local optimum, while DSD effectively achieves superior optimization performance for finding a better solution. DSD models are available to download at https://songhan.github.io/DSD.