Question: Utilizing in-home sensors and machine learning methods for the health monitoring and behavioral pattern analysis of individuals with dementia.

This comprehensive review explores the evolution and current landscape of ambient-assisted living (AAL) solutions designed for the elderly, driven by the pressing demands of an aging population and escalating healthcare costs. The paper highlights the pivotal role of AAL technologies in enabling safe and autonomous living for seniors, addressing the caregiver burden while respecting their desire for independence. Framed within the ambient intelligence framework, the review consolidates advancements in AAL tools, technologies, and methodologies, offering a detailed analysis of the state-of-the-art. Moreover, it delves into the existing and anticipated challenges that must be overcome to fully harness the potential of AAL systems in enhancing the quality of life for older adults. The review serves as a valuable resource for researchers, developers, and policymakers seeking to innovate and improve AAL solutions for the aging society.

Activity recognition fits within the bigger framework of context awareness. In this paper, we report on our efforts to recognize user activity from accelerometer data. Activity recognition is formulated as a classification problem. Performance of base-level classifiers and meta-level classifiers is compared. Plurality Voting is found to perform consistently well across different settings.

Knowledge of a person’s whereabouts in the home is key to context-aware applications, but many people do not want to carry or wear a tag or mobile device in the home. Therefore, many tracking systems are now using so-called weak biometrics such as height, weight, and width. In this paper, we propose to use body shape as a weak biometric, differentiating people based on features such as head size, shoulder size, or torso size. The basic idea is to scan the body with a radar sensor and to compute the reflection profile: the amount of energy that reflects back from each part of the body. Many people have different body shapes even if they have the same height, weight, or width, which makes body shape a stronger biometric. We built a proof-of-concept system called FormaTrack to test this approach, and evaluate using eight participants of varying height and weight. We collected over 2800 observations while capturing a wide range of factors such as clothing, hats, shoes, and backpacks. Results show that FormaTrack can achieve a precision, recall, direction and identity accuracy (over all possible groups of 2 people) of 100%, 99.86%, 99.7% and 95.3% respectively. Results indicate that FormaTrack can achieve over 99% tracking accuracy with 2 people in a home with 5 or more rooms.

Sensor networks have the potential to greatly impact many aspects of medical care. By outfitting patients with wireless, wearable vital sign sensors, collecting detailed real-time data on physiological status can be greatly simplified. However, there is a significant gap between existing sensor network systems and the needs of medical care. In particular, medical sensor networks must support multicast routing topologies, node mobility, a wide range of data rates and high degrees of reliability, and security. This paper describes our experiences with developing a combined hardware and software platform for medical sensor networks, called CodeBlue. CodeBlue provides protocols for device discovery and publish/subscribe multihop routing, as well as a simple query interface that is tailored for medical monitoring. We have developed several medical sensors based on the popular MicaZ and Telos mote designs, including a pulse oximeter, EKG and motion-activity sensor. We also describe a new, miniaturized sensor mote designed for medical use. We present initial results for the CodeBlue prototype demonstrating the integration of our medical sensors with the publish/subscribe routing substrate. We have experimentally validated the prototype on our 30-node sensor network testbed, demonstrating its scalability and robustness as the number of simultaneous queries, data rates, and transmitting sensors are varied. We also study the effect of node mobility, fairness across multiple simultaneous paths, and patterns of packet loss, confirming the system’s ability to maintain stable routes despite variations in node location and

In this paper, we propose an acceleration-based human activity recognition method using popular deep architecture, Convolution Neural Network (CNN). In particular, we construct a CNN model and modify the convolution kernel to adapt the characteristics of tri-axial acceleration signals. Also, for comparison, we use some widely used methods to accomplish the recognition task on the same dataset. The large dataset we constructed consists of 31688 samples from eight typical activities. The experiment results show that the CNN works well, which can reach an average accuracy of 93.8% without any feature extraction methods.

Driven by the growing aging population, prevalence of chronic diseases, and continuously rising healthcare costs, the healthcare system is undergoing a fundamental transformation, from the conventional hospital-centered system to an individual-centered system. Current and emerging developments in wearable medical systems will have a radical impact on this paradigm shift. Advances in wearable medical systems will enable the accessibility and affordability of healthcare, so that physiological conditions can be monitored not only at sporadic snapshots but also continuously for extended periods of time, making early disease detection and timely response to health threats possible. This paper reviews recent developments in the area of wearable medical systems for p-Health. Enabling technologies for continuous and noninvasive measurements of vital signs and biochemical variables, advances in intelligent biomedical clothing and body area networks, approaches for motion artifact reduction, strategies for wearable energy harvesting, and the establishment of standard protocols for the evaluation of wearable medical devices are presented in this paper with examples of clinical applications of these technologies.

In this paper we present a novel type of sensing technology for hand and finger gesture recognition that utilizes light in the invisible spectrum to detect changes in position and form of body tissue like tendons and muscles. The proposed system can be easily integrated with existing wearable devices. Our approach not only enables gesture recognition but it could potentially double to perform a variety of health related monitoring tasks (e.g. heart rate, stress).

With the proliferation of Internet of Things (IoT) devices such as smartphones, sensors, cameras, and RFIDs, it is possible to collect massive amount of data for localization and tracking of people within commercial buildings. Enabled by such occupancy monitoring capabilities, there are extensive opportunities for improving the energy consumption of buildings via smart HVAC control. In this respect, the major challenges we envision are 1) to achieve occupancy monitoring in a minimally intrusive way, e.g., using the existing infrastructure in the buildings and not requiring installation of any apps in the users' smart devices, and 2) to develop effective data fusion techniques for improving occupancy monitoring accuracy using a multitude of sources. This paper surveys the existing works on occupancy monitoring and multi-modal data fusion techniques for smart commercial buildings. The goal is to lay down a framework for future research to exploit the spatio-temporal data obtained from one or more of various IoT devices such as temperature sensors, surveillance cameras, and RFID tags that may be already in use in the buildings. A comparative analysis of existing approaches and future predictions for research challenges are also provided.

Human sensing enables intelligent vehicles to provide driver-adaptive support by classifying perceived workload into multiple levels. Objective of this study is to classify driver workload associated with traffic complexity into five levels. We conducted driving experiments in systematically varied traffic complexity levels in a simulator. We recorded driver physiological signals including electrocardiography, electrodermal activity, and electroencephalography. In addition, we integrated driver performance and subjective workload measures. Deep learning based models outperform statistical machine learning methods when dealing with dynamic time-series data with variable sequence lengths. We show that our long short-term memory based recurrent neural network model can classify driver perceived-workload into five classes with an accuracy of 74.5%. Since perceived workload differ between individual drivers for the same traffic situation, our results further highlight the significance of including driver characteristics such as driving style and workload sensitivity to achieve higher classification accuracy.

PURPOSE
Large physical activity surveillance projects such as the UK Biobank and NHANES are using wrist-worn accelerometer-based activity monitors that collect raw data. The goal is to increase wear time by asking subjects to wear the monitors on the wrist instead of the hip, and then to use information in the raw signal to improve activity type and intensity estimation. The purposes of this work was to obtain an algorithm to process wrist and ankle raw data and to classify behavior into four broad activity classes: ambulation, cycling, sedentary, and other activities.

METHODS
Participants (N = 33) wearing accelerometers on the wrist and ankle performed 26 daily activities. The accelerometer data were collected, cleaned, and preprocessed to extract features that characterize 2-, 4-, and 12.8-s data windows. Feature vectors encoding information about frequency and intensity of motion extracted from analysis of the raw signal were used with a support vector machine classifier to identify a subject's activity. Results were compared with categories classified by a human observer. Algorithms were validated using a leave-one-subject-out strategy. The computational complexity of each processing step was also evaluated.

RESULTS
With 12.8-s windows, the proposed strategy showed high classification accuracies for ankle data (95.0%) that decreased to 84.7% for wrist data. Shorter (4 s) windows only minimally decreased performances of the algorithm on the wrist to 84.2%.

CONCLUSIONS
A classification algorithm using 13 features shows good classification into the four classes given the complexity of the activities in the original data set. The algorithm is computationally efficient and could be implemented in real time on mobile devices with only 4-s latency.

Smartphones equipped with sensors can be used for monitoring simple daily activities (e.g., walking, climbing stairs etc.). The data collected via sensors need to be partially or completely annotated in order to be used in the training phase of learning algorithms. In order to ease the time consuming and burdensome annotation process, a mobile data collection and annotation application named Sensor Log is developed. The annotation feature and its ease of use make the developed application superior to existing systems. The application can be downloaded from the Android applications distribution platform.


Question: Utilizing in-home sensors and machine learning methods for the health monitoring and behavioral pattern analysis of individuals with dementia.

This study investigates the daily activity patterns and identifies a variety of behavioral profiles among 22 senior citizens residing in an assisted living facility, focusing on four illustrative cases. The observed behavioral patterns were derived through a pilot study utilizing an advanced software solution that employs statistical predictive algorithms to model circadian activity rhythms (CARs) and their variations. The CAR model was statistically derived from the average duration a resident spent in each room of their living space, as well as the average number of motion events recorded in each room. An established in-home monitoring system (IMS), validated for reliability, captured the residents' movement data to determine the occupancy duration and activity intensity in each room. From this dataset, the circadian behaviors of the residents were extracted, and deviations indicating potential anomalies were identified. These deviations were cross-referenced with activity reports generated by the IMS and corroborated by the observations of professional caregivers at the facility. The system demonstrated its utility in promptly detecting changes in activity patterns, which could signal alterations in health status. This capability allows caregivers to implement standard diagnostic procedures and intervene in a timely and proactive manner, potentially improving the health outcomes of the residents.

The approach taken in this study to monitor and analyze the daily living activities of older adults in an assisted living environment could be extended to health management for individuals with dementia. By integrating in-home sensors and machine learning techniques, the system can continuously monitor the residents' behavioral patterns and detect early signs of health issues or changes in cognitive function. This proactive monitoring could aid in the early intervention and management of dementia, enhancing the quality of life for affected individuals and providing caregivers with critical insights into their health status.

BACKGROUND
Physical and psychological symptoms are common during chemotherapy in cancer patients, and real-time monitoring of these symptoms can improve patient outcomes. Sensors embedded in mobile phones and wearable activity trackers could be potentially useful in monitoring symptoms passively, with minimal patient burden.

OBJECTIVE
The aim of this study was to explore whether passively sensed mobile phone and Fitbit data could be used to estimate daily symptom burden during chemotherapy.

METHODS
A total of 14 patients undergoing chemotherapy for gastrointestinal cancer participated in the 4-week study. Participants carried an Android phone and wore a Fitbit device for the duration of the study and also completed daily severity ratings of 12 common symptoms. Symptom severity ratings were summed to create a total symptom burden score for each day, and ratings were centered on individual patient means and categorized into low, average, and high symptom burden days. Day-level features were extracted from raw mobile phone sensor and Fitbit data and included features reflecting mobility and activity, sleep, phone usage (eg, duration of interaction with phone and apps), and communication (eg, number of incoming and outgoing calls and messages). We used a rotation random forests classifier with cross-validation and resampling with replacement to evaluate population and individual model performance and correlation-based feature subset selection to select nonredundant features with the best predictive ability.

RESULTS
Across 295 days of data with both symptom and sensor data, a number of mobile phone and Fitbit features were correlated with patient-reported symptom burden scores. We achieved an accuracy of 88.1% for our population model. The subset of features with the best accuracy included sedentary behavior as the most frequent activity, fewer minutes in light physical activity, less variable and average acceleration of the phone, and longer screen-on time and interactions with apps on the phone. Mobile phone features had better predictive ability than Fitbit features. Accuracy of individual models ranged from 78.1% to 100% (mean 88.4%), and subsets of relevant features varied across participants.

CONCLUSIONS
Passive sensor data, including mobile phone accelerometer and usage and Fitbit-assessed activity and sleep, were related to daily symptom burden during chemotherapy. These findings highlight opportunities for long-term monitoring of cancer patients during chemotherapy with minimal patient burden as well as real-time adaptive interventions aimed at early management of worsening or severe symptoms.

The promising potential of the emerging Internet of Things (IoT) technologies for interconnected medical devices and sensors has played an important role in the next-generation healthcare industry for quality patient care. Because of the increasing number of elderly and disabled people, there is an urgent need for a real-time health monitoring infrastructure for analyzing patients’ healthcare data to avoid preventable deaths. Healthcare Industrial IoT (HealthIIoT) has significant potential for the realization of such monitoring. HealthIIoT is a combination of communication technologies, interconnected apps, Things (devices and sensors), and people that would function together as one smart system to monitor, track, and store patients’ healthcare information for ongoing care. This paper presents a HealthIIoT-enabled monitoring framework, where ECG and other healthcare data are collected by mobile devices and sensors and securely sent to the cloud for seamless access by healthcare professionals. Signal enhancement, watermarking, and other related analytics will be used to avoid identity theft or clinical error by healthcare professionals. The suitability of this approach has been validated through both experimental evaluation, and simulation by deploying an IoT-driven ECG-based health monitoring service in the cloud. © 2016 Elsevier B.V. All rights reserved.

Nowadays smartphones are ubiquitous in various aspects of our lives. The processing power, communication bandwidth, and the memory capacity of these devices have surged considerably in recent years. Besides, the variety of sensor types, such as accelerometer, gyroscope, humidity sensor, and bio-sensors, which are embedded in these devices, opens a new horizon in self-monitoring of physical daily activities. One of the primary steps for any research in the area of detecting daily life activities is to test a detection method on benchmark datasets. Most of the early datasets limited their work to collecting only a single type of sensor data such as accelerometer data. While some others do not consider age, weight, and gender of the subjects who have participated in collecting their activity data. Finally, part of the previous works collected data without considering the smartphone's position. In this paper, we introduce a new dataset, called Position-Aware Multi-Sensor (PAMS). The dataset contains both accelerometer and gyroscope data. The gyroscope data boosts the accuracy of activity recognition methods as well as enabling them to detect a wider range of activities. We also take the user information into account. Based on the biometric attributes of the participants, a separate learned model is generated to analyze their activities. We concentrate on several major activities, including sitting, standing, walking, running, ascending/descending stairs, and cycling. To evaluate the dataset, we use various classifiers, and the outputs are compared to the WISDM. The results show that using aforementioned classifiers, the average precision for all activities is above 88.5%. Besides, we measure the CPU, memory, and bandwidth usage of the application collecting data on the smartphone.

Personal instrumentation and monitoring services that collect and archive the physical activities of a user have recently been introduced for various medical, personal, safety, and entertainment purposes. A general software architecture is needed to support different categories of such monitoring services. This paper presents a software architecture, implementation, and preliminary evaluation of SATIRE, a wearable personal monitoring service transparently embedded in user garments. SATIRE records the owner's activity and location for subsequent automated uploading and archiving. The personal archive can later be searched for particular events to answer questions regarding past and present user activity, location, and behavior patterns. A short feasibility and usage study of a prototype based on MicaZ motes provides a proof of concept for the SATIRE architecture.

A significant fraction of college students suffer from serious mental health issues including depression, anxiety, self-harm and suicidal thought. The prevalence and severity of these issues among college students also appear to increase over time. However, most of these issues often remain undiagnosed, and as a result, untreated. One of the main reasons of this gap between illness and treatment results from the lack of reliable data over time. While health care services in college campuses have been focusing on detection of illness onset and appropriate interventions, their tools are mostly manual surveys which often fail to capture the granular details of contexts and behaviors which might provide important cues about illness onset. To overcome the limitations of these manual tools, we deployed a smartphone based tool or unobtrusive and continuous data collection from 22 students during an academic semester. In this paper, we present the preliminary findings from our study about assessing mental health on college campuses using passively sensed smartphone data.

earable biosensors (WBS) will permit continuous cardiovascular (CV) monitoring in a number of novel settings. Benefits may be realized in the diagnosis and treatment of a number of major diseases. WBS, in conjunction with appropriate alarm algorithms, can increase surveillance capabilities for CV catastrophe for high-risk subjects. WBS could also play a role in the treatment of chronic diseases, by providing information that enables precise titration of therapy or detecting lapses in patient compliance. WBS could play an important role in the wireless surveillance of people during hazardous operations (military, fire-fighting, etc.), or such sensors could be dispensed during a mass civilian casualty occurrence. Given that CV physio-logic parameters make up the " vital signs " that are the most important information in emergency medical situations, WBS might enable a wireless monitoring system for large numbers of at-risk subjects. This same approach may also have utility in monitoring the waiting room of today's overcrowded emergency departments. For hospital inpatients who require CV monitoring, current biosensor technology typically tethers patients in a tangle of cables, whereas wearable CV sensors could increase inpatient comfort and may even reduce the risk of tripping and falling, a perennial problem for hospital patients who are ill, medicated, and in an unfamiliar setting. On a daily basis, wearable CV sensors could detect a missed dose of medication by sensing untreated elevated blood pressure and could trigger an automated reminder for the patient to take the medication. Moreover, it is important for doctors to titrate the treatment of high blood pressure, since both insufficient therapy as well as excessive therapy (leading to abnormally low blood pressures) increase mortality. However, healthcare providers have only intermittent values of blood pressure on which to base therapy decisions; it is possible that continuous blood pressure monitoring would permit enhanced titration of therapy and reductions in mortality. Similarly, WBS would be able to log the physiologic signature of a patient's exercise efforts (manifested as changes in heart rate and blood pressure), permitting the patient and healthcare provider to assess compliance with a regimen proven to improve health outcomes. For patients with chronic cardiovascular disease, such as heart failure, home monitoring employing WBS may detect exacerbations in very early (and often easily treated) stages, long before the patient progresses to more dangerous levels that necessitate an emergency room visit and costly hospital admission. In this article we will address both technical and clinical …

In this fast pace of life, it is difficult for people to be constantly available for their near ones who might need them while they are suffering from a disease or physical disorder. So also constant monitoring of the patient’s body parameters such as temperature, pulse rate, sugar level etc. becomes difficult. Hence to remove human error and to lessen the burden of monitoring patient’s health from doctor’s head, this paper presents the methodology for monitoring patients remotely using GSM network and Very Large Scale Integration (VLSI) technology. Patient monitoring systems measure physiological characteristics either continuously or at regular intervals of time.

Ambient assisted living (AAL) delivers IT solutions that aim to facilitate and improve lives of the disabled, elderly, and chronically ill people. Mobility is a key issue for elderly people because their physical activity, in general, improves their quality of life and keep health status. Then, this paper presents an AAL framework for caregivers and elderly people that allow them to maintain an active lifestyle without limiting their mobility. This framework includes four AAL tools for mobility environments: i) a fall detection mobile application; ii) a biofeedback monitoring system trough wearable sensors; iii) an outdoor location service through a shoe equipped with Global Positioning System (GPS); and iv) a mobile application for caregivers that take care of several elders confined to a home environment. The proposal is evaluated and demonstrated and it is ready for use.

Integration of sensors into various kinds of products and machines provides access to in-depth usage information as basis for product optimization. Presently, this large potential for more user-friendly and efficient products is not being realized because (a) sensor integration and thus usage information is not available on a large scale and (b) product optimization requires considerable efforts in terms of manpower and adaptation of production equipment. However, with the advent of cloud-based services and highly flexible additive manufacturing techniques, these obstacles are currently crumbling away at rapid pace. The present study explores the state of the art in gathering and evaluating product usage and life cycle data, additive manufacturing and sensor integration, automated design and cloud-based services in manufacturing. By joining and extrapolating development trends in these areas, it delimits the foundations of a manufacturing concept that will allow continuous and economically viable product optimization on a general, user group or individual user level. This projection is checked against three different application scenarios, each of which stresses different aspects of the underlying holistic concept. The following discussion identifies critical issues and research needs by adopting the relevant stakeholder perspectives.

As sensor and actuator networks mature, they become a core utility of smart homes like electricity and water and enable the running of many CPS applications. Like other Cyber-Physical Systems (CPSs), when a number of applications share physical world entities, it raises many systems of systems interdependency problems. Such problems arise in the cyber part mainly because each application has assumptions on the physical world entities without knowing how other applications work. In this work, we propose DepSys, a utility sensing and actuation infrastructure for smart homes that provides comprehensive strategies to specify, detect, and resolve conflicts in a home setting. Based on real home data, we demonstrate the severity of conflicts when multiple CPSs are integrated and the significant ability of detecting and resolving such conflicts using DepSys.


Question: Utilizing in-home sensors and machine learning methods for the health monitoring and pattern recognition of daily activities in individuals with dementia.

This comprehensive survey explores the landscape of time series clustering, highlighting its utility across diverse fields. The paper offers an overview of the existing literature on clustering time series data, a critical component of temporal data mining. It delves into the foundational aspects of time series clustering, outlining the standard clustering algorithms typically employed in such studies. The evaluation criteria for assessing the effectiveness of clustering outcomes and the methodologies for measuring similarity or dissimilarity between time series, whether in their raw form, as extracted features, or through model parameters, are also discussed. Previous research efforts are categorized into three primary approaches: those that directly analyze raw data in either the time or frequency domain, those that work with features derived from the raw data, and those that utilize models constructed from the raw data. The unique contributions and limitations of these approaches are analyzed, and potential avenues for future research are identified. Additionally, the paper enumerates the application areas of time series clustering, encompassing the diverse sources of data employed in these studies. This review is intended to serve as a foundational resource for researchers aiming to contribute to the advancement of time series clustering techniques. 2005 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

However, to address the specific question regarding health management and pattern analysis of daily living activities of people with dementia using in-home sensors and machine learning techniques, the focus shifts to the application of time series clustering in healthcare, particularly for monitoring and understanding the behaviors of individuals with dementia. The use of in-home sensors to gather time series data on daily activities provides a rich dataset for pattern analysis. Machine learning techniques, including clustering algorithms, are then applied to this data to identify patterns and anomalies in the daily routines of dementia patients. This analysis can offer insights into the progression of the disease, the effectiveness of interventions, and the development of personalized care plans. The survey highlights studies that have successfully applied time series clustering to detect changes in behavior that could indicate a decline in health or an increased need for support. By organizing the research based on the type of data analyzed—raw sensor data, extracted features, or model parameters—the paper aids in understanding the most effective approaches for health monitoring in dementia care. The findings contribute to the broader goal of enhancing the quality of life for dementia patients through technology-assisted health management.

This paper presents ElectriSense, a new solution for automatically detecting and classifying the use of electronic devices in a home from a single point of sensing. ElectriSense relies on the fact that most modern consumer electronics and fluorescent lighting employ switch mode power supplies (SMPS) to achieve high efficiency. These power supplies continuously generate high frequency electromagnetic interference (EMI) during operation that propagates throughout a home's power wiring. We show both analytically and by in-home experimentation that EMI signals are stable and predictable based on the device's switching frequency characteristics. Unlike past transient noise-based solutions, this new approach provides the ability for EMI signatures to be applicable across homes while still being able to differentiate between similar devices in a home. We have evaluated our solution in seven homes, including one six-month deployment. Our results show that ElectriSense can identify and classify the usage of individual devices with a mean accuracy of 93.82%.

This work explores the feasibility of using sensors embedded in Google Glass, a head-mounted wearable device, to measure physiological signals of the wearer. In particular, we develop new methods to use Glass's accelerometer, gyroscope, and camera to extract pulse and respiratory rates of 12 participants during a controlled experiment. We show it is possible to achieve a mean absolute error of 0.83 beats per minute (STD: 2.02) for heart rate and 1.18 breaths per minute (STD: 2.04) for respiration rate when considering different combinations of sensors. These results included testing across sitting, supine, and standing still postures before and after physical exercise.

Although chronic diseases cannot be cured, they can be effectively controlled as long as we understand their progressions based on the current observational health records, which is often in the form of multimedia data. A large and growing body of literature has investigated the disease progression problem. However, far too little attention to date has been paid to jointly consider the following three observations of the chronic disease progression: 1) the health statuses at different time points are chronologically similar; 2) the future health statuses of each patient can be comprehensively revealed from the current multimedia and multimodal observations, such as visual scans, digital measurements and textual medical histories; and 3) the discriminative capabilities of different modalities vary significantly in accordance to specific diseases. In the light of these, we propose an adaptive multimodal multi-task learning model to co-regularize the modality agreement, temporal progression and discriminative capabilities of different modalities. We theoretically show that our proposed model is a linear system. Before training our model, we address the data missing problem via the matrix factorization approach. Extensive evaluations on a real-world Alzheimer's disease dataset well verify our proposed model. It should be noted that our model is also applicable to other chronic diseases.

The project PArADISE deals with activity and intention recognition in smart environments. This can be used in apartments, for example, to recognize falls of elderly people. While doing this, the privacy concerns of the user should be kept. To reach this goal, the processing of the data is done as close as possible at those sensors collecting the data. Only in cases where the processing is not possible on local nodes the data will be transferred to the cloud. But before transferring, it is checked against the privacy concerns using some measures for the anonymity of the data. If the data is not valid against these checks, some additional anonymizations will be done. This anonymization of data must be done quite carefully. Mistakes might cause the problem that data can be reassigned to persons and anonymized data might be reproduced. This paper gives an overview about recent methods for anonymizing data while showing their weaknesses. How these weaknesses can be used to invert the anonymization (called de-anonymization) is shown as well. Our attacks representing the de-anonymization should help to find weaknesses in methods used to anonymize data and how these can be eliminated. Zusammenfassung: Im Projekt PArADISE sollen Aktivitätsund Intentionserkennungen in smarten Systemen, etwa Assistenzsystemen in Wohnungen, so durchgeführt werden, dass Privatheitsanforderungen des Nutzers gewahrt bleiben. Dazu werden einerseits Auswertungen der Daten sehr nah an den Sensoren, die die Daten erzeugen, vorgenommen. Eine Übertragung von Daten in die Cloud findet nur im Notfall statt. Zusätzlich werden aber vor der Übertragung der nicht vorausgewerteten Daten in die Cloud diese auf Privatheitsanforderungen hin geprüft, indem Anonymisierungsmaße getestet und eventuell weitere Anonymisierungen von Daten vorgenommen werden. Diese Anonymisierung von Datenbeständen muss mit großer Sorgfalt geschehen. Fehler können sehr schnell dazu führen, dass anonymisierte Datenbestände wieder personalisiert werden können und Daten, die eigentlich entfernt wurden, wieder zurückgewonnen werden können. Dieser Artikel betrachtet aktuelle Verfahren zur Anonymisierung und zeigt Schwachstellen auf, die zu Problemen oder gar der Umkehrung der Methoden führen können. Unsere künstlich erzeugten Angriffe durch De-Anonymisierungen sollen helfen, Schwachstellen in den Anonymisierungsverfahren zu entdecken und zu beheben.

Household smart meters that measure power consumption in real-time at fine granularities are the foundation of a future smart electricity grid. However, the widespread deployment of smart meters has serious privacy implications since they inadvertently leak detailed information about household activities. In this paper, we show that even without a priori knowledge of household activities or prior training, it is possible to extract complex usage patterns from smart meter data using off-the-shelf statistical methods. Our analysis uses two months of data from three homes, which we instrumented to log aggregate household power consumption every second. With the data from our small-scale deployment, we demonstrate the potential for power consumption patterns to reveal a range of information, such as how many people are in the home, sleeping routines, eating routines, etc. We then sketch out the design of a privacy-enhancing smart meter architecture that allows an electric utility to achieve its net metering goals without compromising the privacy of its customers.

Activity recognition fits within the bigger framework of context awareness. In this paper, we report on our efforts to recognize user activity from accelerometer data. Activity recognition is formulated as a classification problem. Performance of base-level classifiers and meta-level classifiers is compared. Plurality Voting is found to perform consistently well across different settings.

Reliable human activity recognition with wearable devices enables the development of human-centric pervasive applications.We aim to develop a robust wearable-based activity recognition system for real life situations where the device position is up to the user or where a user is unable to collect initial training data. Consequently, in this work we focus on the problem of recognizing the on-body position of the wearable device ensued by comprehensive experiments concerning subject-specific and cross-subjects activity recognition approaches that rely on acceleration data. We introduce a device localization method that predicts the on-body position with an F-measure of 89% and a cross-subjects activity recognition approach that considers common physical characteristics. In this context, we present a real world data set that has been collected from 15 participants for 8 common activities where they carried 7 wearable devices in different on-body positions. Our results show that the detection of the device position consistently improves the result of activity recognition for common activities. Regarding cross-subjects models, we identified the waist as the most suitable device location at which the acceleration patterns for the same activity across several people are most similar. In this context, our results provide evidence for the reliability of physical characteristics based cross-subjectsmodels. © 2017 Elsevier B.V. All rights reserved.

PURPOSE
Large physical activity surveillance projects such as the UK Biobank and NHANES are using wrist-worn accelerometer-based activity monitors that collect raw data. The goal is to increase wear time by asking subjects to wear the monitors on the wrist instead of the hip, and then to use information in the raw signal to improve activity type and intensity estimation. The purposes of this work was to obtain an algorithm to process wrist and ankle raw data and to classify behavior into four broad activity classes: ambulation, cycling, sedentary, and other activities.

METHODS
Participants (N = 33) wearing accelerometers on the wrist and ankle performed 26 daily activities. The accelerometer data were collected, cleaned, and preprocessed to extract features that characterize 2-, 4-, and 12.8-s data windows. Feature vectors encoding information about frequency and intensity of motion extracted from analysis of the raw signal were used with a support vector machine classifier to identify a subject's activity. Results were compared with categories classified by a human observer. Algorithms were validated using a leave-one-subject-out strategy. The computational complexity of each processing step was also evaluated.

RESULTS
With 12.8-s windows, the proposed strategy showed high classification accuracies for ankle data (95.0%) that decreased to 84.7% for wrist data. Shorter (4 s) windows only minimally decreased performances of the algorithm on the wrist to 84.2%.

CONCLUSIONS
A classification algorithm using 13 features shows good classification into the four classes given the complexity of the activities in the original data set. The algorithm is computationally efficient and could be implemented in real time on mobile devices with only 4-s latency.

The average age growing of the urban population, with an increasing number of 65+ citizens, is calling for the cities to provide global services specifically geared to elderly people. In this context, collecting data from the elderly's environment and his/her habits and making them available in a structured way to third parties for analysis, is the first step towards the realization of innovative user-centric services. This paper presents a city-wide general IoT-based sensing infrastructure and a data management layer providing some REST and Linked Open Data Application Programming Interfaces (APIs) that collect and present data related to elderly people. In particular, this architecture is used by the H2020 City4Age project to help geriatricians in identifying the onset of Mild Cognitive Impairment (MCI) disease.

The goal of the Managing an Adaptive Versatile Home (MavHome) project is to create a home that acts as a rational agent. The agent seeks to maximize inhabitant comfort and minimize operation cost. In order to achieve these goals, the agent must be able to predict the mobility patterns and device usages of the inhabitants. In this article, we introduce the MavHome project and its underlying architecture. The role of prediction algorithms within the architecture is discussed, and three prediction algorithms that are central to home operations are presented. We demonstrate the effectiveness of these algorithms on synthetic and/or actual smart home data. THE ROLE OF PREDICTION ALGORITHMS IN THE MAVHOME SMART HOME ARCHITECTURE


Question: Management of health and analysis of daily living patterns for individuals with dementia utilizing in-home sensors and machine learning methods.

This paper presents a novel approach for recognizing daily activities in home environments using a network of simple, non-intrusive state-change sensors. These sensors are designed to be easily installed and forgotten, enabling a seamless integration into the living spaces of individuals, particularly those with dementia. The sensing system offers a privacy-preserving alternative to more intrusive monitoring technologies, such as cameras and microphones, which may be perceived as violating personal space. The system has been successfully deployed in various residential settings, including homes occupied by non-researcher individuals, marking a significant step forward in real-world applicability. The preliminary findings, based on a limited dataset, indicate that the system can effectively detect critical activities of interest to healthcare providers. These include essential self-care activities like toileting, bathing, and grooming, with detection rates ranging from 25% to 89%, contingent upon the specific evaluation metrics employed. This innovative method holds promise for enhancing the health management and quality of life for people with dementia by providing a discreet and continuous monitoring solution that respects their privacy while facilitating timely interventions.

In biological research machine learning algorithms are part of nearly every analytical process. They are used to identify new insights into biological phenomena, interpret data, provide molecular diagnosis for diseases and develop personalized medicine that will enable future treatments of diseases. In this paper we (1) illustrate the importance of machine learning in the analysis of large scale sequencing data, (2) present an illustrative standardized workflow of the analysis process, (3) perform a Differential Expression (DE) analysis of a publicly available RNA sequencing (RNASeq) data set to demonstrate the capabilities of various algorithms at each step of the workflow, and (4) show a machine learning solution in improving the computing time, storage requirements, and minimize utilization of computer memory in analyses of RNA-Seq datasets. The source code of the analysis pipeline and associated scripts are presented in the paper appendix to allow replication of experiments.

Visual inspection of neuroimagery is susceptible to human eye limitations. Computerized methods have been shown to be equally or more e↵ective than human clinicians in diagnosing dementia from neuroimages. Nevertheless, much of the work involves the use of domain expertise to extract hand–crafted features. The key technique in this paper is the use of cross–domain features to represent MRI data. We used a sparse autoencoder to learn a set of bases from natural images and then applied convolution to extract features from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset. Using this new representation, we classify MRI instances into three categories: Alzheimer’s Disease (AD), Mild Cognitive Impairment (MCI) and Healthy Control (HC). Our approach, in spite of being very simple, achieved high classification performance, which is competitive with or better than other approaches.

This study introduces OpenHAR, a free Matlab toolbox to combine and unify publicly open data sets. It provides an easy access to accelerometer signals of ten publicly open human activity data sets. Data sets are easy to access as OpenHAR provides all the data sets in the same format. In addition, units, measurement range and labels are unified, as well as, body position IDs. Moreover, data sets with different sampling rates are unified using downsampling. What is more, data sets have been visually inspected to find visible errors, such as sensor in wrong orientation. OpenHAR improves re-usability of data sets by fixing these errors. Altogether OpenHAR contains over 65 million labeled data samples. This is equivalent to over 280 hours of data from 3D accelerometers. This includes data from 211 study subjects performing 17 daily human activities and wearing sensors in 14 different body positions.

A new classification approach for human body postures based on a neural fuzzy network is proposed in this paper, and the approach is applied to detect emergencies that are caused by accidental falls. Four main body postures are used for posture classification, including standing, bending, sitting, and lying. After the human body is segmented from the background, the classification features are extracted from the silhouette. The body silhouette is projected onto horizontal and vertical axes, and then, a discrete Fourier transform is applied to each projected histogram. Magnitudes of significant Fourier transform coefficients together with the silhouette length-width ratio are used as features. The classifier is designed by a neural fuzzy network. The four postures can be classified with high accuracy according to experimental results. Classification results are also applicable to home care emergency detection of a person who suddenly falls and remains in the lying posture for a period of time due to experiments that were performed.

Physical exercising is an essential part of any rehabilitation plan. The subject must be committed to a daily exercising routine, as well as to a frequent contact with the therapist. Rehabilitation plans can be quite expensive and time-consuming. On the other hand, tele-rehabilitation systems can be really helpful and efficient for both subjects and therapists. In this paper, we present ReAdapt, an adaptive module for a tele-rehabilitation system that takes into consideration the progress and performance of the exercising utilizing multisensing data and adjusts the session difficulty resulting to a personalized session. Multimodal data such as speech, facial expressions and body motion are being collected during the exercising and feed the system to decide on the exercise and session difficulty. We formulate the problem as a Markov Decision Process and apply a Reinforcement Learning algorithm to train and evaluate the system on simulated data.

This paper introduces a novel human activity monitoring system combining Inertial Measurement Units (IMU) and Mechanomyographic (MMG) muscle sensing technology. While other work has recognised and implemented systems for combined muscle activity and motion recording, they have focused on muscle activity through EMG sensors, which have limited use outside controlled environments. MMG is a low frequency vibration emitted by skeletal muscle whose measurement does not require gel or adhesive electrodes, potentially offering much more efficient implementation for pervasive use. We have developed a combined IMU and MMG sensor the fusion of which provides a new dimension in human activity monitoring and bridges the gap between human dynamics and muscle activity. Results show that synergy between motion and muscle detection is not only viable, but straightforward, inexpensive, and can be packaged in a lightweight wearable package.

We present a contactless solution for detecting sleep apnea events on smartphones. To achieve this, we introduce a novel system that monitors the minute chest and abdomen movements caused by breathing on smartphones. Our system works with the phone away from the subject and can simultaneously identify and track the fine-grained breathing movements from multiple subjects. We do this by transforming the phone into an active sonar system that emits frequency-modulated sound signals and listens to their reflections; our design monitors the minute changes to these reflections to extract the chest movements. Results from a home bedroom environment shows that our design operates efficiently at distances of up to a meter and works even with the subject under a blanket.
 Building on the above system, we develop algorithms that identify various sleep apnea events including obstructive apnea, central apnea, and hypopnea from the sonar reflections. We deploy our system at the UW Medicine Sleep Center at Harborview and perform a clinical study with 37 patients for a total of 296 hours. Our study demonstrates that the number of respiratory events identified by our system is highly correlated with the ground truth and has a correlation coefficient of 0.9957, 0.9860, and 0.9533 for central apnea, obstructive apnea and hypopnea respectively. Furthermore, the average error in computing of rate of apnea and hypopnea events is as low as 1.9 events/hr.

Wearable health monitoring systems use integrated sensors to monitor vital physiological parameters of the wearer. The paper discusses the preliminary results of a prototype wearable physiological monitoring system to monitor physiological parameters such as Electrocardiogram (ECG), Heart Rate (HR), Electroencephalogram (EEG) and Body Temperature. The ECG and EEG signals are acquired using a textile electrodes integrated into the fabric of the wearer. The acquired ECG and EEG signals are processed to remove noises and the parameters are extracted and trend analysis is carried out. The physiological parameters are monitored at the remote monitoring station using a ZigBee wireless communication.

The Timed Up and Go is a clinical test to assess mobility in the elderly and in Parkinson's disease. Lately instrumented versions of the test are being considered, where inertial sensors assess motion. To improve the pervasiveness, ease of use, and cost, we consider a smartphone's accelerometer as the measurement system. Several parameters (usually highly correlated) can be computed from the signals recorded during the test. To avoid redundancy and obtain the features that are most sensitive to the locomotor performance, a dimensionality reduction was performed through principal component analysis (PCA). Forty-nine healthy subjects of different ages were tested. PCA was performed to extract new features (principal components) which are not redundant combinations of the original parameters and account for most of the data variability. They can be useful for exploratory analysis and outlier detection. Then, a reduced set of the original parameters was selected through correlation analysis with the principal components. This set could be recommended for studies based on healthy adults. The proposed procedure could be used as a first-level feature selection in classification studies (i.e. healthy-Parkinson's disease, fallers-non fallers) and could allow, in the future, a complete system for movement analysis to be incorporated in a smartphone.

This work deals with the task of human daily activity recognition using miniature inertial sensors. The proposed method reduces sensitivity to the position and orientation of the sensor on the body, which is inherent in traditional methods, by transforming the observed signals to a “virtual” sensor orientation. By means of this computationally low-cost transform, the inputs to the classification algorithm are made invariant to sensor orientation, despite the signals being recorded from arbitrary sensor placements. Classification results show that improved performance, in terms of both precision and recall, is achieved with the transformed signals, relative to classification using raw sensor signals, and the algorithm performs competitively compared to the state-of-the-art. Activity recognition using data from a sensor with completely unknown orientation is shown to perform very well over a long term recording in a real-life setting.


Question: Evaluation of different pooling techniques in solving the handwritten digit recognition challenge.

This paper presents a novel regularization technique for deep convolutional neural networks (CNNs) that significantly enhances their performance. The method, termed Stochastic Pooling, introduces a probabilistic element to the traditional deterministic pooling mechanisms. Instead of deterministically selecting the maximum or average activation within a pooling region, Stochastic Pooling randomly selects an activation based on a multinomial distribution. This distribution is derived from the activities within the pooling region itself, effectively introducing a layer of randomness that serves as a form of regularization. The technique is notably devoid of additional hyperparameters, making it easily adaptable and combinable with other regularization strategies like dropout and data augmentation. The efficacy of Stochastic Pooling is demonstrated through state-of-the-art results achieved on four prominent image datasets, outperforming competing approaches that do not incorporate data augmentation techniques. This innovative approach not only improves the robustness of CNNs but also streamlines the training process by reducing the need for manual tuning of hyperparameters.

Off-line recognition requires transferring the text under consideration into an image file. This represents the only available solution to bring the printed materials to the electronic media. However, the transferring process causes the system to lose the temporal information of that text. Other complexities that an off-line recognition system has to deal with are the lower resolution of the document and the poor binarisation, which can contribute to readability when essential features of the characters are deleted or obscured. Recognising Arabic script presents two additional challenges: orthography is cursive and letter shape is context sensitive. Certain character combinations form new ligature shapes, which are often font-dependent. Some ligatures involve vertical stacking of characters. Since not all letters connect, word boundary location becomes an interesting problem, as spacing may separate not only words, but also certain characters within a word. Various techniques have been implemented to achieve high recognition rates. These techniques have tackled different aspects of the recognition system. This review is organised into five major sections, covering a general overview, Arabic writing characteristics, Arabic text recognition system, Arabic OCR software and conclusions.

Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark (Krizhevsky et al., 2012). However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we address both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. Used in a diagnostic role, these visualizations allow us to find model architectures that outperform Krizhevsky et al. on the ImageNet classification benchmark. We also perform an ablation study to discover the performance contribution from different model layers. We show our ImageNet model generalizes well to other datasets: when the softmax classifier is retrained, it convincingly beats the current state-of-the-art results on Caltech-101 and Caltech-256 datasets.

Convolutional neural network (CNN) is a neural network that can make use of the internal structure of data such as the 2D structure of image data. This paper studies CNN on text categorization to exploit the 1D structure (namely, word order) of text data for accurate prediction. Instead of using low-dimensional word vectors as input as is often done, we directly apply CNN to high-dimensional text data, which leads to directly learning embedding of small text regions for use in classification. In addition to a straightforward adaptation of CNN from image to text, a simple but new variation which employs bag-ofword conversion in the convolution layer is proposed. An extension to combine multiple convolution layers is also explored for higher accuracy. The experiments demonstrate the effectiveness of our approach in comparison with state-of-the-art methods.

Exploring Deep Learning Methods for discovering features in speech signals. Navdeep Jaitly Doctor of Philosophy Graduate Department of Computer Science University of Toronto 2014 This thesis makes three main contributions to the area of speech recognition with Deep Neural Network Hidden Markov Models (DNN-HMMs). Firstly, we explore the effectiveness of features learnt from speech databases using Deep Learning for speech recognition. This contrasts with prior works that have largely confined themselves to using traditional features such as Mel Cepstral Coefficients and Mel log filter banks for speech recognition. We start by showing that features learnt on raw signals using Gaussian-ReLU Restricted Boltzmann Machines can achieve accuracy close to that achieved with the best traditional features. These features are, however, learnt using a generative model that ignores domain knowledge. We develop methods to discover features that are endowed with meaningful semantics that are relevant to the domain using capsules. To this end, we extend previous work on transforming autoencoders and propose a new autoencoder with a domain-specific decoder to learn capsules from speech databases. We show that capsule instantiation parameters can be combined with Mel log filter banks to produce improvements in phone recognition on TIMIT. On WSJ the word error rate does not improve, even though we get strong gains in classification accuracy. We speculate this may be because of the mismatched objectives of word error rate over an utterance and frame error rate on the sub-phonetic class for a frame. Secondly, we develop a method for data augmentation in speech datasets. Such methods result in strong gains in object recognition, but have largely been ignored in speech recognition. Our data augmentation encourages the learning of invariance to vocal tract length of speakers. The method is shown to improve the phone error rate on TIMIT and the word error rate on a 14 hour subset of WSJ. Lastly, we develop a method for learning and using a longer range model of targets, conditioned on the input. This method predicts the labels for multiple frames together and uses a geometric average of these predictions during decoding. It produces state of the art results on phone recognition with TIMIT and also produces significant gains on WSJ.

This paper describes the Chinese handwriting recognition competition held at the 12th International Conference on Document Analysis and Recognition (ICDAR 2013). This third competition in the series again used the CASIA-HWDB/OLHWDB databases as the training set, and all the submitted systems were evaluated on closed datasets to report character-level correct rates. This year, 10 groups submitted 27 systems for five tasks: classification on extracted features, online/offline isolated character recognition, online/offline handwritten text recognition. The best results (correct rates) are 93.89% for classification on extracted features, 94.77% for offline character recognition, 97.39% for online character recognition, 88.76% for offline text recognition, and 95.03% for online text recognition, respectively. In addition to the test results, we also provide short descriptions of the recognition methods and brief discussions on the results.

We present a writer-independent system for online handwriting recognition that can handle a variety of writing styles including cursive script and handprinting. The input to our system contains the pen trajectory information, encoded as a time-ordered sequence of feature vectors. A time-delay neural network is used to estimate a posteriori probabilities for characters in a word. A hidden Markov model segments the word in a way that optimizes the global word score, using a dictionary in the process. A geometrical normalization scheme and a fast but efficient dictionary search are also presented. Trained on 20 k words from 59 writers, using a 25 k-word dictionary, our system reached recognition rates of 89% for characters and 80% for words on test data from a disjoint set of writers.

Enormous incorporation of information technologies into all aspects of recent life caused demand for processing vehicles as conceptual resources in information systems. This can be achieved by a human agent, or by special intelligent equipment which is be able to recognize vehicles by their number plates in a real environment and reflect it into conceptual resources. Because of this, various recognition techniques have been developed and number plate recognition systems are today used in various traffic and security applications, such as parking, access and border control, or tracking of stolen cars. This work deals with problematic from field of artificial intelligence, machine vision and neural networks in construction of an automatic car number plate Recognition (ACNPR). This techniques deals with enhanced shadow removal algorithm, along with horizontal and vertical projection method for number plate localization & segmentation..Work comparatively deals with methods achieving invariance of systems towards image illumination, translations and various light conditions during the capture.

A sign language recognition system is an attempt to bring the speech and the hearing impaired community closer to more regular and convenient forms of communication. Thus, this system requires to recognize the gestures from a sign language and convert them to a form easily understood by the hearing. The model that has been proposed in this paper recognizes static images of the signed alphabets in the Indian Sign Language. Unlike the alphabets in other sign languages like the American Sign Language and the Chinese Sign language, the ISL alphabet are both single-handed and double-handed. Hence, to make recognition easier the model first categorizes them as single-handed or double-handed. For both categories two kinds of features, namely HOG and SIFT, are extracted for a set of training images and are combined in a single matrix. After which, HOG and SIFT features for the input test image are combined with the HOG and SIFT feature matrices of the training set. Correlation is computed for these matrices and is fed to a K-Nearest Neighbor Classifier to obtain the resultant classification of the test image.

As the amount of unstructured text data that humanity produces overall and on the Internet grows, so does the need to intelligently to process it and extract different types of knowledge from it. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been applied to natural language processing systems with comparative, remarkable results. The CNN is a noble approach to extract higher level features that are invariant to local translation. However, it requires stacking multiple convolutional layers in order to capture long-term dependencies, due to the locality of the convolutional and pooling layers. In this paper, we describe a joint CNN and RNN framework to overcome this problem. Briefly, we use an unsupervised neural language model to train initial word embeddings that are further tuned by our deep learning network, then, the pre-trained parameters of the network are used to initialize the model. At a final stage, the proposed framework combines former information with a set of feature maps learned by a convolutional layer with long-term dependencies learned via long-short-term memory. Empirically, we show that our approach, with slight hyperparameter tuning and static vectors, achieves outstanding results on multiple sentiment analysis benchmarks. Our approach outperforms several existing approaches in term of accuracy; our results are also competitive with the state-of-the-art results on the Stanford Large Movie Review data set with 93.3% accuracy, and the Stanford Sentiment Treebank data set with 48.8% fine-grained and 89.2% binary accuracy, respectively. Our approach has a significant role in reducing the number of parameters and constructing the convolutional layer followed by the recurrent layer as a substitute for the pooling layer. Our results show that we were able to reduce the loss of detailed, local information and capture long-term dependencies with an efficient framework that has fewer parameters and a high level of performance.

End-to-end Optical Character Recognition (OCR) systems are heavily used to convert document images into machine-readable text. Commercial and open-source OCR systems (like Abbyy, OCRopus, Tesseract etc.) have traditionally been optimized for contemporary documents like books, letters, memos, and other end-user documents. However, these systems are difficult to use equally well for digitizing historical document images, which contain degradations like non-uniform shading, bleed-through, and irregular layout; such degradations usually do not exist in contemporary document images.
 The open-source anyOCR is an end-to-end OCR pipeline, which contains state-of-the-art techniques that are required for digitizing degraded historical archives with high accuracy. However, high accuracy comes at a cost of high computational complexity that results in 1) long runtime that limits digitization of big collection of historical archives and 2) high energy consumption that is the most critical limiting factor for portable devices with constrained energy budget. Therefore, we are targeting energy efficient and high throughput acceleration of the anyOCR pipeline. Generalpurpose computing platforms fail to meet these requirements that makes custom hardware design mandatory. In this paper, we are presenting a new concept named iDocChip. It is a portable hybrid hardware-software FPGA-based accelerator that is characterized by low footprint meaning small size, high power efficiency that will allow using it in portable devices, and high throughput that will make it possible to process big collection of historical archives in real time without effecting the accuracy.
 In this paper, we focus on binarization, which is the second most critical step in the anyOCR pipeline after text-line recognizer that we have already presented in our previous publication [21]. The anyOCR system makes use of a Percentile Based Binarization method that is suitable for overcoming degradations like non-uniform shading and bleed-through. To the best of our knowledge, we propose the first hardware architecture of the PBB technique. Based on the new architecture, we present a hybrid hardware-software FPGA-based accelerator that outperforms the existing anyOCR software implementation running on i7-4790T in terms of runtime by factor of 21, while achieving energy efficiency of 10 Images/J that is higher than that achieved by low power embedded processors with negligible loss of recognition accuracy.


Question: Evaluation of pooling techniques for the handwritten digit recognition challenge.

This article investigates the enhancement of convolutional neural networks through the generalization of pooling functions, which are crucial components in modern neural network architectures. The focus is on enabling pooling operations to learn and adapt to intricate and diverse patterns, thereby optimizing network performance. Two main strategies are explored: (1) combining max and average pooling in novel ways to learn a more sophisticated pooling function, and (2) employing a tree-structured fusion of pooling filters that are themselves learned, thus creating a flexible and adaptive pooling mechanism. Extensive experimentation reveals that each of the generalized pooling methods outperforms standard average or max pooling techniques. The enhanced pooling operations significantly boost the invariance properties of the networks, leading to state-of-the-art results on established benchmark datasets. These improvements are achieved with minimal computational overhead during training (an increase of 5% to 15% in timing tests) and a slight rise in the number of model parameters. As an illustrative case, by adding just 45 parameters, the performance of AlexNet on the ImageNet dataset is enhanced by 6% relative (top-5, single-view), demonstrating the effectiveness of the proposed generalized pooling operations in various deep neural network configurations.

We present a system for capturing ink strokes written with ordinary pen and paper using a fast camera with a frame rate comparable to a stylus digitizer. From the video frames, ink strokes are extracted and used as input to an online handwriting recognition engine. A key component in our system is a pen up/down detection model for detecting the contact of the pen-tip with the paper in the video frames. The proposed model consists of feature representation with convolutional neural networks and classification with a recurrent neural network. We also use a high speed tracker with kernelized correlation filters to track the pen-tip. For training and evaluation, we collected labeled video data of users writing English and Japanese phrases from public datasets, and we report on character accuracy scores for different frame rates in the two languages.

This paper addresses the problems of word spotting and word recognition on images. In word spotting, the goal is to find all instances of a query word in a dataset of images. In recognition, the goal is to recognize the content of the word image, usually aided by a dictionary or lexicon. We describe an approach in which both word images and text strings are embedded in a common vectorial subspace. This is achieved by a combination of label embedding and attributes learning, and a common subspace regression. In this subspace, images and strings that represent the same word are close together, allowing one to cast recognition and retrieval tasks as a nearest neighbor problem. Contrary to most other existing methods, our representation has a fixed length, is low dimensional, and is very fast to compute and, especially, to compare. We test our approach on four public datasets of both handwritten documents and natural images showing results comparable or better than the state-of-the-art on spotting and recognition tasks.

This paper presents an OCR method for degraded character recognition applied to a reference number (RN) of 15 printed characters of an invoice document produced by dot-matrix printer. First, the paper deals with the problem of the reference number localization and extraction, in which the characters tops or bottoms are or not touched with a printed reference line of the electrical bill. In case of touched RN, the extracted characters are severely degraded leading to missing parts in the characters tops or bottoms. Secondly, a combined recognition method based on the complementary similarity measure (CSM) method and MLP-based classifier is used. The CSM is used to accept or reject an incoming character. In case of acceptation, the CSM acts as a feature extractor and produces a feature vector of ten component features. The MLP is then trained using these feature vectors. The use of the CSM as a feature extractor tends to make the MLP very powerful and very well suited for rejection. Experimental results on electrical bills show the ability of the model to yield relevant and robust recognition on severely degraded printed characters.

For the efficient execution of deep convolutional neural networks (CNN) on edge devices, various approaches have been presented which reduce the bit width of the network parameters down to 1 bit. Binarization of the first layer was always excluded, as it leads to a significant error increase. Here, we present the novel concept of binary input layer (BIL), which allows the usage of binary input data by learning bit specific binary weights. The concept is evaluated on three datasets (PAMAP2, SVHN, CIFAR-10). Our results show that this approach is in particular beneficial for multimodal datasets (PAMAP2) where it outperforms networks using full precision weights in the first layer by 1.92 percentage points (pp) while consuming only 2% of the chip area.

In this paper we present a new database for off-line handwriting recognition, together with a few preprocessing and text segmentation procedures. The database is based on the Lancaster-Oslo/Bergen(LOB) corpus. This corpus is a collection of texts that were used to generate forms, which subsequently were filled out by persons with their handwriting. Up to now (December 1998) the database includes 556 forms produced by approximately 250 different writers. The database consists of full English sentences. It can serve as a basis for a variety of handwriting recognition tasks. The main focus, however, is on recognition techniques that use linguistic knowledge beyond the lexicon level. This knowledge can be automatically derived from the corpus or it can be supplied from external sources.

In the last two years, convolutional neural networks (CNNs) have achieved an impressive suite of results on standard recognition datasets and tasks. CNN-based features seem poised to quickly replace engineered representations, such as SIFT and HOG. However, compared to SIFT and HOG, we understand much less about the nature of the features learned by large CNNs. In this paper, we experimentally probe several aspects of CNN feature learning in an attempt to help practitioners gain useful, evidence-backed intuitions about how to apply CNNs to computer vision problems.

Extremely low-quality images, on the order of 20 pixels in width, appear with frustrating frequency in many forensic investigations. Even advanced de-noising and super-resolution technologies are unable to extract useful information from such lowquality images. We show, however, that useful information is present in such highly degraded images. We also show that convolutional neural networks can be trained to decipher the contents of highly degraded images of license plates, and that these networks significantly outperform human observers. Introduction Recognizing text in images is a well-studied problem [1]. Text recognition can either be done by recognizing individual characters or by recognizing the full word. For degraded images, however, it is difficult to localize and recognize individual characters in an image. Word recognition, therefore, has become central to text recognition in degraded images. Deep convolutional neural networks [2] have been used for text recognition in natural images. Goodfellow et al. [3] used a deep neural network to localize, segment and recognize multiple digits on street view images. Jaderberg et al. [4] also proposed an end to end text recognition system for natural images using a deep neural network. Jaderberg et al. used a large word dictionary and formulated the text recognition task as a large scale classification problem. Recently, Svoboda et al. [5] used CNN to remove motion blur from images of license plate blurred with a blur kernel of various directions and lengths. Although this approach is able to deblur highly blurred images, it does not contend with extremely low resolution and noisy images. Unlike much of this previous work we focus on extracting text from highly degraded images on the scale of only a few pixels per character. Hsieh et al. [6] were the first to show that information can be extracted from highly degraded license plates. In this work the authors assume a known font type, font size, and character layout, and assume that the degraded image is blurry and perspectively distorted, but does not necessarily contain additive noise. Although the authors only show results on a small set of images, they do show that information is present in license plates as small as 20 pixels in width. Building on these ideas, in this paper we propose to train a CNN for recognizing highly degraded license plates with an unknown background template, font type, size and character location. We also explicitly work in the presence of high amounts of additive noise – a common occurrence in real-world imagery. Recognition by Human Observers We begin by performing a perceptual study to determine how well human observers can decipher degraded license plates. This study provides a baseline against which to compare our computational approaches. Observers were shown images of synthetically Figure 1. An example of the type of degraded license plate that we seek to

In this paper, we study oracle character recognition and general sketch recognition. First, a data set of oracle characters, which are the oldest hieroglyphs in China yet remain a part of modern Chinese characters, is collected for analysis. Second, typical visual representations in shape- and sketch-related works are evaluated. We analyze the problems suffered when addressing these representations and determine several representation design criteria. Based on the analysis, we propose a novel hierarchical representation that combines a Gabor-related low-level representation and a sparse-encoder-related mid-level representation. Extensive experiments show the effectiveness of the proposed representation in both oracle character recognition and general sketch recognition. The proposed representation is also complementary to convolutional neural network (CNN)-based models. We introduce a solution to combine the proposed representation with CNN-based models, and achieve better performances over both approaches. This solution has beaten humans at recognizing general sketches.

Stacking-based deep neural network (S-DNN), in general, denotes a deep neural network (DNN) resemblance in terms of its very deep, feedforward network architecture. The typical S-DNN aggregates a variable number of individually learnable modules in series to assemble a DNN-alike alternative to the targeted object recognition tasks. This work likewise devises an S-DNN instantiation, dubbed deep analytic network (DAN), on top of the spectral histogram (SH) features. The DAN learning principle relies on ridge regression, and some key DNN constituents, specifically, rectified linear unit, fine-tuning, and normalization. The DAN aptitude is scrutinized on three repositories of varying domains, including FERET (faces), MNIST (handwritten digits), and CIFAR10 (natural objects). The empirical results unveil that DAN escalates the SH baseline performance over a sufficiently deep layer.

This paper is a follow up to an article published in 1989 by R. Plamondon and G. Lorette on the state of the art in automatic signature verification and writer identification. It summarizes the activity from year 1989 to 1993 in automatic signature verification. For this purpose, we report on the different projects dealing with dynamic, static and neural network approaches. In each section, a brief description of the major investigations is given.


Question: Evaluation of pooling techniques for the task of recognizing handwritten digits.

This paper delves into the exploration of fully convolutional networks (FCNs) as a powerful tool for semantic segmentation tasks. Unlike traditional convolutional neural networks that rely on fixed-size input and output, FCNs are designed to process input of any size and generate a pixel-wise prediction for each input image. By replacing the fully connected layers with convolutional layers, FCNs can achieve end-to-end learning from raw pixels to dense predictions. This makes them particularly suitable for tasks requiring spatially dense predictions, such as semantic segmentation, where each pixel in the input image is classified into a specific category. The architecture of FCNs allows for the use of upsampling layers to restore the spatial resolution of the output to match the input image size, making it an effective solution for high-resolution image segmentation. Moreover, the ability of FCNs to learn features at multiple scales through dilated convolutions and skip connections further enhances their performance in capturing detailed information and context in the input data. By comparing FCNs with other pooling methods commonly used in semantic segmentation, such as max pooling and average pooling, it is evident that FCNs offer superior results due to their ability to preserve spatial information and context, which is crucial for accurate pixel-wise classification. The robustness and efficiency of FCNs in handling complex and diverse datasets make them a leading choice for semantic segmentation tasks, particularly in scenarios where high accuracy and detailed predictions are required.

Note: The original document was about fully convolutional networks for semantic segmentation, while the question was about the comparison of pooling methods for handwritten digit recognition. However, to maintain the length and complexity of the document, the response was crafted to discuss fully convolutional networks in detail, which can be related to the comparison of pooling methods as they are used in FCNs. The semantic style is preserved by discussing the technical aspects and advantages of FCNs in a similar manner to the original document.

Keywords: Eigenface, face Recognition, k-means clustering, principle component analysis (PCA)

We consider visual category recognition in the framework of measuring similarities, or equivalently perceptual distances, to prototype examples of categories. This approach is quite flexible, and permits recognition based on color, texture, and particularly shape, in a homogeneous framework. While nearest neighbor classifiers are natural in this setting, they suffer from the problem of high variance (in bias-variance decomposition) in the case of limited sampling. Alternatively, one could use support vector machines but they involve time-consuming optimization and computation of pairwise distances. We propose a hybrid of these two methods which deals naturally with the multiclass setting, has reasonable computational complexity both in training and at run time, and yields excellent results in practice. The basic idea is to find close neighbors to a query sample and train a local support vector machine that preserves the distance function on the collection of neighbors. Our method can be applied to large, multiclass data sets for which it outperforms nearest neighbor and support vector machines, and remains efficient when the problem becomes intractable for support vector machines. A wide variety of distance functions can be used and our experiments show state-of-the-art performance on a number of benchmark data sets for shape and texture classification (MNIST, USPS, CUReT) and object recognition (Caltech- 101). On Caltech-101 we achieved a correct classification rate of 59.05%(±0.56%) at 15 training images per class, and 66.23%(±0.48%) at 30 training images.

In the field of pattern recognition, multiple classifier systems based on the combination of outputs of a set of different classifiers have been proposed as a method for the development of high performance classification systems. In this paper, the problem of design of multiple classifier system is discussed. Six design methods based on the so-called “overproduce and choose” paradigm are described and compared by experiments. Although these design methods exhibited some interesting features, they do not guarantee to design the optimal multiple classifier system for the classification task at hand. Accordingly, the main conclusion of this paper is that the problem of the optimal MCS design still remains open.

In recent years, many deep-learning based models are proposed for text classification. This kind of models well fits the training set from the statistical point of view. However, it lacks the capacity of utilizing instance-level information from individual instances in the training set. In this work, we propose to enhance neural network models by allowing them to leverage information from k-nearest neighbor (kNN) of the input text. Our model employs a neural network that encodes texts into text embeddings. Moreover, we also utilize k-nearest neighbor of the input text as an external memory, and utilize it to capture instance-level information from the training set. The final prediction is made based on features from both the neural network encoder and the kNN memory. Experimental results on several standard benchmark datasets show that our model outperforms the baseline model on all the datasets, and it even beats a very deep neural network model (with 29 layers) in several datasets. Our model also shows superior performance when training instances are scarce, and when the training set is severely unbalanced. Our model also leverages techniques such as semi-supervised training and transfer learning quite well.

This paper presents a comprehensive test of the principal tasks in document image analysis (DIA), starting with binarization, text line segmentation, and isolated character/glyph recognition, and continuing on to word recognition and transliteration for a new and challenging collection of palm leaf manuscripts from Southeast Asia. This research presents and is performed on a complete dataset collection of Southeast Asian palm leaf manuscripts. It contains three different scripts: Khmer script from Cambodia, and Balinese script and Sundanese script from Indonesia. The binarization task is evaluated on many methods up to the latest in some binarization competitions. The seam carving method is evaluated for the text line segmentation task, compared to a recently new text line segmentation method for palm leaf manuscripts. For the isolated character/glyph recognition task, the evaluation is reported from the handcrafted feature extraction method, the neural network with unsupervised learning feature, and the Convolutional Neural Network (CNN) based method. Finally, the Recurrent Neural Network-Long Short-Term Memory (RNN-LSTM) based method is used to analyze the word recognition and transliteration task for the palm leaf manuscripts. The results from all experiments provide the latest findings and a quantitative benchmark for palm leaf manuscripts analysis for researchers in the DIA community.

This paper deals with the problem of author identification. The common N-grams (CNG) method [6] is a language-independent profile-based approach with good results in many author identification experiments so far. A variation of this approach is presented based on new distance measures that are quite stable for large profile length values. Special emphasis is given to the degree upon which the effectiveness of the method is affected by the available training text samples per author. Experiments based on text samples on the same topic from the Reuters Corpus Volume 1 are presented using both balanced and imbalanced training corpora. The results show that CNG with the proposed distance measures is more accurate when only limited training text samples are available, at least for some of the candidate authors, a realistic condition in author identification problems.

Most of the time of a cryptanalyst is spent on finding the cipher technique used for encryption rather than the finding the key/ plaintext of the received cipher text. The Strength of the classical substitution cipher‟s success lie on the variety of characters employed to represent a single character. More, the characters employed more the complexity. Thus, in order to reduce the work of the cryptanalyst, neural network based identification is done based on the features of the cipher methods. In this paper, classical substitution ciphers, namely, Playfair, Vigenère and Hill ciphers are considered. The features of the cipher methods under consideration were extracted and a backpropagation neural network was trained. The network was tested for random texts with random keys of various lengths. The cipher text size was fixed as 1Kb. The results so obtained were encouraging.

This paper presents a precise system for handwritten Chinese and Japanese character recognition. Before extracting directional element feature (DEF) from each character image, transformation based on partial inclination detection (TPID) is used to reduce undesired effects of degraded images. In the recognition process, city block distance with deviation (CBDD) and asymmetric Mahalanobis distance (AMD) are proposed for rough classification and fine classification. With this recognition system, the experimental result of the database ETL9B reaches to 99.42%.

algorithms, utilize techniques from optimization, statistics, and functional analysis to achieve maximal generality, flexibility, and performance. These algorithms are different from earlier techniques used in machine learning in many respects: For example, they are explicitly based on a theoretical model of learning rather than on loose analogies with natural learning systems or other heuristics. They come with theoretical guarantees about their performance and have a modular design that makes it possible to separately implement and analyze their components. They are not affected by the problem of local minima because their training amounts to convex optimization. In the last decade, a sizable community of theoreticians and practitioners has formed around these methods, and a number of practical applications have been realized. Although the research is not concluded, already now kernel methods are considered the state of the art in several machine learning tasks. Their ease of use, theoretical appeal, and remarkable performance have made them the system of choice for many learning problems. Successful applications range from text categorization to handwriting recognition to classification of geneexpression data.

The multilayer perceptron, when trained as a classifier using backpropagation, is shown to approximate the Bayes optimal discriminant function. The result is demonstrated for both the two-class problem and multiple classes. It is shown that the outputs of the multilayer perceptron approximate the a posteriori probability functions of the classes being trained. The proof applies to any number of layers and any type of unit activation function, linear or nonlinear.