Question: Design, creation, and evaluation of an intelligent lighting system.

This document introduces a novel Java-based home automation system designed for monitoring and controlling household devices through the World Wide Web. The system architecture is centered around an autonomous embedded system board that interfaces with a PC-based server within the home. Household appliances are linked to the input/output ports of the embedded system, allowing their operational status to be relayed to the server. The control and monitoring software is built on a blend of JavaServer pages, JavaBeans, and interactive C, enabling both local and remote management of appliances through a web browser, assuming Internet access is available. The system is designed to be adaptable, accommodating the integration of appliances from various manufacturers without necessitating significant alterations to its core structure. Security is ensured through password protection, preventing unauthorized access to home appliances. Additionally, the system maintains functionality in local control even when the Internet connection is disrupted or the server is offline. This innovative approach to home automation demonstrates the system's robustness and versatility in managing household devices effectively.

A home-based intelligent energy conservation system needs to know what appliances (or loads) are being used in the home and when they are being used in order to provide intelligent feedback or to make intelligent decisions. This analysis task is known as load disaggregation or non-intrusive load monitoring (NILM). The datasets used for NILM research generally contain real power readings, with the data often being too coarse for more sophisticated analysis algorithms, and often covering too short a time period. We present the Almanac of Minutely Power dataset (AMPds) for load disaggregation research; it contains one year of data that includes 11 measurements at one minute intervals for 21 sub-meters. AMPds also includes natural gas and water consumption data. Finally, we use AMPds to present findings from our own load disaggregation algorithm to show that current, rather than real power, is a more effective measure for NILM.

A smart home is an application of ubiquitous computing in which the home environment is monitored by ambient intelligence to provide context-aware services and facilitate remote home control. This paper presents an overview of previous smart home research as well as the associated technologies. A brief discussion on the building blocks of smart homes and their interrelationships is presented. It describes collective information about sensors, multimedia devices, communication protocols, and systems, which are widely used in smart home implementation. Special algorithms from different fields and their significance are explained according to their scope of use in smart homes. This paper also presents a concrete guideline for future researchers to follow in developing a practical and sustainable smart home.

This paper aims to develop a convenient traffic system that allows a smooth movement of cars which will help build a smarter city. The traffic system currently implemented in many areas is not based on the density of traffic and every road is allotted a preset time. This results in traffic congestion due to large red-light delays and timings allotted for roads in a city that should vary during peak on-off hours, but in reality don't. These traditional systems are not adaptable and fail to support traffic during an unexpected situation or during an accident, and this makes them inefficient. In order to calculate the density of traffic various sensors can be used, each having their merits and demerits. In our proposed system Ultrasound Sensors are used along with Image Processing(using live feed from a camera) that works on a Raspberry Pi platform and calculates the vehicle density and dynamically allots time for different levels of traffic. This in turn allows better signal control and effective management of traffic thereby reducing the probability of a collision. By using Internet Of Things(IoT) real time data from the system can be collected, stored and managed on a cloud. This data can be used to interpret the signal duration in-case any of the sensing equipment fail, and also for future analysis.

We report a simulation program for indoor visible light communication environment based on MATLAB and Simulink. The program considers the positions of the transmitters and the reflections at each wall. For visible light communication environment, the illumination light-emitting diode is used not only as a lighting device, but also as a communication device. Using the simulation program, the distributions of illuminance and root-mean-square delay spread are analyzed at bottom surface.

Most existing approaches to characterizing thermal properties of buildings and heat emissions from their elements rely on manual inspection and as such are slow, and labor intensive. This is often a daunting task, which requires several days of on-site inspection. In this paper, we propose a fully automatic approach to construct a 3D thermal point cloud of the building interior reflecting the geometry including walls, floors, and ceilings, as well as structures such as furniture, lights, windows, and plug loads. Our approach is based on a wearable ambulatory backpack comprising multiple sensors such as Light Detection And Ranging (LiDAR) scanners, and Infrared and optical cameras. As the operator wearing the backpack walks through the building, the LiDAR scans are collected and processed in order to compute the 3D geometry of the building. Furthermore, the Infrared cameras are calibrated intrinsically and extrinsically such that the captured images are registered to the captured geometry. Thus, the temperature data in the Infrared images is associated with the geometry resulting in a “thermal 3D point cloud”. The same process can be repeated using optical imagery resulting in a “visible 3D point cloud”. By visualizing the two point clouds simultaneously in interactive rendering tools, we can virtually walk through the thermal and optical 3D point clouds, toggle between them, identify and annotate, “hot” regions, objects, plug loads, thermal and moisture leaks, and document their location with fine spatial granularity in the 3D point clouds.

Homes are rich with information about people's energy consumption, medical health, and personal or family functions. In this paper, we present our experiences deploying large-scale residential sensing systems in over 20 homes. Deploying small-scale systems in homes can be deceptively easy, but in our deployments we encountered a phase transition in which deployment effort increases dramatically as residential deployments scale up in terms of 1) the number of nodes, 2) the length of time, and 3) the number of houses. In this paper, we distill our experiences down to a set of guidelines and design principles to help future deployments avoid the potential pitfalls of large-scale sensing in homes.

This paper presents ElectriSense, a new solution for automatically detecting and classifying the use of electronic devices in a home from a single point of sensing. ElectriSense relies on the fact that most modern consumer electronics and fluorescent lighting employ switch mode power supplies (SMPS) to achieve high efficiency. These power supplies continuously generate high frequency electromagnetic interference (EMI) during operation that propagates throughout a home's power wiring. We show both analytically and by in-home experimentation that EMI signals are stable and predictable based on the device's switching frequency characteristics. Unlike past transient noise-based solutions, this new approach provides the ability for EMI signatures to be applicable across homes while still being able to differentiate between similar devices in a home. We have evaluated our solution in seven homes, including one six-month deployment. Our results show that ElectriSense can identify and classify the usage of individual devices with a mean accuracy of 93.82%.

Health monitoring systems have rapidly evolved during the past two decades and have the potential to change the way health care is currently delivered. Although smart health monitoring systems automate patient monitoring tasks and, thereby improve the patient workflow management, their efficiency in clinical settings is still debatable. This paper presents a review of smart health monitoring systems and an overview of their design and modeling. Furthermore, a critical analysis of the efficiency, clinical acceptability, strategies and recommendations on improving current health monitoring systems will be presented. The main aim is to review current state of the art monitoring systems and to perform extensive and an in-depth analysis of the findings in the area of smart health monitoring systems. In order to achieve this, over fifty different monitoring systems have been selected, categorized, classified and compared. Finally, major advances in the system design level have been discussed, current issues facing health care providers, as well as the potential challenges to health monitoring field will be identified and compared to other similar systems.

Nowadays, the remote Home Automation turns out to be more and more significant and appealing. It improves the value of our lives by automating various electrical appliances or instruments. This paper describes GSM (Global System Messaging) based secured device control system using App Inventor for Android mobile phones. App Inventor is a latest visual programming platform for developing mobile applications for Android-based smart phones. The Android Mobile Phone Platform becomes more and more popular among software developers, because of its powerful capabilities and open architecture. It is a fantastic platform for the real world interface control, as it offers an ample of resources and already incorporates a lot of sensors. No need to write programming codes to develop apps in the App Inventor, instead it provides visual design interface as the way the apps looks and use blocks of interlocking components to control the app’s behaviour. The App Inventor aims to make programming enjoyable and accessible to novices.

OBJECTIVE
To review the history and current applications of artificial intelligence in the intensive care unit.

DATA SOURCES
The MEDLINE database, bibliographies of selected articles, and current texts on the subject.

STUDY SELECTION
The studies that were selected for review used artificial intelligence tools for a variety of intensive care applications, including direct patient care and retrospective database analysis.

DATA EXTRACTION
All literature relevant to the topic was reviewed.

DATA SYNTHESIS
Although some of the earliest artificial intelligence (AI) applications were medically oriented, AI has not been widely accepted in medicine. Despite this, patient demographic, clinical, and billing data are increasingly available in an electronic format and therefore susceptible to analysis by intelligent software. Individual AI tools are specifically suited to different tasks, such as waveform analysis or device control.

CONCLUSIONS
The intensive care environment is particularly suited to the implementation of AI tools because of the wealth of available data and the inherent opportunities for increased efficiency in inpatient care. A variety of new AI tools have become available in recent years that can function as intelligent assistants to clinicians, constantly monitoring electronic data streams for important trends, or adjusting the settings of bedside devices. The integration of these tools into the intensive care unit can be expected to reduce costs and improve patient outcomes.


Question: Design, creation, and evaluation of an intelligent lighting system.

This document introduces CHARADE, an innovative application that enables users to control a computer during presentations through the use of free-hand gestures. To bring this concept to life, the developers have established an interaction model, a standardized notation for gestures, and a set of design principles for creating effective gestural command sets. The overarching goal of CHARADE is to establish interaction paradigms that are well-suited for computerized reality environments. In this application, hand gestures serve not only as a means of interacting with the computer but also as a method of communication with others or for operating various devices. By integrating gesture recognition technology, CHARADE facilitates a seamless and intuitive user experience, making presentations more engaging and interactive. This approach to human-computer interaction has potential applications in diverse settings, from educational lectures to corporate meetings, enhancing user engagement and control.

A novel system for the location of people in an office environment is described. Members of staff wear badges that transmit signals providing information about their location to a centralized location service, through a network of sensors. The paper also examines alternative location techniques, system design issues and applications, particularly relating to telephone call routing. Location systems raise concerns about the privacy of an individual and these issues are also addressed.

How can we innovate smart systems for smart cities, to make data available homogeneously, inexpensively, and flexibly while supporting an array of applications that have yet to exist or be specified?

The IEEE Standards Working Group, IEEE PAR1789 "Recommending practices for modulating current in High Brightness LEDs for mitigating health risks to viewers" has been formed to advise the lighting industry and standards groups about the emerging concern of flicker in LED lighting. This paper intends to introduce new measures and definitions of lamp flicker in lighting. The discussion represents on-going work in IEEE PAR1789 that is vital to designing safe LED lamp drivers.

This paper introduces a new class of interactive interfaces that can be moved around to appear on ordinary objects and surfaces anywhere in a space. By dynamically adapting the form, function, and location of an interface to suit the context of the user, such steerable interfaces have the potential to offer radically new and powerful styles of interaction in intelligent pervasive computing spaces. We propose defining characteristics of steerable interfaces and present the first steerable interface system that combines projection, gesture recognition, user tracking, environment modeling and geometric reasoning components within a system architecture. Our work suggests that there is great promise and rich potential for further research on steerable interfaces.

This paper presents an intelligent traffic control system to pass emergency vehicles smoothly. Each individual vehicle is equipped with special radio frequency identification (RFID) tag (placed at a strategic location), which makes it impossible to remove or destroy. We use RFID reader, NSK EDK-125-TTL, and PIC16F877A system-on-chip to read the RFID tags attached to the vehicle. It counts number of vehicles that passes on a particular path during a specified duration. It also determines the network congestion, and hence the green light duration for that path. If the RFID-tag-read belongs to the stolen vehicle, then a message is sent using GSM SIM300 to the police control room. In addition, when an ambulance is approaching the junction, it will communicate to the traffic controller in the junction to turn ON the green light. This module uses ZigBee modules on CC2500 and PIC16F877A system-on-chip for wireless communications between the ambulance and traffic controller. The prototype was tested under different combinations of inputs in our wireless communication laboratory and experimental results were found as expected.

The Internet of Things (IoT) is entering the daily operation of many industries; applications include but are not limited to smart cities, smart grids, smart homes, physical security, e-health, asset management, and logistics. For example, the concept of smart cities is emerging in multiple continents, where enhanced street lighting controls, infrastructure monitoring, public safety and surveillance, physical security, gunshot detection, meter reading, and transportation analysis and optimization systems are being deployed on a city-wide scale. A related and cost-effective user-level IoT application is the support of IoT-enabled smart buildings. Commercial space has substantial requirements in terms of comfort, usability, security, and energy management. IoT-based systems can support these requirements in an organic manner. In particular, power over Ethernet, as part of an IoT-based solution, offers disruptive opportunities in revolutionizing the in-building connectivity of a large swath of devices. However, a number of deployment-limiting issues currently impact the scope of IoT utilization, including lack of comprehensive end-to-end standards, fragmented cybersecurity solutions, and a relative dearth of fully-developed vertical applications. This paper reviews some of the technical opportunities offered and the technical challenges faced by the IoT in the smart building arena.

In this paper, we present a Web-based intelligent tutoring system, called BITS. The decision making process conducted in our intelligent system is guided by a Bayesian network approach to support students in learning computer programming. Our system takes full advantage of Bayesian networks, which are a formal framework for uncertainty management in Artificial Intelligence based on probability theory. We discuss how to employ Bayesian networks as an inference engine to guide the students’ learning processes . In addition, we describe the architecture of BITS and the role of each module in the system. Whereas many tutoring systems are static HTML Web pages of a class textbook or lecture notes, our intelligent system can help a student nav igate through the online course materials, recommend learning goals, and generate appropriate reading sequences.

A smart home is an application of ubiquitous computing in which the home environment is monitored by ambient intelligence to provide context-aware services and facilitate remote home control. This paper presents an overview of previous smart home research as well as the associated technologies. A brief discussion on the building blocks of smart homes and their interrelationships is presented. It describes collective information about sensors, multimedia devices, communication protocols, and systems, which are widely used in smart home implementation. Special algorithms from different fields and their significance are explained according to their scope of use in smart homes. This paper also presents a concrete guideline for future researchers to follow in developing a practical and sustainable smart home.

Although the exploration of design alternatives is crucial for interaction designers and customization is required for end-users, the current development tools for physical computing devices have focused on single versions of an artifact. We propose the parametric design of devices including their enclosure layouts and programs to address this issue. A Web-based design tool called f3.js is presented as an example implementation, which allows devices assembled from laser-cut panels with sensors and actuator modules to be parametrically created and customized. It enables interaction designers to write code with dedicated APIs, declare parameters, and interactively tune them to produce the enclosure layouts and programs. It also provides a separate user interface for end-users that allows parameter tuning and dynamically generates instructions for device assembly. The parametric design approach and the tool were evaluated through two user studies with interaction designers, university students, and end-users.

The pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. A primary challenge that needs to be tackled to meet this need is the ability to recognize and track functional activities that people perform in their own homes and everyday settings. In this paper, we look at approaches to perform real-time recognition of Activities of Daily Living. We enhance other related research efforts to develop approaches that are effective when activities are interrupted and interleaved. To evaluate the accuracy of our recognition algorithms we assess them using real data collected from participants performing activities in our on-campus smart apartment testbed.


Question: A 20-machine examination of facial expression analysis.

An Extensive Resource for Analyzing Facial Expressions
Over the past ten years, there has been a surge in research aimed at advancing the analysis of facial expressions. However, due to the reliance on small-scale datasets by many researchers, the broader applicability of these methods is uncertain. This paper delineates the multifaceted domain of facial expression analysis, encompassing levels of description, transitions between expressions, eliciting conditions, the reliability and validity of training and testing datasets, individual subject variations, head orientation, scene complexity, image attributes, and correlations with non-verbal behaviors. To address these aspects, we introduce the CMU-Pittsburgh Action Unit (AU)-Coded Facial Expression Image Database. Comprising 2105 digital image sequences from a diverse group of 182 adult subjects, representing different ethnic backgrounds and executing multiple instances of primary Facial Action Coding System (FACS) action units, this database stands as the most extensive and detailed resource available for comparative studies in facial expression analysis to date.

The accuracy of automated human face recognition algorithms can significantly degrade while recognizing same subjects under make-up and disguised appearances. Increasing constraints on enhanced security and surveillance requires enhanced accuracy from face recognition algorithms for faces under disguise and/or makeup. This paper presents a new database for face images under disguised and make-up appearances the development of face recognition algorithms under such covariates. This database has 2460 images from 410 different subjects and is acquired under real environment, focuses on make-up and disguises covariates and also provides ground truth (eye glass, goggle, mustache, beard) for every image. This can enable developed algorithms to automatically quantify their capability for identifying such important disguise attribute during the face recognition We also present comparative experimental results from two popular commercial matchers and from recent publications. Our experimental results suggest significant performance degradation in the capability of these matchers in automatically recognizing these faces. We also analyze face detection accuracy from these matchers. The experimental results underline the challenges in recognizing faces under these covariates. Availability of this new database in public domain will help to advance much needed research and development in recognizing make-up and disguised faces.

In this paper, we present a quantitative, highly structured cortex-simulated model, which can be simply described as feedforward, hierarchical simulation of ventral stream of visual cortex using biologically plausible, computationally convenient spiking neural network system. The motivation comes directly from recent pioneering works on detailed functional decomposition analysis of the feedforward pathway of the ventral stream of visual cortex and developments on artificial spiking neural networks (SNNs). By combining the logical structure of the cortical hierarchy and computing power of the spiking neuron model, a practical framework has been presented. As a proof of principle, we demonstrate our system on several facial expression recognition tasks. The proposed cortical-like feedforward hierarchy framework has the merit of capability of dealing with complicated pattern recognition problems, suggesting that, by combining the cognitive models with modern neurocomputational approaches, the neurosystematic approach to the study of cortex-like mechanism has the potential to extend our knowledge of brain mechanisms underlying the cognitive analysis and to advance theoretical models of how we recognize face or, more specifically, perceive other people's facial expression in a rich, dynamic, and complex environment, providing a new starting point for improved models of visual cortex-like mechanism.

Parameterized models can produce realistic, manipulable images of human faces— with a surprisingly small number of parameters.

ses a facial expression recognition system based on Gabor feature using a novel r bank. Traditionally, a global Gabor filter bank with 5 frequencies and 8 ten used to extract the Gabor feature. A lot of time will be involved to extract mensions of such Gabor feature vector are prohibitively high. A novel local Gabor art of frequency and orientation parameters is proposed. In order to evaluate the he local Gabor filter bank, we first employed a two-stage feature compression LDA to select and compress the Gabor feature, then adopted minimum distance nize facial expression. Experimental results show that the method is effective for eduction and good recognition performance in comparison with traditional entire . The best average recognition rate achieves 97.33% for JAFFE facial expression abor filter bank, feature extraction, PCA, LDA, facial expression recognition. deliver rich information about human emotion and play an essential role in human In order to facilitate a more intelligent and natural human machine interface of new cts, automatic facial expression recognition [1][18][20] had been studied world en years, which has become a very active research area in computer vision and n. There are many approaches have been proposed for facial expression analysis ages and image sequences [12][18] in the literature. we focus on the recognition of facial expression from single digital images with feature extraction. A number of approaches have been developed for extracting by: Motorola Labs Research Foundation (No.303D804372), NSFC (No.60275005), GDNSF 105938).

Estimating human age group automatically via facial image analysis has lots of potential real-world applications, such as human computer interaction and multimedia communication. However, It is still a challenging problem for the existing computer vision systems to automatically and effectively estimate human age group. The aging process is determined by not only the person’s gene, but also many external factors, such as health, living style, living location, and weather conditions. Males and females may also age differently. An age group classification system for facial images is proposed in this paper. Five age groups including babies, children, young adults, middle-aged adults, and old adults, are used in the classification system. The process of the system is divided into threephases: location, feature extraction, and age classification. Geometric features are used to distinguish whether the face is baby or child. Wrinkle features are used to classify the image into one of three adult groupsyoung adults, middle-aged adults, and old adults.

The article describes a database of emotional speech. Ten actors (5 female and 5 male) simulated the emotions, producing 10 German utterances (5 short and 5 longer sentences) which could be used in everyday communication and are interpretable in all applied emotions. The recordings were taken in an anechoic chamber with high-quality recording equipment. In addition to the sound electro-glottograms were recorded. The speech material comprises about 800 sentences (seven emotions * ten actors * ten sentences + some second versions). The complete database was evaluated in a perception test regarding the recognisability of emotions and their naturalness. Utterances recognised better than 80% and judged as natural by more than 60% of the listeners were phonetically labelled in a narrow transcription with special markers for voice-quality, phonatory and articulatory settings and articulatory features. The database can be accessed by the public via the internet (http://www.expressive-speech.net/emodb/).

We present the first shared task on detecting the intensity of emotion felt by the speaker of a tweet. We create the first datasets of tweets annotated for anger, fear, joy, and sadness intensities using a technique called best–worst scaling (BWS). We show that the annotations lead to reliable fine-grained intensity scores (rankings of tweets by intensity). The data was partitioned into training, development, and test sets for the competition. Twenty-two teams participated in the shared task, with the best system obtaining a Pearson correlation of 0.747 with the gold intensity scores. We summarize the machine learning setups, resources, and tools used by the participating teams, with a focus on the techniques and resources that are particularly useful for the task. The emotion intensity dataset and the shared task are helping improve our understanding of how we convey more or less intense emotions through language.

A major challenge in physiology-based emotion recognition is to establish an effective emotion recognizer for multi-users in the user-independent scenario. The recognition result is not satisfied because it ignores the difference in individual response pattern, which can be attributed to IRS (Individual Response Specificity) and SRS(Stimuli Response Specificity) in psychophysiology. To improve the performance of emotion recognition, this paper proposes a Group-Based IRS model by adaptively matching a suitable recognizer in accordance with user's IRS level. Specifically, the users are put into distinct groups by using cluster analysis techniques, where users within the same group have similar IRS level than other groups. Then physiological data of users from each group is utilized to build the corresponding emotion recognizers. After categorizing a new user into one group according to his IRS level, the new user's emotion state is predicted by the corresponding emotion recognizer. To validate our model, the affective physiological data was collected from 11 subjects in four induced emotions(neutral, sadness, fear and pleasure), three-channel bio-sensors were used to measure users electrocardiogram (ECG), galvanic skin response (GSR) and photo plethysmography (PPG). The results show that the recognition precision in Group-based IRS model is higher than general model.

Face detection has been one of the most studied topics in computer vision literature. Given an arbitrary image, the goal of face detection is to determine whether or not there are any faces in the image and, if present, return the image location and extent of each face. While this appears to be a trivial task for human beings, it is a very challenging task for computers. The difficulty associated with face detection can be attributed to many variations in scale, location, view point, illumination, occlusions, etc. Although there have been hundreds of reported approaches to face detection, if one were asked to name a single face detection algorithm that has the most impact in recent decades, it will most likely be the Viola and Jones face detection, which is capable of processing images extremely rapidly and achieve high detection rates. This project is going to study and understand the Viola-Jones algorithm by implementing the whole detection framework and based on the implementation, conduct experiment to hopefully further improve the performance.

Abstract Speech is an effective medium to express emotions and attitude through language. Finding the emotional content from a speech signal and identify the emotions from the speech utterances is an important task for the researchers. Speech emotion recognition has considered as an important research area over the last decade. Many researchers have been attracted due to the automated analysis of human affective behaviour. Therefore a number of systems, algorithms, and classifiers have been developed and outlined for the identification of emotional content of a speech from a person’s speech. In this study, available literature on various databases, different features and classifiers have been taken in to consideration for speech emotion recognition from assorted languages.


Question: A 20-machine analysis approach for the interpretation of facial expressions.

This paper discusses advancements in real-time face detection and facial expression recognition, exploring their implications for enhancing human-computer interaction through computer-animated agents and robots. These interactive entities introduce a social aspect to computing, prompting us to reconsider the potential applications of technology in everyday life. Face-to-face communication is characterized by rapid exchanges, occurring within approximately 40 milliseconds, necessitating the use of rich, sensory-based perceptual cues over slower symbolic reasoning.

The study highlights the development of a system capable of automatically detecting frontal faces in real-time video streams and coding them according to seven distinct emotional states: neutral, anger, disgust, fear, joy, sadness, and surprise. The face detection mechanism utilizes a cascade of feature detectors, trained with boosting techniques, to identify faces. Once a face is located, the system extracts image patches, which are then transformed into a Gabor representation. This representation is subsequently analyzed by a set of support vector machine (SVM) classifiers. A unique integration of Adaboost and SVMs boosts the system's performance.

To assess the system's accuracy, it was tested on the Cohn-Kanade dataset, a collection of posed facial expressions. The results showed high generalization capabilities to new subjects, with a 7-way forced choice correct classification. A noteworthy feature of the system is its ability to smoothly track changes in facial expressions over time, offering a valuable tool for analyzing facial expression dynamics in a non-intrusive manner.

The system has been successfully integrated into various platforms, including Sony's Aibo pet robot, ATR's RoboVie, and CU animator. Ongoing evaluations are being conducted for applications such as automatic reading tutors and assessments of human-robot interaction, demonstrating the system's potential to enrich human-computer engagement across multiple domains.

Morphometric measures and geometrical features are widely used to descript faces. Generally, they are extracted punctually from landmarks, namely anthropometric reference points. The aims are various, such as face recognition, facial expression recognition, face detection, study of changes in facial morphology due to growth, or dysmorphologies. Most of the time, landmarks were extracted with the help of an algorithm or manually located on the faces. Then, measures are computed or geometrical features are extracted to perform the scope of the study. This paper is intended as a survey collecting and explaining all these features, in order to provide a structured user database of the potential parameters and their charateristics. Firstly, facial soft-tissue landmarks are defined and contextualized; then the various morphometric measures are introduced and some results are given; lastly, the most important measures are compared to identify the best one for face recognition applications.

People routinely make various trait judgments from facial appearance, and such judgments affect important social outcomes. These judgments are highly correlated with each other, reflecting the fact that valence evaluation permeates trait judgments from faces. Trustworthiness judgments best approximate this evaluation, consistent with evidence about the involvement of the amygdala in the implicit evaluation of face trustworthiness. Based on computer modeling and behavioral experiments, I argue that face evaluation is an extension of functionally adaptive systems for understanding the communicative meaning of emotional expressions. Specifically, in the absence of diagnostic emotional cues, trustworthiness judgments are an attempt to infer behavioral intentions signaling approach/avoidance behaviors. Correspondingly, these judgments are derived from facial features that resemble emotional expressions signaling such behaviors: happiness and anger for the positive and negative ends of the trustworthiness continuum, respectively. The emotion overgeneralization hypothesis can explain highly efficient but not necessarily accurate trait judgments from faces, a pattern that appears puzzling from an evolutionary point of view and also generates novel predictions about brain responses to faces. Specifically, this hypothesis predicts a nonlinear response in the amygdala to face trustworthiness, confirmed in functional magnetic resonance imaging (fMRI) studies, and dissociations between processing of facial identity and face evaluation, confirmed in studies with developmental prosopagnosics. I conclude with some methodological implications for the study of face evaluation, focusing on the advantages of formally modeling representation of faces on social dimensions.

The work described in this paper represents the study and the attempt to make a contribution to one of the most stimulating and promising sectors in the field of emotion recognition, which is health care management. Multidisciplinary studies in artificial intelligence, augmented reality and psychology stressed out the importance of emotions in communication and awareness. The intent is to recognize human emotions, processing images streamed in real-time from a mobile device. The adopted techniques involve the use of open source libraries of visual recognition and machine learning approaches based on convolutional neural networks (CNN).

Facial expression synthesis has drawn much attention in the field of computer graphics and pattern recognition. It has been widely used in face animation and recognition. However, it is still challenging due to the high-level semantic presence of large and non-linear face geometry variations. This paper proposes a Geometry-Guided Generative Adversarial Network (G2-GAN) for continuously-adjusting and identity-preserving facial expression synthesis. We employ facial geometry (fiducial points) as a controllable condition to guide facial texture synthesis with specific expression. A pair of generative adversarial subnetworks is jointly trained towards opposite tasks: expression removal and expression synthesis. The paired networks form a mapping cycle between neutral expression and arbitrary expressions, with which the proposed approach can be conducted among unpaired data. The proposed paired networks also facilitate other applications such as face transfer, expression interpolation and expression-invariant face recognition. Experimental results on several facial expression databases show that our method can generate compelling perceptual results on different expression editing tasks.

Between October 2000 and December 2000 we collected a database of over 40,000 facial images of 68 people. Using the CMU 3D Room we imaged each person across 13 different poses, under 43 different illumination conditions, and with 4 different expressions. We call this database the CMU Pose, Illumination, and Expression (PIE) database. In this paper we describe the imaging hardware, the collection procedure, the organization of the database, several potential uses of the database, and how to obtain the database.

The ability to recognize affective states of a person we are communicating with is the core of emotional intelligence. Emotional intelligenceisafacetofhumanintelligencethathasbeenarguedtobe indispensable and perhaps the most important for successful interpersonal social interaction. This paper argues that next-generation human–computer interaction (HCI) designs need to include the essence of emotional intelligence—the ability to recognize a user’s affective states—in order to become more human-like, more effective, and more efficient. Affective arousal modulates all nonverbal communicative cues (facial expressions, body movements, and vocal and physiological reactions). In a face-to-face interaction, humans detect and interpret those interactive signals of their communicator with little or no effort. Yet design and development of an automated system that accomplishes these tasks is rather difficult. This paper surveys the past work in solving these problems by a computer and provides a set of recommendations for developing the first part of an intelligent multimodal HCI—an automatic personalized analyzer of a user’s nonverbal affective feedback.

Facial Expression Recognition is still one of the challenging fields in pattern recognition and machine learning science. Despite efforts made in developing various methods for this topic, existing approaches lack generalizability and almost all studies focus on more traditional hand-crafted features extraction to characterize facial expressions. Moreover, effective classifiers to model the spatial and temporary patterns embedded in facial expressions ignore the effects of facial attributes, such as age, on expression recognition even though research indicates that facial expression manifestation varies with ages. Although there are large amount of benchmark datasets available for the recognition of facial expressions, only few datasets contains faces of older adults. Consequently the current scientific literature has not exhausted this topic. Recently, deep learning methods have been attracting more and more researchers due to their great success in various computer vision tasks, mainly because they avoid a process of feature definition and extraction which is often very difficult due to the wide variability of the facial expressions. Based on the deep learning theory, a neural network for facial expression recognition in older adults is constructed by combining a Stacked Denoising Auto-Encoder method to pre-train the network and a supervised training that provides a fine-tuning adjustment of the network. For the supervised classification layer, the -class softmax classifier was implemented, where is the number of expressions to be recognized. The performance are evaluated on two benchmark datasets (FACES and Lifespan), that are the only ones that contain facial expressions of the elderly. The achieved results show the superiority of the proposed deep learning approach compared to the conventional non-deep learning based facial expression recognition methods used in this context.

We describe a deep learning based method for estimating 3D facial expression coefficients. Unlike previous work, our process does not relay on facial landmark detection methods as a proxy step. Recent methods have shown that a CNN can be trained to regress accurate and discriminative 3D morphable model (3DMM) representations, directly from image intensities. By foregoing landmark detection, these methods were able to estimate shapes for occluded faces appearing in unprecedented viewing conditions. We build on those methods by showing that facial expressions can also be estimated by a robust, deep, landmark-free approach. Our ExpNet CNN is applied directly to the intensities of a face image and regresses a 29D vector of 3D expression coefficients. We propose a unique method for collecting data to train our network, leveraging on the robustness of deep networks to training label noise. We further offer a novel means of evaluating the accuracy of estimated expression coefficients: by measuring how well they capture facial emotions on the CK+ and EmotiW-17 emotion recognition benchmarks. We show that our ExpNet produces expression coefficients which better discriminate between facial emotions than those obtained using state of the art, facial landmark detectors. Moreover, this advantage grows as image scales drop, demonstrating that our ExpNet is more robust to scale changes than landmark detectors. Finally, our ExpNet is orders of magnitude faster than its alternatives.

The face is one of the most important stimuli carrying social meaning. Thanks to the fast analysis of faces, we are able to judge physical attractiveness and features of their owners' personality, intentions, and mood. From one's facial expression we can gain information about danger present in the environment. It is obvious that the ability to process efficiently one's face is crucial for survival. Therefore, it seems natural that in the human brain there exist structures specialized for face processing. In this article, we present recent findings from studies on the neuronal mechanisms of face perception and recognition in the light of current theoretical models. Results from brain imaging (fMRI, PET) and electrophysiology (ERP, MEG) show that in face perception particular regions (i.e. FFA, STS, IOA, AMTG, prefrontal and orbitofrontal cortex) are involved. These results are confirmed by behavioral data and clinical observations as well as by animal studies. The developmental findings reviewed in this article lead us to suppose that the ability to analyze face-like stimuli is hard-wired and improves during development. Still, experience with faces is not sufficient for an individual to become an expert in face perception. This thesis is supported by the investigation of individuals with developmental disabilities, especially with autistic spectrum disorders (ASD).

Images containing faces are essential to intelligent vision-based human computer interaction, and research efforts in face processing include face recognition, face tracking, pose estimation, and expression recognition. However, many reported methods assume that the faces in an image or an image sequence have been identified and localized. To build fully automated systems that analyze the information contained in face images, robust and efficient face detection algorithms are required. Given a single image, the goal of face detection is to identify all image regions which contain a face regardless of its three-dimensional position, orientation, and lighting conditions. Such a problem is challenging because faces are nonrigid and have a high degree of variability in size, shape, color, and texture. Numerous techniques have been developed to detect faces in a single image, and the purpose of this paper is to categorize and evaluate these algorithms. We also discuss relevant issues such as data collection, evaluation metrics, and benchmarking. After analyzing these algorithms and identifying their limitations, we conclude with several promising directions for


Question: A 20-machine analysis approach for interpreting facial expressions.

This paper provides a comprehensive survey on the automatic recognition and analysis of human faces and facial expressions, highlighting the challenges and complexities involved in replicating human capabilities through automated systems. Despite the ease with which humans detect and identify faces, automating this process presents significant difficulties due to the multitude of subproblems that need to be addressed. These include the detection of patterns as faces, the identification of individual faces, the analysis of facial expressions, and the classification of faces based on their physical features. The development of systems capable of performing these tasks has potential applications in various fields, such as criminal identification, authentication in secure systems, and more. While much of the research has focused on identification, this paper also examines the state of the art in solving the broader range of problems related to face and expression analysis. The remarkable capabilities of the human visual system in these areas are discussed, serving as a benchmark for automated systems. Additionally, the paper introduces some novel approaches to these problems, offering insights into the future directions of research in facial recognition and analysis.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design
An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed

The importance of the face in social interaction and social intelligence is widely recognized in anthropology. Yet the adaptive functions of human facial expression remain largely unknown. An evolutionary model of human facial expression as behavioral adaptation can be constructed, given the current knowledge of the phenotypic variation, ecological contexts, and fitness consequences of facial behavior. Studies of facial expression are available, but results are not typically framed in an evolutionary perspective. This review identifies the relevant physical phenomena of facial expression and integrates the study of this behavior with the anthropological study of communication and sociality in general. Anthropological issues with relevance to the evolutionary study of facial expression include: facial expressions as coordinated, stereotyped behavioral phenotypes, the unique contexts and functions of different facial expressions, the relationship of facial expression to speech, the value of facial expressions as signals, and the relationship of facial expression to social intelligence in humans and in nonhuman primates. Human smiling is used as an example of adaptation, and testable hypotheses concerning the human smile, as well as other expressions, are proposed.

We have developed a near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals. The computational approach taken in this system is motivated by both physiology and information theory, as well as by the practical requirements of near-real-time performance and accuracy. Our approach treats the face recognition problem as an intrinsically two-dimensional (2-D) recognition problem rather than requiring recovery of three-dimensional geometry, taking advantage of the fact that faces are normally upright and thus may be described by a small set of 2-D characteristic views. The system functions by projecting face images onto a feature space that spans the significant variations among known face images. The significant features are known as "eigenfaces," because they are the eigenvectors (principal components) of the set of faces; they do not necessarily correspond to features such as eyes, ears, and noses. The projection operation characterizes an individual face by a weighted sum of the eigenface features, and so to recognize a particular face it is necessary only to compare these weights to those of known individuals. Some particular advantages of our approach are that it provides for the ability to learn and later recognize new faces in an unsupervised manner, and that it is easy to implement using a neural network architecture.

In this paper, a new dynamic facial expression recognition method is proposed. Dynamic facial expression recognition is formulated as a longitudinal groupwise registration problem. The main contributions of this method lie in the following aspects: 1) subject-specific facial feature movements of different expressions are described by a diffeomorphic growth model; 2) salient longitudinal facial expression atlas is built for each expression by a sparse groupwise image registration method, which can describe the overall facial feature changes among the whole population and can suppress the bias due to large intersubject facial variations; and 3) both the image appearance information in spatial domain and topological evolution information in temporal domain are used to guide recognition by a sparse representation method. The proposed framework has been extensively evaluated on five databases for different applications: the extended Cohn-Kanade, MMI, FERA, and AFEW databases for dynamic facial expression recognition, and UNBC-McMaster database for spontaneous pain expression monitoring. This framework is also compared with several state-of-the-art dynamic facial expression recognition methods. The experimental results demonstrate that the recognition rates of the new method are consistently higher than other methods under comparison.

More than 15 years after the early studies in Affective Computing (AC), [1] the problem of detecting and modeling emotions in the context of human-computer interaction (HCI) remains complex and largely unexplored. The detection and modeling of emotion is, primarily, the study and use of artificial intelligence (AI) techniques for the construction of computational models of emotion. The key challenges one faces when attempting to model emotion [2] are inherent in the vague definitions and fuzzy boundaries of emotion, and in the modeling methodology followed. In this context, open research questions are still present in all key components of the modeling process. These include, first, the appropriateness of the modeling tool employed to map emotional manifestations and responses to annotated affective states; second, the processing of signals that express these manifestations (i.e., model input); and third, the way affective annotation (i.e., model output) is handled. This paper touches upon all three key components of an affective model (i.e., input, model, output) and introduces the use of deep learning (DL) [3], [4], [5] methodologies for affective modeling from multiple physiological signals.

Most 3D face processing systems require feature detection and localisation, for example to crop, register, analyse or recognise faces. The three features often used in the literature are the tip of the nose, and the two inner corner of the eyes. Failure to localise these landmarks can cause the system to fail and they become very difficult to detect under large pose variations or when occlusion is present. In this paper, we present a proof-of-concept for a face labelling system capable of overcoming this problem, as a larger number of landmarks are employed. A set of points containing hand-placed landmarks is used as input data. The aim here is to retrieve the landmark's labels when some part of the face is missing. By using graph matching techniques to reduce the number of candidates, and translation and unit-quaternion clustering to determine a final correspondence, we evaluate the accuracy at which landmarks can be retrieved under changes in expression, orientation and in the presence of occlusions.

Abstract Speech is an effective medium to express emotions and attitude through language. Finding the emotional content from a speech signal and identify the emotions from the speech utterances is an important task for the researchers. Speech emotion recognition has considered as an important research area over the last decade. Many researchers have been attracted due to the automated analysis of human affective behaviour. Therefore a number of systems, algorithms, and classifiers have been developed and outlined for the identification of emotional content of a speech from a person’s speech. In this study, available literature on various databases, different features and classifiers have been taken in to consideration for speech emotion recognition from assorted languages.

Face recognition is receiving a significant attention due to the need of facing important challenges when developing real applications under unconstrained environments. The three most important challenges are facial occlusion, the problem of dealing with a single sample per subject (SSPS) and facial expression. This paper describes and analyzes various strategies that have been developed recently for overcoming these three major challenges that seriously affect the performance of real face recognition systems. This survey is organized in three parts. In the first part, approaches to tackle the challenge of facial occlusion are classified, illustrated and compared. The second part briefly describes the SSPS problem and the associated solutions. In the third part, facial expression challenge is illustrated. In addition, pros and cons of each technique are stated. Finally, several improvements for future research are suggested, providing a useful perspective for addressing new research in face recognition.

Here, we develop an audiovisual deep residual network for multimodal apparent personality trait recognition. The network is trained end-to-end for predicting the Big Five personality traits of people from their videos. That is, the network does not require any feature engineering or visual analysis such as face detection, face landmark alignment or facial expression recognition. Recently, the network won the third place in the ChaLearn First Impressions Challenge with a test accuracy of 0.9109.

Multimodal recognition of affective states is a difficult problem, unless the recording conditions are carefully controlled. For recognition “in the wild”, large variances in face pose and illumination, cluttered backgrounds, occlusions, audio and video noise, as well as issues with subtle cues of expression are some of the issues to target. In this paper, we describe a multimodal approach for video-based emotion recognition in the wild. We propose using summarizing functionals of complementary visual descriptors for video modeling. These features include deep convolutional neural network (CNN) based features obtained via transfer learning, for which we illustrate the importance of flexible registration and fine-tuning. Our approach combines audio and visual features with least squares regression based classifiers and weighted score level fusion. We report state-of-the-art results on the EmotiW Challenge for “in the wild” facial expression recognition. Our approach scales to other problems, and ranked top in the ChaLearn-LAP First Impressions Challenge 2016 from video clips collected in the wild.

Very large online music databases have recently been created by vendors, but they generally lack content-based retrieval methods. One exception is Allmusic.com which offers browsing by musical emotion, using human experts to classify several thousand songs into 183 moods. In this paper, machine learning techniques are used instead of human experts to extract emotions in Music. The classification is based on a psychological model of emotion that is extended to 23 specific emotion categories. Our results for mining the lyrical text of songs for specific emotion are promising, generate classification models that are human-comprehensible, and generate results that correspond to commonsense intuitions about specific emotions. Mining lyrics focused in this paper is one aspect of research which combines different classifiers of musical emotion such as acoustics and lyrical text.


Question: 20 machine examination of facial expressions.

Advancing Toward Emotionally Intelligent Human-Computer Interaction

The recognition of emotional states in human communication is a fundamental aspect of emotional intelligence, which is crucial for effective interpersonal relationships. Emotional intelligence, often considered the most significant component of human intelligence, is essential for successful social interactions. This paper posits that future designs in human-computer interaction (HCI) must incorporate the key element of emotional intelligence—the capability to discern a user's emotional states—to enhance the human-likeness, efficiency, and effectiveness of these systems. Affective arousal influences all nonverbal communication signals, including facial expressions, body language, and vocal and physiological responses. While humans naturally and effortlessly interpret these cues during face-to-face interactions, developing an automated system capable of performing these tasks remains a significant challenge.

This paper reviews previous efforts in enabling computers to recognize and interpret these nonverbal emotional signals and offers guidelines for the creation of an advanced, multimodal HCI component—an automated, personalized analyzer of a user's nonverbal emotional feedback. Such a system would be a pivotal first step in the evolution of emotionally intelligent HCI, allowing for more nuanced and empathetic interactions between humans and computers. By integrating this capability, HCI systems can better adapt to the emotional needs of users, potentially leading to more personalized, engaging, and supportive experiences.

We describe a deep learning based method for estimating 3D facial expression coefficients. Unlike previous work, our process does not relay on facial landmark detection methods as a proxy step. Recent methods have shown that a CNN can be trained to regress accurate and discriminative 3D morphable model (3DMM) representations, directly from image intensities. By foregoing landmark detection, these methods were able to estimate shapes for occluded faces appearing in unprecedented viewing conditions. We build on those methods by showing that facial expressions can also be estimated by a robust, deep, landmark-free approach. Our ExpNet CNN is applied directly to the intensities of a face image and regresses a 29D vector of 3D expression coefficients. We propose a unique method for collecting data to train our network, leveraging on the robustness of deep networks to training label noise. We further offer a novel means of evaluating the accuracy of estimated expression coefficients: by measuring how well they capture facial emotions on the CK+ and EmotiW-17 emotion recognition benchmarks. We show that our ExpNet produces expression coefficients which better discriminate between facial emotions than those obtained using state of the art, facial landmark detectors. Moreover, this advantage grows as image scales drop, demonstrating that our ExpNet is more robust to scale changes than landmark detectors. Finally, our ExpNet is orders of magnitude faster than its alternatives.

For survivors of childhood sexual abuse (CSA), verbal disclosure is often complex and painful. The authors examined the voluntary disclosure-nondisclosure of CSA in relation to nonverbal expressions of emotion in the face. Consistent with hypotheses derived from recent theorizing about the moral nature of emotion, CSA survivors who did not voluntarily disclose CSA showed greater facial expressions of shame, whereas CSA survivors who voluntarily disclosed CSA expressed greater disgust. Expressions of disgust also signaled sexual abuse accompanied by violence. Consistent with recent theorizing about smiling behavior, CSA nondisclosers made more polite smiles, whereas nonabused participants expressed greater genuine positive emotion. Discussion addressed the implications of these findings for the study of disclosure of traumatic events, facial expression, and the links between morality and emotion.

We propose that social attitudes, and in particular implicit prejudice, bias people's perceptions of the facial emotion displayed by others. To test this hypothesis, we employed a facial emotion change-detection task in which European American participants detected the offset (Study 1) or onset (Study 2) of facial anger in both Black and White targets. Higher implicit (but not explicit) prejudice was associated with a greater readiness to perceive anger in Black faces, but neither explicit nor implicit prejudice predicted anger perceptions regarding similar White faces. This pattern indicates that European Americans high in implicit racial prejudice are biased to perceive threatening affect in Black but not White faces, suggesting that the deleterious effects of stereotypes may take hold extremely early in social interaction.

This paper presents an approach to automatic visual emotion recognition from two modalities: face and body. Firstly, individual classifiers are trained from individual modalities. Secondly, we fuse facial expression and affective body gesture information first at a feature-level, in which the data from both modalities are combined before classification, and later at a decision-level, in which we integrate the outputs of the monomodal systems by the use of suitable criteria. We then evaluate these two fusion approaches, in terms of performance over monomodal emotion recognition based on facial expression modality only. In the experiments performed the emotion classification using the two modalities achieved a better recognition accuracy outperforming the classification using the individual facial modality. Moreover, fusion at the feature-level proved better recognition than fusion at the decision-level.

Deep Neural Networks (DNNs) have shown to outperform traditional methods in various visual recognition tasks including Facial Expression Recognition (FER). In spite of efforts made to improve the accuracy of FER systems using DNN, existing methods still are not generalizable enough in practical applications. This paper proposes a 3D Convolutional Neural Network method for FER in videos. This new network architecture consists of 3D Inception-ResNet layers followed by an LSTM unit that together extracts the spatial relations within facial images as well as the temporal relations between different frames in the video. Facial landmark points are also used as inputs to our network which emphasize on the importance of facial components rather than the facial regions that may not contribute significantly to generating facial expressions. Our proposed method is evaluated using four publicly available databases in subject-independent and cross-database tasks and outperforms state-of-the-art methods.

Spoken dialogue systems have received increased interest because they are potentially much more natural and powerful methods of communicating with machines than are current graphics-based interfaces. Wired for Speech presents basic research in the psychological and sociological aspects of voice synthesis and recognition. Its major lesson is that people attribute human characteristics to spoken dialogue systems for reasons related to human evolution. But although it contains interesting basic research, the book is mainly aimed at giving technological or marketing advice to those seeking to use voice interfaces when creating commercial applications. The book is oriented around a series of simple experiments designed to show just how pervasive psychological and social influences can be on the opinions and behaviors of people confronted with voice interfaces. Each chapter describes a basic research hypothesis, introduces an experiment to test it, and discusses its implications for designing voice interfaces: gender, personality, accent, ethnicity, emotion, number of distinct voices, use of " I " by the system, voices in concert with faces, mixed synthetic and recorded voices, context, and the effects of errors in human–computer cooperation. Although Wired for Speech is very accessible , especially to the non-scientist, it is written with an unusual bibliography style for an academic book: All references and details are given in a notes section at the end of the book, making up one third of the content. This narrative exposition style will probably not satisfy either type of reader: Scientists will be frustrated at the imprecision in argumen-tation, lack of detail in the book itself, and continually having to refer to the notes. The lack of detail also prevents the book from serving as a reference work. Meanwhile those needing advice when implementing voice interfaces will be puzzled at references to Chomsky, Grice, and Zemlin. Thus the book seems to suffer from not knowing its audience well, which is odd as this is precisely the lesson that the book tries to impart. Another complaint from the scientific point of view is the obsession this particular book has with casting every experimental result as advice for the business and marketing side of voice interfaces, typically concentrating on Web-based e-marketing examples such as buying books on line. Most of the experiments have sample sizes of between 40 and 50, but the authors seem ready to invite multi-billion-dollar businesses to immediately base their deployed systems on these results. Finally, and fundamentally, this …

system is the change in electrical properties of skin due to variation in physiological and psychological conditions. The change is caused by the degree to which a person's sweat glands are active. Psychological status of a person tends to make the glands active and this change the skin resistance. Drier the skin is, higher will be the skin resistance. This variation in skin resistance ranges from 5kΩ to 25kΩ. In the current work, a subject whose Galvanic skin response (GSR) is to be measured, has been shown/played a movie clipping, images or recorded audio signals. Depending upon the theme, emotion will be evoked in the subject. Due to the change in emotion, GSR varies. This variation in GSR is recorded for a fixed time interval. In the present work, a total of 75 subjects are selected. People from different age groups and social background, both male and female have been carefully considered. Data acquisition and analysis is carried out by using LabVIEW. The results obtained are convincing and follows the ground reality with a high level of accuracy. As the subject also gives the feedback about the emotions he/she experienced, the results obtained are validated.

In this paper, we present our latest progress in Emotion Recognition techniques, which combines acoustic features and facial features in both non-temporal and temporal mode. This paper presents the details of our techniques used in the Audio-Video Emotion Recognition subtask in the 2018 Emotion Recognition in the Wild (EmotiW) Challenge. After the multimodal results fusion, our final accuracy in Acted Facial Expression in Wild (AFEW) test dataset achieves 61.87%, which is 1.53% higher than the best results last year. Such improvements prove the effectiveness of our methods.

Some writers assume--and others deny--that all human beings distinguish emotions from nonemotions and divide the emotions into happiness, anger, fear, and so on. A review of ethnographic and cross-cultural studies on (a) emotion lexicons, (b) the emotions inferred from facial expressions, and (c) dimensions implicit in comparative judgments of emotions indicated both similarities and differences in how the emotions are categorized in different languages and cultures. Five hypotheses are reviewed: (a) Basic categories of emotion are pancultural, subordinate categories culture specific; (b) emotional focal points are pancultural, boundaries culture specific; (c) emotion categories evolved from a single primitive category of physiological arousal; (d) most emotion categories are culture specific but can be defined by pancultural semantic primitives; and (e) an emotion category is a script with both culture-specific and pancultural components.

We present a real-time face tracker in this paper The system has achieved a rate of 30% frameshecond using an HP-9000 workstation with a framegrabber and a Canon VC-CI camera. It can track a person 'sface while the person moves freely (e.g., walks, jumps, sits down and stands up) in a room. Three types of models have been employed in developing the system. First, we present a stochastic model to characterize skin-color distributions of human faces. The information provided by the model is sufJicient for tracking a human face in various poses and views. This model is adaptable to different people and different lighting conditions in real-time. Second, a motion model e's used to estimate image motion and to predict search window. Third, a camera model is used toprediet and to compensate for camera motion. The system can be applied to tele-conferencing and many HCI applications including lip-reading and gaze tracking. The principle in developing this system can be extended to other tracking problems such as tracking the human hand.


Question: A 20-machine analysis approach for evaluating facial expressions.

A novel technique for the analysis, identification, and interpretation of parameterized gestures is introduced, focusing on gestures that exhibit a systematic spatial variation, such as a pointing gesture where the parameter is the direction. This approach extends the conventional hidden Markov model (HMM) for gesture recognition by incorporating a global parametric variation into the output probabilities of the HMM states. A linear model of dependence is used to formulate an expectation-maximization (EM) algorithm for training the parametric HMM (PHMM). During testing, a similar EM algorithm is employed to simultaneously maximize the output likelihood of the PHMM for a given sequence and estimate the quantifying parameters. Results are presented using both visually derived and directly measured three-dimensional hand position data as input, showing that the PHMM outperforms standard HMM techniques in recognition accuracy and exhibits enhanced robustness in parameter estimation in the presence of input noise. Furthermore, the PHMM is extended to accommodate arbitrary smooth (nonlinear) dependencies, necessitating the use of a generalized expectation-maximization (GEM) algorithm for both training and the simultaneous recognition of the gesture and estimation of the parameter value. Experimental outcomes on a pointing gesture, where the nonlinear approach allows for the natural spherical coordinate parameterization of pointing direction, are discussed. Key terms include gesture recognition, hidden Markov models, expectation-maximization algorithm, time-series modeling, and computer vision.

Automatic recognition of people has received much attention during the recent years due to its many applications in different fields such as law enforcement, security applications or video indexing. Face recognition is an important and very challenging technique to automatic people recognition. Up to date, there is no technique that provides a robust solution to all situations and different applications that face recognition may encounter. In general, we can make sure that performance of a face recognition system is determined by how to extract feature vector exactly and to classify them into a group accurately. It, therefore, is necessary for us to closely look at the feature extractor and classifier. In this paper, Principle Component Analysis (PCA) is used to play a key role in feature extractor and the SVMs are used to tackle the face recognition problem. Support Vector Machines (SVMs) have been recently proposed as a new classifier for pattern recognition. We illustrate the potential of SVMs on the Cambridge ORL Face database, which consists of 400 images of 40 individuals, containing quite a high degree of variability in expression, pose, and facial details. The SVMs that have been used included the Linear (LSVM), Polynomial (PSVM), and Radial Basis Function (RBFSVM) SVMs. We provide experimental evidence which show that Polynomial and Radial Basis Function (RBF) SVMs performs better than Linear SVM on the ORL Face Dataset when both are used with one against all classification. We also compared the SVMs based recognition with the standard eigenface approach using the Multi-Layer Perceptron (MLP) Classification criterion.

In this paper, we address the issue of personality and interaction style recognition from profile pictures in Facebook. We recruited volunteers among Facebook users and collected a dataset of profile pictures, labeled with gold standard self-assessed personality and interaction style labels. Then, we exploited a bag-of-visual-words technique to extract features from pictures. Finally, different machine learning approaches were used to test the effectiveness of these features in predicting personality and interaction style traits. Our good results show that this task is very promising, because profile pictures convey a lot of information about a user and are directly connected to impression formation and identity management.

In this paper, an analysis of the effect of partial occlusion on facial expression recognition is investigated. The classification from partially occluded images in one of the six basic facial expressions is performed using a method based on Gabor wavelets texture information extraction, a supervised image decomposition method based on Discriminant Non-negative Matrix Factorization and a shape-based method that exploits the geometrical displacement of certain facial features. We demonstrate how partial occlusion affects the above mentioned methods in the classification of the six basic facial expressions, and indicate the way partial occlusion affects human observers when recognizing facial expressions. An attempt to specify which part of the face (left, right, lower or upper region) contains more discriminant information for each facial expression, is also made and conclusions regarding the pairs of facial expressions misclassifications that each type of occlusion introduces, are drawn. 2008 Published by Elsevier B.V.

In this paper, a novel algorithm is proposed for facial expression recognition by integrating curvelet transform and online sequential extreme learning machine (OSELM) with radial basis function (RBF) hidden node having optimal network architecture. In the proposed algorithm, the curvelet transform is firstly applied to each region of the face image divided into local regions instead of whole face image to reduce the curvelet coefficients too huge to classify. Feature set is then generated by calculating the entropy, the standard deviation and the mean of curvelet coefficients of each region. Finally, spherical clustering (SC) method is employed to the feature set to automatically determine the optimal hidden node number and RBF hidden node parameters of OSELM by aim of increasing classification accuracy and reducing the required time to select the hidden node number. So, the learning machine is called as OSELM-SC. It is constructed two groups of experiments: The aim of the first one is to evaluate the classification performance of OSELM-SC on the benchmark datasets, i.e., image segment, satellite image and DNA. The second one is to test the performance of the proposed facial expression recognition algorithm on the Japanese Female Facial Expression database and the Cohn-Kanade database. The obtained experimental results are compared against the state-of-the-art methods. The results demonstrate that the proposed algorithm can produce effective facial expression features and exhibit good recognition accuracy and robustness.

Our objective is to index people in a TV-Content. In this context, because of multi-face shots and non-speaking face shots, it is difficult to determine which face is speaking. There is no guaranteed synchronization between sequences of a person’s appearance and sequences of his or her speech. In this work, we want to separate talking and non-talking faces by detecting lip motion. We propose a method to detect the lip motion by measuring the degree of disorder of pixel directions around the lip. Results of experiments on a TV-Show database show that a high correct classification rate can be achieved by the proposed method.

Detection of human emotions is an essential part of affect-aware human-computer interaction (HCI). In daily conversations, the preferred way of describing affects is by using categorical emotion labels (e.g., sad, anger, surprise). In categorical emotion classification, multiple descriptors (with different degrees of relevance) can be assigned to a sample. Perceptual evaluations have relied on primary and secondary emotions to capture the ambiguous nature of spontaneous recordings. Primary emotion is the most relevant category felt by the evaluator. Secondary emotions capture other emotional cues also conveyed in the stimulus. In most cases, the labels collected from the secondary emotions are discarded, since assigning a single class label to a sample is preferred from an application perspective. In this work, we take advantage of both types of annotations to improve the performance of emotion classification. We collect the labels from all the annotations available for a sample and generate primary and secondary emotion labels. A classifier is then trained using multitask learning with both primary and secondary emotions. We experimentally show that considering secondary emotion labels during the learning process leads to relative improvements of 7.9% in F1-score for an 8-class emotion classification task.

Emotion recognition in speech is a topic on which little research has been done to-date. In this paper, we discuss why emotion recognition in speech is a significant and applicable research topic, and present a system for emotion recognition using oneclass-in-one neural networks. By using a large database of phoneme balanced words, our system is speakerand context-independent. We achieve a recognition rate of approximately 50% when testing eight emotions.

We consider the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise. We cast the recognition problem as one of classifying among multiple linear regression models and argue that new theory from sparse signal representation offers the key to addressing this problem. Based on a sparse representation computed by C1-minimization, we propose a general classification algorithm for (image-based) object recognition. This new framework provides new insights into two crucial issues in face recognition: feature extraction and robustness to occlusion. For feature extraction, we show that if sparsity in the recognition problem is properly harnessed, the choice of features is no longer critical. What is critical, however, is whether the number of features is sufficiently large and whether the sparse representation is correctly computed. Unconventional features such as downsampled images and random projections perform just as well as conventional features such as eigenfaces and Laplacianfaces, as long as the dimension of the feature space surpasses certain threshold, predicted by the theory of sparse representation. This framework can handle errors due to occlusion and corruption uniformly by exploiting the fact that these errors are often sparse with respect to the standard (pixel) basis. The theory of sparse representation helps predict how much occlusion the recognition algorithm can handle and how to choose the training images to maximize robustness to occlusion. We conduct extensive experiments on publicly available databases to verify the efficacy of the proposed algorithm and corroborate the above claims.

In this contribution we introduce speech emotion recognition by use of continuous hidden Markov models. Two methods are propagated and compared throughout the paper. Within the first method a global statistics framework of an utterance is classified by Gaussian mixture models using derived features of the raw pitch and energy contour of the speech signal. A second method introduces increased temporal complexity applying continuous hidden Markov models considering several states using low-level instantaneous features instead of global statistics. The paper addresses the design of working recognition engines and results achieved with respect to the alluded alternatives. A speech corpus consisting of acted and spontaneous emotion samples in German and English language is described in detail. Both engines have been tested and trained using this equivalent speech corpus. Results in recognition of seven discrete emotions exceeded 86% recognition rate. As a basis of comparison the similar judgment of human deciders classifying the same corpus at 79.8% recognition rate was analyzed.

Faces constitute a unique and widely used category of stimuli. In spite of their importance, there are few collections of faces for use in research, none of which adequately represent the different ages of faces across the lifespan. This lack of a range of ages has limited the majority of researchers to using predominantly young faces as stimuli even when their hypotheses concern both young and old participants. We describe a database of 575 individual faces ranging from ages 18 to 93. Our database was developed to be more representative of age groups across the lifespan, with a special emphasis on recruiting older adults. The resulting database has faces of 218 adults age 18-29, 76 adults age 30-49, 123 adults age 50-69, and 158 adults age 70 and older. These faces may be acquired for research purposes from http://agingmind.cns.uiuc.edu/facedb/. This will allow researchers interested in using facial stimuli access to a wider age range of adult faces than has previously been available.