Question: Speech emotion recognition utilizing the k-nearest neighbor algorithm in speech processing.

This document introduces a comprehensive German emotional speech database designed for research in emotion recognition. The database was created by ten professional actors (five females and five males) who portrayed seven distinct emotions through 10 German sentences, chosen for their commonality in daily communication and their interpretability across all expressed emotions. To ensure high-quality recordings, the speech samples were captured in an anechoic chamber using state-of-the-art recording equipment. Besides the audio recordings, electro-glottograms were also collected to provide additional insights into the vocal mechanisms during speech production. The total dataset consists of approximately 800 sentences (seven emotions * ten actors * ten sentences, plus additional takes for some sentences). A perception test was conducted to evaluate the database, assessing the recognizability of emotions and their perceived naturalness. Sentences recognized with an accuracy of over 80% and deemed natural by more than 60% of the listeners were selected for further analysis. These chosen utterances were then phonetically transcribed in detail, incorporating markers for voice quality, phonatory, articulatory settings, and articulatory features. The entire database is accessible to the public online, making it a valuable resource for researchers studying speech processing and emotion recognition algorithms, such as the k-nearest neighbor algorithm.

Facial Expression Recognition is still one of the challenging fields in pattern recognition and machine learning science. Despite efforts made in developing various methods for this topic, existing approaches lack generalizability and almost all studies focus on more traditional hand-crafted features extraction to characterize facial expressions. Moreover, effective classifiers to model the spatial and temporary patterns embedded in facial expressions ignore the effects of facial attributes, such as age, on expression recognition even though research indicates that facial expression manifestation varies with ages. Although there are large amount of benchmark datasets available for the recognition of facial expressions, only few datasets contains faces of older adults. Consequently the current scientific literature has not exhausted this topic. Recently, deep learning methods have been attracting more and more researchers due to their great success in various computer vision tasks, mainly because they avoid a process of feature definition and extraction which is often very difficult due to the wide variability of the facial expressions. Based on the deep learning theory, a neural network for facial expression recognition in older adults is constructed by combining a Stacked Denoising Auto-Encoder method to pre-train the network and a supervised training that provides a fine-tuning adjustment of the network. For the supervised classification layer, the -class softmax classifier was implemented, where is the number of expressions to be recognized. The performance are evaluated on two benchmark datasets (FACES and Lifespan), that are the only ones that contain facial expressions of the elderly. The achieved results show the superiority of the proposed deep learning approach compared to the conventional non-deep learning based facial expression recognition methods used in this context.

Audio signal classification (ASC) consists of extracting relevant features from a sound, and of using these features to identify into which of a set of classes the sound is most likely to fit. The feature extraction and grouping algorithms used can be quite diverse depending on the classification domain of the application. This paper presents background necessary to understand the general research domain of ASC, including signal processing, spectral analysis, psychoacoustics and auditory scene analysis. Also presented are the basic elements of classification systems. Perceptual and physical features are discussed, as well as clustering algorithms and analysis duration. Neural nets and hidden Markov models are discussed as they relate to ASC. These techniques are presented with an overview of the current state of the ASC research literature.

A set of simple new procedures has been developed to enable the real-time manipulation of speech parameters. The proposed method uses pitch-adaptive spectral analysis combined with a surface reconstruction method in the time± frequency region. The method also consists of a fundamental frequency (F0) extraction using instantaneous frequency calculation based on a new concept called `fundamentalness'. The proposed procedures preserve the details of time± frequency surfaces while almost perfectly removing ®ne structures due to signal periodicity. This close-to-perfect elimination of interferences and smooth F0 trajectory allow for over 600% manipulation of such speech parameters as pitch, vocal tract length, and speaking rate, while maintaining high reproductive quality. Ó 1999 Elsevier Science B.V. All rights reserved.

A novel, data-driven approach to voice activity detection is presented. The approach is based on Long Short-Term Memory Recurrent Neural Networks trained on standard RASTA-PLP frontend features. To approximate real-life scenarios, large amounts of noisy speech instances are mixed by using both read and spontaneous speech from the TIMIT and Buckeye corpora, and adding real long term recordings of diverse noise types. The approach is evaluated on unseen synthetically mixed test data as well as a real-life test set consisting of four full-length Hollywood movies. A frame-wise Equal Error Rate (EER) of 33.2% is obtained for the four movies and an EER of 9.6% is obtained for the synthetic test data at a peak SNR of 0 dB, clearly outperforming three state-of-the-art reference algorithms under the same conditions.

This paper presents new Speaker Identification and Speaker Verification systems based on the use of new feature vectors extracted from the speech signal. The proposed structure combine between the most successful Mel Frequency Cepstral Coefficients and new features which are the Short Time Zero Crossing Rate of the signal. A comparison between speaker recognition systems based on Gaussian mixture models using the well known Mel Frequency Cepstral Coefficients and the novel systems based on the use of a combination between both reduced Mel Frequency Cepstral Coefficients features vectors and Short Time Zero Crossing Rate features is given. This comparison proves that the use of the new reduced feature vectors help to improve the system’s performance and also help to reduce the time and memory complexity of the system which is required for realistic applications that suffer from computational resource limitation. The experiments were performed on speakers from TIMIT database for different training durations. The suggested systems performances are evaluated against the baseline systems. The increase of the proposed systems performances are well observed for identification experiments and the decrease of Equal Error Rates are also remarkable for verification experiments. Experimental results demonstrate the effectiveness of the new approach which avoids the use of more complex algorithms or the combination of different approaches requiring lengthy calculation. Keywords—GMM; speaker verification; speaker recognition; speaker identification

This paper introduces a new corpus of read English speech, suitable for training and evaluating speech recognition systems. The LibriSpeech corpus is derived from audiobooks that are part of the LibriVox project, and contains 1000 hours of speech sampled at 16 kHz. We have made the corpus freely available for download, along with separately prepared language-model training data and pre-built language models. We show that acoustic models trained on LibriSpeech give lower error rate on the Wall Street Journal (WSJ) test sets than models trained on WSJ itself. We are also releasing Kaldi scripts that make it easy to build these systems.

As is known for centuries, humans exhibit an electrical profile. This profile is altered through various psychological and physiological processes, which can be measured through biosignals; e.g., electromyography (EMG) and electrodermal activity (EDA). These biosignals can reveal our emotions and, as such, can serve as an advanced man-machine interface (MMI) for empathic consumer products. However, such a MMI requires the correct classification of biosignals to emotion classes. This chapter starts with an introduction on biosignals for emotion detection. Next, a state-of-the-art review is presented on automatic emotion classification. Moreover, guidelines are presented for affective MMI. Subsequently, a research is presented that explores the use of EDA and three facial EMG signals to determine neutral, positive, negative, and mixed emotions, using recordings of 21 people. A range of techniques is tested, which resulted in a generic framework for automated emotion classification with up to 61.31% correct classification of the four emotion classes, without the need of personal profiles. Among various other directives for future research, the results emphasize the need for parallel processing of multiple biosignals. That men are machines (whatever else they may be) has long been suspected; but not till our generation have men fairly felt in concrete just what wonderful psycho-neuro-physical mechanisms they are. William James (1893; 1842 – 1910) A. Fred, J. Filipe, and H. Gamboa (Eds.): BIOSTEC 2009, CCIS 52, pp. 21–47, 2010. c © Springer-Verlag Berlin Heidelberg 2010 22 E.L. van den Broek et al.

The advances in automatic facial expression recognition make possible to mine and characterize large amounts of data, opening a wide research domain on behavioral understanding. In this paper, we leverage the use of a state-of-the-art facial expression recognition technology to characterize users of a popular type of online social video, conversational vlogs. First, we propose the use of several activity cues to characterize vloggers based on frame-by-frame estimates of facial expressions of emotion. Then, we present results for the task of automatically predicting vloggers' personality impressions using facial expressions and the Big-Five traits. Our results are promising, specially for the case of the Extraversion impression, and in addition our work poses interesting questions regarding the representation of multiple natural facial expressions occurring in conversational video.

This paper proposes an algorithm for the recognition and separation of speech signals in non-stationary noise, such as another speaker. We present a method to combine hidden Markov models (HMMs) trained for the speech and noise into a factorial HMM to model the mixture signal. Robustness is obtained by separating the speech and noise signals in a feature domain, which discards unnecessary information. We use mel-cepstral coefficients (MFCCs) as features, and estimate the distribution of mixture MFCCs from the distributions of the target speech and noise. A decoding algorithm is proposed for finding the state transition paths and estimating gains for the speech and noise from a mixture signal. Simulations were carried out using speech material where two speakers were mixed at various levels, and even for high noise level (9 dB above the speech level), the method produced relatively good (60% word recognition accuracy) results. Audio demonstrations are available at www.cs.tut.fi/ ̃tuomasv.

Modeling the phases of audio signals has received significantly less attention in comparison to the modeling of magnitudes. This paper proposes to use linear least squares and neural networks to predict phases from the neighboring points only in the phase spectrum. The simulation results show that there is a structure in the phase components which could be used in further analysis algorithms based on the phase spectrum.


Question: Speech emotion recognition employing a k-nearest neighbor algorithm approach.

This paper introduces a novel method for the classification of emotions in speech signals, utilizing short-time log frequency power coefficients (LFPC) and discrete hidden Markov models (HMMs). Traditional approaches to emotion classification in speech often employ statistics of fundamental frequency, energy contour, duration of silence, and voice quality. However, these methods experience significant performance declines when attempting to classify more than two emotion categories. The proposed method offers a text-independent solution, representing speech signals through LFPC and employing a discrete HMM as the classifier. Emotions are categorized into six distinct classes: Anger, Disgust, Fear, Joy, Sadness, and Surprise. A database of 60 emotional utterances, sourced from twelve speakers, is utilized for both training and testing the system. The performance of LFPC is benchmarked against linear prediction cepstral coefficients (LPCC) and mel-frequency cepstral coefficients (MFCC), which are commonly used in speech recognition systems. The results indicate an average classification accuracy of 78% and a peak accuracy of 96% for the six emotion categories, significantly surpassing the 17% probability of a random selection for a set of six categories. The findings also highlight the superiority of LFPC as a feature parameter for emotion classification over traditional methods. (C) 2003 Elsevier B.V. All rights reserved.

In a different context, the k-nearest neighbor (k-NN) algorithm has been explored for speech emotion recognition, offering a non-parametric approach to classify emotions based on proximity to training data points. This method, unlike HMMs, does not require the modeling of the temporal dynamics of speech but instead relies on the distance between feature vectors in a high-dimensional space. The k-NN algorithm can be particularly effective in scenarios where the emotional content of speech is characterized by distinct, localized features, making it a promising alternative to HMMs in certain applications of speech emotion recognition.

The Face Expression Based Emoticon Identification (FEBEI) system is an open source extension to the Tracker.js framework which converts a human facial expression to the best matching emoticon. The contribution of this project was to build this robust classifier which can identify facial expression in real time without any reliance on an external server or computation node. An entirely client-side JavaScript implementation has clear privacy benefits as well as the avoidance of any lag inherent in uploading and downloading images. We accomplished this by utilizing several computationally efficient methods. Tracking.js provided a Viola Jones based face detector which we used to pass facial images to our own implementation of an eigenemotion detection system which was trained to distinguish between happy and angry faces. We have implemented a similar eigenface classifier in python and have trained a Convoluted Neural Network (CNN) to classify emotions to provide a comparative view of its advantages. We aim to make FEBEI easily extendable so that the developer community will be able to add classifiers for more emoticon.

Nearly perfect speech recognition was observed under conditions of greatly reduced spectral information. Temporal envelopes of speech were extracted from broad frequency bands and were used to modulate noises of the same bandwidths. This manipulation preserved temporal envelope cues in each band but restricted the listener to severely degraded information on the distribution of spectral energy. The identification of consonants, vowels, and words in simple sentences improved markedly as the number of bands increased; high speech recognition performance was obtained with only three bands of modulated noise. Thus, the presentation of a dynamic temporal pattern in only a few broad spectral regions is sufficient for the recognition of speech.

In this paper, we present our latest progress in Emotion Recognition techniques, which combines acoustic features and facial features in both non-temporal and temporal mode. This paper presents the details of our techniques used in the Audio-Video Emotion Recognition subtask in the 2018 Emotion Recognition in the Wild (EmotiW) Challenge. After the multimodal results fusion, our final accuracy in Acted Facial Expression in Wild (AFEW) test dataset achieves 61.87%, which is 1.53% higher than the best results last year. Such improvements prove the effectiveness of our methods.

A sign language recognition system is an attempt to bring the speech and the hearing impaired community closer to more regular and convenient forms of communication. Thus, this system requires to recognize the gestures from a sign language and convert them to a form easily understood by the hearing. The model that has been proposed in this paper recognizes static images of the signed alphabets in the Indian Sign Language. Unlike the alphabets in other sign languages like the American Sign Language and the Chinese Sign language, the ISL alphabet are both single-handed and double-handed. Hence, to make recognition easier the model first categorizes them as single-handed or double-handed. For both categories two kinds of features, namely HOG and SIFT, are extracted for a set of training images and are combined in a single matrix. After which, HOG and SIFT features for the input test image are combined with the HOG and SIFT feature matrices of the training set. Correlation is computed for these matrices and is fed to a K-Nearest Neighbor Classifier to obtain the resultant classification of the test image.

In this paper, we address the problem of automatically recognizing emotions in still images. While most of current work focus on improving whole-image representations using CNNs, we argue that discovering affective regions and supplementing local features will boost the performance, which is inspired by the observation that both global distributions and salient objects carry massive sentiments. We propose an algorithm to discover affective regions via deep framework, in which we use an off-the-shelf tool to generate N object proposals from a query image and rank these proposals with their objectness scores. Then, each proposal's sentiment score is computed using a pre-trained and fine-tuned CNN model. We combine both scores and select top K regions from the N candidates. These K regions are regarded as the most affective ones of the input image. Finally, we extract deep features from the whole-image and the selected regions, respectively, and sentiment label is predicted. The experiments show that our method is able to detect the affective local regions and achieve state-of-the-art performances on several popular datasets.

Depression is a major mental health disorder that is rapidly affecting lives worldwide. Depression not only impacts emotional but also physical and psychological state of the person. Its symptoms include lack of interest in daily activities, feeling low, anxiety, frustration, loss of weight and even feeling of selfhatred. This report describes work done by us for Audio Visual Emotion Challenge (AVEC) 2017 during our second year BTech summer internship. With the increase in demand to detect depression automatically with the help of machine learning algorithms, we present our multimodal feature extraction and decision level fusion approach for the same. Features are extracted by processing on the provided Distress Analysis Interview Corpus-Wizard of Oz (DAIC-WOZ) database. Gaussian Mixture Model (GMM) clustering and Fisher vector approach were applied on the visual data; statistical descriptors on gaze, pose; low level audio features and head pose and text features were also extracted. Classification is done on fused as well as independent features using Support Vector Machine (SVM) and neural networks. The results obtained were able to cross the provided baseline on validation data set by 17% on audio features and 24.5% on video features. Keywords—AVEC 2017, SVM, Depression, Neural network, RMSE, MAE, fusion, speech processing.

Over the last 20 years, several different techniques have been proposed for computer recognition of human faces. The purpose of this paper is to compare two simple but general strategies on a common database (frontal images of faces of 47 people: 26 males and 21 females, four images per person). We have developed and implemented two new algorithms; the first one is based on the computation of a set of geometrical features, such as nose width and length, mouth position, and chin shape, and the second one is based on almost-grey-level template matching. The results obtained on the testing sets (about 90% correct recognition using geometrical features and perfect recognition using template matching) favor our implementation of the template-matching approach.

0957-4174/$ see front matter 2008 Elsevier Ltd. A doi:10.1016/j.eswa.2008.09.042 q Part of the content of this paper has been publi International Conference on Multimedia, 2005. * Corresponding author. Tel.: +886 2 29393091x67 E-mail address: mkshan@cs.nccu.edu.tw (M.-K. Sh With the growth of digital music, the development of music recommendation is helpful for users to pick desirable music pieces from a huge repository of music. The existing music recommendation approaches are based on a user’s preference on music. However, sometimes, it might better meet users’ requirement to recommend music pieces according to emotions. In this paper, we propose a novel framework for emotion-based music recommendation. The core of the recommendation framework is the construction of the music emotion model by affinity discovery from film music, which plays an important role in conveying emotions in film. We investigate the music feature extraction and propose the Music Affinity Graph and Music Affinity Graph-Plus algorithms for the construction of music emotion model. Experimental result shows the proposed emotion-based music recommendation achieves 85% accuracy in average. 2008 Elsevier Ltd. All rights reserved.

CH2118-8/85/0000-0387 $1.00 © 1985 IEEE 387 ABSTRACT. In this study a vector quantIzation (VQ) codebook was system. In the other, Shore and Burton 112] used word-based VQ used as an efficient means of characterizing the short-time spectral codebooks and reported good performance in speaker-trained isolatedfeatures of a speaker. A set of such codebooks were then used to word recognition experiments. Here, instead of using word-based VQ recognize the identity of an unknown speaker from his/her unlabelled spoken utterances based on a minimum distance (distortion) codebooks to characterize the phonetic contents of isolated words, we propose to use speaker-based VQ codebooks to characterize the classification rule. A series of speaker recognition experiments was variability of short-time acoustic features of speakers. performed using a 100-talker (50 male and 50 female) telephone recording database consisting of isolated digit utterances. For ten random but different isolated digits, over 98% speaker identification H. Speaker-based VQ Codebook Approach to Speaker accuracy was achieved. The effects, on performance, of different Characterization and Recognition system parameters such as codebook sizes, the number of test digits, phonetic richness of the text, and difference in recording sessions Were also studied in detail. A set of short-time raw feature vectors of a speaker can be used directly to represent the essential acoustical, phonological or physiological characteristics of that speaker if the training set includes sufficient variations. However such a direct representation is not practical when the number of training vectors is large. The memory requirements for storage and computational complexity in the recognition phase eventually become prohibitively high. Therefore an Automatic speaker recognition has long been an interesting and challenging problem to speech researchers [1-101. The problem, efficient way of compressing the training data had to be found. In order to compress the original data to a small set of representative depending upon the nature of the final task, can be classified into two points, we used a VQ codebook with a small number of codebook different categories: speaker verification and speaker identification. In entries. a speaker verification task, the recognizer is asked to verify an identity claim made by an unknown speaker and a decision to reject or accept the identity claim is made. In a speaker identification task the recognizer is asked to decide which out of a population of N speakers is best classified as the unknown speaker. The decision may include a choice of "none of the above" (i.e., a choice that the specific speaker is not in a given closed set of speakers). The input speech material used for speaker recognition can be either text-dependent (text-constrained) or text-independent (text-free). In the text-dependent mode the speaker is asked to utter a prescribed text. The utterance is then The speaker-based VQ codebook generation can be summarized as follows: Given a set of I training feature vectors, {a1,a2 a) characterizing the variability of a speaker, we want to find a partitioning of the feature vector space, {S1,S2 SM}, for that particular speaker where, 5, the whole feature space is represented as S S1 US2 U . . . US. Each partition, S, forms a convex, nonoverlapping region and every vector inside S is represented by the respo5'tis'1g ceatcoid. vector, b1, cf 5,. The p titioning is done in such a way that the average distortion

Recently, the physiological signals have been analyzed more and more, especially in the context of everyday life activities such as watching video or looking at pictures. Tracking these signals gives access to the mental state of the user (interest, tiredness, stress) but also to his emotions (sadness, fright, happiness). The analysis of the reader's physiological signals during reading can provide a better understanding of the reader's feelings but also a better understanding of the documents. Our main research direction is to find the relationship between a change in the reader's physiological signal and the content of the reading. As a first step, we investigate whether it is possible to distinguish a manga (Japanese comic book) from another by analyzing the physiological signals of the reader. We use 3 different manga genres (horror, romance, comedy) and try to predict which one is read by analyzing the features extracted from the physiological signals of the reader. Our method uses the blood volume pulse, the electrodermal activity and the skin temperature of the reader while reading. We show that by using these physiological signals with a support vector machine we can retrieve which manga has been read with a 90% average accuracy.


Question: Emotion identification through speech processing utilizing the k-nearest neighbor algorithm.

This paper delves into the underexplored area of emotion recognition through speech, highlighting its relevance and practical implications. We propose an innovative system for identifying emotions in spoken language, employing one-class-in-one neural networks. To ensure the system's versatility, we utilize a comprehensive database featuring phonetically balanced words, making our approach speaker-independent and context-independent. Our results demonstrate a promising recognition accuracy of roughly 50% across eight distinct emotions, showcasing the system's potential in the field of speech emotion recognition.

**Question**: A Comparative Study of Machine Learning Algorithms for Predicting Software Defects

**Documents**: A comparative study of machine learning algorithms for software defect prediction
This paper presents a comparative study of machine learning algorithms for software defect prediction. We compare the performance of several algorithms including Support Vector Machines (SVM), Random Forest (RF), and Neural Networks (NN). We evaluate these algorithms on a large dataset of open-source software projects. Our results show that RF outperforms the other algorithms in terms of accuracy, precision, and recall. We also find that the performance of all algorithms improves with the size of the dataset. This study provides insights into the effectiveness of different machine learning algorithms for software defect prediction and highlights the importance of dataset size.

**Similar Documents**: In this comprehensive analysis, we conduct a detailed comparison of various machine learning algorithms for the prediction of software defects. Our investigation encompasses prominent techniques such as Support Vector Machines (SVM), Random Forest (RF), and Neural Networks (NN). Utilizing a substantial dataset derived from open-source software initiatives, we assess the efficacy of these algorithms. Our findings reveal that RF exhibits superior performance compared to SVM and NN, excelling in accuracy, precision, and recall metrics. Notably, we observe that the predictive capabilities of all algorithms enhance as the dataset's size increases. This study not only underscores the varying degrees of effectiveness of different machine learning algorithms in software defect prediction but also underscores the pivotal role of dataset scale in determining algorithmic performance.

This paper explores the role of voice quality in the communication of emotions, moods and attitudes. Listeners reactions to an utterance synthesised with seven different voice qualities were elicited in terms of pairs of opposing affective attributes. The voice qualities included harsh voice, tense voice, modal voice, breathy voice, whispery voice, creaky voice and lax–creaky voice. These were synthesised using a formant synthesiser, and the voice source parameter settings were guided by prior analytic studies as well as auditory judgements. Results offer support for some past observations on the association of voice quality and affect, and suggest a number of refinements in some cases. Listeners ratings further suggest that these qualities are considerably more effective in signalling milder affective states than the strong emotions. It is clear that there is no one-to-one mapping between voice quality and affect: rather a given quality tends to be associated with a cluster of affective attributes. 2002 Elsevier Science B.V. All rights reserved.

The author investigates automatic recognition of singers’ gender and age through audio features using deep neural network (DNN). Features of each singing voice, fundamental frequency and Mel-Frequency Cepstrum Coefficients (MFCC) are extracted for neural network training. 10,000 singing voice from Smule’s Sing! Karaoke app is used for training and evaluation, and the DNN-based method achieves an average recall of 91% for gender classification and 36% for age identification.

The Development Workflow There are two major stages within isolated word recognition: a training stage and a testing stage. Training involves “teaching” the system by building its dictionary, an acoustic model for each word that the system needs to recognize. In our example, the dictionary comprises the digits ‘zero’ to ‘nine’. In the testing stage we use acoustic models of these digits to recognize isolated words using a classification algorithm. The development workflow consists of three steps: Speech acquisition • Speech analysis • User interface development •

Nowadays, the automatic detection of emotions is employed by many applications across different fields like security informatics, e-learning, humor detection, targeted advertising, etc. Many of these applications focus on social media. In this study, we address the problem of emotion detection in Arabic tweets. We focus on the supervised approach for this problem where a classifier is trained on an already labeled dataset. Typically, such a training set is manually annotated, which is expensive and time consuming. We propose to use an automatic approach to annotate the training data based on using emojis, which are a new generation of emoticons. We show that such an approach produces classifiers that are more accurate than the ones trained on a manually annotated dataset. To achieve our goal, a dataset of emotional Arabic tweets is constructed, where the emotion classes under consideration are: anger, disgust, joy and sadness. Moreover, we consider two classifiers: Support Vector Machine (SVM) and Multinomial Naive Bayes (MNB). The results of the tests show that the automatic labeling approaches using SVM and MNB outperform manual labeling approaches.

A novel, data-driven approach to voice activity detection is presented. The approach is based on Long Short-Term Memory Recurrent Neural Networks trained on standard RASTA-PLP frontend features. To approximate real-life scenarios, large amounts of noisy speech instances are mixed by using both read and spontaneous speech from the TIMIT and Buckeye corpora, and adding real long term recordings of diverse noise types. The approach is evaluated on unseen synthetically mixed test data as well as a real-life test set consisting of four full-length Hollywood movies. A frame-wise Equal Error Rate (EER) of 33.2% is obtained for the four movies and an EER of 9.6% is obtained for the synthetic test data at a peak SNR of 0 dB, clearly outperforming three state-of-the-art reference algorithms under the same conditions.

While many lexica annotated with words polarity are available for sentiment analysis, very few tackle the harder task of emotion analysis and are usually quite limited in coverage. In this paper, we present a novel approach for extracting – in a totally automated way – a highcoverage and high-precision lexicon of roughly 37 thousand terms annotated with emotion scores, called DepecheMood. Our approach exploits in an original way ‘crowd-sourced’ affective annotation implicitly provided by readers of news articles from rappler.com. By providing new state-of-the-art performances in unsupervised settings for regression and classification tasks, even using a naı̈ve approach, our experiments show the beneficial impact of harvesting social media data for affective lexicon building.

0957-4174/$ see front matter 2008 Elsevier Ltd. A doi:10.1016/j.eswa.2008.09.042 q Part of the content of this paper has been publi International Conference on Multimedia, 2005. * Corresponding author. Tel.: +886 2 29393091x67 E-mail address: mkshan@cs.nccu.edu.tw (M.-K. Sh With the growth of digital music, the development of music recommendation is helpful for users to pick desirable music pieces from a huge repository of music. The existing music recommendation approaches are based on a user’s preference on music. However, sometimes, it might better meet users’ requirement to recommend music pieces according to emotions. In this paper, we propose a novel framework for emotion-based music recommendation. The core of the recommendation framework is the construction of the music emotion model by affinity discovery from film music, which plays an important role in conveying emotions in film. We investigate the music feature extraction and propose the Music Affinity Graph and Music Affinity Graph-Plus algorithms for the construction of music emotion model. Experimental result shows the proposed emotion-based music recommendation achieves 85% accuracy in average. 2008 Elsevier Ltd. All rights reserved.

This paper reports the results of our experiments on speaker identification in the SCOTUS corpus, which includes oral arguments from the Supreme Court of the United States. Our main findings are as follows: 1) a combination of Gaussian mixture models and monophone HMM models attains near-100% textindependent identification accuracy on utterances that are longer than one second; 2) the sampling rate of 11025 Hz achieves the best performance (higher sampling rates are harmful); and a sampling rate as low as 2000 Hz still achieves more than 90% accuracy; 3) a distance score based on likelihood numbers was used to measure the variability of phones among speakers; we found that the most variable phone is the phone UH (as in good), and the velar nasal NG is more variable than the other two nasal sounds M and N; 4.) our models achieved “perfect” forced alignment on very long speech segments (40 minutes). These findings and their significance are discussed.

This paper introduces a new corpus of read English speech, suitable for training and evaluating speech recognition systems. The LibriSpeech corpus is derived from audiobooks that are part of the LibriVox project, and contains 1000 hours of speech sampled at 16 kHz. We have made the corpus freely available for download, along with separately prepared language-model training data and pre-built language models. We show that acoustic models trained on LibriSpeech give lower error rate on the Wall Street Journal (WSJ) test sets than models trained on WSJ itself. We are also releasing Kaldi scripts that make it easy to build these systems.

In this paper we propose a scheme for developing a voice conversion system that converts the speech signal uttered by a source speaker to a speech signal having the voice characteristics of the target speaker. In particular, we address the issue of transformation of the vocal tract system features from one speaker to another. Formants are used to represent the vocal tract system features and a formant vocoder is used for synthesis. The scheme consists of a formant analysis phase, followed by a learning phase in which the implicit formant transformation is captured by a neural network. The transformed formants together with the pitch contour modified to suit the average pitch of the target speaker are used to synthesize speech with the desired vocal tract system characteristics.


Question: A method for visual gaze tracking using a single, affordable camera.

This paper explores the integration of driver gaze tracking with road scene analysis for advanced driver assistance systems (ADAS). The goal is to enhance driving safety by monitoring the driver's visual attention and correlating it with critical road information. While it's challenging to ascertain a driver's thoughts, their gaze direction offers valuable insights. By continuously tracking where the driver is looking and comparing this data with the visual field, the system can infer the driver's focus. The paper describes a comprehensive approach that automates the detection and recognition of road signs, particularly focusing on speed limit signs. This system operates in real-time, analyzing the driver's gaze direction in relation to these signs. If the system detects that the driver has potentially missed a sign, it provides immediate feedback, alerting the driver to the missed information. This innovative method not only monitors the driver's actions but also their visual engagement with the road environment, thereby offering a more holistic approach to driver assistance. The system's effectiveness in enhancing situational awareness and response times is discussed, highlighting its potential to significantly improve driving safety.

In this article a new method for the calibration of a vision system which consists of two (or more) cameras is presented. The proposed method, which uses simple properties of vanishing points, is divided into two steps. In the first step, the intrinsic parameters of each camera, that is, the focal length and the location of the intersection between the optical axis and the image plane, are recovered from a single image of a cube. In the second step, the extrinsic parameters of a pair of cameras, that is, the rotation matrix and the translation vector which describe the rigid motion between the coordinate systems fixed in the two cameras are estimated from an image stereo pair of a suitable planar pattern. Firstly, by matching the corresponding vanishing points in the two images the rotation matrix can be computed, then the translation vector is estimated by means of a simple triangulation. The robustness of the method against noise is discussed, and the conditions for optimal estimation of the rotation matrix are derived. Extensive experimentation shows that the precision that can be achieved with the proposed method is sufficient to efficiently perform machine vision tasks that require camera calibration, like depth from stereo and motion from image sequence.

Face detection and eyes extraction has an important role in many applications such as face recognition, facial expression analysis, security login etc. Detection of human face and facial structures like eyes, nose are the complex procedure for the computer. This paper proposes an algorithm for face detection and eyes extraction from frontal face images using Sobel edge detection and morphological operations. The proposed approach is divided into three phases; preprocessing, identification of face region, and extraction of eyes. Resizing of images and gray scale image conversion is achieved in preprocessing. Face region identification is accomplished by Sobel edge detection and morphological operations. In the last phase, eyes are extracted from the face region with the help of morphological operations. The experiments are conducted on 120, 75, 40 images of IMM frontal face database, FEI face database and IMM face database respectively. The face detection accuracy is 100%, 100%, 97.50% and the eyes extraction accuracy rate is 92.50%, 90.66%, 92.50% respectively.

Gaze reflects how humans process visual scenes and is therefore increasingly used in computer vision systems. Previous works demonstrated the potential of gaze for object-centric tasks, such as object localization and recognition, but it remains unclear if gaze can also be beneficial for scene-centric tasks, such as image captioning. We present a new perspective on gaze-assisted image captioning by studying the interplay between human gaze and the attention mechanism of deep neural networks. Using a public large-scale gaze dataset, we first assess the relationship between state-of-the-art object and scene recognition models, bottom-up visual saliency, and human gaze. We then propose a novel split attention model for image captioning. Our model integrates human gaze information into an attention-based long short-term memory architecture, and allows the algorithm to allocate attention selectively to both fixated and non-fixated image regions. Through evaluation on the COCO/SALICON datasets we show that our method improves image captioning performance and that gaze can complement machine attention for semantic scene understanding tasks.

Activity tracking using a wearable device is an emerging research field. Large-scale studies on activity tracking performed with eyewear-type wearable devices remains a challenging area owing to the negative effect such devices have on users' looks. To cope with this challenge, JINS Inc., an eyewear retailer in Japan, has developed a state-of-the-art smart eyewear called JINS MEME. The design of JINS MEME is almost the same as that of Wellington-type eyeglasses so that people can use it in their daily lives. JINS MEME is equipped with sensors to detect a user's eye movement and head motion. In addition to these functions of JINS MEME, JINS developed an application to measure concentration levels of users. In this demonstration, users will experience wearing the JINS MEME glasses and their concentration will be measured while they perform a certain task at our booth.

We explore the combination of above-surface sensing with eye tracking to facilitate concurrent interaction with multiple regions on touch screens. Conventional touch input relies on positional accuracy, thereby requiring tight visual monitoring of one's own motor action. In contrast, above-surface sensing and eye tracking provides information about how user's hands and gaze are distributed across the interface. In these situations we facilitate interaction by 1) showing the visual feedback of the hand hover near user's gaze point and 2) decrease the requisite of positional accuracy by employing gestural information. We contribute input and visual feedback techniques that combine these modalities and demonstrate their use in example applications. A controlled study showed the effectiveness of our techniques for manipulation tasks against conventional touch, while the effectiveness in acquisition tasks depended on the amount of mid-air motion, leading to our conclusion that the techniques can benefit interacting with multiple interface regions.

The objective of this paper is to propose a new homography-based approach to image-based visual tracking and servoing. The visual tracking algorithm proposed in the paper is based on a new efficient second-order minimization method. Theoretical analysis and comparative experiments with other tracking approaches show that the proposed method has a higher convergence rate than standard first-order minimization techniques. Therefore, it is well adapted to real-time robotic applications. The output of the visual tracking is a homography linking the current and the reference image of a planar target. Using the homography, a task function isomorphic to the camera pose has been designed. A new image-based control law is proposed which does not need any measure of the 3D structure of the observed target (e.g. the normal to the plane). The theoretical proof of the existence of the isomorphism between the task function and the camera pose and the theoretical proof of the stability of the control law are provided. The experimental results, obtained with a 6 d.o.f. robot, show the advantages of the proposed method with respect to the existing approaches. KEY WORDS—visual tracking, visual servoing, efficient second-order minimization, homography-based control law

Ego-pose estimation, i.e., estimating a person’s 3D pose with a single wearable camera, has many potential applications in activity monitoring. For these applications, both accurate and physically plausible estimates are desired, with the latter often overlooked by existing work. Traditional computer vision-based approaches using temporal smoothing only take into account the kinematics of the motion without considering the physics that underlies the dynamics of motion, which leads to pose estimates that are physically invalid. Motivated by this, we propose a novel control-based approach to model human motion with physics simulation and use imitation learning to learn a videoconditioned control policy for ego-pose estimation. Our imitation learning framework allows us to perform domain adaption to transfer our policy trained on simulation data to real-world data. Our experiments with real egocentric videos show that our method can estimate both accurate and physically plausible 3D ego-pose sequences without observing the cameras wearer’s body.

This paper deals with the problem of tracking football players in a football match using data from a single moving camera. Tracking footballers from a single video source is di cult: not only do the football players occlude each other, but they frequently enter and leave the camera's eld of view, making initialisation and destruction of a player's tracking a di cult task. The system presented here uses particle lters to track players. The multiple state estimates used by a particle lter provide an elegant method for maintaining tracking of players following an occlusion. Automated tracking can be achieved by creating and stopping particle lters depending on the input player data.

To fill the semantic gap between the predictive power of computational saliency models and human behavior, this paper proposes to predict where people look at using spatial-aware object-level cues. While object-level saliency has been recently suggested by psychophysics experiments and shown effective with a few computational models, the spatial relationship between the objects has not yet been explored in this context. We in this work for the first time explicitly model such spatial relationship, as well as leveraging semantic information of an image to enhance object-level saliency modeling. The core computational module is a graphlet-based (i.e., graphlets are moderate-sized connected subgraphs) deep architecture, which hierarchically learns a saliency map from raw image pixels to object-level graphlets (oGLs) and further to spatial-level graphlets (sGLs). Eye tracking data are also used to leverage human experience in saliency prediction. Experimental results demonstrate that the proposed oGLs and sGLs well capture object-level and spatial-level cues relating to saliency, and the resulting saliency model performs competitively compared with the state-of-the-art.

Altered visual exploration of faces likely contributes to social cognition deficits seen in autism. To investigate the relationship between face gaze and social cognition in autism, we measured both face gaze and how facial regions were actually used during emotion judgments from faces. Compared to IQ-matched healthy controls, nine high-functioning adults with autism failed to make use of information from the eye region of faces, instead relying primarily on information from the mouth. Face gaze accounted for the increased reliance on the mouth, and partially accounted for the deficit in using information from the eyes. These findings provide a novel quantitative assessment of how people with autism utilize information in faces when making social judgments.


Question: A method for visual gaze tracking using a single low-cost camera.

This document outlines a robust technique for identifying and interpreting road signs in both grayscale and color images captured by a single camera affixed to a moving vehicle. The methodology is executed in three sequential steps. Initially, utilizing prior knowledge of the scene or color hints (when present), the search area for road signs is narrowed down to a specific region within the image. Subsequently, a geometric analysis of the detected edges is conducted, aiming to isolate potential circular and triangular sign candidates. Finally, a recognition phase is initiated, where each candidate is cross-correlated against a database of known signs. If validation is achieved, the candidate is categorized accordingly. Extensive testing has confirmed the method's resilience to low-level noise, which can often distort edge detection and contour delineation, making it effective in a variety of settings, from busy urban streets to rural roads and highways. Performance enhancements have been realized through temporal integration, utilizing Kalman filtering techniques to refine the information extracted from the images. This advanced approach holds significant potential for the enhancement of driving assistance systems.

Automatic recognition of human actions from video has been studied for many years. Although still very difficult in uncontrolled scenarios, it has been successful in more restricted settings (e.g., fixed viewpoint, no occlusions) with recognition rates approaching 100%. However, the best-performing methods are complex and computationally-demanding and thus not well-suited for real-time deployments. This paper proposes to leverage the Kinect camera for close-range gesture recognition using two methods. Both methods use feature vectors that are derived from the skeleton model provided by the Kinect SDK in real-time. Although both methods perform nearest-neighbor classification, one method does this in the space of features using the Euclidean distance metric, while the other method does this in the space of feature covariances using a log-Euclidean metric. Both methods recognize 8 hand gestures in real time achieving correct-classification rates of over 99% on a dataset of 20 subjects but the method based on Euclidean distance requires feature-vector collections to be of the same size, is sensitive to temporal misalignment, and has higher computation and storage requirements.

Robot vision refers to the capability of a robot to visually perceive the environment and use this information for execution of different tasks. Visual feedback has been used extensively for robot navigation and obstacle avoidance. In the recent years, there are also examples that include interaction with people and manipulation of objects. In this paper, we review some of the work that goes beyond of using artificial landmarks and fiducial markers for the purpose of implementing vision based control in robots. We discuss different application areas, both from the systems perspective and individual problems such as object tracking and recognition.

We present a mobile vision system for multi-person tracking in busy environments. Specifically, the system integrates continuous visual odometry computation with tracking-by-detection in order to track pedestrians in spite of frequent occlusions and egomotion of the camera rig. To achieve reliable performance under real-world conditions, it has long been advocated to extract and combine as much visual information as possible. We propose a way to closely integrate the vision modules for visual odometry, pedestrian detection, depth estimation, and tracking. The integration naturally leads to several cognitive feedback loops between the modules. Among others, we propose a novel feedback connection from the object detector to visual odometry which utilizes the semantic knowledge of detection to stabilize localization. Feedback loops always carry the danger that erroneous feedback from one module is amplified and causes the entire system to become instable. We therefore incorporate automatic failure detection and recovery, allowing the system to continue when a module becomes unreliable. The approach is experimentally evaluated on several long and difficult video sequences from busy inner-city locations. Our results show that the proposed integration makes it possible to deliver stable tracking performance in scenes of previously infeasible complexity.

In recent years researchers have presented a number of new methods for recovering physiological parameters using just low-cost digital cameras and image processing. The ubiquity of digital cameras presents the possibility for many new, low-cost applications of vital sign monitoring. In this paper we present a review of the work on remote photoplethysmographic (PPG) imaging using digital cameras. This review specifically focuses on the state-of-the-art in PPG imaging where: 1) measures beyond pulse rate are evaluated, 2) non-ideal conditions (e.g., the presence of motion artifacts) are explored, and 3) use cases in relevant environments are demonstrated. We discuss gaps within the literature and future challenges for the research community. To aid in the continuing advancement of PPG imaging research, we are making available a website with the references collected for this review as well as information on available code and datasets of interest. It is our hope that this website will become a valuable resource for the PPG imaging community. The site can be found at: http://web.mit.edu/~djmcduff/www/ remote-physiology.html.

Avoiding high computational costs and calibration issues involved in stereo-vision-based algorithms, this paper proposes real-time monocular-vision-based techniques for simultaneous vehicle detection and inter-vehicle distance estimation, in which the performance and robustness of the system remain competitive, even for highly challenging benchmark datasets. This paper develops a collision warning system by detecting vehicles ahead and, by identifying safety distances to assist a distracted driver, prior to occurrence of an imminent crash. We introduce adaptive global Haar-like features for vehicle detection, tail-light segmentation, virtual symmetry detection, intervehicle distance estimation, as well as an efficient single-sensor multifeature fusion technique to enhance the accuracy and robustness of our algorithm. The proposed algorithm is able to detect vehicles ahead at both day or night and also for short- and long-range distances. Experimental results under various weather and lighting conditions (including sunny, rainy, foggy, or snowy) show that the proposed algorithm outperforms state-of-the-art algorithms.

The eye's blinking is a significant indicator of the sleepiness. The existing systems of blink detection and sleepiness analysis require usually to fix camera on spectacle frame or a special helmet that is not convenient and can affect the obtained result. In this paper, the infrared camera-based contact-less system is proposed to evaluate the human's sleepiness. The infrared light switching is used to detect the pupil in each frame and, as a result, the blink event. The active pan-tilt unit is used to make possible free head movements. The algorithm is pointed out to process the camera frames in order to distinguish the involuntary blinks from voluntary ones. Preliminary experimental tests are shown with the intent to validate the proposed hardware and software system pointed out.

A new transform is presented that utilizes local radial symmetry to highlight points of interest within a scene. Its lowcomputational complexity and fast runtimes makes this method well-suited for real-time vision applications. The performance of the transform is demonstrated on a wide variety of images and compared with leading techniques from the literature. Both as a facial feature detector and as a generic region of interest detector the new transform is seen to offer equal or superior performance to contemporary techniques at a relatively low-computational cost. A real-time implementation of the transform is presented running at over 60 frames per second on a standard Pentium III PC.

Human eyes can recognize person identities based on some small salient regions. However, such valuable salient information is often hidden when computing similarities of images with existing approaches. Moreover, many existing approaches learn discriminative features and handle drastic viewpoint change in a supervised way and require labeling new training data for a different pair of camera views. In this paper, we propose a novel perspective for person re-identification based on unsupervised salience learning. Distinctive features are extracted without requiring identity labels in the training procedure. First, we apply adjacency constrained patch matching to build dense correspondence between image pairs, which shows effectiveness in handling misalignment caused by large viewpoint and pose variations. Second, we learn human salience in an unsupervised manner. To improve the performance of person re-identification, human salience is incorporated in patch matching to find reliable and discriminative matched patches. The effectiveness of our approach is validated on the widely used VIPeR dataset and ETHZ dataset.

With the invention of the low-cost Microsoft Kinect sensor, high-resolution depth and visual (RGB) sensing has become available for widespread use. The complementary nature of the depth and visual information provided by the Kinect sensor opens up new opportunities to solve fundamental problems in computer vision. This paper presents a comprehensive review of recent Kinect-based computer vision algorithms and applications. The reviewed approaches are classified according to the type of vision problems that can be addressed or enhanced by means of the Kinect sensor. The covered topics include preprocessing, object tracking and recognition, human activity analysis, hand gesture analysis, and indoor 3-D mapping. For each category of methods, we outline their main algorithmic contributions and summarize their advantages/differences compared to their RGB counterparts. Finally, we give an overview of the challenges in this field and future research trends. This paper is expected to serve as a tutorial and source of references for Kinect-based computer vision researchers.

This study develops a framework for the detection and analysis of stress/anxiety emotional states through video-recorded facial cues. A thorough experimental protocol was established to induce systematic variability in affective states (neutral, relaxed and stressed/anxious) through a variety of external and internal stressors. The analysis was focused mainly on non-voluntary and semi-voluntary facial cues in order to estimate the emotion representation more objectively. Features under investigation included eye-related events, mouth activity, head motion parameters and heart rate estimated through camerabased photoplethysmography. A feature selection procedure was employed to select the most robust features followed by classification schemes discriminating between stress/anxiety and neutral states nxiety tress link rate ead motion with reference to a relaxed state in each experimental phase. In addition, a ranking transformation was proposed utilizing self reports in order to investigate the correlation of facial parameters with a participant perceived amount of stress/anxiety. The results indicated that, specific facial cues, derived from eye activity, mouth activity, head movements and camera based heart activity achieve good accuracy and are suitable as discriminative indicators of stress and anxiety.


Question: A method for visual gaze tracking using a single low-cost camera.

This comprehensive review explores the field of face recognition, a prominent application in image analysis and understanding that has garnered substantial interest, particularly over the past few years. Two factors contribute to this heightened focus: the extensive potential for commercial and law enforcement applications, and the maturation of viable technologies after three decades of research. Despite advancements, current machine recognition systems still face limitations, especially in real-world applications. For instance, the recognition of faces in outdoor settings with varying illumination and pose poses significant challenges, indicating that these systems are yet to match the capabilities of human perception. The paper offers a critical and up-to-date overview of research in still- and video-based face recognition. Our dual objectives in writing this review are to provide an updated analysis of existing literature and to shed light on the methodologies of face recognition by machines. To ensure a thorough examination, we classify recognition techniques and provide detailed explanations of notable methods within each category. Additionally, we discuss related areas such as psychophysical studies, system evaluation, and the complexities associated with illumination and pose variations.

**Question**: An efficient method for the numerical simulation of the dynamics of a single electron in a high intensity laser field

**Documents**: Particle swarm optimization and the non-convex economic dispatch problems
This paper proposes a particle swarm optimization (PSO) algorithm for solving non-convex economic dispatch (ED) problems. The proposed algorithm is applied to non-convex ED problems with valve-point effects, multiple fuels, and transmission losses. The valve-point effect on the generation cost function is represented by a nonlinear function with a sinusoidal term. The multiple fuel ED problem involves the use of two fuels in the same generating unit. The transmission loss is considered in the ED problem by using the loss formula. The proposed PSO algorithm is applied to four test systems: two systems with valve-point effect, one system with multiple fuels, and one system with transmission losses. The results obtained by the proposed PSO algorithm are compared with those obtained by other optimization methods. The proposed PSO algorithm provides better solutions than other optimization methods for non-convex ED problems.

**Similar Documents**: This article introduces the application of the Particle Swarm Optimization (PSO) algorithm in addressing non-convex economic dispatch (ED) problems, particularly focusing on scenarios with valve-point effects, multiple fuels, and transmission losses. The PSO algorithm is demonstrated on ED problems characterized by non-convex generation cost functions that include a sinusoidal term to represent the valve-point effect. The multiple fuel ED problem is tackled by considering the simultaneous use

Gaze reflects how humans process visual scenes and is therefore increasingly used in computer vision systems. Previous works demonstrated the potential of gaze for object-centric tasks, such as object localization and recognition, but it remains unclear if gaze can also be beneficial for scene-centric tasks, such as image captioning. We present a new perspective on gaze-assisted image captioning by studying the interplay between human gaze and the attention mechanism of deep neural networks. Using a public large-scale gaze dataset, we first assess the relationship between state-of-the-art object and scene recognition models, bottom-up visual saliency, and human gaze. We then propose a novel split attention model for image captioning. Our model integrates human gaze information into an attention-based long short-term memory architecture, and allows the algorithm to allocate attention selectively to both fixated and non-fixated image regions. Through evaluation on the COCO/SALICON datasets we show that our method improves image captioning performance and that gaze can complement machine attention for semantic scene understanding tasks.

We present a new motion tracking technique to robustly reconstruct non-rigid geometries and motions from a single view depth input recorded by a consumer depth sensor. The idea is based on the observation that most non-rigid motions (especially human-related motions) are intrinsically involved in articulate motion subspace. To take this advantage, we propose a novel <inline-formula><tex-math notation="LaTeX">$L_0$</tex-math><alternatives> <inline-graphic xlink:href="xu-ieq2-2688331.gif"/></alternatives></inline-formula> based motion regularizer with an iterative solver that implicitly constrains local deformations with articulate structures, leading to reduced solution space and physical plausible deformations. The <inline-formula><tex-math notation="LaTeX">$L_0$</tex-math> <alternatives><inline-graphic xlink:href="xu-ieq3-2688331.gif"/></alternatives></inline-formula> strategy is integrated into the available non-rigid motion tracking pipeline, and gradually extracts articulate joints information online with the tracking, which corrects the tracking errors in the results. The information of the articulate joints is used in the following tracking procedure to further improve the tracking accuracy and prevent tracking failures. Extensive experiments over complex human body motions with occlusions, facial and hand motions demonstrate that our approach substantially improves the robustness and accuracy in motion tracking.

We present a new anatomically-constrained local face model and fitting approach for tracking 3D faces from 2D motion data in very high quality. In contrast to traditional global face models, often built from a large set of blendshapes, we propose a local deformation model composed of many small subspaces spatially distributed over the face. Our local model offers far more flexibility and expressiveness than global blendshape models, even with a much smaller model size. This flexibility would typically come at the cost of reduced robustness, in particular during the under-constrained task of monocular reconstruction. However, a key contribution of this work is that we consider the face anatomy and introduce subspace skin thickness constraints into our model, which constrain the face to only valid expressions and helps counteract depth ambiguities in monocular tracking. Given our new model, we present a novel fitting optimization that allows 3D facial performance reconstruction from a single view at extremely high quality, far beyond previous fitting approaches. Our model is flexible, and can be applied also when only sparse motion data is available, for example with marker-based motion capture or even face posing from artistic sketches. Furthermore, by incorporating anatomical constraints we can automatically estimate the rigid motion of the skull, obtaining a rigid stabilization of the performance for free. We demonstrate our model and single-view fitting method on a number of examples, including, for the first time, extreme local skin deformation caused by external forces such as wind, captured from a single high-speed camera.

The visual scanpaths of five high-functioning adult autistic males and five adult male controls were recorded using an infrared corneal reflection technique as they viewed photographs of human faces. Analyses of the scanpath data revealed marked differences in the scanpaths of the two groups. The autistic participants viewed nonfeature areas of the faces significantly more often and core feature areas of the faces (i.e., eyes, nose, and mouth) significantly less often than did control participants. Across both groups of participants, scanpaths generally did not differ as a function of the instructions given to the participants (i.e., "Please look at the faces in any manner you wish." vs. "Please identify the emotions portrayed in these faces."). Autistic participants showed a deficit in emotion recognition, but this effect was driven primarily by deficits in the recognition of fear. Collectively, these results indicate disorganized processing of face stimuli in autistic individuals and suggest a mechanism that may subserve the social information processing deficits that characterize autism spectrum disorders.

We present a real-time performance-driven facial animation system based on 3D shape regression. In this system, the 3D positions of facial landmark points are inferred by a regressor from 2D video frames of an ordinary web camera. From these 3D points, the pose and expressions of the face are recovered by fitting a user-specific blendshape model to them. The main technical contribution of this work is the 3D regression algorithm that learns an accurate, user-specific face alignment model from an easily acquired set of training data, generated from images of the user performing a sequence of predefined facial poses and expressions. Experiments show that our system can accurately recover 3D face shapes even for fast motions, non-frontal faces, and exaggerated expressions. In addition, some capacity to handle partial occlusions and changing lighting conditions is demonstrated.

This paper presents the design and analysis of a computer vision-aided motion sensing algorithm for wheeled mobile robot's indoor navigation. The algorithm is realized using two vision cameras attached on a wheeled mobile robot. The first camera is positioned at front-looking direction while the second camera is positioned at downward-looking direction. An algorithm is developed to process the images acquired from the cameras to yield the mobile robot's positions and orientations. The proposed algorithm is implemented on a wheeled mobile robot for real-world effectiveness testing. Results are compared and shown the accuracy of the proposed algorithm. At the end of the paper, an artificial landmark approach is introduced to improve the navigation efficiency. Future work involved implementing the proposed artificial landmark for indoor navigation applications with minimized accumulated errors.

In this letter, we address the autonomous flight of a small quadrotor, enabling tracking of a moving object. The 15-cm diameter, 250-g robot relies only on onboard sensors (a single camera and an inertial measurement unit) and computers, and can detect, localize, and track moving objects. Our key contributions include the relative pose estimate of a spherical target as well as the planning algorithm, which considers the dynamics of the underactuated robot, the actuator limitations, and the field of view constraints. We show simulation and experimental results to demonstrate feasibility and performance, as well as robustness to abrupt variations in target motion.

2D Face recognition systems bound to fail on images with varying pose angles and occlusions. Many pose invariant methods are proposed in recent years but they are still not able to achieve very good accuracies. So in order to achieve a better accuracy we need to extend algorithms over 3D faces. Due to the high cost involved in acquisition of 3D faces we developed our approach for low-cost and low-quality Microsoft Kinect Sensor and propose an algorithm to produce better results than existing 2D Face recognition techniques even after compromising on the quality of the images from the sensor. Our proposed algorithm is based on modified SURF descriptors on RGB images combined with various enhancements on automatically generated training images using Depth and Color images. We compare our results obtained with State Of The Art Techniques obtained on publicly available RGB-D Face databases. Our System obtained recognition rate of 98.07% on 30° CurtinFace Database, 89.28% on EURECOM Database, 98.00% on 15° Internal Database and 81.00% on 30° Internal Database.

A large number of road accidents in the world occur due to driver’s not taking proper breaks while driving long distances. This System is used to detect fatigue behaviour of the driver well and before any accident occurs. In this system driver’s facial appearance is extracted via high speed camera which is placed in car in front of driver. This camera will capture the frames at every instant of time. Each of this frame will be processed using image processing algorithm. The mouth and eye region is separated from each of the frames and analyzed for concluding whether the driver is dozy or not. The system draws conclusion if the eyes are found closed for five consecutive frame. If doziness is detected, a signal alert is triggered. In case accidents occurs, message is sent to the desired number.

In this paper we present a markerless augmented reality (MAR) approach based on real-time 3D reconstruction using a low-cost depth camera, the Kinect. We intend to use this MAR approach for 3D medical visualization. In the actual state of developing, our approach is based on the KinectFusion algorithm with some minor adaptations, as long as our goal is not reconstruct scenes but use the reconstructed model as a reference to MAR. Keywords-Augmented Reality; 3D Reconstruction; Kinect;


Question: A method for visual gaze tracking utilizing a single, affordable camera.

This paper introduces a novel algorithm for detecting points of interest in visual scenes, leveraging the concept of local radial symmetry. The proposed transform is characterized by its low computational complexity and rapid execution times, making it particularly suitable for real-time vision systems. The algorithm's effectiveness is showcased through its application to a diverse set of images, with comparisons to established methods from the field. In both facial feature detection and general region of interest identification, the new transform is shown to match or outperform current techniques, all while maintaining a low computational footprint. A practical demonstration of the transform's real-time capabilities is provided, achieving over 60 frames per second on a standard Pentium III personal computer. This breakthrough approach offers a cost-effective and efficient solution for real-time gaze tracking using a single low-cost camera, demonstrating its potential in enhancing the accuracy and speed of visual tracking systems.

In this paper, we present a novel model for saliency prediction under a unified framework of feature integration. The model distinguishes itself by directly learning from natural images and automatically incorporating higher-level semantic information in a scalable manner for gaze prediction. Unlike most existing saliency models that rely on specific features or object detectors, our model learns multiple stages of features that mimic the hierarchical organization of the ventral stream in the visual cortex and integrate them by adapting their weights based on the ground-truth fixation data. To accomplish this, we utilize a multi-layer sparse network to learn low-, midand high-level features from natural images and train a linear support vector machine (SVM) for weight adaption and feature integration. Experimental results show that our model could learn high-level semantic features like faces and texts and can perform competitively among existing approaches in predicting eye fixations. & 2014 Elsevier B.V. All rights reserved.

We present the design and construction of a handheld mirror simulation device. The perception of the world reflected through a mirror depends on the viewer's position with respect to the mirror and the 3-D geometry of the world. In order to simulate a real mirror on a computer screen, images of the observed world, consistent with the viewer's position, must be synthesized and displayed in realtime. Our system is build around a LCD screen manipulated by the user, a single camera fixed on the screen, and a tracking device. The continuous input video stream and tracker data is used to synthesize, in real-time, a continuous video stream displayed on the LCD screen. The synthesized video stream is a close approximation of what the user would see on the screen surface if it were a real mirror. Our system provides a generic interface for applications involving rich, first-person interaction, such as the Virtual Daguerreotype.

We present a study of depth-disparity calibration for augmented reality applications using binocular optical see-through displays. Two techniques were proposed and compared. The "paired-eyes" technique leverages the Panum's fusional area to help viewer find alignment between the virtual and physical objects. The "separate-eyes" technique eliminates the need of binocular fusion and involves using both eyes sequentially to check the virtual-physical object alignment on retinal images. We conducted a user study to measure the calibration results and assess the subjective experience of users with the proposed techniques.

By using the knowledge of facial structure and temperature distribution, this paper proposes an automatic eye localization method from long wave infrared thermal images both with eyeglasses and without eyeglasses. First, with the help of support vector machine classifier, three gray-projection features are defined to determine whether a subject is with eyeglasses. For subjects with eyeglasses, the locations of valleys in the projection curve are used to perform eye localization. For subjects without eyeglasses, a facial structure consisting of 15 sub-regions is proposed to extract Haar-like features. Eight classifiers are learned from the features selected by Adaboost algorithm for left and right eye, respectively. A vote strategy is employed to find the most likely eyes. To evaluate the effectiveness of our approach, experiments are performed on NVIE and Equinox databases. The eyeglass detection results on NVIE database and Equinox database are 99.36% and 95%, respectively, which demonstrate the effectiveness and robustness of our eyeglass detection method. Eye localization results of withindatabase experiments and cross-database experiments on these two databases are very comparable with the previous results in this field, verifying the effectiveness and the generalization ability of our approach. & 2013 Elsevier Ltd. All rights reserved.

A real-time solution for estimating and tracking the 3D pose of a rigid object is presented for image-based visual servo with natural landmarks. The many state-of-the-art technologies that are available for recognizing the 3D pose of an object in a natural setting are not suitable for real-time servo due to their time lags. This paper demonstrates that a real-time solution of 3D pose estimation become feasible by combining a fast tracker such as KLT [7] [8] with a method of determining the 3D coordinates of tracking points on an object at the time of SIFT based tracking point initiation, assuming that a 3D geometric model with SIFT description of an object is known a-priori. Keeping track of tracking points with KLT, removing the tracking point outliers automatically, and reinitiating the tracking points using SIFT once deteriorated, the 3D pose of an object can be estimated and tracked in real-time. This method can be applied to both mono and stereo camera based 3D pose estimation and tracking. The former guarantees higher frame rates with about 1 ms of local pose estimation, while the latter assures of more precise pose results but with about 16 ms of local pose estimation. The experimental investigations have shown the effectiveness of the proposed approach with real-time performance.

The eye's blinking is a significant indicator of the sleepiness. The existing systems of blink detection and sleepiness analysis require usually to fix camera on spectacle frame or a special helmet that is not convenient and can affect the obtained result. In this paper, the infrared camera-based contact-less system is proposed to evaluate the human's sleepiness. The infrared light switching is used to detect the pupil in each frame and, as a result, the blink event. The active pan-tilt unit is used to make possible free head movements. The algorithm is pointed out to process the camera frames in order to distinguish the involuntary blinks from voluntary ones. Preliminary experimental tests are shown with the intent to validate the proposed hardware and software system pointed out.

In this paper, we describe how to establish an embedded framework for real-time top-view people counting. The development of our system consists of two parts, i.e. establishing an embedded signal processing platform and designing a people counting algorithm for the embedded system. For the hardware platform construction, we use Kinect as the camera and exploit NVIDIA Jetson TK1 board as the embedded processing platform. We describe how to build a channel to make Kinect for windows version 2.0 communicate with Jetson TK1. Based on the embedded system, we adapt a water filling based scheme for top-view people counting, which integrates head detection based on water drop, people tracking and counting. Gaussian Mixture Model is used to construct and update the background model. The moving people in each video frame are extracted using background subtraction method. Additionally, the water filling algorithm is used to segment head area as Region Of Interest(ROI). Tracking and counting people are performed by calculating the distance of ROI center point before and after the frame. The whole framework is flexible and practical for real-time application.

In recent years, many user-interface devices appear for managing variety the physical interactions. The Microsft Kinect camera is a revolutionary and useful depth camera giving new user experience of interactive gaming on the Xbox platform through gesture or motion detection. In this paper we present an approach for control the Quadrotor AR.Drone using the Microsft Kinect sensor.

In this paper we propose a novel approach for detecting and tracking objects in videos with variable background i.e. videos captured by moving cameras without any additional sensor. The performance of tracking in videos with variable background depends on the successful detection of an object in variable background. The most attractive feature of detecting an object in variable background is that it does not depend on any a priori information of the scene. In a video captured by a moving camera, both the background and foreground are changing in each frame of the image sequence. So for these videos, modeling a single background with traditional background modeling methods is infeasible and thus the detection of actual moving object in a variable background is a challenging task. To detect actual moving object in this work, spatio-temporal blobs have been generated in each frame by spatio-temporal analysis of the image sequence using a three-dimensional Gabor filter. Then individual blobs, which are parts of one object are merged using Minimum Spanning Tree to form the moving object in the variable background. The height, width and four-bin grayvalue histogram of the object are calculated as its features and an object is tracked in each frame using these features to generate the trajectories of the object through the video sequence. In this work, problem of data association during tracking is solved by Linear Assignment Problem and occlusion is handled by the application of kalman filter. The major advantage of our method over most of the existing tracking algorithms is that, the proposed method does not require initialization in the first frame or training on sample data to perform. Performance of the algorithm has been tested on benchmark videos and very satisfactory result has been achieved. The performance of the algorithm is also comparable and superior with respect to some benchmark algorithms.

This paper presents a technique for automatically detecting human faces in digital color images. This is two-step process which first detects regions contain human skin in the color image and then extracts information from these regions which might indicate the location of a face in the image. The skin detection is performed using a skin filter which relies on color and texture information. The face detection is performed on a grayscale image containing only the detected skin areas. A combination of thresh holding and mathematical morphology are used to extract object features that would indicate the presence of a face. The face detection process works predictably and fairly reliably, as test results show.


Question: A method for visual gaze tracking using a single affordable camera.

This paper explores a vision-based eye-gaze tracking system designed for human-computer interaction interfaces. The research emphasizes the importance of eye movement as a key component in developing efficient computer interfaces. The focus is on non-intrusive eye-gaze tracking that accommodates minor head movements. To address the issue of head movement, a small 2D marker is utilized as a reference point for compensation. The iris center is selected as the primary feature for tracking eye movement. Following the acquisition of eye movement data, the gaze point is calculated. The effectiveness of the system is demonstrated through preliminary experiments conducted using a screen pointing application, showcasing its potential in enhancing human-computer interaction.

**Question**: The role of the environment in the evolution of cooperation

**Documents**: Cooperation in social dilemmas: The role of spatial structure, voluntary interactions, and individual learning
Cooperation in social dilemmas is a fundamental problem in evolutionary biology. Spatial structure, voluntary interactions, and individual learning are three key factors that can promote cooperation. Here we study their combined effects on cooperation in a spatial public goods game with voluntary participation. We show that cooperation can be promoted by spatial structure and individual learning, but the effect of voluntary interactions is less clear. When the cost of voluntary participation is low, voluntary interactions can promote cooperation, but when the cost is high, voluntary interactions can inhibit cooperation. Our results suggest that the role of the environment in the evolution of cooperation is complex and depends on the interplay between different factors.

**Similar Documents**: This study investigates the role of cooperation in social dilemmas, a critical area within evolutionary biology. Three pivotal elements—spatial structure, voluntary interactions, and individual learning—are identified as significant in fostering cooperative behavior. The research examines their collective impact on cooperation through a spatial public goods game that incorporates voluntary participation. It reveals that spatial structure and individual learning can significantly enhance cooperation. However, the effect of voluntary interactions is nuanced; while they can promote cooperation when the cost is low, they may hinder it when the cost is high. These findings underscore the intricate nature of environmental influences on the evolution of cooperation, highlighting the interdependence of various factors.

**Question**: The impact of technology on the future of work

**Documents**: The impact of technology on the future of work: A review of the literature
The rapid pace of technological change is transforming the nature of work. This paper reviews the literature on the impact of technology on the future of work. We find that technology is likely to have a significant impact on the future of work, both in terms of

In recent years researchers have presented a number of new methods for recovering physiological parameters using just low-cost digital cameras and image processing. The ubiquity of digital cameras presents the possibility for many new, low-cost applications of vital sign monitoring. In this paper we present a review of the work on remote photoplethysmographic (PPG) imaging using digital cameras. This review specifically focuses on the state-of-the-art in PPG imaging where: 1) measures beyond pulse rate are evaluated, 2) non-ideal conditions (e.g., the presence of motion artifacts) are explored, and 3) use cases in relevant environments are demonstrated. We discuss gaps within the literature and future challenges for the research community. To aid in the continuing advancement of PPG imaging research, we are making available a website with the references collected for this review as well as information on available code and datasets of interest. It is our hope that this website will become a valuable resource for the PPG imaging community. The site can be found at: http://web.mit.edu/~djmcduff/www/ remote-physiology.html.

Face detection has been one of the most studied topics in computer vision literature. Given an arbitrary image, the goal of face detection is to determine whether or not there are any faces in the image and, if present, return the image location and extent of each face. While this appears to be a trivial task for human beings, it is a very challenging task for computers. The difficulty associated with face detection can be attributed to many variations in scale, location, view point, illumination, occlusions, etc. Although there have been hundreds of reported approaches to face detection, if one were asked to name a single face detection algorithm that has the most impact in recent decades, it will most likely be the Viola and Jones face detection, which is capable of processing images extremely rapidly and achieve high detection rates. This project is going to study and understand the Viola-Jones algorithm by implementing the whole detection framework and based on the implementation, conduct experiment to hopefully further improve the performance.

This paper proposes a method for detecting moving obstacles on roads, by using a vehicle mounted monocular camera. To detect various moving obstacles, such as vehicles and pedestrians, the ego-motion of the vehicle is initially estimated from images captured by the camera. There are two problems in ego-motion estimation. Firstly, a typical road scene contains moving obstacles. This causes false estimation of the ego-motion. Secondly, roads possess fewer features, when compared to the number associated with background structures. This reduces the accuracy of ego-motion estimation. In our approach, the ego-motion is estimated from the correspondences of dispersed feature points extracted from various regions other than those that contain moving obstacles. After estimating the ego-motion, any moving obstacles are detected by tracking the feature points over consecutive frames. In our experiments, it has been shown that the proposed method is able to detect moving obstacles

This paper proposes and validates a real-time on-road vehicle detection system, which uses a single camera for the purpose of intelligent driver assistance. A three-step vehicle detection framework is presented to detect and track the target vehicle within an image. In the first step, probable vehicle locations are hypothesized using pattern recognition. The vehicle candidates are then verified in the hypothesis verification step. In this step, lane detection is used to filter vehicle candidates that are not within the lane region of interest. In the final step tracking and online learning are implemented to optimize the detection algorithm during misdetection and temporary occlusion. Good detection performance and accuracy was observed in highway driving environments with minimal shadows.

This article presents an image-based visual servoing system for indoor visual tracking of 3D moving objects by an Unmanned Aerial Vehicle. This system autonomously follows a 3D moving target object, maintaining it with a fixed distance and centered on its image plane. The initial setup is tested in a detailed simulation environment. The system is then validated on flights in indoor scenarios using the Parrot AR.Drone and the CMT tracker, demonstrating the robustness of the system to differences in object features, environmental clutter, and target trajectory. The obtained results indicate that the proposed system is suitable for complex controls task, such object surveillance and pursuit.

The 3D tracking of the articulation of human hands is a theoretically interesting and challenging problem with numerous and diverse applications [1]. Several attempts have been made to address the problem by considering markerless visual data. Existing approaches can be categorized into modeland appearance-based [1]. In this work, we propose a model-based approach to the problem (Fig. 1). We formulate it as an optimization problem that minimizes the discrepancy between the 3D structure and appearance of hypothesized 3D hand model instances, and actual visual observations. Optimization is performed with a variant of an existing stochastic optimization method (Particle Swarm Optimization PSO) [2]. The most computationally demanding parts of the process have been implemented to run efficiently on a GPU [3]. The resulting system performs hand articulations tracking at a rate of 15Hz.

This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the “Integral Image” which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers[6]. The third contribution is a method for combining increasingly more complex classifiers in a “cascade” which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.

Altered visual exploration of faces likely contributes to social cognition deficits seen in autism. To investigate the relationship between face gaze and social cognition in autism, we measured both face gaze and how facial regions were actually used during emotion judgments from faces. Compared to IQ-matched healthy controls, nine high-functioning adults with autism failed to make use of information from the eye region of faces, instead relying primarily on information from the mouth. Face gaze accounted for the increased reliance on the mouth, and partially accounted for the deficit in using information from the eyes. These findings provide a novel quantitative assessment of how people with autism utilize information in faces when making social judgments.

Substantial literature exists regarding how eye-tracking systems can be used to measure cognitive load and how these measurements can be useful for adapting training in real time. Much of the published literature discusses the applications and limitations of the research and typically provides recommendations for improvement. This review assesses these articles collectively to provide a clearer solution for implementing eye-tracking systems into a training environment. Although limitations exist for using eye tracking as an interface tool, gaze and pupillary response have been successfully used to reflect changes in cognitive load and are starting to be incorporated into adaptive training systems, although issues are still present with differentiating pupil responses from simultaneous psychological effects. Additionally, current eye-tracking systems and data analysis software have proven accurate enough for general use, but issues including system cost and software integration prevent this technology from becoming commercialized for use in common instructional settings.

In this work, we evaluate the feasibility of tracking-based interaction using a mobile phone's or tablet's camera in order to create and edit 3D objects in augmented reality applications. We present a feasibility study investigating if and how gestures made with your finger can be used to create such objects. A revised interface design is evaluated in a user study with 24 subjects that reveals a high usability and entertainment value, but also identifies issues such as ergonomic discomfort and imprecise input for complex tasks. Hence, our results suggest huge potential for this type of interaction in the entertainment, edutainment, and leisure domain, but limited usefulness for serious applications.