Question: Attire and Individuals - A social signal processing viewpoint.

This paper delves into the realm of Social Signal Processing (SSP), a domain that has gained prominence in recent years. It emphasizes the pivotal role of social intelligence in human interaction, underscoring its significance for personal and professional success. The paper posits that the future of computing must incorporate the nuances of social intelligence, particularly the capacity to interpret and respond to social signals such as turn-taking, politeness, and disagreement. Despite the intuitive understanding humans have of social signals and the recent advancements in machine analysis of behavioral cues—like eye blinks, smiles, crossed arms, and laughter—the creation of fully automated SSP systems remains a complex challenge. The paper provides an extensive review of previous research efforts aimed at enabling computers to process social signals. It also synthesizes findings from social psychology and offers a series of recommendations to facilitate the evolution of the next generation of socially-aware computing technologies, underscoring the need for systems that can effectively navigate the subtleties of human social interaction.

**Question**: A novel framework for understanding the impact of social media on political polarization

**Documents**: The Impact of Social Media on Political Polarization: A Meta-Analysis
The role of social media in political polarization has become a topic of intense interest and debate. This paper presents a meta-analysis of the existing literature on the relationship between social media usage and political polarization. The analysis includes studies conducted in various countries and settings, using different methods and measures of polarization. The results suggest that social media can contribute to increased political polarization by facilitating the formation of echo chambers, where individuals are exposed primarily to information and opinions that reinforce their preexisting beliefs. However, the impact of social media on polarization is not uniform and can vary depending on the context and the individual characteristics of the users. The paper discusses the implications of these findings for understanding the role of social media in shaping political discourse and offers suggestions for future research.

**Similar Documents**: This comprehensive review paper explores the intricate link between social media and political polarization, a subject that has garnered significant attention and controversy in recent times. Through a meta-analysis encompassing a wide range of studies conducted across different nations and environments, utilizing diverse methodologies and measures of polarization, the paper aims to clarify the nature of this relationship. The findings indicate that social media platforms can exacerbate political polarization by fostering echo chambers, environments where users predominantly encounter information and viewpoints that echo their preconceived notions. Yet, the effect of social media on polarization is nuanced, exhibiting variability based on contextual factors and the unique attributes of individual users. The paper delves into

Two quasi-experiments investigated the relation between specific adult female facial features and the attraction, attribution, and altruistic responses of adult males. Precise measurements were obtained of the relative size of 24 facial features in an international sample of photographs of 50 females. Male subjects provided ratings of the attractiveness of each of the females. Positively correlated with attractiveness ratings were the neonate features of large eyes, small nose, and small chin; the maturity features of prominent cheekbones and narrow cheeks; and the expressive features of high eyebrows, large pupils, and large smile. A second study asked males to rate the personal characteristics of 16 previously measured females. The males were also asked to indicate the females for whom they would be most inclined to perform altruistic behaviors, and select for dating, sexual behavior, and childrearing. The second study replicated the correlations of feature measurements with attractiveness. Facial features also predicted personality attributions, altruistic inclinations, and reproductive interest. Sociobiological interpretations are discussed.

We address the problem of attribute-based people search in real surveillance environments. The system we developed is capable of answering user queries such as "show me all people with a beard and sunglasses, wearing a white hat and a patterned blue shirt, from all metro cameras in the downtown area, from 2pm to 4pm last Saturday". In this paper, we describe the lessons we learned from practical deployments of our system, and how we made our algorithms achieve the accuracy and efficiency required by many police departments around the world. In particular, we show that a novel set of multimodal integral filters and proper normalization of attribute scores are critical to obtain good performance. We conduct a comprehensive experimental analysis on video footage captured from a large set of surveillance cameras monitoring metro chokepoints, in both crowded and normal activity periods. Moreover, we show impressive results using images from the recent Boston marathon bombing event, where our system can rapidly retrieve the two suspects based on their attributes from a database containing more than one thousand people present at the event.

Conformity is a type of social influence involving a change in opinion or behavior in order to fit in with a group. Employing several social networks as the source for our experimental data, we study how the effect of conformity plays a role in changing users' online behavior. We formally define several major types of conformity in individual, peer, and group levels. We propose Confluence model to formalize the effects of social conformity into a probabilistic model. Confluence can distinguish and quantify the effects of the different types of conformities. To scale up to large social networks, we propose a distributed learning method that can construct the Confluence model efficiently with near-linear speedup. Our experimental results on four different types of large social networks, i.e., Flickr, Gowalla, Weibo and Co-Author, verify the existence of the conformity phenomena. Leveraging the conformity information, Confluence can accurately predict actions of users. Our experiments show that Confluence significantly improves the prediction accuracy by up to 5-10% compared with several alternative methods.

My problem is that I have been persecuted by an integer. For seven years this number has followed me around, has intruded in my most private data, and has assaulted me from the pages of our most public journals. This number assumes a variety of disguises, being sometimes a little larger and sometimes a little smaller than usual, but never changing so much as to be unrecognizable. The persistence with which this number plagues me is far more than a random accident. There is, to quote a famous senator, a design behind it, some pattern governing its appearances. Either there really is something unusual about the number or else I am suffering from delusions of persecution. I shall begin my case history by telling you about some experiments that tested how accurately people can assign numbers to the magnitudes of various aspects of a stimulus. In the traditional language of psychology these would be called experiments in absolute

Under what conditions will a bystander intervene to try to stop a violent attack by one person on another? It is generally believed that the greater the size of the crowd of bystanders, the less the chance that any of them will intervene. A complementary model is that social identity is critical as an explanatory variable. For example, when the bystander shares common social identity with the victim the probability of intervention is enhanced, other things being equal. However, it is generally not possible to study such hypotheses experimentally for practical and ethical reasons. Here we show that an experiment that depicts a violent incident at life-size in immersive virtual reality lends support to the social identity explanation. 40 male supporters of Arsenal Football Club in England were recruited for a two-factor between-groups experiment: the victim was either an Arsenal supporter or not (in-group/out-group), and looked towards the participant for help or not during the confrontation. The response variables were the numbers of verbal and physical interventions by the participant during the violent argument. The number of physical interventions had a significantly greater mean in the in-group condition compared to the out-group. The more that participants perceived that the Victim was looking to them for help the greater the number of interventions in the in-group but not in the out-group. These results are supported by standard statistical analysis of variance, with more detailed findings obtained by a symbolic regression procedure based on genetic programming. Verbal interventions made during their experience, and analysis of post-experiment interview data suggest that in-group members were more prone to confrontational intervention compared to the out-group who were more prone to make statements to try to diffuse the situation.

OBJECTIVES
A women's breast is a sex-specific and aesthetic bodily attribute. It is suggested that breast morphology signals maturity, health, and fecundity. The perception of a woman's attractiveness and age depends on various cues, such as breast size or areola pigmentation. Conducted in Poland and Papua, the current study investigated how breast attractiveness, and the further estimate of a woman's age based on her breast's appearance, is affected by the occurrence of breast ptosis (ie, sagginess, droopiness).

METHODS
In the Polish sample, 57 women and 50 men (N = 107) were presented with sketches of breasts manipulated to represent different stages of ptosis based on two different breast ptosis classifications. The participants were asked to rate the breast attractiveness and age of the woman whose breasts were depicted in each sketch. In Papua, 45 men aged 20 to 75 years took part in the study, which was conducted using only one of the classifications of breast ptosis.

RESULTS
Regardless of the classification used, the results showed that the assessed attractiveness of the breasts decreased as the estimated age increased with respect to the more ptotic breasts depicted in the sketches. The results for Papuan raters were the same as for the Polish sample.

CONCLUSIONS
Breast ptosis may be yet another physical trait that affects the perception and preferences of a potential sexual partner. The consistency in ratings between Polish and Papuan raters suggests that the tendency to assess ptotic breasts with aging and a loss of attractiveness is cross-culturally universal.

Existing research suggests that individual personality differences are correlated with a user's speed and accuracy in solving problems with different types of complex visualization systems. We extend this research by isolating factors in personality traits as well as in the visualizations that could have contributed to the observed correlation. We focus on a personality trait known as "locus of control” (LOC), which represents a person's tendency to see themselves as controlled by or in control of external events. To isolate variables of the visualization design, we control extraneous factors such as color, interaction, and labeling. We conduct a user study with four visualizations that gradually shift from a list metaphor to a containment metaphor and compare the participants' speed, accuracy, and preference with their locus of control and other personality factors. Our findings demonstrate that there is indeed a correlation between the two: participants with an internal locus of control perform more poorly with visualizations that employ a containment metaphor, while those with an external locus of control perform well with such visualizations. These results provide evidence for the externalization theory of visualization. Finally, we propose applications of these findings to adaptive visual analytics and visualization evaluation.

Attributes such as size, weight or color are at the core of conceptualization, i.e., the formal representation of entities or events in the real world. In natural language, formal attributes find their counterpart in attribute nouns which can be used in order to generalize over individual properties (e.g., big or small in case of size, blue or red in case of color). In order to ascribe such properties to entities or events, adjectivenoun phrases are a very frequent linguistic pattern (e.g., a blue shirt, a big lion). In these constructions, attribute meaning is conveyed only implicitly, i.e., without being overtly realized at the phrasal surface. This thesis is about modeling attribute meaning in adjectives and nouns in a distributional semantics framework. This implies the acquisition of meaning representations for adjectives, nouns and their phrasal combination from corpora of natural language text in an unsupervised manner, without tedious handcrafting or manual annotation efforts. These phrase representations can be used to predict implicit attribute meaning from adjective-noun phrases – a problem which will be referred to as attribute selection throughout this thesis. The approach to attribute selection proposed in this thesis is framed in structured distributional models. We model adjective and noun meanings as distinct semantic vectors in the same semantic space spanned by attributes as dimensions of meaning. Based on these word representations, we make use of vector composition operations in order to construct a phrase representation from which the most prominent attribute(s) being expressed in the compositional semantics of the adjective-noun phrase can be selected by means of an unsupervised selection function. This approach not only accounts for the linguistic principle of compositionality that underlies adjective-noun phrases, but also avoids inherent sparsity issues that result from the fact that the relationship between an adjective, a noun and a particular attribute is rarely explicitly observed in corpora. The attribute models developed in this thesis aim at a reconciliation of the conflict between specificity and sparsity in distributional semantic models. For this purpose, we compare various instantiations of attribute models capitalizing on pattern-based and dependency-based distributional information as well as attribute-specific latent topics induced from a weakly supervised adaptation of Latent Dirichlet Allocation. Moreover, we propose a novel framework of distributional enrichment in order to enhance structured vector representations by incorporating additional lexical information from complementary distributional sources. In applying distributional enrichment to distributional attribute models, we follow the idea to augment structured

While behavioural difficulties in facial emotion recognition (FER) have been observed in individuals with Autism Spectrum Disorder (ASD), behavioural studies alone are not suited to elucidate the specific nature of FER challenges in ASD. Eye tracking (ET) and electroencephalography (EEG) provide insights in to the attentional and neurological correlates of performance, and may therefore provide insight in to the mechanisms underpinning FER in ASD. Given that these processes develop over the course of the developmental trajectory, there is a need to synthesise findings in regard to the developmental stages to determine how the maturation of these systems may impact FER in ASD. We conducted a systematic review of fifty-four studies investigating ET or EEG meeting inclusion criteria. Findings indicate divergence of visual processing pathways in individuals with ASD. Altered function of the social brain in ASD impacts the processing of facial emotion across the developmental trajectory, resulting in observable differences in ET and EEG outcomes.

The face is one of the most powerful channels of nonverbal communication. Facial expression provides cues about emotion, intention, alertness, pain, personality, regulates interpersonal behavior, and communicates psychiatric and biomedical status among other functions. Within the past 15 years, there has been increasing interest in automated facial expression analysis within the computer vision and machine learning communities. This chapter reviews fundamental approaches to facial measurement by behavioral scientists and current efforts in automated facial expression recognition. We consider challenges, review databases available to the research community, approaches to feature detection, tracking, and representation, and both supervised and unsupervised learning. keywords : Facial expression analysis, Action unit recognition, Active Appearance Models, temporal clustering.


Question: Clothing and individuals - a social signal processing viewpoint.

In the Intersection of Fashion and Identity: Uncovering the Signatures of Style
The garments we adorn ourselves with and our self-identities are inextricably linked, broadcasting subtle indicators of our financial status, profession, and social persona. This paper delves into the narratives our attire narrates about our individual styles. We commence by conceptualizing an interactive online game, termed Style Showdown, to harness the collective discernment of human participants in evaluating style. Through this game, we gather a novel dataset encompassing ensembles of clothing, each paired with style assessments across five distinctive categories: hipster, bohemian, vintage vixen, preppy, and gothic. Subsequently, we employ these data to cultivate models capable of discerning between styles, both among classes and within a single style. Moreover, we probe methodologies to pinpoint clothing components that are universally emblematic of a specific style, alongside techniques to identify items within an ensemble that may insinuate a particular stylistic inclination.

The game, Style Showdown, is designed to capture the nuances of style perception, allowing participants to rate outfits based on the aforementioned style categories. The dataset, rich with diverse fashion expressions, serves as a foundation for our analytical models. Our approach to between-class and within-class classification of styles showcases the potential for computational analysis in understanding the subtleties of fashion. By identifying the key elements that define each style, we not only enrich the field of social signal processing but also provide insights into the multifaceted relationship between clothing and personal identity. This exploration contributes to a broader understanding of how fashion serves as a medium for self-expression and social signaling in contemporary society.

A direct form of experiential understanding of others, "intentional attunement", is achieved by modeling their behavior as intentional experiences on the basis of the activation of shared neural systems underpinning what the others do an feel and what we do and feel. This modeling mechanism is embodied simulation. In parallel with the detached sensory description of the observed social stimuli, internal representations of the body states associated with actions, emotions, and sensations are evoked in the observer, as if he/she would be doing a similar action or experiencing a similar emotion or sensation. Mirror neuron systems are likely the neural correlate of this mechanism. By means of a shared neural state realized in two different bodies, the "objectual other" becomes "another self". A defective intentional attunement caused by a lack of embodies simulation might cause some of the social impairments of autistic individuals.

The authors tested the prediction that women prefer clothing that is more revealing and sexy when fertility is highest within the ovulatory cycle. Eighty-eight women reported to the lab twice: once on a low-fertility day of the cycle and once on a high-fertility day (confirmed using hormone tests). In each session, participants posed for full-body photographs in the clothing they wore to the lab, and they drew illustrations to indicate an outfit they would wear to a social event that evening. Although each data source supported the prediction, the authors found the most dramatic changes in clothing choice in the illustrations. Ovulatory shifts in clothing choice were moderated by sociosexuality, attractiveness, relationship status, and relationship satisfaction. Sexually unrestricted women, for example, showed greater shifts in preference for revealing clothing worn to the laboratory near ovulation. The authors suggest that clothing preference shifts could reflect an increase in female-female competition near ovulation.

Spoken dialogue systems have received increased interest because they are potentially much more natural and powerful methods of communicating with machines than are current graphics-based interfaces. Wired for Speech presents basic research in the psychological and sociological aspects of voice synthesis and recognition. Its major lesson is that people attribute human characteristics to spoken dialogue systems for reasons related to human evolution. But although it contains interesting basic research, the book is mainly aimed at giving technological or marketing advice to those seeking to use voice interfaces when creating commercial applications. The book is oriented around a series of simple experiments designed to show just how pervasive psychological and social influences can be on the opinions and behaviors of people confronted with voice interfaces. Each chapter describes a basic research hypothesis, introduces an experiment to test it, and discusses its implications for designing voice interfaces: gender, personality, accent, ethnicity, emotion, number of distinct voices, use of " I " by the system, voices in concert with faces, mixed synthetic and recorded voices, context, and the effects of errors in human–computer cooperation. Although Wired for Speech is very accessible , especially to the non-scientist, it is written with an unusual bibliography style for an academic book: All references and details are given in a notes section at the end of the book, making up one third of the content. This narrative exposition style will probably not satisfy either type of reader: Scientists will be frustrated at the imprecision in argumen-tation, lack of detail in the book itself, and continually having to refer to the notes. The lack of detail also prevents the book from serving as a reference work. Meanwhile those needing advice when implementing voice interfaces will be puzzled at references to Chomsky, Grice, and Zemlin. Thus the book seems to suffer from not knowing its audience well, which is odd as this is precisely the lesson that the book tries to impart. Another complaint from the scientific point of view is the obsession this particular book has with casting every experimental result as advice for the business and marketing side of voice interfaces, typically concentrating on Web-based e-marketing examples such as buying books on line. Most of the experiments have sample sizes of between 40 and 50, but the authors seem ready to invite multi-billion-dollar businesses to immediately base their deployed systems on these results. Finally, and fundamentally, this …

We present a distributed representation of pose and appearance of people called the “poselet activation vector”. First we show that this representation can be used to estimate the pose of people defined by the 3D orientations of the head and torso in the challenging PASCAL VOC 2010 person detection dataset. Our method is robust to clutter, aspect and viewpoint variation and works even when body parts like faces and limbs are occluded or hard to localize. We combine this representation with other sources of information like interaction with objects and other people in the image and use it for action recognition. We report competitive results on the PASCAL VOC 2010 static image action classification challenge.

We live life in the network. We check our e-mails regularly, make mobile phone calls from almost any location ... make purchases with credit cards ... [and] maintain friendships through online social networks. ... These transactions leave digital traces that can be compiled into comprehensive pictures of both individual and group behavior, with the potential to transform our understanding of our lives, organizations, and societies.

The words people use in their daily lives can reveal important aspects of their social and psychological worlds. With advances in computer technology, text analysis allows researchers to reliably and quickly assess features of what people say as well as subtleties in their linguistic styles. Following a brief review of several text analysis programs, we summarize some of the evidence that links natural word use to personality, social and situational fluctuations, and psychological interventions. Of particular interest are findings that point to the psychological value of studying particles-parts of speech that include pronouns, articles, prepositions, conjunctives, and auxiliary verbs. Particles, which serve as the glue that holds nouns and regular verbs together, can serve as markers of emotional state, social identity, and cognitive styles.

Audio signal classification (ASC) consists of extracting relevant features from a sound, and of using these features to identify into which of a set of classes the sound is most likely to fit. The feature extraction and grouping algorithms used can be quite diverse depending on the classification domain of the application. This paper presents background necessary to understand the general research domain of ASC, including signal processing, spectral analysis, psychoacoustics and auditory scene analysis. Also presented are the basic elements of classification systems. Perceptual and physical features are discussed, as well as clustering algorithms and analysis duration. Neural nets and hidden Markov models are discussed as they relate to ASC. These techniques are presented with an overview of the current state of the ASC research literature.

Synopsis Few systematic studies have examined individual differences in women's use of facial make-up or the possible psychosocial effects of such use. In the present investigation, the researchers developed several measures of the amount and the situational and temporal patterning of cosmetics use. Among forty-two female college students, differential use was associated with a number of selected personality variables-public self-consciousness, public body-consciousness, social anxiety, and various body-image factors. Through the imagery-induction of cosmetics use and non-use conditions, cosmetics users' self-evaluative responses were studied. In a variety of imagined situations, subjects reported being more self-confident and sociable when wearing as opposed to not wearing their customary cosmetics. Findings were discussed in the context of the role of cosmetics in self-image enhancement and social impression management.

The face is one of the most powerful channels of nonverbal communication. Facial expression provides cues about emotion, intention, alertness, pain, personality, regulates interpersonal behavior, and communicates psychiatric and biomedical status among other functions. Within the past 15 years, there has been increasing interest in automated facial expression analysis within the computer vision and machine learning communities. This chapter reviews fundamental approaches to facial measurement by behavioral scientists and current efforts in automated facial expression recognition. We consider challenges, review databases available to the research community, approaches to feature detection, tracking, and representation, and both supervised and unsupervised learning. keywords : Facial expression analysis, Action unit recognition, Active Appearance Models, temporal clustering.

The use of the gesture system in our daily life as a natural human-human interaction has inspired the researchers to simulate and utilize this gift in human-machine interaction which is appealing and can take place the bore interaction ones that existed such as television, radio, and various home appliances as well as virtual reality will worth and deserve its name. This kind of interaction ensures promising and satisfying outcomes if applied in systematic approach, and supports unadorned human hand when transferring the message to these devices which is easiest, comfort and desired rather than the communication that requires frills to deliver the message to such devices. With the rapid emergence of 3d applications and virtual environments in computer system the need for a new type of interaction device arises. This is because the traditional devices such as mouse, keyboard and joystick become inefficient and cumbersome within this virtual environments.in other words evolution of user interfaces shapes the change in the HumanComputer Interaction (HCI).Intuitive and naturalness characteristics of “Hand Gestures” in the HCI have been the driving force and motivation to develop an interaction device which can replace current unwieldy tools. A survey on the methods of analysing, modelling and recognizing hand gestures in the context of the HCI is provided in this paper.


Question: Clothes and individuals - A social signal processing viewpoint.

This paper introduces the novel task of "Exact Street to Shop" matching, aimed at bridging the gap between real-world clothing images captured in uncontrolled environments and their exact counterparts found in online retail settings. The challenge lies in reconciling the stark visual discrepancies between the spontaneous, candid street photographs, featuring individuals wearing attire in various contexts, and the meticulously staged, professional images from online stores, which may depict the same items on models, mannequins, or in isolation. To address this task, we have assembled a comprehensive dataset, comprising 404,683 meticulously curated online shop images sourced from 25 diverse retailers, and 20,357 street photographs. This dataset uniquely offers 39,479 precise matches between garments depicted in street and shop images. We propose and evaluate three distinct approaches to tackle the "Exact Street to Shop" retrieval problem, including two foundational deep learning methods as baselines, and an innovative technique for learning a domain-specific similarity measure that effectively captures the nuances between street and shop domains. Our empirical findings reveal that the learned similarity metric significantly outperforms the baseline methods, which rely on conventional deep learning representations, thereby highlighting the efficacy of our domain-adapted approach in this complex matching task.

Wearable computers are one of the new technologies that are expected to be a part of users' lives extensively in near future. While some of the users have positive attitudes towards these new products, some users may reject to use them due to different reasons. User experience is subjective, and effected by various parameters. Among these the first impression, namely the perceived qualities has an important impact on product acceptance. This paper aims to explore the perceived qualities of wearables and define the relations between them. An empirical study is conducted, to find out the hierarchy and meaningful relationships between the perceived qualities of smart wearables. The study is based on personal construct theory and data is presented by Cross-Impact Analysis. The patterns behind affection and affected qualities are explored to understand the design requirements for the best integration of wearables into daily lives.

We have developed a near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals. The computational approach taken in this system is motivated by both physiology and information theory, as well as by the practical requirements of near-real-time performance and accuracy. Our approach treats the face recognition problem as an intrinsically two-dimensional (2-D) recognition problem rather than requiring recovery of three-dimensional geometry, taking advantage of the fact that faces are normally upright and thus may be described by a small set of 2-D characteristic views. The system functions by projecting face images onto a feature space that spans the significant variations among known face images. The significant features are known as "eigenfaces," because they are the eigenvectors (principal components) of the set of faces; they do not necessarily correspond to features such as eyes, ears, and noses. The projection operation characterizes an individual face by a weighted sum of the eigenface features, and so to recognize a particular face it is necessary only to compare these weights to those of known individuals. Some particular advantages of our approach are that it provides for the ability to learn and later recognize new faces in an unsupervised manner, and that it is easy to implement using a neural network architecture.

Computer animated agents and robots bring a social dimension to human computer interaction and force us to think in new ways about how computers could be used in daily life. Face to face communication is a real-time process operating at a a time scale in the order of 40 milliseconds. The level of uncertainty at this time scale is considerable, making it necessary for humans and machines to rely on sensory rich perceptual primitives rather than slow symbolic inference processes. In this paper we present progress on one such perceptual primitive. The system automatically detects frontal faces in the video stream and codes them with respect to 7 dimensions in real time: neutral, anger, disgust, fear, joy, sadness, surprise. The face finder employs a cascade of feature detectors trained with boosting techniques [15, 2]. The expression recognizer receives image patches located by the face detector. A Gabor representation of the patch is formed and then processed by a bank of SVM classifiers. A novel combination of Adaboost and SVM's enhances performance. The system was tested on the Cohn-Kanade dataset of posed facial expressions [6]. The generalization performance to new subjects for a 7- way forced choice correct. Most interestingly the outputs of the classifier change smoothly as a function of time, providing a potentially valuable representation to code facial expression dynamics in a fully automatic and unobtrusive manner. The system has been deployed on a wide variety of platforms including Sony's Aibo pet robot, ATR's RoboVie, and CU animator, and is currently being evaluated for applications including automatic reading tutors, assessment of human-robot interaction.

Knowledge of a person’s whereabouts in the home is key to context-aware applications, but many people do not want to carry or wear a tag or mobile device in the home. Therefore, many tracking systems are now using so-called weak biometrics such as height, weight, and width. In this paper, we propose to use body shape as a weak biometric, differentiating people based on features such as head size, shoulder size, or torso size. The basic idea is to scan the body with a radar sensor and to compute the reflection profile: the amount of energy that reflects back from each part of the body. Many people have different body shapes even if they have the same height, weight, or width, which makes body shape a stronger biometric. We built a proof-of-concept system called FormaTrack to test this approach, and evaluate using eight participants of varying height and weight. We collected over 2800 observations while capturing a wide range of factors such as clothing, hats, shoes, and backpacks. Results show that FormaTrack can achieve a precision, recall, direction and identity accuracy (over all possible groups of 2 people) of 100%, 99.86%, 99.7% and 95.3% respectively. Results indicate that FormaTrack can achieve over 99% tracking accuracy with 2 people in a home with 5 or more rooms.

In this work, we present a fashion-focused Creative Commons dataset, which is designed to contain a mix of general images as well as a large component of images that are focused on fashion (i.e., relevant to particular clothing items or fashion accessories). The dataset contains 4810 images and related metadata. Furthermore, a ground truth on image's tags is presented. Ground truth generation for large-scale datasets is a necessary but expensive task. Traditional expert based approaches have become an expensive and non-scalable solution. For this reason, we turn to crowdsourcing techniques in order to collect ground truth labels; in particular we make use of the commercial crowdsourcing platform, Amazon Mechanical Turk (AMT). Two different groups of annotators (i.e., trusted annotators known to the authors and crowdsourcing workers on AMT) participated in the ground truth creation. Annotation agreement between the two groups is analyzed. Applications of the dataset in different contexts are discussed. This dataset contributes to research areas such as crowdsourcing for multimedia, multimedia content analysis, and design of systems that can elicit fashion preferences from users.

The electrocardiogram (ECG also called EKG) trace expresses cardiac features that are unique to an individual. The ECG processing followed a logical series of experiments with quantifiable metrics. Data filters were designed based upon the observed noise sources. Fiducial points were identified on the filtered data and extracted digitally for each heartbeat. From the fiducial points, stable features were computed that characterize the uniqueness of an individual. The tests show that the extracted features are independent of sensor location, invariant to the individual’s state of anxiety, and unique to an individual. 2004 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

Interpersonal relation defines the association, e.g., warm, friendliness, and dominance, between two or more people. We investigate if such fine-grained and high-level relation traits can be characterized and quantified from face images in the wild. We address this challenging problem by first studying a deep network architecture for robust recognition of facial expressions. Unlike existing models that typically learn from facial expression labels alone, we devise an effective multitask network that is capable of learning from rich auxiliary attributes such as gender, age, and head pose, beyond just facial expression data. While conventional supervised training requires datasets with complete labels (e.g., all samples must be labeled with gender, age, and expression), we show that this requirement can be relaxed via a novel attribute propagation method. The approach further allows us to leverage the inherent correspondences between heterogeneous attribute sources despite the disparate distributions of different datasets. With the network we demonstrate state-of-the-art results on existing facial expression recognition benchmarks. To predict inter-personal relation, we use the expression recognition network as branches for a Siamese model. Extensive experiments show that our model is capable of mining mutual context of faces for accurate fine-grained interpersonal prediction.

A major challenge in physiology-based emotion recognition is to establish an effective emotion recognizer for multi-users in the user-independent scenario. The recognition result is not satisfied because it ignores the difference in individual response pattern, which can be attributed to IRS (Individual Response Specificity) and SRS(Stimuli Response Specificity) in psychophysiology. To improve the performance of emotion recognition, this paper proposes a Group-Based IRS model by adaptively matching a suitable recognizer in accordance with user's IRS level. Specifically, the users are put into distinct groups by using cluster analysis techniques, where users within the same group have similar IRS level than other groups. Then physiological data of users from each group is utilized to build the corresponding emotion recognizers. After categorizing a new user into one group according to his IRS level, the new user's emotion state is predicted by the corresponding emotion recognizer. To validate our model, the affective physiological data was collected from 11 subjects in four induced emotions(neutral, sadness, fear and pleasure), three-channel bio-sensors were used to measure users electrocardiogram (ECG), galvanic skin response (GSR) and photo plethysmography (PPG). The results show that the recognition precision in Group-based IRS model is higher than general model.

Awareness of the emotion of those who communicate with others is a fundamental challenge in building affective intelligent systems. Emotion is a complex state of the mind influenced by external events, physiological changes, or relationships with others. Because emotions can represent a user's internal context or intention, researchers suggested various methods to measure the user's emotions from analysis of physiological signals, facial expressions, or voice. However, existing methods have practical limitations to be used with consumer devices, such as smartphones; they may cause inconvenience to users and require special equipment such as a skin conductance sensor. Our approach is to recognize emotions of the user by inconspicuously collecting and analyzing user-generated data from different types of sensors on the smartphone. To achieve this, we adopted a machine learning approach to gather, analyze and classify device usage patterns, and developed a social network service client for Android smartphones which unobtrusively find various behavioral patterns and the current context of users. Also, we conducted a pilot study to gather real-world data which imply various behaviors and situations of a participant in her/his everyday life. From these data, we extracted 10 features and applied them to build a Bayesian Network classifier for emotion recognition. Experimental results show that our system can classify user emotions into 7 classes such as happiness, surprise, anger, disgust, sadness, fear, and neutral with a surprisingly high accuracy. The proposed system applied to a smartphone demonstrated the feasibility of an unobtrusive emotion recognition approach and a user scenario for emotion-oriented social communication between users.

In this paper we shall present a further development of a previously published theory concerning opinion influence processes in social groups (7). This further development has enabled us to extend the theory to deal with other areas, in addition to opinion formation, in which social comparison is important. Specifically, we shall develop below how the theory applies to the appraisal and evaluation of abilities as well as opinions. Such theories and hypotheses in the area of social psychology are frequently viewed in terms of how “plausible” they seem. “Plausibility” usually means whether or not the theory or hypothesis fits one’s intuition or one’s common sense. In this meaning much of the theory which is to be presented here is not” plausible “. The theory does, however, explain a considerable amount of data and leads to testable derivations. Three experiments, specifically designed to test predictions from this extension of the theory, have now been completed (5, 12, 19). They all provide good corroboration. We will in the following pages develop the theory and present the relevant data.


Question: Attire and Individuals - A social signal processing viewpoint.

Fashion Lens: Precise Clothing Style Recognition and Retrieval
Amidst the burgeoning popularity of smartphones and tablets, search methodologies have transcended text to encompass more diverse formats such as images and voice. This evolution is particularly impactful in the fashion industry, where visual search provides a more intuitive interface for capturing intricate visual elements of style that are often beyond the descriptive capacity of text. Nevertheless, accurately identifying and matching these stylistic attributes remains a daunting task due to the inherent variability and malleability of clothing. In this paper, we introduce a novel fine-grained learning model and multimedia retrieval architecture to tackle this challenge. Initially, we construct an attribute vocabulary by leveraging human annotations sourced from a unique fine-grained clothing dataset. This vocabulary then serves as the foundation for training a sophisticated visual recognition system, specifically tailored to discern clothing styles with precision. We showcase our system's capabilities through benchmark recognition and retrieval outcomes on the Women's Fashion Coat Dataset, and further demonstrate the potential for mobile applications in attribute-based multimedia retrieval for clothing items, as well as image annotation.

We describe a complete system for animating realistic clothing on synthetic bodies of any shape and pose without manual intervention. The key component of the method is a model of clothing called DRAPE (DRessing Any PErson) that is learned from a physics-based simulation of clothing on bodies of different shapes and poses. The DRAPE model has the desirable property of "factoring" clothing deformations due to body shape from those due to pose variation. This factorization provides an approximation to the physical clothing deformation and greatly simplifies clothing synthesis. Given a parameterized model of the human body with known shape and pose parameters, we describe an algorithm that dresses the body with a garment that is customized to fit and possesses realistic wrinkles. DRAPE can be used to dress static bodies or animated sequences with a learned model of the cloth dynamics. Since the method is fully automated, it is appropriate for dressing large numbers of virtual characters of varying shape. The method is significantly more efficient than physical simulation.

Prior research has assumed that touch has a persuasive effect only if it provides attribute or structural information about a product. Under this view, the role of touch as a persuasive tool is limited. The main purpose of this research is to investigate the persuasive influence of touch as an affective tool in the absence of useful product-related information. The authors find that for people who are motivated to touch because it is fun or interesting, a communication that incorporates touch leads to increased affective response and increased persuasion, particularly when the touch provides neutral or positive sensory feedback. People who are not motivated to touch for fun will also be persuaded by a communication that incorporates touch when they are able to make sense of how the touch is related to the message. The authors explore the effectiveness of different types of touch in generating an affective response, and they replicate the effects on attitudes and behavior in a real-world setting. This research suggests that the marketing implications of touch are more substantial than previously believed. The authors present research implications for direct marketing, product packaging, point-of-purchase displays, and print advertising.

There is an increased interest in artificially intelligent technology that surrounds us and takes decisions on our behalf. This creates the need for such technology to be able to communicate with humans and understand natural language and non-verbal behaviour that may carry information about our complex physical world. Artificial agents today still have little knowledge about the physical space that surrounds us and about the objects or concepts within our attention. We are still lacking computational methods in understanding the context of human conversation that involves objects and locations around us. Can we use multimodal cues from human perception of the real world as an example of language learning for robots? Can artificial agents and robots learn about the physical world by observing how humans interact with it and how they refer to it and attend during their conversations? This PhD project’s focus is on combining spoken language and non-verbal behaviour extracted by multi-party dialogue in order to increase context awareness and spatial understanding for artificial agents.

The image memorability consists in the faculty of an image to be recalled after a period of time. Recently, the memorability of an image database was measured and some factors responsible for this memorability were highlighted. In this paper, we investigate the role of visual attention in image memorability around two axis. The first one is experimental and uses results of eye-tracking performed on a set of images of different memorability scores. The second investigation axis is predictive and we show that attention-related features can advantageously replace low-level features in image memorability prediction. From our work it appears that the role of visual attention is important and should be more taken into account along with other low-level features.

This thesis investigates the use of wearable sensors to recognize human activity. The activity of the user is one example of context information – others include the user’s location or the state of his environment – which can help computer applications to adapt to the user depending on the situation. In this thesis we use wearable sensors – mainly accelerometers – to record, model and recognize human activities. Using wearable sensors allows continuous recording of activities across different locations and independent from external infrastructure. There are many possible applications for activity recognition with wearable sensors, for instance in the areas of healthcare, elderly care, personal fitness, entertainment, or performing arts. In this thesis we focus on two particular research challenges in activity recognition, namely the need for less supervision, and the recognition of high-level activities. We make several contributions towards addressing these challenges. Our first contribution is an analysis of features for activity recognition. Using a data set of activities such as walking, standing, sitting, or hopping, we analyze the performance of commonly used features and window lengths over which the features are computed. Our results indicate that different features perform well for different activities, and that in order to achieve best recognition performance, features and window lengths should be chosen specific for each activity. In order to reduce the need for labeled training data, we propose an unsupervised algorithm which can discover structure in unlabeled recordings of activities. The approach identifies correlated subsets in feature space, and represents these subsets with low-dimensional models. We show that the discovered subsets often correspond to distinct activities, and that the resulting models can be used for recognition of activities in unknown data. In a separate study, we show that the approach can be effectively deployed in a semi-supervised learning framework. More specifically, we combine the approach with a discriminant classifier, and show that this scheme allows high recognition rates even when using only a small amount of labeled training data. Recognition of higher-level activities such as shopping, doing housework, or commuting is challenging, as these activities are composed of changing sub-activities and vary strongly across individuals. We present one study in which we recorded 10h of three different high-level activities, investigating to which extent methods for low-level activities can be scaled to the recognition of high-level activities. Our results indicate that for settings as ours, traditional supervised approaches in combination with data from wearable accelerometers can achieve recognition rates of more than 90%. While unsupervised techniques are desirable for short-term activities, they become crucial for long-term activities, for which annotation is often impractical or impossible. To this end we propose an unsupervised approach based on topic models that allows to discover high-level structure in human activity data. The discovered activity patterns correlate with daily routines such as commuting, office work, or lunch routine, and they can be used to recognize such routines in unknown data.

Sentiment analysis is crucial for extracting social signals from social media content. Due to the prevalence of images in social media, image sentiment analysis is receiving increasing attention in recent years. However, most existing systems are black-boxes that do not provide insight on how image content invokes sentiment and emotion in the viewers. On the other hand, psychological studies have confirmed that salient objects in an image often invoke emotions. In this work, we investigate more fine-grained and more comprehensive interaction between visual saliency and visual sentiment. In particular, we partition images in several meta-level scene-type dimensions that are relevant to most images, including: open-closed, natural-manmade, indoor-outdoor, and face-noface. Facilitated by state of the art saliency detection algorithm and sentiment classification algorithm, we examine how the sentiment of the salient region(s) in an image relates to the overall sentiment of the image. The experiments on a representative image emotion dataset have shown interesting correlation between saliency and sentiment in different scene types and shed light on the mechanism of visual sentiment evocation.

In paradigms in which participants state their ideal romantic-partner preferences or examine vignettes and photographs, men value physical attractiveness more than women do, and women value earning prospects more than men do. Yet it remains unclear if these preferences remain sex differentiated in predicting desire for real-life potential partners (i.e., individuals whom one has actually met). In the present study, the authors explored this possibility using speed dating and longitudinal follow-up procedures. Replicating previous research, participants exhibited traditional sex differences when stating the importance of physical attractiveness and earning prospects in an ideal partner and ideal speed date. However, data revealed no sex differences in the associations between participants' romantic interest in real-life potential partners (met during and outside of speed dating) and the attractiveness and earning prospects of those partners. Furthermore, participants' ideal preferences, assessed before the speed-dating event, failed to predict what inspired their actual desire at the event. Results are discussed within the context of R. E. Nisbett and T. D. Wilson's (1977) seminal article: Even regarding such a consequential aspect of mental life as romantic-partner preferences, people may lack introspective awareness of what influences their judgments and behavior.

Simulation models of facial expressions propose that sensorimotor regions may increase the clarity of facial expressions representations in extrastriate areas. We monitored the event-related potential marker of visual working memory (VWM) representations, namely the sustained posterior contralateral negativity (SPCN), also termed contralateral delay activity, while participants performed a change detection task including to-be-memorized faces with different intensities of anger. In one condition participants could freely use their facial mimicry during the encoding/VWM maintenance of the faces while in a different condition participants had their facial mimicry blocked by a gel. Notably, SPCN amplitude was reduced for faces in the blocked mimicry condition when compared to the free mimicry condition. This modulation interacted with the empathy levels of participants such that only participants with medium-high empathy scores showed such reduction of the SPCN amplitude when their mimicry was blocked. The SPCN amplitude was larger for full expressions when compared to neutral and subtle expressions, while subtle expressions elicited lower SPCN amplitudes than neutral faces. These findings provide evidence of a functional link between mimicry and VWM for faces and further shed light on how this memory system may receive feedbacks from sensorimotor regions during the processing of facial expressions.

The words people use in their daily lives can reveal important aspects of their social and psychological worlds. With advances in computer technology, text analysis allows researchers to reliably and quickly assess features of what people say as well as subtleties in their linguistic styles. Following a brief review of several text analysis programs, we summarize some of the evidence that links natural word use to personality, social and situational fluctuations, and psychological interventions. Of particular interest are findings that point to the psychological value of studying particles-parts of speech that include pronouns, articles, prepositions, conjunctives, and auxiliary verbs. Particles, which serve as the glue that holds nouns and regular verbs together, can serve as markers of emotional state, social identity, and cognitive styles.

The face is one of the most important stimuli carrying social meaning. Thanks to the fast analysis of faces, we are able to judge physical attractiveness and features of their owners' personality, intentions, and mood. From one's facial expression we can gain information about danger present in the environment. It is obvious that the ability to process efficiently one's face is crucial for survival. Therefore, it seems natural that in the human brain there exist structures specialized for face processing. In this article, we present recent findings from studies on the neuronal mechanisms of face perception and recognition in the light of current theoretical models. Results from brain imaging (fMRI, PET) and electrophysiology (ERP, MEG) show that in face perception particular regions (i.e. FFA, STS, IOA, AMTG, prefrontal and orbitofrontal cortex) are involved. These results are confirmed by behavioral data and clinical observations as well as by animal studies. The developmental findings reviewed in this article lead us to suppose that the ability to analyze face-like stimuli is hard-wired and improves during development. Still, experience with faces is not sufficient for an individual to become an expert in face perception. This thesis is supported by the investigation of individuals with developmental disabilities, especially with autistic spectrum disorders (ASD).


Question: Clothing and individuals - a social signal processing viewpoint.

Integrating Matching-CNN with KNN: A Quasi-Parametric Framework for Human Parsing

Recent advancements in human parsing, the process of segmenting a human image into distinct semantic regions (e.g., hat, bag, left arm, face), have showcased the strengths of both parametric and non-parametric approaches. This paper introduces a novel quasi-parametric human parsing model that amalgamates the benefits of both methodologies, enabling the use of annotated data for supervision and the flexibility to incorporate newly annotated (potentially rare) images. We propose a method that combines the K Nearest Neighbor (KNN) non-parametric framework with the parametric Matching Convolutional Neural Network (M-CNN) to predict the confidence and displacement of the best-matching region in a test image for a specific semantic region found in one of the KNN images.

The proposed quasi-parametric model begins by identifying the KNN images for a given test image from a corpus of annotated or manually-parsed human images. Subsequently, each semantic region in the KNN images is matched with the test image using M-CNN, which calculates a confidence score for the match. The matched regions from all KNN images are then integrated, and a superpixel smoothing procedure is applied to refine the segmentation, yielding the final human parsing outcome. A key distinction of M-CNN from traditional CNNs lies in its use of cross-image matching filters, specifically designed to characterize the correspondence between the test image and the semantic regions of the KNN images. These filters are implemented across various convolutional layers, each focusing on capturing a distinct range of displacements.

The effectiveness of the quasi-parametric model is substantiated through extensive evaluations on a large dataset featuring 7,700 annotated human images, showcasing a substantial performance enhancement over existing state-of-the-art methods for human parsing tasks.

Recently, Wi-Fi has caught tremendous attention for its ubiquity. Motivated by Wi-Fi’s low cost and privacy preservation, researchers have been putting lots of investigation into its potential on action recognition and even person identification. In this paper, for bringing a new modality for multimedia community, we offer an comprehensive overview on these two topics in Wi-Fi. Also, through looking at these two topics from an unprecedented perspective, we could achieve generality instead of designing specific ad-hoc features for each scenario. Observing the great resemblance of Channel State Information (CSI, a finegrained information captured from the received Wi-Fi signal) to texture, we propose a brand-new framework based on computer vision methods. To minimize the effect of location dependency embedded in CSI, we propose a novel de-noising method based on Singular Value Decomposition (SVD) to eliminate the background energy and effectively extract the channel information of signals reflected by human bodies. From the experiments conducted, we demonstrate the feasibility and efficacy of the proposed methods. Also, we conclude the factors that would affect the performance and highlight a few potential issues that require further deliberations.

The authors tested the prediction that women prefer clothing that is more revealing and sexy when fertility is highest within the ovulatory cycle. Eighty-eight women reported to the lab twice: once on a low-fertility day of the cycle and once on a high-fertility day (confirmed using hormone tests). In each session, participants posed for full-body photographs in the clothing they wore to the lab, and they drew illustrations to indicate an outfit they would wear to a social event that evening. Although each data source supported the prediction, the authors found the most dramatic changes in clothing choice in the illustrations. Ovulatory shifts in clothing choice were moderated by sociosexuality, attractiveness, relationship status, and relationship satisfaction. Sexually unrestricted women, for example, showed greater shifts in preference for revealing clothing worn to the laboratory near ovulation. The authors suggest that clothing preference shifts could reflect an increase in female-female competition near ovulation.

We propose a novel methodology for re-identification, based on Pictorial Structures (PS). Whenever face or other biometric information is missing, humans recognize an individual by selectively focusing on the body parts, looking for part-to-part correspondences. We want to take inspiration from this strategy in a re-identification context, using PS to achieve this objective. For single image re-identification, we adopt PS to localize the parts, extract and match their descriptors. When multiple images of a single individual are available, we propose a new algorithm to customize the fit of PS on that specific person, leading to what we call a Custom Pictorial Structure (CPS). CPS learns the appearance of an individual, improving the localization of its parts, thus obtaining more reliable visual characteristics for re-identification. It is based on the statistical learning of pixel attributes collected through spatio-temporal reasoning. The use of PS and CPS leads to state-of-the-art results on all the available public benchmarks, and opens a fresh new direction for research on re-identification.

We report on the results of a study in which 19 new mobile phone users were closely tracked for the first six weeks after service acquisition. Results show that new users tend to rapidly modify their perceptions of social appropriateness around mobile phone use, that actual nature of use frequently differs from what users initially predict, and that comprehension of service-oriented technologies can be problematic. We describe instances and features of mobile telephony practice. When in use, mobile phones occupy multiple social spaces simultaneously, spaces with norms that sometimes conflict: the physical space of the mobile phone user and the virtual space of the conversation.

An overview of the key challenges facing the practice of medicine today is presented along with the need for technological solutions that can "prevent" problems. Then, the development of the Wearable Motherboard/spl trade/ (Smart Shirt) as a platform for sensors and monitoring devices that can unobtrusively monitor the health and well being of individuals (directly and/or remotely) is described. This is followed by a discussion of the applications and impact of this technology in the continuum of life-from preventing SIDS to facilitating independent living for senior citizens. Finally, the future advancements in the area of wearable, yet comfortable, systems that can continue the transformation of healthcare - all aimed at enhancing the quality of life for humans - are presented.

Research on the perception of faces has focused on the size, shape, and configuration of inherited features or the biological phenotype, and largely ignored the effects of adornment, or the extended phenotype. Research on the evolution of signaling has shown that animals frequently alter visual features, including color cues, to attract, intimidate or protect themselves from conspecifics. Humans engage in conscious manipulation of visual signals using cultural tools in real time rather than genetic changes over evolutionary time. Here, we investigate one tool, the use of color cosmetics. In two studies, we asked viewers to rate the same female faces with or without color cosmetics, and we varied the style of makeup from minimal (natural), to moderate (professional), to dramatic (glamorous). Each look provided increasing luminance contrast between the facial features and surrounding skin. Faces were shown for 250 ms or for unlimited inspection time, and subjects rated them for attractiveness, competence, likeability and trustworthiness. At 250 ms, cosmetics had significant positive effects on all outcomes. Length of inspection time did not change the effect for competence or attractiveness. However, with longer inspection time, the effect of cosmetics on likability and trust varied by specific makeup looks, indicating that cosmetics could impact automatic and deliberative judgments differently. The results suggest that cosmetics can create supernormal facial stimuli, and that one way they may do so is by exaggerating cues to sexual dimorphism. Our results provide evidence that judgments of facial trustworthiness and attractiveness are at least partially separable, that beauty has a significant positive effect on judgment of competence, a universal dimension of social cognition, but has a more nuanced effect on the other universal dimension of social warmth, and that the extended phenotype significantly influences perception of biologically important signals at first glance and at longer inspection.

This paper discusses the problem of recognizing interaction-level human activities from a first-person viewpoint. The goal is to enable an observer (e.g., a robot or a wearable camera) to understand 'what activity others are performing to it' from continuous video inputs. These include friendly interactions such as 'a person hugging the observer' as well as hostile interactions like 'punching the observer' or 'throwing objects to the observer', whose videos involve a large amount of camera ego-motion caused by physical interactions. The paper investigates multi-channel kernels to integrate global and local motion information, and presents a new activity learning/recognition methodology that explicitly considers temporal structures displayed in first-person activity videos. In our experiments, we not only show classification results with segmented videos, but also confirm that our new approach is able to detect activities from continuous videos reliably.

Interpersonal relation defines the association, e.g., warm, friendliness, and dominance, between two or more people. We investigate if such fine-grained and high-level relation traits can be characterized and quantified from face images in the wild. We address this challenging problem by first studying a deep network architecture for robust recognition of facial expressions. Unlike existing models that typically learn from facial expression labels alone, we devise an effective multitask network that is capable of learning from rich auxiliary attributes such as gender, age, and head pose, beyond just facial expression data. While conventional supervised training requires datasets with complete labels (e.g., all samples must be labeled with gender, age, and expression), we show that this requirement can be relaxed via a novel attribute propagation method. The approach further allows us to leverage the inherent correspondences between heterogeneous attribute sources despite the disparate distributions of different datasets. With the network we demonstrate state-of-the-art results on existing facial expression recognition benchmarks. To predict inter-personal relation, we use the expression recognition network as branches for a Siamese model. Extensive experiments show that our model is capable of mining mutual context of faces for accurate fine-grained interpersonal prediction.

Protecting social norms as confidentiality wanes.

We introduce a generative model for learning person and costume specific detectors from labeled examples. We demonstrate the model on the task of localizing and naming actors in long video sequences. More specifically, the actor's head and shoulders are each represented as a constellation of optional color regions. Detection can proceed despite changes in view-point and partial occlusions. We explain how to learn the models from a small number of labeled key frames or video tracks, and how to detect novel appearances of the actors in a maximum likelihood framework. We present results on a challenging movie example, with 81% recall in actor detection (coverage) and 89% precision in actor identification (naming).


Question: Music emotion recognition: examining the significance of individual differences.

This document explores a novel strategy for music emotion classification, focusing on the inherent subjectivity of human perception. Given the complexity of associating emotions with music due to individual differences in emotional responses, a deterministic classification method is insufficient. Instead, the paper proposes a probabilistic approach to assess the degree to which a music segment aligns with a specific emotion. By utilizing two fuzzy classifiers, the system quantifies the intensity of emotion associated with each segment, which not only enhances classification accuracy but also enables the tracking of emotional fluctuations within a song. The outcomes highlight the efficacy of this fuzzy classification technique in addressing the variability and complexity of music emotion recognition.

**Question**: Evaluating the impact of social media on mental health: a longitudinal study

**Documents**: The effects of social media on mental health: a cross-sectional study
The use of social media has been linked to various mental health issues, including depression, anxiety, and poor sleep quality. However, most of the studies on this topic have been cross-sectional, making it difficult to determine causality. In this study, we aimed to explore the association between social media use and mental health by conducting a cross-sectional survey among young adults. The results showed that higher levels of social media use were associated with increased symptoms of depression and anxiety, as well as poorer sleep quality. These findings suggest that social media use may have negative implications for mental health, but further longitudinal studies are needed to establish causality.

**Similar Documents**: This research delves into the relationship between social media usage and its potential impacts on mental health, a topic of growing concern in the digital age. While previous studies have suggested links between social media and mental health problems such as depression, anxiety, and disrupted sleep patterns, the cross-sectional nature of these investigations limits their ability to establish causal relationships. In our study, we surveyed young adults to examine the correlation between social media engagement and mental health outcomes. Our findings revealed that increased social media use correlates with heightened depressive and anxiety symptoms, as well as reduced sleep quality. Although these results underscore the possible detrimental effects of social media on mental health, the need for longitudinal research to confirm causality remains a critical next step in this area of inquiry.

We argue that an evaluation of system behavior at the level of the music is required to usefully address the fundamental problems of music genre recognition (MGR), and indeed other tasks of music information retrieval, such as autotagging. A recent review of works in MGR since 1995 shows that most (82 %) measure the capacity of a system to recognize genre by its classification accuracy. After reviewing evaluation in MGR, we show that neither classification accuracy, nor recall and precision, nor confusion tables, necessarily reflect the capacity of a system to recognize genre in musical signals. Hence, such figures of merit cannot be used to reliably rank, promote or discount the genre recognition performance of MGR systems if genre recognition (rather than identification by irrelevant confounding factors) is the objective. This motivates the development of a richer experimental toolbox for evaluating any system designed to intelligently extract information from music signals.

In this paper, we present our latest progress in Emotion Recognition techniques, which combines acoustic features and facial features in both non-temporal and temporal mode. This paper presents the details of our techniques used in the Audio-Video Emotion Recognition subtask in the 2018 Emotion Recognition in the Wild (EmotiW) Challenge. After the multimodal results fusion, our final accuracy in Acted Facial Expression in Wild (AFEW) test dataset achieves 61.87%, which is 1.53% higher than the best results last year. Such improvements prove the effectiveness of our methods.

Future Work Dataset and Features Music has been an integral part of our culture all throughout human history. In 2012 alone, the U.S. music industry generated $15 billion. Having a fundamental understanding of what makes a song popular has major implications to businesses that thrive on popular music, namely radio stations, record labels, and digital and physical music market places. Many private companies in these industries have solutions for this problem, but details have been kept private for competitive reasons. For our project, we will predict song popularity based on an song’s audio features and metadata. Methods & Results

Emotion and topic were manipulated in original song lyrics. Participants completed Batcho's and Holbrook's nostalgia surveys and rated 6 sets of lyrics for happiness, sadness, anger, nostalgia, meaning, liking, and relevance. Nostalgic lyrics were characterized by bittersweet reverie, loss of the past, identity, and meaning. Contrary to theories linking nostalgia to pathology, participants who scored high on Batcho's measure of personal nostalgia preferred happy lyrics, found them more meaningful, and related more closely to them. Consistent with theories relating nostalgia to social connectedness, high-nostalgia respondents preferred other-directed to solitary themes. Historical nostalgia was associated with relating more closely to sad lyrics.

We used positron emission tomography to study neural mechanisms underlying intensely pleasant emotional responses to music. Cerebral blood flow changes were measured in response to subject-selected music that elicited the highly pleasurable experience of "shivers-down-the-spine" or "chills." Subjective reports of chills were accompanied by changes in heart rate, electromyogram, and respiration. As intensity of these chills increased, cerebral blood flow increases and decreases were observed in brain regions thought to be involved in reward/motivation, emotion, and arousal, including ventral striatum, midbrain, amygdala, orbitofrontal cortex, and ventral medial prefrontal cortex. These brain structures are known to be active in response to other euphoria-inducing stimuli, such as food, sex, and drugs of abuse. This finding links music with biologically relevant, survival-related stimuli via their common recruitment of brain circuitry involved in pleasure and reward.

The ability to classify the genre of a song is an easy task for the human ear. After listening to a song for just several seconds, it is often not difficult to take note of that song’s characteristics and subsequently identify this genre. However, this task is one that computers have historically not been able to solve well, at least not until the advent of more sophisticated machine learning techniques. Our team explored several of these techniques over the course of this project and successfully built a system that achieves 61.873% accuracy in classifying music genres. This paper discusses the methods we used for exploratory data analysis, feature selection, hyperparameter optimization, and eventual implementation of several algorithms for classification.

This article faces the problem of how different audio features and segmentation methods work with different music genres. A new annotated corpus of Chinese traditional Jingju music is presented. We incorporate this dataset with two existing music datasets from the literature in an integrated retrieval system to evaluate existing features, structural hypotheses, and segmentation algorithms outside a Western bias. A harmonic-percussive source separation technique is introduced to the feature extraction process and brings significant improvement to the segmentation. Results show that different features capture the structural patterns of different music genres in different ways. Novelty- or homogeneity-based segmentation algorithms and timbre features can surpass the investigated alternatives for the structure analysis of Jingju due to their lack of harmonic repetition patterns. Findings indicate that the design of audio features and segmentation algorithms as well as the consideration of contextual information related to the music corpora should be accounted dependently in an effective segmentation system.

This paper targets the automatic detection of personality traits in a meeting environment by means of audio and visual features; information about the relational context is captured by means of acoustic features designed to that purpose. Two personality traits are considered: Extraversion (from the Big Five) and the Locus of Control. The classification task is applied to thin slices of behaviour, in the form of 1-minute sequences. SVM were used to test the performances of several training and testing instance setups, including a restricted set of audio features obtained through feature selection. The outcomes improve considerably over existing results, provide evidence about the feasibility of the multimodal analysis of personality, the role of social context, and pave the way to further studies addressing different features setups and/or targeting different personality traits.

Automatic emotion recognition is an interdisciplinary research field which deals with the algorithmic detection of human affect, e.g. anger or sadness, from a variety of sources, such as speech or facial gestures. Apart from the obvious usage for industry applications in human-robot interaction, acquiring the emotional state of a person automatically also is of great potential for the health domain, especially in psychology and psychiatry. Here, evaluation of human emotion is often done using oral feedback or questionnaires during doctor-patient sessions. However, this can be perceived as intrusive by the patient. Furthermore, the evaluation can only be done in a noncontinuous manner, e.g. once a week during therapy sessions. In contrast, using automatic emotion detection, the affect state of a person can be evaluated in a continuous non-intrusive manner, for example to detect early on-sets of depression. An additional benefit of automatic emotion recognition is the objectivity of such an approach, which is not influenced by the perception of the patient and the doctor. To reach the goal of objectivity, it is important, that the source of the emotion is not easily manipulable, e.g. as in the speech modality. To circumvent this caveat, novel approaches in emotion detection research the potential of using physiological measures, such as galvanic skin sensors or pulse meters. In this paper we outline a way of detecting emotion from brain waves, i.e., EEG data. While EEG allows for a continuous, real-time automatic emotion recognition, it furthermore has the charm of measuring the affect close to the point of emergence: the brain. Using EEG data for emotion detection is nevertheless a challenging task: Which features, EEG channel locations and frequency bands are best suited for is an issue of ongoing research. In this paper we evaluate the use of state of the art feature extraction, feature selection and classification algorithms for EEG emotion classification using data from the de facto standard dataset, DEAP. Moreover, we present results that help choose methods to enhance classification performance while simultaneously reducing computational complexity.

This work is an attempt towards robust automated analysis of Indian classical ragas through machine learning and signal processing tools and techniques. Indian classical music has a definite heirarchical structure where macro level concepts like thaats and raga are defined in terms of micro entities like swaras and shrutis. Swaras or notes in Indian music are defined only in terms of their relation to one another (akin to the movable do-re-mi-fa system), and an inference must be made from patterns of sounds, rather than their absolute frequency structure. We have developed methods to perform scale-independent raga identification using a random forest classifier on swara histograms and achieved state-of-the-art results for the same. The approach is robust as it directly works on partly noisy raga recordings from Youtube videos without knowledge of the scale used, whereas previous work in this direction often use audios generated in a controlled environment with the desired scale. The current work demonstrates the approach for 8 ragas namely Darbari, Khamaj, Malhar, Sohini, Bahar, Basant, Bhairavi and Yaman and we have achieved an average identification accuracy of 94.28% through the framework.


Question: Music emotion recognition: investigating the significance of individual differences.

This paper introduces an automated system for detecting and tracking the mood of music audio signals, a critical aspect in music psychology, understanding, and retrieval. Drawing from Western music psychological theories, the proposed hierarchical framework is designed to identify the emotional expression inherent in music clips. The framework's unique strength lies in its ability to prioritize the most relevant features for different detection tasks. To capture the essence of a music clip's emotional profile, three feature sets are extracted: intensity, timbre, and rhythm. Intensity is characterized by energy levels across various subbands, timbre by spectral shape and contrast features, and rhythm by three components that resonate with emotional responses—strength, regularity, and tempo. Given the dynamic nature of mood in classical music, the framework is further enhanced to track mood changes within a piece by segmenting it into sections with consistent emotional expression. Early evaluations demonstrate the system's effectiveness, achieving an average accuracy of 86.3% in mood detection across 800 diverse music clips and an average recall rate of 84.1% for mood boundaries in nine test pieces. These results underscore the framework's potential for enhancing music-related applications through accurate emotional analysis.

In this work, we explore the degree to which personality information can be used to model newcomer retention, investment, intensity of engagement, and distribution of activity in a recommender community. Prior work shows that Big-Five Personality traits can explain variation in user behavior in other contexts. Building on this, we carry out and report on an analysis of 1008 MovieLens users with identified personality profiles. We find that Introverts and low Agreeableness users are more likely to survive into the second and subsequent sessions compared to their respective counterparts; Introverts and low Conscientiousness users are a significantly more active population compared to their respective counterparts; High Openness and High Neuroticism users contribute (tag) significantly more compared to their counterparts, but their counterparts consume (browse and bookmark) more; and low Agreeableness users are more likely to rate whereas high Agreeableness users are more likely to tag. These results show how modeling newcomer behavior from user personality can be useful for recommender systems designers as they customize the system to guide people towards tasks that need to be done or tasks the users will find rewarding and also decide which users to invest retention efforts in.

We argue that an evaluation of system behavior at the level of the music is required to usefully address the fundamental problems of music genre recognition (MGR), and indeed other tasks of music information retrieval, such as autotagging. A recent review of works in MGR since 1995 shows that most (82 %) measure the capacity of a system to recognize genre by its classification accuracy. After reviewing evaluation in MGR, we show that neither classification accuracy, nor recall and precision, nor confusion tables, necessarily reflect the capacity of a system to recognize genre in musical signals. Hence, such figures of merit cannot be used to reliably rank, promote or discount the genre recognition performance of MGR systems if genre recognition (rather than identification by irrelevant confounding factors) is the objective. This motivates the development of a richer experimental toolbox for evaluating any system designed to intelligently extract information from music signals.

We used a new theory of the biological basis of the Big Five personality traits to generate hypotheses about the association of each trait with the volume of different brain regions. Controlling for age, sex, and whole-brain volume, results from structural magnetic resonance imaging of 116 healthy adults supported our hypotheses for four of the five traits: Extraversion, Neuroticism, Agreeableness, and Conscientiousness. Extraversion covaried with volume of medial orbitofrontal cortex, a brain region involved in processing reward information. Neuroticism covaried with volume of brain regions associated with threat, punishment, and negative affect. Agreeableness covaried with volume in regions that process information about the intentions and mental states of other individuals. Conscientiousness covaried with volume in lateral prefrontal cortex, a region involved in planning and the voluntary control of behavior. These findings support our biologically based, explanatory model of the Big Five and demonstrate the potential of personality neuroscience (i.e., the systematic study of individual differences in personality using neuroscience methods) as a discipline.

This study investigated the role of ability and trait emotional intelligence (EI), fluid intelligence, and personality traits in career decision-making self-efficacy, career indecision and indecisiveness. The Advanced Progressive Matrices, Big Five Questionnaire, Mayer-Salovey-Caruso Emotional Intelligence Test, Bar-On Emotional Intelligence Inventory, Trait Emotional Intelligence Questionnaire, Career Decision SelfEfficacy Scale: Short Form, Career Decision-Making Difficulties Questionnaire, and Indecisiveness Scale were administered to 194 Italian high school students. These results highlighted the potentially important role of self-reported EI in career decisions. 2014 Elsevier Ltd. All rights reserved.

Despite the enormous interest in emotion classification from speech, the impact of noise on emotion classification is not well understood. This is important because, due to the tremendous advancement of the smartphone technology, it can be a powerful medium for speech emotion recognition in the outside laboratory natural environment, which is likely to incorporate background noise in the speech. We capitalize on the current breakthrough of Recurrent Neural Network (RNN) and seek to investigate its performance for emotion classification from noisy speech. We particularly focus on the recently proposed Gated Recurrent Unit (GRU), which is yet to be explored for emotion recognition from speech. Experiments conducted with speech compounded with eight different types of noises reveal that GRU incurs an 18.16% smaller run-time while performing quite comparably to the Long Short-Term Memory (LSTM), which is the most popular Recurrent Neural Network proposed to date. This result is promising for any embedded platform in general and will initiate further studies to utilize GRU to its full potential for emotion recognition on smartphones.

system is the change in electrical properties of skin due to variation in physiological and psychological conditions. The change is caused by the degree to which a person's sweat glands are active. Psychological status of a person tends to make the glands active and this change the skin resistance. Drier the skin is, higher will be the skin resistance. This variation in skin resistance ranges from 5kΩ to 25kΩ. In the current work, a subject whose Galvanic skin response (GSR) is to be measured, has been shown/played a movie clipping, images or recorded audio signals. Depending upon the theme, emotion will be evoked in the subject. Due to the change in emotion, GSR varies. This variation in GSR is recorded for a fixed time interval. In the present work, a total of 75 subjects are selected. People from different age groups and social background, both male and female have been carefully considered. Data acquisition and analysis is carried out by using LabVIEW. The results obtained are convincing and follows the ground reality with a high level of accuracy. As the subject also gives the feedback about the emotions he/she experienced, the results obtained are validated.

Previous studies have demonstrated that synchronising movements with other people can influence affiliative behaviour towards them. While research has focused on synchronisation with visually observed movement, synchronisation with a partner who is heard may have similar effects. We replicate findings showing that synchronisation can influence ratings of likeability of a partner, but demonstrate that this is possible with virtual interaction, involving a video of a partner. Participants performed instructed synchrony in time to sounds instead of the observable actions of another person. Results show significantly higher ratings of likeability of a partner after moving at the same time as sounds attributed to that partner, compared with moving in between sounds. Objectively quantified synchrony also correlated with ratings of likeability. Belief that sounds were made by another person was manipulated in Experiment 2, and results demonstrate that when sounds are attributed to a computer, ratings of likeability are not affected by moving in or out of time. These findings demonstrate that interaction with sound can be experienced as social interaction in the absence of genuine interpersonal contact, which may help explain why people enjoy engaging with recorded music.

Robots have the potential to facilitate future therapies for children on the autism spectrum. However, existing robots are limited in their ability to automatically perceive and respond to human affect, which is necessary for establishing and maintaining engaging interactions. Their inference challenge is made even harder by the fact that many individuals with autism have atypical and unusually diverse styles of expressing their affective-cognitive states. To tackle the heterogeneity in children with autism, we used the latest advances in deep learning to formulate a personalized machine learning (ML) framework for automatic perception of the children’s affective states and engagement during robot-assisted autism therapy. Instead of using the traditional one-size-fits-all ML approach, we personalized our framework to each child using their contextual information (demographics and behavioral assessment scores) and individual characteristics. We evaluated this framework on a multimodal (audio, video, and autonomic physiology) data set of 35 children (ages 3 to 13) with autism, from two cultures (Asia and Europe), and achieved an average agreement (intraclass correlation) of ~60% with human experts in the estimation of affect and engagement, also outperforming nonpersonalized ML solutions. These results demonstrate the feasibility of robot perception of affect and engagement in children with autism and have implications for the design of future autism therapies.

In 1993, it was found that Mozart’s music temporarily enhanced performance on spatialtemporal reasoning tasks. This effect later became termed the Mozart Effect. Since then, the mechanisms of the Mozart Effect have been hotly debated among the scientific community. Recent research has called for studying the Mozart Effect by analyzing music as a series of components. The present study took the components of music into account by testing if predictable music could enhance spatial-temporal reasoning compared to non-predictable music. Participants were administered a task designed to test spatial-temporal reasoning after listening to either predictable or non-predictable music. Additionally, as a control condition, they performed the same task after listening to a short story. Listening to music did not affect reasoning performance, regardless of the predictability of the music. The results indicate that predictability in music alone may not be a major element of the Mozart Effect.

Neuroimagingstudies with positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) have begun to describe the functional neuroanatomy of emotion. Taken separately, specific studies vary in task dimensions and in type(s) of emotion studied and are limited by statistical power and sensitivity. By examining findings across studies, we sought to determine if common or segregated patterns of activations exist across various emotional tasks. We reviewed 55 PET and fMRI activation studies (yielding 761 individual peaks) which investigated emotion in healthy subjects. Peak activation coordinates were transformed into a standard space and plotted onto canonical 3-D brain renderings. We divided the brain into 20 nonoverlapping regions, and characterized each region by its responsiveness across individual emotions (positive, negative, happiness, fear, anger, sadness, disgust), to different induction methods (visual, auditory, recall/imagery), and in emotional tasks with and without cognitive demand. Our review yielded the following summary observations: (1) The medial prefrontal cortex had a general role in emotional processing; (2) fear specifically engaged the amygdala; (3) sadness was associated with activity in the subcallosal cingulate; (4) emotional induction by visual stimuli activated the occipital cortex and the amygdala; (5) induction by emotional recall/imagery recruited the anterior cingulate and insula; (6) emotional tasks with cognitive demand also involved the anterior cingulate and insula. This review provides a critical comparison of findings across individual studies and suggests that separate brain regions are involved in different aspects of emotion.