Question: Fingerprint authentication through the integration of optical and capacitive sensor technologies.

In the face of escalating identity fraud and the growing importance of automated personal identification systems, biometrics, particularly fingerprint-based identification, has garnered significant interest. Traditional fingerprint representation methods suffer from two critical flaws. For a substantial portion of the population, extracting representations based on the explicit detection of complete ridge structures in fingerprints poses a challenge for automated systems. Additionally, the widely adopted minutiae-based representation fails to harness a considerable portion of the rich, discriminatory information embedded in fingerprints. Local ridge structures are more complex than what minutiae can fully capture. Moreover, minutiae-based matching struggles with the rapid comparison of two fingerprint images that contain a varying number of unregistered minutiae points. To address these issues, a novel filter-based algorithm has been developed, employing a Gabor filterbank to effectively capture both local and global details in a fingerprint, condensed into a compact, fixed-length FingerCode. The matching process, based on the Euclidean distance between corresponding FingerCodes, is notably swift. Our algorithm achieves a verification accuracy that is nearly on par with the top-performing minutiae-based algorithms reported in the literature. Notably, our system outperforms a leading minutiae-based system when the application's performance requirements do not necessitate an exceedingly low false acceptance rate. Furthermore, we demonstrate that combining the decisions of matchers based on complementary fingerprint information—minutiae-based and filter-based—can enhance matching performance.

Current face biometric systems are vulnerable to spoofing attacks. A spoofing attack occurs when a person tries to masquerade as someone else by falsifying data and thereby gaining illegitimate access. Inspired by image quality assessment, characterisation of printing artefacts and differences in light reflection, the authors propose to approach the problem of spoofing detection from texture analysis point of view. Indeed, face prints usually contain printing quality defects that can be well detected using texture and local shape features. Hence, the authors present a novel approach based on analysing facial image for detecting whether there is a live person in front of the camera or a face print. The proposed approach analyses the texture and gradient structures of the facial images using a set of low-level feature descriptors, fast linear classification scheme and score level fusion. Compared to many previous works, the authors proposed approach is robust and does not require user-cooperation. In addition, the texture features that are used for spoofing detection can also be used for face recognition. This provides a unique feature space for coupling spoofing detection and face recognition. Extensive experimental analysis on three publicly available databases showed excellent results compared to existing works.

Recently MEMS Capacitive Pressure Sensor gains more advantage over micromachined piezoresistive pressure sensor due to high sensitivity, low power consumption, free from temperature effects, IC compatibility, etc,. The spectrum of capacitive pressure sensor application is increasing, hence it is essential to review the path of technological development and further prospective of micromachined capacitive pressure sensor. This paper focuses on the review of various types of capacitive pressure sensor principle, MEMS materials used in fabrication, procedures adopted in microfabrication for silicon and polymer material diaphragm, bonding and packaging techniques used. Selected result on capacitive sensitivity, effect of temperature on capacitive sensitivity was also presented. Finally, the development of smart sensor was discussed. MEMS Capacitive pressure sensor, Review on pressure sensor, CDPS, MEMS Fabrication, MEMS Material, pressure sensor, MEMS (Micro Electro Mechanical System)

The recent introduction of depth cameras like Leap Motion Controller allows researchers to exploit the depth information to recognize hand gesture more robustly. This paper proposes a novel hand gesture recognition system with Leap Motion Controller. A series of features are extracted from Leap Motion tracking data, we feed these features along with HOG feature extracted from sensor images into a multiclass SVM classifier to recognize performed gesture, dimension reduction and feature weighted fusion are also discussed. Our results show that our model is much more accurate than previous work.

Recent advances in sensing technology have enabled a new generation of tabletop displays that can sense multiple points of input from several users simultaneously. However, apart from a few demonstration techniques [17], current user interfaces do not take advantage of this increased input bandwidth. We present a variety of multifinger and whole hand gestural interaction techniques for these displays that leverage and extend the types of actions that people perform when interacting on real physical tabletops. Apart from gestural input techniques, we also explore interaction and visualization techniques for supporting shared spaces, awareness, and privacy. These techniques are demonstrated within a prototype room furniture layout application, called RoomPlanner.

This paper presents the results of a 36 participant empirical comparison of touch mode-switching. Six techniques are evaluated, spanning current and future techniques: long press, non-dominant hand, two-fingers, hard press, knuckle, and thumb-on-finger. Two poses are controlled for, seated with the tablet on a desk and standing with the tablet held on the forearm. Findings indicate pose has no effect on mode switching time and little effect on error rate; using two-fingers is fastest while long press is much slower; non-preferred hand and thumb-on-finger also rate highly in subjective scores. The experiment protocol is based on Li et al.'s pen mode-switching study, enabling a comparison of touch and pen mode switching. Among the common techniques, the non-dominant hand is faster than pressure with touch, whereas no significant difference had been found for pen. Our work addresses the lack of empirical evidence comparing touch mode-switching techniques and provides guidance to practitioners when choosing techniques and to researchers when designing new mode-switching methods.

In this paper we examine the state of the art in tactile sensing for mechatronics[ We de_ne a tactile sensor as a device or system that can measure a given property of an object or contact event through physical contact between the sensor and the object[ We consider any property that can be measured through contact\ including the shape of an object\ texture\ temperature\ hardness\ moisture content\ etc[ A comprehensive search of the literature revealed that there was a signi_cant increase in publications on tactile sensing from 0880 onwards[ Considerable e}ort in the 0879s was spent investigating transduction techniques and developing new sensors\ whilst emphasis in more recent research has focused on experiments using tactile sensors to perform a variety of tasks[ This paper reports on progress in tactile sensing in the following areas] cutaneous sensors\ sensing _ngers\ soft materials\ industrial robot grippers\ multi_ngered hands\ probes and whiskers\ analysis of sensing devices\ haptic perception\ processing sensory data and new application areas[ We conclude that the predominant choice of transduction method is piezoelectric\ with arrays using resistive or capacitive sensing[ We found that increased emphasis on understanding tactile sensing and perception issues has opened up potential for new application areas[ The predicted growth in applications in industrial automation has not eventuated[ New applications for tactile sensing including surgery\ rehabilitation and service robotics\ and food processing automation show considerable potential and are now receiving signi_cant levels of research attention[ Þ 0887 Elsevier Science Ltd[ All rights reserved[

ARTICLE INFO In this system an electro-mechanical robot is designed and controlled using hand gesture in real time. The system is designed on microcontroller platform using Keil and MPLAB tools. Hand gesture recognition is done on the principle of resistance change sensed through flex sensor. These sensors are integrated in a hand gloves from which input to the system is given. The designed system is divided into two sections as transmitter and receiver. The transmitter section will be in hand gloves from which the data is sensed and processed through PIC16F7487 and send serially to the receiver section. RF technology is used to transmit the data at the receiver section at the frequency of 2.4 GHz. ARM 7 (LPC2148) processor is used to receive the data. Here from the received data, the character is predicted and matched with the closet character from which the character is identified and displayed on LCD. The various case studies is prepared for the designed system and tested in real time. The proposed system can be used for the various applications such as in unmanned machines, industries, handicapped personnel etc. Keywords— a Sensor gloves, flex sensors, PIC controller, ARM processor etc Article History Received:20 th September 2015 Received in revised form :

In this paper we present a secure and unclonable embedded system design that can target either an FPGA or an ASIC technology. The premise of the security is that the executed machine code and the executing environment (the embedded processor) will authenticate each other at a per-instruction basis using Physical Unclonable Functions (PUFs) that are built into the processor. The PUFs ensure that the execution of the binary code may only proceed if the binary is compiled with the correct intrinsic knowledge of the PUFs, and that such intrinsic knowledge is virtually unique to each processor and therefore unclonable. We will explain how to implement and integrate the PUFs into the processor's execution environment such that each instruction is authenticated and de-obfuscated on-demand and how to transform an ordinary binary executable into PUF-aware, obfuscated binaries. We will also present a prototype system on a Xilinx Spartan6-based FPGA board.

In this study we show that by the current state-of-the-art synthetically generated fingerprints can easily be discriminated from real fingerprints. We propose a method based on second order extended minutiae histograms (MHs) which can distinguish between real and synthetic prints with very high accuracy. MHs provide a fixed-length feature vector for a fingerprint which are invariant under rotation and translation. This ’test of realness’ can be applied to synthetic fingerprints produced by any method. In this work, tests are conducted on the 12 publicly available databases of FVC2000, FVC2002 and FVC2004 which are well established benchmarks for evaluating the performance of fingerprint recognition algorithms; 3 of these 12 databases consist of artificial fingerprints generated by the SFinGe software. Additionally, we evaluate the discriminative performance on a database of synthetic fingerprints generated by the software of Bicz versus real fingerprint images. We conclude with suggestions for the improvement of synthetic fingerprint generation.

Fingerprint is one of the well accepted biometric traits for building automatic human recognition system. These systems mainly involves matching of two fingerprints based on their minutiae points. Therefore, extraction of reliable or true minutiae from the fingerprint image is critically important. This paper proposed an novel algorithm for automatic extraction of highly reliable minutiae points from a fingerprint image. It utilizes frequency domain image enhancement and heuristics. Experiments have been conducted on two databases. FVC2002 which is a publicly available fingerprint database containing 800 fingerprint images of 100 persons and IITK-Sel500FP which is an in-house database containing 1000 fingerprint images of 500 persons. Minutiae points extracted by the proposed algorithm are benchmarked against the manually marked ones and the comparison is done with an open source minutiae extractor mindtct of NIST. Experimental results have shown that the proposed algorithm has significantly higher confidence of extracted minutiae points being true.


Question: Fingerprint authentication through integration of optical and capacitive sensor data.

This paper presents the development of an efficient on-line fingerprint verification system, designed to address the limitations of manual fingerprint verification, which is often laborious, time-intensive, and costly, failing to meet current performance standards. An Automated Fingerprint Identification System (AFIS) is crucial for both forensic and civilian purposes, such as criminal identification, access control, and ATM card authentication. The proposed system comprises two primary stages: minutia extraction and minutia matching. An enhanced minutia extraction algorithm, based on the method by Ratha et al., is implemented for the rapid and accurate extraction of features from fingerprint images acquired through an on-line inkless scanner. For the minutia matching phase, an innovative alignment-based elastic matching algorithm is developed. This algorithm efficiently identifies minutiae correspondences between the input image and stored templates, without the need for exhaustive searches. It also possesses the capability to adaptively compensate for nonlinear deformations and imprecise pose transformations between fingerprints. The system's performance was evaluated using fingerprint images obtained from two distinct inkless scanners. Results indicated acceptable verification accuracy, with an average complete fingerprint verification process taking approximately eight seconds on a SPARC 20 workstation. These findings demonstrate the system's capability to achieve high accuracy in real-time on-line verification. Keywords: Biometrics, fingerprint recognition, minutia extraction, orientation field, ridge detection, matching algorithms.

A successful grasp requires careful balancing of the contact forces. Deducing whether a particular grasp will be successful from indirect measurements, such as vision, is therefore quite challenging, and direct sensing of contacts through touch sensing provides an appealing avenue toward more successful and consistent robotic grasping. However, in order to fully evaluate the value of touch sensing for grasp outcome prediction, we must understand how touch sensing can influence outcome prediction accuracy when combined with other modalities. Doing so using conventional model-based techniques is exceptionally difficult. In this work, we investigate the question of whether touch sensing aids in predicting grasp outcomes within a multimodal sensing framework that combines vision and touch. To that end, we collected more than 9,000 grasping trials using a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger, and evaluated visuo-tactile deep neural network models to directly predict grasp outcomes from either modality individually, and from both modalities together. Our experimental results indicate that incorporating tactile readings substantially improve grasping performance.

The human fingertip is extremely sensitive to lateral (shear) forces that arise in surface exploration. We and others have developed haptic displays that work by modulating surface friction via electrostatic attraction. Despite the demonstrated ability of these displays to render haptic environments, an understanding of the fingertip-surface interface is lacking. We have developed a tribometer for measuring the lateral frictional forces on a fingertip under well-controlled conditions. We show an expected square law dependence of frictional force (and inferred electrostatic normal force) on actuation voltage, although we observe a large person to person variability. We model an expected dependence of the frictional force on the frequency of the actuation voltage, predicting a first order cut off below about 500Hz. However, our measurements are unambiguously at odds with the model's predictions.

In this paper, we propose the Perceptual Shape Decomposition (PSD) to detect fingers for a Kinect-based hand gesture recognition system. The PSD is formulated as a discrete optimization problem by removing all negative minima with minimum cost. Experiments show that our PSD is perceptually relevant and robust against distortion and hand variations, and thus improves the recognition system performance. key words: Kinect camerta, hand gesture recognition, perceptual decomposition, finger detection

A feature extraction method using the chaincode representation of fingerprint ridge contours is presented for use by Automatic Fingerprint Identification Systems. The representation allows efficient image quality enhancement and detection of fine minutiae feature points. For image enhancement a given fingerprint image is first binarized after a quick averaging to generate its chaincode representation. The direction field is estimated from a set of selected chaincodes. The original gray scale image is then enhanced using connected component analysis and a dynamic filtering scheme that takes advantage of the knowledge gained from the estimated direction flow of the contours. For feature extraction, the enhanced fingerprint image is carefully binarized using a local-global binarization algorithm for generating the chaincode representation. The minutiae are generated using a sophisticated ridge contour following procedure. Visual inspection of the experiment images shows that the method is very effective.

We present SnapApp, a novel unlock concept for mobile devices that reduces authentication overhead with a time-constrained quick-access option. SnapApp provides two unlock methods at once: While PIN entry enables full access to the device, users can also bypass authentication with a short sliding gesture ("Snap"). This grants access for a limited amount of time (e.g. 30 seconds). The device then automatically locks itself upon expiration. Our concept further explores limiting the possible number of Snaps in a row, and configuring blacklists for app use during short access (e.g. to exclude banking apps). We discuss opportunities and challenges of this concept based on a 30-day field study with 18 participants, including data logging and experience sampling methods. Snaps significantly reduced unlock times, and our app was perceived to offer a good tradeoff. Conceptual challenges include, for example, supporting users in configuring their blacklists.

^ IEEE ConsumEr ElECtronICs magazInE 19 By early 2016, Pabellon achieved another major milestone: a first touchfree control panel using solely the body’s electrolytes to control a matrix of magnetic pixels. Unlike conventional passive near-field communication technology that relies on receiver loop antennas for detection and identification, the magnetic pixel needs only our body salts to work. Each pixel represents a discrete sensing element. When arrayed or matrixed, the collection of pixels can be used to position index fingers approaching but not touching the surface. Pabellon’s powertransfer emitter turns into a touch-free pad. Unlike motion and other opticalsensing methods, the position of objects like fingers can be resolved by small changes in the magnetic near field. The ability to finely control the vast amount of things we contact on a daily basis may not require contact alone. The magnetic pixel has a very exciting future in the look-but-don’t-touch world that Pabellon envisions.

J. M. Valeton I nstitute for Perception TNO Kampweg 5 3769 DE Soesterberg, The Netherlands Abstract. Integration of images from different sensing modalities can produce information that cannot be obtained by viewing the sensor outputs separately and consecutively. This paper introduces a hierarchical i mage merging scheme based on a multiresolution contrast decomposition (the ratio of a low-pass pyramid). The composite images produced by this scheme preserve those details from the input images that are most relevant to visual perception. The method is tested by merging parallel registered thermal and visual images. The results show that the fused images present a more detailed representation of the depicted scene. Detection, recognition, and search tasks may therefore benefit from this new image representation.

This paper presents a novel minutiae matching method that describes elastic distortions in 4ngerprints by means of a thin-plate spline model, which is estimated using a local and a global matching stage. After registration of the 4ngerprints according to the estimated model, the number of matching minutiae can be counted using very tight matching thresholds. For deformed 4ngerprints, the algorithm gives considerably higher matching scores compared to rigid matching algorithms, while only taking 100 ms on a 1 GHz P-III machine. Furthermore, it is shown that the observed deformations are di6erent from those described by theoretical models proposed in the literature. ? 2003 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.

A facial recognition system is a computer application for automatically identifying or verifying a person from a digital image or a video frame from a video source. One of the way is to do this is by comparing selected facial features from the image and a facial database. It is typically used in security systems and can be compared to other biometrics such as fingerprint or eye iris recognition systems. In this paper we focus on 3-D facial recognition system and biometric facial recognition system. We do critics on facial recognition system giving effectiveness and weaknesses. This paper also introduces scope of recognition system in India. Face recognition has received substantial attention from researchers in biometrics, pattern recognition field and computer vision communities. Face recognition can be applied in Security measure at Air ports, Passport verification, Criminals list verification in police department, Visa processing, Verification of Electoral identification and Card Security measure at ATM’s. Principal Component Analysis (PCA) is a technique among the most common feature extraction techniques used in Face Recognition. In this paper, a face recognition system for personal identification and verification using Principal Component Analysis with different distance classifiers is proposed. The test results in the ORL face database produces interesting results from the point of view of recognition success, rate, and robustness of the face recognition algorithm. Different classifiers were used to match the image of a person to a class (a subject) obtained from the training data. These classifiers are: the City-Block Distance Classifier, the Euclidian distance classifier, the Squared Euclidian Distance Classifier, and the Squared Chebyshev distance Classifier.

Smoothing fingerprint ridge orientation involves a principal discrepancy. Too little smoothing can result in noisy orientation fields (OF), too much smoothing will harm high curvature areas, especially singular points (SP). In this paper we present a fingerprint ridge orientation model based on Legendre polynomials. The motivation for the proposed method can be found by analysing the almost exclusively used method in literature for orientation smoothing, proposed by Witkin and Kass (1987) more than two decades ago. The main contribution of this paper argues that the vectorial data (sine and cosine data) should be smoothed in a coupled way and the approximation error should not be evaluated employing vectorial data. For evaluating the proposed method we use a Poincare-Index based SP detection algorithm. The experiments show, that in comparison to competing methods the proposed method has improved orientation smoothing capabilities, especially in high curvature areas.


Question: Fingerprint authentication through the integration of optical and capacitive sensor technologies.

In the realm of biometric recognition, the Handbook of Fingerprint Recognition offers a comprehensive guide to integration strategies for combining multiple modalities, such as fingerprint, face, and hand geometry data. Among the popular approaches, combination strategies, also known as combination schemes, stand out for their ability to merge outputs from individual classifiers. These strategies often employ majority vote rules, where the input pattern is assigned to the class with the most votes. For two classifiers, a logical AND or OR operator is typically used, and for more than two, AND/OR rules can be combined. For instance, a biometric system might work on "fingerprint OR (face AND hand geometry)," requiring a user to present either a fingerprint or both face and hand geometry for recognition.

Class set reduction, logistic regression, and Borda counts are widely used for integrating classifiers based on rank labels. Class set reduction selects a subset of classes to be as small as possible while still containing the true class, and these subsets from multiple modalities are combined using union or intersection. Logistic regression and Borda counts are part of the class set reordering methods, aiming to derive a consensus ranking that places the true class at the top. This is particularly useful in indexing/retrieval systems, where the top element of the ordered candidate list is the most likely match.

Combining confidence values from various modalities is often achieved through sum, mean, median, product, minimum, and maximum rules. Kittler et al. have developed a theoretical framework to understand the underlying mathematics of these schemes, showing that sum or mean schemes often perform well in practice. However, using the sum rule requires normalizing confidences from different modalities into a common domain. This might involve simple operations like inverting the sign of distance scores to align with similarity scores, or more complex non-linear normalization based on training data, where distributions of confidence values are estimated and then translated, scaled, and remapped to a fixed interval.

Weighted sum rules are also common, where different scaling factors for each modality are calculated from training data to maximize the combined classifier's accuracy. This is particularly useful when component classifiers have different strengths. Theoretical analyses by Daugman and Hong, Jain, and Pankanti have shown that combining a strong and weak biometric using AND/OR rules can degrade performance, but confidence level fusion can significantly improve overall performance, even in such cases.

Kittler et al. introduced a sensitivity analysis, demonstrating that the sum rule is less sensitive to individual classifier error rates in estimating posterior probabilities than other rules

A fully printable, passive and planar chipless RFID tag on thin flexible substrate is presented. The chipless tag encodes data into the spectral signature using a multiresonating circuit. Both amplitude and phase components of the spectral signature are used for data encoding. The tag comprises of a microstrip multiresonating circuit with 6 multi-spiral resonators and two cross-polarized UWB disc loaded monopole transmitting and receiving antennas. The chipless tag is designed to be printed on the Australian banknote and provide an extra layer of security against counterfeiting.

In this letter, using the active dual-mode twisted string actuation (TSA) mechanism and tiny tension sensors on the tendon strings, an anthropomorphic robot hand is newly designed in a compact manner. Thanks to the active dual-mode TSA mechanism, which is a miniaturized transmission, the proposed robot hand dose has a wide range of operation in terms of the grasping force and speed. It experimentally produces maximally the fingertip force 31.3 N and minimally the closing time of 0.5 s in average. Also, tiny tension sensor with the dimension of 4.7 (width) × 4.0 (height) × 10.75 (length) mm is newly presented and embedded at the fingertips in order to measure the tension on the tendon strings, which would allow the grasping force control. The kinetic and kinematic analyses are performed and the performance is verified by experiments.

Today security concerns are on the rise in all areas industries such as banks, governmental applications, healthcare industry, militaryorganization, educational institutions etc, with one common weak link being “passwords”. Several proper strategies for using passwords have been proposed. Some of which are very difficult to use and others might not meet the company’s security concerns. The usage of passwords for authentication is no longer sufficient and stronger authentication schemes are necessary. Two factor authentication uses elements or devices such as tokens and ATM cards. We have proposed to resolve the password problem, which could be inconvenient for the user and costly for the service providers to combat otherwise. To avoid the usage of additional device, the mobile phone is adopted as security token. In this paper several different authentication solutions using the mobile as authentication token are discussed, where these solutions vary in complexity, strength, security and user friendliness. One of the authentication schemes (OTP solution) is implemented to verify their usability. Hence, a classification and evaluation of the different solutions is provided according to defined criteria. Keywords‐ Two-factor authentication, Security token, UPIF, IMSI, IMEI, one time password (OTP).

Security concerns increase as the technology for falsification advances. There are strong evidences that a difficult to falsify biometric trait, the human heartbeat, can be used for identity recognition. Existing solutions for biometric recognition from electrocardiogram (ECG) signals are based on temporal and amplitude distances between detected fiducial points. Such methods rely heavily on the accuracy of fiducial detection, which is still an open problem due to the difficulty in exact localization of wave boundaries. This paper presents a systematic analysis for human identification from ECG data. A fiducial-detection-based framework that incorporates analytic and appearance attributes is first introduced. The appearance-based approach needs detection of one fiducial point only. Further, to completely relax the detection of fiducial points, a new approach based on autocorrelation (AC) in conjunction with discrete cosine transform (DCT) is proposed. Experimentation demonstrates that the AC/DCT method produces comparable recognition accuracy with the fiducial-detection-based approach.

Among various skin detection methods, Skin Probability Map (SPM) method is an effective one. Though SPM method possesses high true acceptance rate (TAR), its false acceptance rate (FAR) is unacceptable in some cases. The reason is that SPM method only use pixel-level color information. This paper proposes an improved skin detection method that integrates color, texture and space information. After color filter, a texture filter is constructed based on texture features extracted form Gabor wavelet transform. Texture filter will further filter non-skin pixels, meanwhile, it may also filter some skin pixel. To compensate the loss, after texture filter, a marker driven watershed transform is then used to grow already obtained skin regions. Experimental results show that the proposed method can achieve better performance than that of SPM.

In this paper, analyses on sensitivity and link budget have been presented to guide the design of high-sensitivity noncontact vital sign detector. Important design issues such as flicker noise, baseband bandwidth, and gain budget have been discussed with practical considerations of analog-to-digital interface and signal processing methods in noncontact vital sign detection. Based on the analyses, a direct-conversion 5.8-GHz radar sensor chip with 1-GHz bandwidth was designed and fabricated. This radar sensor chip is software configurable to set the operation point and detection range for optimal performance. It integrates all the analog functions on-chip so that the output can be directly sampled for digital signal processing. Measurement results show that the fabricated chip has a sensitivity of better than -101 dBm for ideal detection in the absence of random body movement. Experiments have been performed successfully in laboratory environment to detect the vital signs of human subjects.

A fingerprint classification algorithm is presented in this paper. Fingerprints are classified into five categories: arch, tented arch, left loop, right loop and whorl. The algorithm extracts singular points (cores and deltas) in a fingerprint image and performs classification based on the number and locations of the detected singular points. The classifier is invariant to rotation, translation and small amounts of scale changes. The classifier is rule-based, where the rules are generated independent of a given data set. The classifier was tested on 4000 images in the NIST-4 database and on 5400 images in the NIST-9 database. For he NIST-4 database, classification accuracies of 85.4% for the five-class problem and 91.1% for the four-class problem (with arch and tented arch placed in the same category) were achieved. Using a reject option, the four-class classification error can be reduced to less than 6% with 10% fingerprint images rejected. Similar classification performance was obtained on the NIST-9 database.

In the today's mobile communications scenario, smartphones offer new capabilities to develop sophisticated applications that seem to make daily life easier and more convenient for users. Such applications, which may involve mobile ticketing, identification, access control operations, etc., are often accessible through social network aggregators, that assume a fundamental role in the federated identity management space. While this makes modern smartphones very powerful devices, it also makes them very attractive targets for spyware injection. This kind of malware is able to bypass classic authentication measures and steal user credentials even when a secure element is used, and can, therefore, perform unauthorized mobile access to social network services without the user's consent. Such an event allows stealing sensitive information or even a full identity theft. In this work, we address this issue by introducing BrightPass, a novel authentication mechanism based on screen brightness. BrightPass allows users to authenticate safely with a PIN-based confirmation in the presence of specific operations on sensitive data. We compare BrightPass with existing schemes, in order to show its usability and security within the social network arena. Furthermore, we empirically assess the security of BrightPass through experimentation. Our tests indicate that BrightPass protects the PIN code against automatic submissions carried out by malware while granting fast authentication phases and reduced error rates.

The recent introduction of depth cameras like Leap Motion Controller allows researchers to exploit the depth information to recognize hand gesture more robustly. This paper proposes a novel hand gesture recognition system with Leap Motion Controller. A series of features are extracted from Leap Motion tracking data, we feed these features along with HOG feature extracted from sensor images into a multiclass SVM classifier to recognize performed gesture, dimension reduction and feature weighted fusion are also discussed. Our results show that our model is much more accurate than previous work.

This paper presents a biometric user authentication based on a person’s gait. Unlike most previous gait recognition approaches, which are based on machine vision techniques, in our approach gait patterns are extracted from a physical device attached to the lower leg. From the output of the device accelerations in three directions: vertical, forward-backward, and sideways motion of the lower leg are obtained. A combination of these accelerations is used for authentication. Applying two different methods, histogram similarity and cycle length, equal error rates (EER) of 5% and 9% were achieved, respectively.


Question: Fingerprint authentication through integration of optical and capacitive sensor data.

This document details the FVC2002, the second iteration of the Fingerprint Verification Competition, which was meticulously planned by the organizers to assess the latest advancements in the field of fingerprint pattern recognition. Building upon the insights and feedback gathered from the inaugural FVC2000, the organizers were able to enhance the structure and execution of FVC2002, attracting a more extensive participation from both academic and commercial entities (with a total of 33 submitted algorithms). The paper delves into the specifics of the FVC2002 database, outlining the testing protocol and highlighting the key distinctions between FVC2000 and its successor. The comprehensive evaluation of algorithm performance will be showcased at the 16th International Conference on Pattern Recognition (ICPR). This competition underscores the commitment to innovation and excellence in fingerprint verification technology, inviting participants to push the boundaries of what is currently achievable in this critical area of biometric identification.

In this paper, we presented the design and development of a new integrated device for measuring heart rate using fingertip to improve estimating the heart rate. As heart related diseases are increasing day by day, the need for an accurate and affordable heart rate measuring device or heart monitor is essential to ensure quality of health. However, most heart rate measuring tools and environments are expensive and do not follow ergonomics. Our proposed Heart Rate Measuring (HRM) device is economical and user friendly and uses optical technology to detect the flow of blood through index finger. Three phases are used to detect pulses on the fingertip that include pulse detection, signal extraction, and pulse amplification. Qualitative and quantitative performance evaluation of the device on real signals shows accuracy in heart rate estimation, even under intense of physical activity. We compared the performance of HRM device with Electrocardiogram reports and manual pulse measurement of heartbeat of 90 human subjects of different ages. The results showed that the error rate of the device is negligible.

User authentication is one of the fundamental procedures to ensure secure communications and share system resources over an insecure public network channel. Thus, a simple and efficient authentication mechanism is required for securing the network system in the real environment. In general, the password-based authentication mechanism provides the basic capability to prevent unauthorized access. Especially, the purpose of the one-time password is to make it more difficult to gain unauthorized access to restricted resources. Instead of using the password file as conventional authentication systems, many researchers have devoted to implement various one-time password schemes using smart cards, time-synchronized token or short message service in order to reduce the risk of tampering and maintenance cost. However, these schemes are impractical because of the far from ubiquitous hardware devices or the infrastructure requirements. To remedy these weaknesses, the attraction of the QR-code technique can be introduced into our one-time password authentication protocol. Not the same as before, the proposed scheme based on QR code not only eliminates the usage of the password verification table, but also is a cost effective solution since most internet users already have mobile phones. For this reason, instead of carrying around a separate hardware token for each security domain, the superiority of handiness benefit from the mobile phone makes our approach more practical and convenient.

This paper proposes a novel method for offline text-independent writer identification by using convolutional neural network (CNN) and joint Bayesian, which consists of two stages, i.e. feature extraction and writer identification. In the stage of feature extraction, since a large number of data is essential to train an effective CNN model with high generalizability and the amount of handwriting is limited in writer identification, a data augmentation technique is first developed to generate thousands of handwriting images for each writer. Then a deep CNN network is designed to extract discriminative features to represent the properties of different writing styles, which is trained by using the generated handwriting images. In the stage of writer identification, the training dataset is used to train the CNN model for feature extraction and the joint Bayesian technique is employed to accomplish the task of writer identification based on the extracted CNN features. The proposed method is tested on two standard benchmark datasets, i.e. ICDAR2013 and CVL dataset. Experimental results demonstrate that the proposed method gets the best performance compared to the state-of-the-art approaches.

Automatic access control systems (ACS) based on the human biometrics or physical tokens are widely employed in public and private areas. Yet these systems, in their conventional forms, are restricted to active interaction from the users. In scenarios where users are not cooperating with the system, these systems are challenged. Failure in cooperation with the biometric systems might be intentional or because the users are incapable of handling the interaction procedure with the biometric system or simply forget to cooperate with it, due to for example, illness like dementia. This work introduces a challenging bimodal database, including face and hand information of the users when they approach a door to open it by its handle in a noncooperative context. We have defined two (an easy and a challenging) protocols on how to use the database. We have reported results on many baseline methods, including deep learning techniques as well as conventional methods on the database. The obtained results show the merit of the proposed database and the challenging nature of access control with non-cooperative users.

J. M. Valeton I nstitute for Perception TNO Kampweg 5 3769 DE Soesterberg, The Netherlands Abstract. Integration of images from different sensing modalities can produce information that cannot be obtained by viewing the sensor outputs separately and consecutively. This paper introduces a hierarchical i mage merging scheme based on a multiresolution contrast decomposition (the ratio of a low-pass pyramid). The composite images produced by this scheme preserve those details from the input images that are most relevant to visual perception. The method is tested by merging parallel registered thermal and visual images. The results show that the fused images present a more detailed representation of the depicted scene. Detection, recognition, and search tasks may therefore benefit from this new image representation.

Biometric systems are quite common in our everyday life. Despite the higher difficulty to circumvent them, nowadays criminals are developing techniques to accurately simulate physical, physiological, and behavioral traits of valid users, process known as spoofing attack. In this context, robust countermeasure methods must be developed and integrated with the traditional biometric applications in order to prevent such frauds. Despite face being a promising trait due to its convenience and acceptability, face recognition systems can be easily fooled with common printed photographs. Most of state-of-the-art antispoofing techniques for face recognition applications extract handcrafted texture features from images, mainly based on the efficient local binary patterns (LBP) descriptor, to characterize them. However, recent results indicate that high-level (deep) features are more robust for such complex tasks. In this brief, a novel approach for face spoofing detection that extracts deep texture features from images by integrating the LBP descriptor to a modified convolutional neural network is proposed. Experiments on the NUAA spoofing database indicate that such deep neural network (called LBPnet) and an extended version of it (n-LBPnet) outperform other state-of-the-art techniques, presenting great results in terms of attack detection.

Research in person identification and authentication has attracted significant attention from the researchers and scientists. This paper presents a biometric user authentication based on a person's foot step. The advantage of this recognition method over other types of biometrics is that it enables unobtrusive user authentication where other types of biometrics are not available. Firstly the ground reaction force data was extracted using force plate to gather ground reaction force for individuals. Later we utilized the discrete wavelet transform to de-noise the experimental data and in the final step, histograms were used to identify different person's foot step. The experimental results show improvements in identification accuracies compared to previously reported work.

A multi-modal spectroscopy IC combining impedance spectroscopy (IMPS) and multi-wavelength near-infrared spectroscopy (mNIRS) is proposed for high precision non-invasive glucose level estimation. A combination of IMPS and mNIRS can compensate for the glucose estimation error to improve its accuracy. The IMPS circuit measures dielectric characteristics of the tissue using the RLC resonant frequency and the resonant impedance to estimate the glucose level. To accurately find resonant frequency, a 2-step frequency sweep sinusoidal oscillator (FSSO) is proposed: 1) 8-level coarse frequency switching (fSTEP = 9.4 kHz) in 10-76 kHz, and 2) fine analog frequency sweep in the range of 18.9 kHz. During the frequency sweep, the adaptive gain control loop stabilizes the output voltage swing (400 mVp-p). To improve accuracy of mNIRS, three wavelengths, 850 nm, 950 nm, and 1,300 nm, are used. For highly accurate glucose estimation, the measurement data of the IMPS and mNIRS are combined by an artificial neural network (ANN) in external DSP. The proposed ANN method reduces the mean absolute relative difference to 8.3% from 15% of IMPS, and 15-20% of mNIRS in 80-180 mg/dL blood glucose level. The proposed multi-modal spectroscopy IC occupies 12.5 mm 2 in a 0.18 μm 1P6M CMOS technology and dissipates a peak power of 38 mW with the maximum radiant emitting power of 12.1 mW.

Recent research proposes augmenting capacitive touch pads with tangible objects, enabling a new generation of mobile applications enhanced with tangible objects, such as game pieces and tangible controllers. In this paper, we extend the concept to capacitive tangibles consisting of multiple parts, such as stackable gaming pieces and tangible widgets with moving parts. We achieve this using a system of wires and connectors inside each block that causes the capacitance of the bottom-most block to reflect the entire assembly. We demonstrate three types of tangibles, called CapStones, Zebra Dials and Zebra Sliders that work with current consumer hardware and investigate what designs may become possible as touchscreen hardware evolves.

system is the change in electrical properties of skin due to variation in physiological and psychological conditions. The change is caused by the degree to which a person's sweat glands are active. Psychological status of a person tends to make the glands active and this change the skin resistance. Drier the skin is, higher will be the skin resistance. This variation in skin resistance ranges from 5kΩ to 25kΩ. In the current work, a subject whose Galvanic skin response (GSR) is to be measured, has been shown/played a movie clipping, images or recorded audio signals. Depending upon the theme, emotion will be evoked in the subject. Due to the change in emotion, GSR varies. This variation in GSR is recorded for a fixed time interval. In the present work, a total of 75 subjects are selected. People from different age groups and social background, both male and female have been carefully considered. Data acquisition and analysis is carried out by using LabVIEW. The results obtained are convincing and follows the ground reality with a high level of accuracy. As the subject also gives the feedback about the emotions he/she experienced, the results obtained are validated.


Question: Fingerprint authentication through integration of optical and capacitive sensor data.

This paper delves into the fundamental concept of polynomial-time solvable problems, requiring a precise definition of what constitutes a "problem." An abstract problem Q is characterized as a binary relation between a set I of problem instances and a set S of problem solutions. For instance, an instance of the SHORTEST-PATH problem consists of a graph and two vertices, with a solution being a sequence of vertices in the graph, potentially including the empty sequence to denote the absence of a path. The SHORTEST-PATH problem itself is the relation that pairs each instance of a graph and two vertices with the shortest path in the graph that connects them. Given that shortest paths may not be unique, an instance can have multiple solutions. This broad definition of an abstract problem exceeds our requirements, as the theory of NP-completeness focuses on decision problems—those with a yes/no solution. In this context, an abstract decision problem can be viewed as a function mapping the instance set I to the solution set {0, 1}. For example, the PATH problem related to SHORTEST-PATH is defined such that if i = 〈G,u, v,k〉 is an instance of PATH, then PATH(i) = 1 (yes) if the shortest path from u to v has at most k edges, and PATH(i) = 0 (no) otherwise. Although many abstract problems are optimization problems, seeking to minimize or maximize some value, they can typically be easily reformulated as decision problems of comparable difficulty.

A novel approach to fingerprint verification through the fusion of optical and capacitive sensors is introduced in this paper. Unlike traditional methods that rely on a single sensor type, this technique combines the strengths of both optical and capacitive sensors to enhance accuracy and reliability. Optical sensors are adept at capturing high-resolution images of the fingerprint pattern, while capacitive sensors excel in detecting the live tissue beneath the skin, making them less susceptible to spoofing attempts. By fusing the data from both types of sensors, the system can achieve a more comprehensive and secure biometric verification. This fusion process involves complex algorithms that analyze the distinct characteristics provided by each sensor, ensuring that the final decision is based on a thorough and accurate assessment of the fingerprint. The proposed method is evaluated against various attack vectors and compared with existing single-sensor systems, demonstrating its superior performance in terms of both speed and accuracy.

In this study, we propose integrated control of a robotic hand and arm using only proximity sensing from the fingertips. An integrated control scheme for the fingers and for the arm enables quick control of the position and posture of the arm by placing the fingertips adjacent to the surface of an object to be grasped. The arm control scheme enables adjustments based on errors in hand position and posture that would be impossible to achieve by finger motions alone, thus allowing the fingers to grasp an object in a laterally symmetric grasp. This can prevent grasp failures such as a finger pushing the object out of the hand or knocking the object over. Proposed control of the arm and hand allowed correction of position errors on the order of several centimeters. For example, an object on a workbench that is in an uncertain positional relation with the robot, with an inexpensive optical sensor such as a Kinect, which only provides coarse image data, would be sufficient for grasping an object.

This paper proposes a smart system using both face recognition and sound localization techniques to identify the faces of visitors from a door phone in much efficient and accurate ways. This system is effectively used to recognize the faces when their locations are out of the boundaries of the camera scope of the door phone. The smart door phone system proposed in this paper uses a visitor's voice source to rotate the camera to his face and then to recognize the face accurately. The integrated system has been designed with one FPGA(Field Programmable Gate Array) chip and tested for actual use in door phone environments.

We explore the combination of above-surface sensing with eye tracking to facilitate concurrent interaction with multiple regions on touch screens. Conventional touch input relies on positional accuracy, thereby requiring tight visual monitoring of one's own motor action. In contrast, above-surface sensing and eye tracking provides information about how user's hands and gaze are distributed across the interface. In these situations we facilitate interaction by 1) showing the visual feedback of the hand hover near user's gaze point and 2) decrease the requisite of positional accuracy by employing gestural information. We contribute input and visual feedback techniques that combine these modalities and demonstrate their use in example applications. A controlled study showed the effectiveness of our techniques for manipulation tasks against conventional touch, while the effectiveness in acquisition tasks depended on the amount of mid-air motion, leading to our conclusion that the techniques can benefit interacting with multiple interface regions.

In the today's mobile communications scenario, smartphones offer new capabilities to develop sophisticated applications that seem to make daily life easier and more convenient for users. Such applications, which may involve mobile ticketing, identification, access control operations, etc., are often accessible through social network aggregators, that assume a fundamental role in the federated identity management space. While this makes modern smartphones very powerful devices, it also makes them very attractive targets for spyware injection. This kind of malware is able to bypass classic authentication measures and steal user credentials even when a secure element is used, and can, therefore, perform unauthorized mobile access to social network services without the user's consent. Such an event allows stealing sensitive information or even a full identity theft. In this work, we address this issue by introducing BrightPass, a novel authentication mechanism based on screen brightness. BrightPass allows users to authenticate safely with a PIN-based confirmation in the presence of specific operations on sensitive data. We compare BrightPass with existing schemes, in order to show its usability and security within the social network arena. Furthermore, we empirically assess the security of BrightPass through experimentation. Our tests indicate that BrightPass protects the PIN code against automatic submissions carried out by malware while granting fast authentication phases and reduced error rates.

This paper presents an approach to people identification using gait based on floor pressure data. By using a large area high resolution pressure sensing floor, we were able to obtain 3D trajectories of the center of foot pressures over a footstep which contain both the 1D pressure profile and 2D position trajectories of the COP. Based on the 3D COP trajectories a set of features are then extracted and used for people identification together with other features such as stride length and cadence. The Fisher linear discriminant is used as the classifier. Encouraging results have been obtained using the proposed method with an average recognition rate of 94% and false alarm rate of 3% using pair-wise footstep data from 10 subjects.

Thin, flexible, robust capacitive pressure sensors have been the subject of research in many fields where axial strain sensing with high spatial resolution and pressure resolution is desirable for small loads, such as tactile robotics and biomechanics. Simple capacitive pressure sensors have been designed and implemented on flexible substrates in general agreement with performance predicted by an analytical model. Two designs are demonstrated for comparison. The first design uses standard flex circuit technology, and the second design uses photolithography techniques to fabricate capacitive sensors with higher spatial and higher pressure resolution. Sensor arrays of varying sensor size and spacing are tested with applied loads from 0 to 1 MPa. Pressure resolution and linearity of the sensors are significantly improved with the miniaturized, custom fabricated sensor array compared to standard flexible circuit technology.

An ultrathin, stretchable, and conformal sensor system for skin-mounted sweat measurement is characterized and demonstrated in this paper. As an epidermal device, the sweat sensor is mechanically designed for comfortable wear on the skin by employing interdigitated electrodes connected via stretchable serpentine-shaped conductors. Experimental results show that the sensor is sensitive to measuring frequency, sweat level and stretching deformation. It was found that 20kHz signals provide the most sensitive performance: electrical impedance changes 50% while sweat level increases from 20 to 80. In addition, sensor elongation from 15 up to 50% affected the measurement sensitivity of both electrical impedance and capacitance.

We examine three biometric authentication modalities -- voice, face and gesture -- as well as password entry, on a mobile device, to explore the relative demands on user time, effort, error and task disruption. Our laboratory study provided observations of user actions, strategies, and reactions to the authentication methods. Face and voice biometrics conditions were faster than password entry. Speaking a PIN was the fastest for biometric sample entry, but short-term memory recall was better in the face verification condition. None of the authentication conditions were considered very usable. In conditions that combined two biometric entry methods, the time to acquire the biometric samples was shorter than if acquired separately but they were very unpopular and had high memory task error rates. These quantitative results demonstrate cognitive and motor differences between biometric authentication modalities, and inform policy decisions in selecting authentication methods.

Among various skin detection methods, Skin Probability Map (SPM) method is an effective one. Though SPM method possesses high true acceptance rate (TAR), its false acceptance rate (FAR) is unacceptable in some cases. The reason is that SPM method only use pixel-level color information. This paper proposes an improved skin detection method that integrates color, texture and space information. After color filter, a texture filter is constructed based on texture features extracted form Gabor wavelet transform. Texture filter will further filter non-skin pixels, meanwhile, it may also filter some skin pixel. To compensate the loss, after texture filter, a marker driven watershed transform is then used to grow already obtained skin regions. Experimental results show that the proposed method can achieve better performance than that of SPM.

The vulnerability of face recognition systems to presentation attacks (also known as direct attacks or spoof attacks) has received a great deal of interest from the biometric community. The rapid evolution of face recognition systems into real-time applications has raised new concerns about their ability to resist presentation attacks, particularly in unattended application scenarios such as automated border control. The goal of a presentation attack is to subvert the face recognition system by presenting a facial biometric artifact. Popular face biometric artifacts include a printed photo, the electronic display of a facial photo, replaying video using an electronic display, and 3D face masks. These have demonstrated a high security risk for state-of-the-art face recognition systems. However, several presentation attack detection (PAD) algorithms (also known as countermeasures or antispoofing methods) have been proposed that can automatically detect and mitigate such targeted attacks. The goal of this survey is to present a systematic overview of the existing work on face presentation attack detection that has been carried out. This paper describes the various aspects of face presentation attacks, including different types of face artifacts, state-of-the-art PAD algorithms and an overview of the respective research labs working in this domain, vulnerability assessments and performance evaluation metrics, the outcomes of competitions, the availability of public databases for benchmarking new PAD algorithms in a reproducible manner, and finally a summary of the relevant international standardization in this field. Furthermore, we discuss the open challenges and future work that need to be addressed in this evolving field of biometrics.


Question: Assessment datasets for analyzing Twitter sentiment: An overview and introduction to a new collection, the STS-Gold.

This study explores the potential of aggregated Twitter sentiment as a tool for analyzing debate performance, focusing on the 2008 U.S. presidential debate as a case study. With the integration of social media platforms like Twitter into television broadcasting, a new dimension of social video experiences has emerged, offering unique insights into public reactions. Our research delves into the sentiment expressed on Twitter during the debate, employing a systematic methodology to analyze the temporal dynamics of sentiment. We present visual representations and metrics designed to aid journalists and public affairs professionals in identifying sentiment trends, anomalies, and controversial topics. These tools can enhance the understanding of audience reactions and inform the development of visual analytic systems tailored to social media events, providing a comprehensive view of public sentiment in real-time.

In this paper we investigate the use of a multimodal feature learning approach, using neural network based models such as Skip-gram and Denoising Autoencoders, to address sentiment analysis of micro-blogging content, such as Twitter short messages, that are composed by a short text and, possibly, an image. The approach used in this work is motivated by the recent advances in: i) training language models based on neural networks that have proved to be extremely efficient when dealing with web-scale text corpora, and have shown very good performances when dealing with syntactic and semantic word similarities; ii) unsupervised learning, with neural networks, of robust visual features, that are recoverable from partial observations that may be due to occlusions or noisy and heavily modified images. We propose a novel architecture that incorporates these neural networks, testing it on several standard Twitter datasets, and showing that the approach is efficient and obtains good classification results.

The growing number of Twitter users makes it a valuable source of information to study what is happening right now. Users often use Twitter to report real-life events. Here we are only interested in following the financial community. This paper focuses on detecting events popularity through sentiment analysis of tweets published by the financial community on the Twitter universe. The detection of events popularity on Twitter makes this a non-trivial task due to noisy content that often are the tweets. This work aims to filter out all the noisy tweets in order to analyze only the tweets that influence the financial market, more specifically the thirty companies that compose the Dow Jones Average. To perform these tasks, in this paper it is proposed a methodology that starts from the financial community of Twitter and then filters the collected tweets, makes the sentiment analysis of the tweets and finally detects the important events in the life of companies. © 2016 Elsevier Ltd. All rights reserved.

Real-time interaction, which enables live discussions, has become a key feature of most Web applications. In such an environment, the ability to automatically analyze user opinions and sentiments as discussions develop is a powerful resource known as real time sentiment analysis. However, this task comes with several challenges, including the need to deal with highly dynamic textual content that is characterized by changes in vocabulary and its subjective meaning and the lack of labeled data needed to support supervised classifiers. In this paper, we propose a transfer learning strategy to perform real time sentiment analysis. We identify a task - opinion holder bias prediction - which is strongly related to the sentiment analysis task; however, in constrast to sentiment analysis, it builds accurate models since the underlying relational data follows a stationary distribution.
 Instead of learning textual models to predict content polarity (i.e., the traditional sentiment analysis approach), we first measure the bias of social media users toward a topic, by solving a relational learning task over a network of users connected by endorsements (e.g., retweets in Twitter). We then analyze sentiments by transferring user biases to textual features. This approach works because while new terms may arise and old terms may change their meaning, user bias tends to be more consistent over time as a basic property of human behavior. Thus, we adopted user bias as the basis for building accurate classification models. We applied our model to posts collected from Twitter on two topics: the 2010 Brazilian Presidential Elections and the 2010 season of Brazilian Soccer League. Our results show that knowing the bias of only 10% of users generates an F1 accuracy level ranging from 80% to 90% in predicting user sentiment in tweets.

Identifying sentiments (the affective parts of opinions) is a challenging problem. We present a system that, given a topic, automatically finds the people who hold opinions about that topic and the sentiment of each opinion. The system contains a module for determining word sentiment and another for combining sentiments within a sentence. We experiment with various models of classifying and combining sentiment at word and sentence levels, with promising results.

This article introduces a sentiment analysis approach that adopts the way humans read, interpret, and extract sentiment from text. Our motivation builds on the assumption that human interpretation should lead to the most accurate assessment of sentiment in text. We call this automated process Human Reading for Sentiment (HRS). Previous research in sentiment analysis has produced many frameworks that can fit one or more of the HRS aspects; however, none of these methods has addressed them all in one approach. HRS provides a meta-framework for developing new sentiment analysis methods or improving existing ones. The proposed framework provides a theoretical lens for zooming in and evaluating aspects of any sentiment analysis method to identify gaps for improvements towards matching the human reading process. Key steps in HRS include the automation of humans low-level and high-level cognitive text processing. This methodology paves the way towards the integration of psychology with computational linguistics and machine learning to employ models of pragmatics and discourse analysis for sentiment analysis. HRS is tested with two state-of-the-art methods; one is based on feature engineering, and the other is based on deep learning. HRS highlighted the gaps in both methods and showed improvements for both.

CHAPTER 18 Lexicons for Sentiment and Affect Extraction " [W]e write, not with the fingers, but with the whole person. The nerve which controls the pen winds itself about every fibre of our being, threads the heart, pierces the liver. " Virginia Woolf, Orlando " She runs the gamut of emotions from A to B. " Dorothy Parker, reviewing Hepburn's performance in Little Women In this chapter we turn to tools for interpreting affective meaning, extending our affective study of sentiment analysis in Chapter 7. We use the word 'affective', following the tradition in affective computing (Picard, 1995) to mean emotion, sentiment, personality , mood, and attitudes. Affective meaning is closely related to subjectivity, subjectivity the study of a speaker or writer's evaluations, opinions, emotions, and speculations (Wiebe et al., 1999). How should affective meaning be defined? One influential typology of affec-tive states comes from Scherer (2000), who defines each class of affective states by factors like its cognition realization and time course: Emotion: Relatively brief episode of response to the evaluation of an external or internal event as being of major significance. Mood: Diffuse affect state, most pronounced as change in subjective feeling, of low intensity but relatively long duration, often without apparent cause. Interpersonal stance: Affective stance taken toward another person in a specific interaction, colouring the interpersonal exchange in that situation. Figure 18.1 The Scherer typology of affective states, with descriptions from Scherer (2000).

In this paper we present a detailed scheme for annotating expressions of opinions, beliefs, emotions, sentiment and speculation (private states) in the news and other discourse. We explore inter-annotator agreement for individual private state expressions, and show that these low-level annotations are useful for producing higher-level subjective sentence annotations.

The Twitter social network provides a constant stream of concise data, useful within both geospatial and temporal domains. Previous studies have attempted to mine and find recent news topics within the data set. Although the efforts were a step in the right direction, we believe more is needed to accomplish a useful and interesting task: using live Twitter data to automatically identify breaking news events in near real-time. In this paper, we present methods to collect data, identify breaking news topics, and display results in a geo-temporal visualization. Our goal is to provide a solution that discovers breaking news faster than traditional reporting mediums.

Retrieving information from social networks is the first and primordial step many data analysis fields such as Natural Language Processing, Sentiment Analysis and Machine Learning. Important data science tasks relay on historical data gathering for further predictive results. Most of the recent works use Twitter API, a public platform for collecting public streams of information, which allows querying chronological tweets for no more than three weeks old. In this paper, we present a new methodology for collecting historical tweets within any date range using web scraping techniques bypassing for Twitter API restrictions.

The emergence of online social networks in the past few years has generated an enormous amount of information about potentially any subject. Valuable data containing users' opinions and thoughts are available on those repositories and several sentiment analysis techniques have been proposed that address the problem of understanding the opinion orientation of the users' postings. In this paper, we take a different perspective to the problem through a user centric approach, which uses a graph to model users (and their postings) and applies link mining techniques to infer opinions of users. Preliminary experiments on a Twitter corpus have shown promising results.


Question: Assessment datasets for analyzing Twitter sentiment: A review and introduction to the STS-Gold collection.

This paper focuses on the challenge of part-of-speech tagging for English content sourced from Twitter, a leading microblogging platform. A custom tagset is designed, and a dataset is meticulously annotated to facilitate this linguistic analysis. Through the implementation of sophisticated features and rigorous experiments, tagging accuracy reaches an impressive level, approaching 90%. The researchers have generously released both the annotated data and the tagging tools to the academic community, with the intention of enhancing and broadening text analysis capabilities for Twitter and comparable social media datasets. This contribution aims to support more nuanced and detailed studies of social media language use, sentiment, and discourse, thereby enriching the field of computational social science.

Social media constitute a challenging new source of information for intelligence gathering and decision making. Twitter is one of the most popular social media sites and often becomes the primary source of information. Twitter messages are short and well suited for knowledge discovery. Twitter provides both researchers and practitioners a free Application Programming Interface (API) which allows them to gather and analyse large data sets of tweets. Twitter information aren't solely tweet texts, as Twitter’s API provides a lot of info to perform attention-grabbing analysis studies. The paper concisely describes method of knowledge gathering and therefore the main areas of knowledge mining, information discovery and information visual image from Twitter information. In this paper we can create a twitter app from which we can fetch the real time twitter tweets on a particular topic and stored it into R and then we can apply several text mining steps on the tweets to pre-process the tweets text and than we

Sentiment analysis seeks to identify the viewpoint(s) underlying a text span; an example application is classifying a movie review as “thumbs up” or “thumbs down”. To determine this sentiment polarity, we propose a novel machine-learning method that applies text-categorization techniques to just the subjective portions of the document. Extracting these portions can be implemented using efficient techniques for finding minimum cuts in graphs; this greatly facilitates incorporation of cross-sentence contextual constraints. Publication info: Proceedings of the ACL, 2004.

Message-level and word-level polarity classification are two popular tasks in Twitter sentiment analysis. They have been commonly addressed by training supervised models from labelled data. The main limitation of these models is the high cost of data annotation. Transferring existing labels from a related problem domain is one possible solution for this problem. In this paper, we study how to transfer sentiment labels from the word domain to the tweet domain and vice versa by making their corresponding instances compatible. We model instances of these two domains as the aggregation of instances from the other (i.e., tweets are treated as collections of the words they contain and words are treated as collections of the tweets in which they occur) and perform aggregation by averaging the corresponding constituents. We study two different setups for averaging tweet and word vectors: 1) representing tweets by standard NLP features such as unigrams and part-of-speech tags and words by averaging the vectors of the tweets in which they occur, and 2) representing words using skip-gram embeddings and tweets as the average embedding vector of their words. A consequence of our approach is that instances of both domains reside in the same feature space. Thus, a sentiment classifier trained on labelled data from one domain can be used to classify instances from the other one. We evaluate this approach in two transfer learning tasks: 1) sentiment classification of tweets by applying a word-level sentiment classifier, and 2) induction of a polarity lexicon by applying a tweet-level polarity classifier. Our results show that the proposed model can successfully classify words and tweets after transfer.

Twitter is one of world most famous social media. There are many statement expresed in Twitter like happiness, sadness, public information, etc. Unfortunately, people may got angry to each other and write it down as a tweet on Twitter. Some tweet may contain Indonesian swear words. It's serious problem because many Indonesians may not tolerated swear words. Some Indonesian swear words may have multiple means, not always an Indonesian swear word means insulting. Twitter has provide tweet's data by account, trending topics, and advance keyword. This work try to analyze many tweet about political news, political event, and some Indonesian famous person because the tweet assumed contains many Indonesian swear word. The derived tweets will process in text mining and then analyzed by classification process using Naive Bayes Classifier, Nearest Neighbor, and Decision Tree to detect Indonesian swear word. This work expected to discover the high accurate classification model. It means, the model can differentiate the real meaning of Indonesian swear word contained in tweet.

This work examines the predictive power of public data by aggregating information from multiple online sources. Our sources include microblogging sites like Twitter, online message boards like Yahoo! Finance, and traditional news articles. The subject of prediction are daily stock price movements from Standard & Poor’s 500 index (S&P 500) during a period from June 2011 to November 2011. To forecast price movements we filter messages by stocks, apply state-of-the-art sentiment analysis to message texts, and aggregate message sentiments to generate trading signals for daily buy and sell decisions. We evaluate prediction quality through a simple trading model considering real-world limitations like transaction costs or broker commission fees. Considering 833 virtual trades, our model outperformed the S&P 500 and achieved a positive return on investment of up to ~0.49% per trade or ~0.24% when adjusted by market, depending on supposed trading costs.

The use of the web as a universal communication platform generates large volumes of data (Big data), which in many cases, need to be processed so that they can become useful knowledge in face of the sceptics who have doubts about the credibility of such information. The use of web data that comes from educational contexts needs to be addressed, since that large amount of unstructured information is not being valued, losing valuable information that can be used. To solve this problem, we propose the use of data mining techniques such as sentiment analysis to validate the information that comes from the educational platforms. The objective of this research is to propose a methodology that allows the user to apply sentiment analysis in a simple way, because although some researchers have done it, very few do with data in the educational context. The results obtained prove that the proposal can be used in similar cases.

Sentiment Analysis deals with the analysis of emotions, opinions and facts in the sentences which are expressed by the people. It allows us to track attitudes and feelings of the people by analyzing blogs, comments, reviews and tweets about all the aspects. The development of Internet has strong influence in all types of industries like tourism, healthcare and any business. The availability of Internet has changed the way of accessing the information and sharing their experience among users. Social media provide this information and these comments are trusted by other users. This paper recognizes the use and impact of social media on healthcare industry by analyzing the users’ feelings expressed in the form of free text, thereby gives the quality indicators of services or features related with them. In this paper, a sentiment classifier model using improved Term Frequency Inverse Document Frequency (TFIDF) method and linear regression model has been proposed to classify online reviews, tweets or customer feedback for various features. The model involves the process of gathering online user reviews about hospitals and analyzes those reviews in terms of sentiments expressed. Information Extraction process filters irrelevant reviews, extracts sentimental words of features identified and quantifies the sentiment of features using sentiment dictionary. Emotionally expressed positive or negative words are assigned weights using the classification prescribed in the dictionary. The sentiment analysis on tweets/reviews is done for various features using Natural Language Processing (NLP) and Information Retrieval (IR) techniques. The proposed linear regression model using the senti-score predicts the star rating of the feature of service. The statistical results show that improved TF-IDF method gives better accuracy when compared with TF and TF-IDF methods, used for representing the text. The senti-score obtained as a result of text analysis (user feedback) on features gives not only the opinion summarization but also the comparative results on various features of different competitors. This information can be used by business to focus on the low scored features so as to improve their business and ensure a very high level of user satisfaction.

The proliferation of Internet has not only generated huge volumes of unstructured information in the form of web documents, but a large amount of text is also generated in the form of emails, blogs, and feedbacks etc. The data generated from online communication acts as potential gold mines for discovering knowledge. Text analytics has matured and is being successfully employed to mine important information from unstructured text documents. Most of these techniques use Natural Language Processing techniques which assume that the underlying text is clean and correct. Statistical techniques, though not as accurate as linguistic mechanisms, are also employed for the purpose to overcome the dependence on clean text. The chief bottleneck for designing statistical mechanisms is however its dependence on appropriately annotated training data. None of these methodologies are suitable for mining information from online communication text data due to the fact that they are often noisy. These texts are informally written. They suffer from spelling mistakes, grammatical errors, improper punctuation and irrational capitalization. This paper focuses on opinion extraction from noisy text data. It is aimed at extracting and consolidating opinions of customers from blogs and feedbacks, at multiple levels of granularity. Ours is a hybrid approach, in which we initially employ a semi-supervised method to learn domain knowledge from a training repository which contains both noisy and clean text. Thereafter we employ localized linguistic techniques to extract opinion expressions from noisy text. We have developed a system based on this approach, which provides the user with a platform to analyze opinion expressions extracted from a repository.

The popularity of social media in recent years has increased a lot and created new opportunities to study the interactions of different groups of people. Because of this it comes to possible to study the user‟s opinion. Two popular topics in the study of social networks are community detection and finding trends. Trend Analysis is an ongoing field of research in Data mining field, May referred as an “Opinion Mining” by some researchers. Trend Analysis is the computational treatment of opinions, sentiments and subjectivity of text. The related field to Trend Analysis contains finding emotions in a statement which attracted researchers recently are discussed. The main target of this survey is to give nearly full image of Trend Analysis along with the community detection and their related fields with brief details. The main contributions of this paper include the overall analysis to bridge the concept between trend analyses with community detection. To find out the anomaly detection in a community with using the concept of trend analysis is a difficult task.

The growth and diffusion of online social media have been enormously increased in recent years, as well as the research and commercial interests toward these rising sources of information as a direct public expression of the communities. Moreover, the depth and the quality of data that can be harvested by monitoring and analysis tools have evolved significantly. In particular, Twitter has revealed to be one of the most widespread microblogging services for instantly publishing and sharing opinions, feedbacks, ratings etc., contributing in the development of the emerging role of users as sensors. However, due to the huge amount of data to be collected and analyzed and limitations on data access imposed by Twitter public APIs, more efficient requirements are needed for analytics tools, both in terms of data ingestion and processing, as well as for the computation of analysis metrics, to be provided for deeper statistic insights and further investigations. In this paper, the Twitter Vigilance platform is presented, realized by the DISIT Lab at University of Florence. Twitter Vigilance has been designed as a cross-domain, multi-user tool for collecting and analyzing Twitter data, providing aggregated metrics based on the volume of tweets and retweets, users' influence network, Natural Language Processing and Sentiment Analysis of textual content. The proposed architecture has been validated against a dataset of about 270 million tweets showing a high efficiency in recovering Twitter data. For this reason it has been adopted by a number of researchers as a study platform for social media analysis, early warning, etc.


Question: Assessment collections for Twitter emotion analysis: An overview and an introduction to a novel dataset, the STS-Gold.

Innovative Semantic Integration for Enhanced Twitter Sentiment Analysis

The rapid dissemination of public sentiment through platforms like Twitter presents businesses with a valuable tool for gauging public opinion on brands, corporate strategies, and leadership. To capitalize on this, a plethora of techniques and features have been developed to train sentiment classifiers tailored to Twitter data, yielding diverse outcomes. This paper introduces a pioneering method that integrates semantic information as an additional layer in the training process for sentiment analysis. Specifically, for each entity (such as "iPhone") mentioned in tweets, we append its corresponding semantic concept ("Apple product") as a supplementary feature, and assess its correlation with sentiments, whether negative or positive. We validate this methodology by applying it to sentiment prediction across three distinct Twitter datasets. Our findings reveal a significant boost in F harmonic accuracy, with an average uplift of approximately 6.5% and 4.8% for detecting negative and positive sentiments, respectively, when compared to baseline models relying solely on unigrams and part-of-speech tagging. Furthermore, we juxtapose our approach against a technique centered on sentiment-bearing topic analysis, observing that semantic features yield superior Recall and F score in the classification of negative sentiments, and improved Precision albeit with a lower Recall and F score for positive sentiment classification. This underscores the efficacy of incorporating semantic features in sentiment analysis for Twitter data, enhancing the accuracy and reliability of sentiment detection.

The advancement in technology indicates that there is an opportunity to enhance Human-Computer Interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users’ self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as defined by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.

Sentiment analysis (SA) has become one of the most active and progressively popular areas in information retrieval and text mining due to the expansion of the World Wide Web (WWW). SA deals with the computational treatment or the classification of user’s sentiments, opinions and emotions hidden within the text. Aspect extraction is the most vital and extensively explored phase of SA to carry out the classification of sentiments in precise manners. During the last decade, enormous number of research has focused on identifying and extracting aspects. Therefore, in this survey, a comprehensive overview has been attempted for different aspect extraction techniques and approaches. These techniques have been categorized in accordance with the adopted approach. Despite being a traditional survey, a comprehensive comparative analysis is conducted among different approaches of aspect extraction, which not only elaborates the performance of any technique but also guides the reader to compare the accuracy with other state-of-the-art and most recent approaches.

We present a classifier to predict contextual polarity of subjective phrases in a sentence. Our approach features lexical scoring derived from the Dictionary of Affect in Language (DAL) and extended through WordNet, allowing us to automatically score the vast majority of words in our input avoiding the need for manual labeling. We augment lexical scoring with n-gram analysis to capture the effect of context. We combine DAL scores with syntactic constituents and then extract ngrams of constituents from all sentences. We also use the polarity of all syntactic constituents within the sentence as features. Our results show significant improvement over a majority class baseline as well as a more difficult baseline consisting of lexical n-grams.

Online social media, such as Reddit, has become an important resource to share personal experiences and communicate with others. Among other personal information, some social media users communicate about mental health problems they are experiencing, with the intention of getting advice, support or empathy from other users. Here, we investigate the language of Reddit posts specific to mental health, to define linguistic characteristics that could be helpful for further applications. The latter include attempting to identify posts that need urgent attention due to their nature, e.g. when someone announces their intentions of ending their life by suicide or harming others. Our results show that there are a variety of linguistic features that are discriminative across mental health user communities and that can be further exploited in subsequent classification tasks. Furthermore, while negative sentiment is almost uniformly expressed across the entire data set, we demonstrate that there are also condition-specific vocabularies used in social media to communicate about particular disorders. Source code and related materials are available from: https: //github.com/gkotsis/ reddit-mental-health.

This paper describes the shared task for the English Twitter geolocation prediction associated with WNUT 2016. We discuss details of the task settings, data preparation and participant systems. The derived dataset and performance figures from each system provide baselines for future research in this realm.

Sentiment in social media is increasingly considered as an important resource for customer segmentation, market understanding, and tackling other socio-economic issues. However, sentiment in social media is difficult to measure since user-generated content is usually short and informal. Although many traditional sentiment analysis methods have been proposed, identifying slang sentiment words remains untackled. One of the reasons is that slang sentiment words are not available in existing dictionaries or sentiment lexicons. To this end, we propose to build the first sentiment dictionary of slang words to aid sentiment analysis of social media content. It is laborious and time-consuming to collect and label the sentiment polarity of a comprehensive list of slang words. We present an approach to leverage web resources to construct an extensive Slang Sentiment word Dictionary (SlangSD) that is easy to maintain and extend. SlangSD is publicly available1 for research purposes. We empirically show the advantages of using SlangSD, the newly-built slang sentiment word dictionary for sentiment classification, and provide examples demonstrating its ease of use with an existing sentiment system.

Twitter is a social media that developed rapidly in today’s modern world. As millions of Twitter messages are sent day by day, the value and importance of developing a new technique for detecting spammers become significant. Moreover, legitimate users are affected by means of spams in the form of unwanted URLs, irrelevant messages, etc. Another hot topic of research is sentiment analysis that is based on each tweet sent by the user and opinion mining of the customer reviews. Most commonly natural language processing is used for sentiment analysis. The text is collected from user’s tweets by opinion mining and automatic sentiment analysis that are oriented with ternary classifications, such as “positive,” “neutral,” and “negative.” Due to limited size, unstructured nature, misspells, slangs, and abbreviations, it is more challenging for researchers to find sentiments for Twitter data. In this paper, we collected 600 million public tweets using URL-based security tool and feature generation is applied for sentiment analysis. The ternary classification is processed based on preprocessing technique, and the results of tweets sent by the users are obtained. We use a hybridization technique using two optimization algorithms and one machine learning classifier, namely particle swarm optimization and genetic algorithm and decision tree for classification accuracy by sentiment analysis. The results are compared with previous works, and our proposed method shows a better analysis than that of other classifiers.

Very large online music databases have recently been created by vendors, but they generally lack content-based retrieval methods. One exception is Allmusic.com which offers browsing by musical emotion, using human experts to classify several thousand songs into 183 moods. In this paper, machine learning techniques are used instead of human experts to extract emotions in Music. The classification is based on a psychological model of emotion that is extended to 23 specific emotion categories. Our results for mining the lyrical text of songs for specific emotion are promising, generate classification models that are human-comprehensible, and generate results that correspond to commonsense intuitions about specific emotions. Mining lyrics focused in this paper is one aspect of research which combines different classifiers of musical emotion such as acoustics and lyrical text.

Use of data generated through social media for health studies is gradually increasing. Twitter is a short-text message system developed 6 years ago, now with more than 100 million users generating over 300 million Tweets every day. Twitter may be used to gain real-world insights to promote healthy behaviors. The purposes of this paper are to describe a practical approach to analyzing Tweet contents and to illustrate an application of the approach to the topic of physical activity. The approach includes five steps: (1) selecting keywords to gather an initial set of Tweets to analyze; (2) importing data; (3) preparing data; (4) analyzing data (topic, sentiment, and ecologic context); and (5) interpreting data. The steps are implemented using tools that are publically available and free of charge and designed for use by researchers with limited programming skills. Content mining of Tweets can contribute to addressing challenges in health behavior research.

Social media outlets such as Twitter have become an important forum for peer interaction. Thus the ability to classify latent user attributes, including gender, age, regional origin, and political orientation solely from Twitter user language or similar highly informal content has important applications in advertising, personalization, and recommendation. This paper includes a novel investigation of stacked-SVM-based classification algorithms over a rich set of original features, applied to classifying these four user attributes. It also includes extensive analysis of features and approaches that are effective and not effective in classifying user attributes in Twitter-style informal written genres as distinct from the other primarily spoken genres previously studied in the user-property classification literature. Our models, singly and in ensemble, significantly outperform baseline models in all cases. A detailed analysis of model components and features provides an often entertaining insight into distinctive language-usage variation across gender, age, regional origin and political orientation in modern informal communication.