Question: Making 3D Eyeglasses Try-on feasible and user-friendly.

A Practical System for Virtual Eyeglasses Try-On with 3D Head Modeling
This paper introduces a practical system for virtual try-on of eyeglasses, utilizing a 3D model of the user's face and head. The 3D head model is reconstructed from a single image of the user, providing a personalized and accurate representation. The system then overlays 3D glasses models onto this head model, ensuring a precise fit. Real-time head movement tracking enables the 3D head model, complete with the eyeglasses, to rotate in accordance with the user's movements, offering an immersive and interactive try-on experience. This approach not only enhances the realism of the virtual try-on but also makes it more accessible and user-friendly for potential eyewear buyers.

This paper proposes a visuo-haptic feedback system that can provide haptic feedback to multiple users on an LCD monitor using electrostatic adhesion and built-in capacitive sensors. The prototype system consists of a 40-inch LCD monitor, an ITO electrode sheet arranged on the monitor, and multiple contact pads for electrostatic adhesion. By applying low-frequency haptic voltage and high-frequency sensing voltage to each pad, the system realizes passive haptic feedback, as well as position sensing using the same components. The passive haptic feedback force exceeds 1 N at 300 V<inline-formula><tex-math notation="LaTeX">$_\mathrm{rms}$</tex-math><alternatives> <inline-graphic xlink:type="simple" xlink:href="nakamura-ieq1-2556660.gif"/></alternatives></inline-formula>. The performance of the sensing system is investigated in terms of interference, which shows the haptic voltage and existence of multiple pads can affect the sensing and can shift the output by a few millimeters. A pilot application, virtual hockey game, successfully demonstrates the sensing and haptic rendering capability of the proposed system. The effect of the interference on the demonstration is discussed.

Mobile Augmented Reality (MAR) blends the real world with digital objects especially in ubiquitous devices such as smartphones. The MAR applications provide an intelligent interface for users. In this, valuable digital information is advertised in physical spaces. However, the success of these applications is tied directly to the degree of user acceptance. This makes understanding the needs and expectations of the MAR’s potential users of paramount importance for designing and building the proper application. The objective of the paper is to expose an important gap in the development of novel applications in the virtual world. Previous research has shown that it is essential to study and understand the needs and expectations of the potential users of the upcoming application or system. Studying user needs and expectations before offering the developed application ensures a minimum level of acceptance and, of course, success. This paper presents a detailed study comprising of a userexperience (UX) evaluation of different prototypes through the use of three different UX evaluation methods. This kind of evaluation allows new developments to offer systems, which do not fail. The main contributions of this study are that it: 1) solicits expectations when consumers use MAR applications, 2) assesses the UX over different prototypes using three different metrics, 3) provides methodological insights on UX evaluation experiments and, 4) is useful for anyone who wants to develop handheld applications after understanding user expectations and how his experience should progress. The results of the study show that users value concreteness, realizability, personalization, novelty, intuitiveness and the usefulness of presented information. Paying attention to these factors can help develop more acceptable MAR applications and lead to more novel future designs.

Camera calibration patterns and fiducial markers are basic technique for 3D measurement to create Computer Graphics (CG) and Augmented Reality (AR). A checkerboard is widely used as a calibration pattern and a circle or ring grid provides higher precision [Datta et al. 2009], and matrix codes are common as AR markers.

Nowadays, GPS-based car navigation systems mainly use speech and aerial views of simplified road maps to guide drivers to their destination. However, drivers often experience difficulties in linking the simple 2D aerial map with the visual impression that they get from the real environment, which is inherently ground-level based. Therefore, supplying realistically textured 3D city models at ground-level proves very useful for pre-visualizing an upcoming traffic situation. Because this pre-visualization can be rendered from the expected future viewpoints of the driver, the latter will more easily understand the required maneuver. 3D city models can be reconstructed from the imagery recorded by surveying vehicles. The vastness of image material gathered by these vehicles, however, puts extreme demands on vision algorithms to ensure their practical usability. Algorithms need to be as fast as possible and should result in compact, memory efficient 3D city models for future ease of distribution and visualization. For the considered application, these are not contradictory demands. Simplified geometry assumptions can speed up vision algorithms while automatically guaranteeing compact geometry models. We present a novel city modeling framework which builds upon this philosophy to create 3D content at high speed which could allow for pre-visualization of any conceivable traffic situation by car navigation modules.

3D face recognition has become a trending research direction in both industry and academia. It inherits advantages from traditional 2D face recognition, such as the natural recognition process and a wide range of applications. Moreover, 3D face recognition systems could accurately recognize human faces even under dim lights and with variant facial positions and expressions, in such conditions 2D face recognition systems would have immense difficulty to operate. This paper summarizes the history and the most recent progresses in 3D face recognition research domain. The frontier research results are introduced in three categories: pose-invariant recognition, expression-invariant recognition, and occlusion-invariant recognition. To promote future research, this paper collects information about publicly available 3D face databases. This paper also lists important open problems.

In this paper, we introduce the DEEP-SEE FACE framework, an assistive device designed to improve cognition, interaction, and communication of visually impaired (VI) people in social encounters. The proposed approach jointly exploits computer vision algorithms (region proposal networks, ATLAS tracking and global, and low-level image descriptors) and deep convolutional neural networks in order to detect, track, and recognize, in real-time, various persons existent in the video streams. The major contribution of the paper concerns a global, fixed-size face representation that takes into the account of various video frames while remaining independent of the length of the image sequence. To this purpose, we introduce an effective weight adaptation scheme that is able to determine the relevance assigned to each face instance, depending on the frame degree of motion/camera blur, scale variation, and compression artifacts. Another relevant contribution involves a hard negative mining stage that helps us differentiating between known and unknown face identities. The experimental results, carried out on a large-scale data set, validate the proposed methodology with an average accuracy and recognition rates superior to 92%. When tested in real life, indoor/outdoor scenarios, the DEEP-SEE FACE prototype proves to be effective and easy to use, allowing the VI people to access visual information during social events.

The increasingly widespread use of simulations in education underlines the fact that these teaching tools compared to other methods or media allow students to approach and experience new topics in a more realistic way. This realism enhances their learning and understanding of complex subjects. So far it has been difficult to interactively simulate three-dimensional dynamic learning content. In this field, the continuing development of Virtual Reality (VR) offers new opportunities for educators to convey a wide variety of subjects. It is the aim of this paper to characterize the nature of Virtual Reality as an educational setting for simulations, and to show that Studierstube, our multi-user collaborative Virtual Environment, comprises the necessary features for applying VR-techniques to educational purposes. We further discuss the general applicability of VR to various fields of education and demonstrate its specific application as a tool for teaching elementary three-dimensional geometry.

We propose an intuitive user interaction system with virtual objects by directly using user's hands through a physics simulation. This system aims to provide a more direct way to manipulate and interact with virtual objects and to induce more realistic reactions of the virtual objects in the interaction than the existing methods. The key contribution of this paper is to model the deformations of the user's hands in two ways: global deformation by the hand poses and the local deformation by contacts with objects. By the two deformation models, objects can be manipulated with better stability and robustness without any predefinition of interaction methods depending on the object shapes, by solely using a physics simulation. The experiment shows that the proposed method enables a user to intuitively manipulate virtual object accurately.

This paper describes performance evaluation of a wearable augmented reality system for natural outdoor environments. Applied Research Associates (ARA), as prime integrator on the DARPA ULTRA-Vis (Urban Leader Tactical, Response, Awareness, and Visualization) program, is developing a soldier-worn system to provide intuitive ‘heads-up’ visualization of tactically-relevant geo-registered icons. Our system combines a novel pose estimation capability, a helmet-mounted see-through display, and a wearable processing unit to accurately overlay geo-registered iconography (e.g., navigation waypoints, sensor points of interest, blue forces, aircraft) on the soldier’s view of reality. We achieve accurate pose estimation through fusion of inertial, magnetic, GPS, terrain data, and computer-vision inputs. We leverage a helmet-mounted camera and custom computer vision algorithms to provide terrain-based measurements of absolute orientation (i.e., orientation of the helmet with respect to the earth). These orientation measurements, which leverage mountainous terrain horizon geometry and mission planning landmarks, enable our system to operate robustly in the presence of external and body-worn magnetic disturbances. Current field testing activities across a variety of mountainous environments indicate that we can achieve high icon geo-registration accuracy (<10mrad) using these vision-based methods.

In this paper, a new technique for modeling textured 3D faces is introduced. 3D faces can either be generated automatically from one or more photographs, or modeled directly through an intuitive user interface. Users are assisted in two key problems of computer aided face modeling. First, new face images or new 3D face models can be registered automatically by computing dense one-to-one correspondence to an internal face model. Second, the approach regulates the naturalness of modeled faces avoiding faces with an “unlikely” appearance. Starting from an example set of 3D face models, we derive a morphable face model by transforming the shape and texture of the examples into a vector space representation. New faces and expressions can be modeled by forming linear combinations of the prototypes. Shape and texture constraints derived from the statistics of our example faces are used to guide manual modeling or automated matching algorithms. We show 3D face reconstructions from single images and their applications for photo-realistic image manipulations. We also demonstrate face manipulations according to complex parameters such as gender, fullness of a face or its distinctiveness.


Question: Making 3D eyeglasses try-on feasible in real-world scenarios.

This paper introduces a mixed reality platform designed for the virtual try-on of three-dimensional eyeglasses. The system enables users to effortlessly select and superimpose various virtual eyewear models onto their faces, automatically adjusting the fit to ensure a realistic appearance. Users can visualize their reflection, complete with the 3D virtual glasses, akin to looking into a mirror. A key innovation is the resolution of the occlusion challenge, which ensures that only the visible portions of the glasses, not obscured by the face, are displayed. This technology holds significant potential for online shopping experiences, allowing customers to narrow down a vast selection of eyewear models to a manageable number before physically trying them on at a retail location.

We propose a method to remove objects such as people and cars from multi-view urban image datasets, enabling free-viewpoint IBR in the edited scenes. Our method combines information from multi-view 3D reconstruction with image inpainting techniques, by formulating the problem as an optimization of a global patch-based objective function. We use Image-Based Rendering (IBR) techniques to reproject information from neighboring views, and 3D multi-view stereo reconstruction to perform multiview coherent initialization for inpainting of pixels not filled by reprojection. Our algorithm performs multi-view consistent inpainting for color and 3D by blending reprojections with patch-based image inpainting. We run our algorithm on casually captured datasets, and Google StreetViewdata, removing objects cars, people and pillars, showing that our approach produces results of sufficient quality for free-viewpoint IBR on "cleaned up" scenes, as well as IBR scene editing, such as limited motion of real objects.

Person-independent and pose-invariant estimation of eye-gaze is important for situation analysis and for automated video annotation. We propose a fast cascade regression based method that first estimates the location of a dense set of markers and their visibility, then reconstructs face shape by fitting a part-based 3D model. Next, the reconstructed 3D shape is used to estimate a canonical view of the eyes for 3D gaze estimation. The model operates in a feature space that naturally encodes local ordinal properties of pixel intensities leading to photometric invariant estimation of gaze. To evaluate the algorithm in comparison with alternative approaches, three publicly-available databases were used, Boston University Head Tracking, Multi-View Gaze and CAVE Gaze datasets. Precision for head pose and gaze averaged 4 degrees or less for pitch, yaw, and roll. The algorithm outperformed alternative methods in both datasets.

This paper is an overview of technologies that fall under the term augmented reality. Augmented reality refers to a system in which the physical surroundings of a person are mixed with real-time computer generated information creating an enhanced perception of surrounding environment. Being partly virtual and real, augmented reality applications have quite extreme requirements to be practical to use. It also has very much potential in numerous different application areas. These issues make augmented reality both an interesting and challenging subject from scientific and business perspectives.

The present study investigated the ability to inter actively locate points in a three dimensional computer environment using a six degree of freedom input device. Four different visu al feedback modes were tested: fixed viewpoint monoscopic persp ective, fixed viewpoint stereoscopic perspective, head-tracked mo noscopic perspective and head-tracked stereoscopic perspecti ve. Targets were located at plus and minus 10cm along the X, Y or Z axes, from a fixed starting location. Data about the time to complete the task and positioning accuracy (error) are gathered for each trial. In addition, subjective feedback regarding the apparat us, visual mode and task difficulty was solicited from subject s. The results indicate that stereoscopic performance is superior to monoscopic performance and that asymmetries exist both across and within axes. Head tracking had no appreciable effect upon performance. Subjective feedback regarding performance is usuall y consistent with objective measures, but some inconsistencies a re present.

KinectFusion enables a user holding and moving a standard Kinect camera to rapidly create detailed 3D reconstructions of an indoor scene. Only the depth data from Kinect is used to track the 3D pose of the sensor and reconstruct, geometrically precise, 3D models of the physical scene in real-time. The capabilities of KinectFusion, as well as the novel GPU-based pipeline are described in full. Uses of the core system for low-cost handheld scanning, and geometry-aware augmented reality and physics-based interactions are shown. Novel extensions to the core GPU pipeline demonstrate object segmentation and user interaction directly in front of the sensor, without degrading camera tracking or reconstruction. These extensions are used to enable real-time multi-touch interactions anywhere, allowing any planar or non-planar reconstructed physical surface to be appropriated for touch.

Single image reflection separation is an ill-posed problem since two scenes, a transmitted scene and a reflected scene, need to be inferred from a single observation. To make the problem tractable, in this work we assume that categories of two scenes are known. It allows us to address the problem by generating both scenes that belong to the categories while their contents are constrained to match with the observed image. A novel network architecture is proposed to render realistic images of both scenes based on adversarial learning. The network can be trained in a weakly supervised manner, i.e., it learns to separate an observed image without corresponding ground truth images of transmission and reflection scenes which are difficult to collect in practice. Experimental results on real and synthetic datasets demonstrate that the proposed algorithm performs favorably against existing methods.

Machine learning is enabling a myriad innovations, including new algorithms for cancer diagnosis and self-driving cars. The broad use of machine learning makes it important to understand the extent to which machine-learning algorithms are subject to attack, particularly when used in applications where physical security or safety is at risk.
 In this paper, we focus on facial biometric systems, which are widely used in surveillance and access control. We define and investigate a novel class of attacks: attacks that are physically realizable and inconspicuous, and allow an attacker to evade recognition or impersonate another individual. We develop a systematic method to automatically generate such attacks, which are realized through printing a pair of eyeglass frames. When worn by the attacker whose image is supplied to a state-of-the-art face-recognition algorithm, the eyeglasses allow her to evade being recognized or to impersonate another individual. Our investigation focuses on white-box face-recognition systems, but we also demonstrate how similar techniques can be used in black-box scenarios, as well as to avoid face detection.

The goal of this work is to represent objects in an RGB-D scene with corresponding 3D models from a library. We approach this problem by first detecting and segmenting object instances in the scene and then using a convolutional neural network (CNN) to predict the pose of the object. This CNN is trained using pixel surface normals in images containing renderings of synthetic objects. When tested on real data, our method outperforms alternative algorithms trained on real data. We then use this coarse pose estimate along with the inferred pixel support to align a small number of prototypical models to the data, and place into the scene the model that fits best. We observe a 48% relative improvement in performance at the task of 3D detection over the current state-of-the-art [34], while being an order of magnitude faster.

Nowadays, GPS-based car navigation systems mainly use speech and aerial views of simplified road maps to guide drivers to their destination. However, drivers often experience difficulties in linking the simple 2D aerial map with the visual impression that they get from the real environment, which is inherently ground-level based. Therefore, supplying realistically textured 3D city models at ground-level proves very useful for pre-visualizing an upcoming traffic situation. Because this pre-visualization can be rendered from the expected future viewpoints of the driver, the latter will more easily understand the required maneuver. 3D city models can be reconstructed from the imagery recorded by surveying vehicles. The vastness of image material gathered by these vehicles, however, puts extreme demands on vision algorithms to ensure their practical usability. Algorithms need to be as fast as possible and should result in compact, memory efficient 3D city models for future ease of distribution and visualization. For the considered application, these are not contradictory demands. Simplified geometry assumptions can speed up vision algorithms while automatically guaranteeing compact geometry models. We present a novel city modeling framework which builds upon this philosophy to create 3D content at high speed which could allow for pre-visualization of any conceivable traffic situation by car navigation modules.

This paper describes the implementation of a LCD stripe projection system, based on a new processing scheme called line shift processing. One advantage of the method is, that the projection device can, but is not required to be stable over time nor does it have to be calibrated. The new method therefore allows us to use an off-the-shelf multimedia projector to acquire dense 3-D data (several 100K Triangles) of large objects in a matter of seconds. Since we are able to control our source of illumination, we can also acquire registered color information using standard monochrome cameras. Besides the description of the system some practical examples are given. The calibration of the system is also addressed.


Question: Making 3D Eyeglasses Try-on feasible and user-friendly.

This paper presents an innovative method for generating novel perspectives of three-dimensional objects from a solitary real-world image, a requirement in various domains such as computer graphics and object recognition. Traditionally, this task has been tackled using complex 3D models; however, our recent work [11, 6, 5] has introduced simpler techniques that operate under certain constraints. Our methodology hinges on image transformations that are tailored to specific object categories and can be trained using sample views of other objects within the same class. Building upon the concept of linear object classes initially proposed by Poggio and Vetter [12], we introduce an advanced technique in this paper. For linear object classes, we demonstrate that precise linear transformations can be derived from a set of 2D prototypical views. Our approach is first showcased on synthetic objects to illustrate its efficacy, followed by preliminary results indicating that the method can effectively "rotate" high-resolution face images, starting from a single 2D image. This development paves the way for practical applications in virtual try-on systems, particularly for 3D eyeglasses, enabling users to visualize how different frames would look on their face from multiple angles.

We propose a virtual try-on method based on generative adversarial networks (GANs). By considering clothing regions, this method enables us to reflect the pattern of clothes better than Conditional Analogy GAN (CAGAN), an existing virtual try-on method based on GANs. Our method first obtains the clothing region on a person by using a human parsing model learned with a large-scale dataset. Next, using the acquired region, the clothing part is removed from a human image. A desired clothing image is added to the blank area. The network learns how to apply new clothing to the area of people’s clothing. Results demonstrate the possibility of reflecting a clothing pattern. Furthermore, an image of the clothes that the person is originally wearing becomes unnecessary during testing. In experiments, we generate images using images gathered from Zaland (a fashion E-commerce site).

Volumetric 3D displays are frequently purported to lack the ability to reconstruct scenes with viewer-position-dependent effects such as occlusion. To counter these claims, a swept-screen 198-view horizontal-parallax-only 3D display is reported here that is capable of viewer-position-dependent effects. A digital projector illuminates a rotating vertical diffuser with a series of multiperspective 768 x 768 pixel renderings of a 3D scene. Evidence of near-far object occlusion is reported. The aggregate virtual screen surface for a stationary observer is described, as are guidelines to construct a full-parallax system and the theoretical ability of the present system to project imagery outside of the volume swept by the screen.

In this paper, we explore the challenges in applying and investigate methodologies to improve direct-touch interaction on intangible displays. Direct-touch interaction simplifies object manipulation, because it combines the input and display into a single integrated interface. While traditional tangible display-based direct-touch technology is commonplace, similar direct-touch interaction within an intangible display paradigm presents many challenges. Given the lack of tactile feedback, direct-touch interaction on an intangible display may show poor performance even on the simplest of target acquisition tasks. In order to study this problem, we have created a prototype of an intangible display. In the initial study, we collected user discrepancy data corresponding to the interpretation of 3D location of targets shown on our intangible display. The result showed that participants performed poorly in determining the z-coordinate of the targets and were imprecise in their execution of screen touches within the system. Thirty percent of positioning operations showed errors larger than 30mm from the actual surface. This finding triggered our interest to design a second study, in which we quantified task time in the presence of visual and audio feedback. The pseudo-shadow visual feedback was shown to be helpful both in improving user performance and satisfaction.

In augmented reality displays, X-Ray visualization techniques make hidden objects visible through combining the physical view with an artificial rendering of the hidden information. An important step in X-Ray visualization is to decide which parts of the physical scene should be kept and which should be replaced by overlays. The combination should provide users with essential perceptual cues to understand the relationship of depth between hidden information and the physical scene. In this paper we present an approach that addresses this decision in unknown environments by analyzing camera images of the physical scene and using the extracted information for occlusion management. Pixels are grouped into perceptually coherent image regions and a set of parameters is determined for each region. The parameters change the X-Ray visualization for either preserving existing structures or generating synthetic structures. Finally, users can customize the overall opacity of foreground regions to adapt the visualization.

We describe an augmented reality, optical see-through display based on a DMD chip with an extremely fast (16 kHz) binary update rate. We combine the techniques of post-rendering 2-D offsets and just-in-time tracking updates with a novel modulation technique for turning binary pixels into perceived gray scale. These processing elements, implemented in an FPGA, are physically mounted along with the optical display elements in a head tracked rig through which users view synthetic imagery superimposed on their real environment. The combination of mechanical tracking at near-zero latency with reconfigurable display processing has given us a measured average of 80 μs of end-to-end latency (from head motion to change in photons from the display) and also a versatile test platform for extremely-low-latency display systems. We have used it to examine the trade-offs between image quality and cost (i.e. power and logical complexity) and have found that quality can be maintained with a fairly simple display modulation scheme.

A number of software solutions for reconstructing 3D models from multi-view image sets have been released in recent years. Based on an unordered collection of photographs, most of these solutions extract 3D models using structure-from-motion (SFM) algorithms. In this work, we compare the resulting 3D models qualitatively and quantitatively. To achieve these objectives, we have developed different methods of comparison for all software solutions. We discuss the perfomance and existing drawbacks. Particular attention is paid to the ability to create printable 3D models or 3D models usable for other applications.

We present Gazture, a light-weight gaze based real-time gesture control system on commercial tablets. Unlike existing approaches that require dedicated hardware (e.g., high resolution camera), high computation overhead (powerful CPU) or specific user behavior (keeping head steady), Gazture provides gesture recognition based on easy-to-control user gaze input with a small overhead. To achieve this goal, Gazture incorporates a two-layer structure: The first layer focuses on real-time gaze estimation with acceptable tracking accuracy while incurring a small overhead. The second layer implements a robust gesture recognition algorithm while compensating gaze estimation error. To address user posture change while using mobile device, we design a online transfer function based method to convert current eye features into corresponding eye features in reference posture, which then facilitates efficient gaze position estimation. We implement Gazture on Lenovo Tab3 8 Plus tablet with Android 6.0.1, and evaluate its performance in different scenarios. The evaluation results show that Gazture can achieve a high accuracy in gesture recognition while incurring a low overhead.

In this paper, we introduce a real-time face recognition (and announcement) system targeted at aiding the blind and low-vision people. The system uses a Microsoft Kinect sensor as a wearable device, performs face detection, and uses temporal coherence along with a simple biometric procedure to generate a sound associated with the identified person, virtualized at his/her estimated 3-D location. Our approach uses a variation of the K-nearest neighbors algorithm over histogram of oriented gradient descriptors dimensionally reduced by principal component analysis. The results show that our approach, on average, outperforms traditional face recognition methods while requiring much less computational resources (memory, processing power, and battery life) when compared with existing techniques in the literature, deeming it suitable for the wearable hardware constraints. We also show the performance of the system in the dark, using depth-only information acquired with Kinect's infrared camera. The validation uses a new dataset available for download, with 600 videos of 30 people, containing variation of illumination, background, and movement patterns. Experiments with existing datasets in the literature are also considered. Finally, we conducted user experience evaluations on both blindfolded and visually impaired users, showing encouraging results.

This paper describes an easy-to-use system to estimate the shape of a human body and his/her clothes. The system uses a Kinect to capture the human's RGB and depth information from different views. Using the depth data, a non-rigid deformation method is devised to compensate motions between different views, thus to align and complete the dressed shape. Given the reconstructed dressed shape, the skin regions are recognized by a skin classifier from the RGB images, and these skin regions are taken as a tight constraints for the body estimation. Subsequently, the body shape is estimated from the skin regions of the dressed shape by leveraging a statistical model of human body. After the body estimation, the body shape is non-rigidly deformed to fit the dressed shape, so as to extract the cloth field of the dressed shape. We demonstrate our system and the therein algorithms by several experiments. The results show the effectiveness of the proposed method. & 2014 Elsevier B.V. All rights reserved.

We present a simple, fast solution for reflectance acquisition using tools that fit into a pocket. Our method captures video of a flat target surface from a fixed video camera lit by a hand-held, moving, linear light source. After processing, we obtain an SVBRDF.
 We introduce a BRDF chart, analogous to a color "checker" chart, which arranges a set of known-BRDF reference tiles over a small card. A sequence of light responses from the chart tiles as well as from points on the target is captured and matched to reconstruct the target's appearance.
 We develop a new algorithm for BRDF reconstruction which works directly on these LDR responses, without knowing the light or camera position, or acquiring HDR lighting. It compensates for spatial variation caused by the local (finite distance) camera and light position by warping responses over time to align them to a specular reference. After alignment, we find an optimal linear combination of the Lambertian and purely specular reference responses to match each target point's response. The same weights are then applied to the corresponding (known) reference BRDFs to reconstruct the target point's BRDF. We extend the basic algorithm to also recover varying surface normals by adding two spherical caps for diffuse and specular references to the BRDF chart.
 We demonstrate convincing results obtained after less than 30 seconds of data capture, using commercial mobile phone cameras in a casual environment.


Question: Making 3D eyeglasses try-on feasible in practical scenarios.

This article outlines a novel method for efficient object detection in images, capable of delivering high detection rates at remarkable speeds. The approach is characterized by three significant innovations. Firstly, it introduces the "Integral Image" representation, which enables the rapid calculation of the features used in the detection process. Secondly, it employs a learning algorithm, rooted in AdaBoost, to sift through a vast set of visual features, selecting only the most critical ones to produce highly efficient classifiers. Thirdly, it proposes a cascading mechanism for classifier combination, which allows for the quick elimination of non-object areas in the image, while allocating more computational resources to regions that are more likely to contain objects. This cascade acts as an object-specific focus-of-attention system, distinct from previous methods, as it provides statistical assurances that the discarded regions are unlikely to contain the object of interest. In the context of face detection, the system matches or surpasses the performance of leading systems. Remarkably, it operates in real-time, achieving 15 frames per second without relying on image differencing or skin color detection techniques. This method, therefore, represents a significant advancement in the field of visual object detection, offering both speed and accuracy in a single framework.

In this paper we show that misclassification attacks against face-recognition systems based on deep neural networks (DNNs) are more dangerous than previously demonstrated, even in contexts where the adversary can manipulate only her physical appearance (versus directly manipulating the image input to the DNN). Specifically, we show how to create eyeglasses that, when worn, can succeed in targeted (impersonation) or untargeted (dodging) attacks while improving on previous work in one or more of three facets: (i) inconspicuousness to onlooking observers, which we test through a user study; (ii) robustness of the attack against proposed defenses; and (iii) scalability in the sense of decoupling eyeglass creation from the subject who will wear them, i.e., by creating “universal” sets of eyeglasses that facilitate misclassification. Central to these improvements are adversarial generative nets, a method we propose to generate physically realizable attack artifacts (here, eyeglasses) automatically.

We explore the development of an experimental augmented reality application that provides benefits to professional mechanics performing maintenance and repair tasks in a field setting. We developed a prototype that supports military mechanics conducting routine maintenance tasks inside an armored vehicle turret, and evaluated it with a user study. Our prototype uses a tracked headworn display to augment a mechanic's natural view with text, labels, arrows, and animated sequences designed to facilitate task comprehension, localization, and execution. A within-subject controlled user study examined professional military mechanics using our system to complete 18 common tasks under field conditions. These tasks included installing and removing fasteners and indicator lights, and connecting cables, all within the cramped interior of an armored personnel carrier turret. An augmented reality condition was tested against two baseline conditions: the same headworn display providing untracked text and graphics and a fixed flat panel display representing an improved version of the laptop-based documentation currently employed in practice. The augmented reality condition allowed mechanics to locate tasks more quickly than when using either baseline, and in some instances, resulted in less overall head movement. A qualitative survey showed that mechanics found the augmented reality condition intuitive and satisfying for the tested sequence of tasks.

Funding information National Natural Science Foundation of China, Grant/Award Number: 61732015; National Key R&D Program of China, Grant/Award Number: 2017YFB1002600; Key Research and Development Program of Zhejiang Province, Grant/Award Number: 2018C01090; National Natural Science Foundation of China, Grant/Award Number: 61472355 Abstract We present a fast and automatic method to fit a given 3D garment onto a human model with various shapes and poses, without using a reference human model. Our approach uses a novel skeleton-based error metric to find the pose that best fits the input garment. Specifically, we first generate the skeleton of the given human model and its corresponding skinning weights. Then, we iteratively rotate each bone to find its best position to fit the garment. After that, we rig the surface of the human model according to the transformations of the skeleton. Potential penetrations are resolved using collision handling and physically based simulation. Finally, we restore the human model back to the original pose in order to obtain the desired fitting result. Our experiment results show that besides its efficiency and automation, our method is about two orders of magnitudes faster than existing approaches, and it can handle various garments, including jacket, trousers, skirt, a suit of clothing, and even multilayered clothing.

Typical stereo displays provide incorrect focus cues because the light comes from a single surface. We describe a prototype stereo display comprising two independent fixed-viewpoint volumetric displays. Like autostereoscopic volumetric displays, fixed-viewpoint volumetric displays generate near-correct focus cues without tracking eye position, because light comes from sources at the correct focal distances. (In our prototype, from three image planes at different physical distances.) Unlike autostereoscopic volumetric displays, however, fixed-viewpoint volumetric displays retain the qualities of modern projective graphics: view-dependent lighting effects such as occlusion, specularity, and reflection are correctly depicted; modern graphics processor and 2-D display technology can be utilized; and realistic fields of view and depths of field can be implemented. While not a practical solution for general-purpose viewing, our prototype display is a proof of concept and a platform for ongoing vision research. The design, implementation, and verification of this stereo display are described, including a novel technique of filtering along visual lines using 1-D texture mapping.

We present a method to acquire the reflectance field of a human face and use these measurements to render the face under arbitrary changes in lighting and viewpoint. We first acquire images of the face from a small set of viewpoints under a dense sampling of incident illumination directions using a light stage. We then construct a reflectance function image for each observed image pixel from its values over the space of illumination directions. From the reflectance functions, we can directly generate images of the face from the original viewpoints in any form of sampled or computed illumination. To change the viewpoint, we use a model of skin reflectance to estimate the appearance of the reflectance functions for novel viewpoints. We demonstrate the technique with synthetic renderings of a person's face under novel illumination and viewpoints.

This paper introduces a system for estimating the attention of a driver wearing a first person view camera using salient objects to improve gaze estimation. A challenging data set of pedestrians crossing intersections has been captured using Google Glass worn by a driver. A challenge unique to first person view from cars is that the interior of the car can take up a large part of the image. The proposed system automatically filters out the dashboard of the car, along with other parts of the instrumentation. The remaining area is used as a region of interest for a pedestrian detector. Two cameras looking at the driver are used to determine the direction of the driver's gaze, by examining the eye corners and the center of the iris. This coarse gaze estimation is then linked to the detected pedestrians to determine which pedestrian the driver is focused on at any given time.

We make the Texas 3D Face Recognition Database available to researchers in three dimensional (3D) face recognition and other related areas. This database contains 1149 pairs of high resolution, pose normalized, preprocessed, and perfectly aligned color and range images of 118 adult human subjects acquired using a stereo camera. The images are accompanied with information about the subjects' gender, ethnicity, facial expression, and the locations of 25 manually located anthropometric facial fiducial points. Specific partitions of the data for developing and evaluating 3D face recognition algorithms are also included.

In recent years, projection mapping has attracted much attention in a variety of fields. Generally, however, the objects in projection mapping are limited to rigid and static or quasistatic objects. Dynamic projection mapping onto a deformable object could remarkably expand the possibilities. In order to achieve such a projection mapping, it is necessary to recognize the deformation of the object even when it is occluded. However, it is still a challenging problem to achieve this task in real-time with low latency. In this paper, we propose an efficient, high-speed tracking method utilizing high-frame-rate imaging. Our method is able to track an array of dot markers arranged on a deformable object even when there is external occlusion caused by the user interaction and self-occlusion caused by the deformation of the object itself. Additionally, our method can be applied to a stretchable object. Dynamic projection mapping with our method showed robust and consistent display onto a sheet of paper and cloth with a tracking performance of about 0.2 ms per frame, with the result that the projected pattern appeared to be printed on the deformable object.

In Augmented Reality, see-through HMDs superimpose virtual 3D objects on the real world. This technology has the potential to enhance a user's perception and interaction with the real world. However, many Augmented Reality applications will not be accepted until we can accurately register virtual objects with their real counterparts. In previous systems, such registration was achieved only from a limited range of viewpoints, when the user kept his head still. This paper offers improved registration in two areas. First, our system demonstrates accurate static registration across a wide variety of viewing angles and positions. An optoelectronic tracker provides the required range and accuracy. Three calibration steps determine the viewing parameters. Second, dynamic errors that occur when the user moves his head are reduced by predicting future head locations. Inertial sensors mounted on the HMD aid head-motion prediction. Accurate determination of prediction distances requires low-overhead operating systems and eliminating unpredictable sources of latency. On average, prediction with inertial sensors produces errors 2-3 times lower than prediction without inertial sensors and 5-10 times lower than using no prediction at all. Future steps that may further improve registration are outlined.

Parameterized models can produce realistic, manipulable images of human faces— with a surprisingly small number of parameters.


Question: A deep semantic frame-based method for detecting deceptive opinion spam.

This paper focuses on identifying and detecting individuals who post spam reviews for products, referred to as review spammers. We have pinpointed several distinctive behaviors that are common among review spammers and have developed models to capture these behaviors for spammer detection. Specifically, we have modeled the following behaviors: Firstly, spammers often concentrate their efforts on particular products or product categories to maximize their influence. Secondly, their rating patterns tend to diverge significantly from those of genuine reviewers. To quantify the likelihood of a user being a spammer, we have devised scoring mechanisms and applied them to a dataset of reviews from Amazon. Subsequently, we have shortlisted a group of reviewers with the highest spam suspicion scores for detailed examination by our evaluators. This evaluation was facilitated by a custom-built web-based software designed specifically for conducting user evaluation experiments. Our findings reveal that our proposed ranking and supervised learning approaches are highly effective in unmasking spammers, outperforming a baseline method that relies solely on the helpfulness votes received by reviews. Additionally, we have demonstrated that the spammers we identified have a more substantial impact on product ratings than reviewers whose contributions are deemed unhelpful.

This paper presents the results of systematic experimentation on the impact in duplicate question detection of different types of questions across both a number of established approaches and a novel, superior one used to address this language processing task. This study permits to gain a novel insight on the different levels of robustness of the diverse detection methods with respect to different conditions of their application, including the ones that approximate real usage scenarios.

We present an effective end-to-end memory network model that jointly (i) predicts whether a given document can be considered as relevant evidence for a given claim, and (ii) extracts snippets of evidence that can be used to reason about the factuality of the target claim. Our model combines the advantages of convolutional and recurrent neural networks as part of a memory network. We further introduce a similarity matrix at the inference level of the memory network in order to extract snippets of evidence for input claims more accurately. Our experiments on a public benchmark dataset, FakeNewsChallenge, demonstrate the effectiveness of our approach.

The analysis of user generated content on social media and the accurate specification of user opinions towards products and events is quite valuable to many applications. With the proliferation of Web 2.0 and the rapid growth of user-generated content on the web, approaches on aspect level sentiment analysis that yield fine grained information are of great interest. In this work, a classifier ensemble approach for aspect based sentiment analysis is presented. The approach is generic and utilizes latent dirichlet allocation to model a topic and to specify the main aspects that users address. Then, each comment is further analyzed and word dependencies that indicate the interactions between words and aspects are extracted. An ensemble classifier formulated by naive bayes, maximum entropy and support vector machines is designed to recognize the polarity of the user's comment towards each aspect. The evaluation results show sound improvement compared to individual classifiers and indicate that the ensemble system is scalable and accurate in analyzing user generated content and in specifying users' opinions and attitudes.

Past works on personality detection has shown that psycho-linguistic features, frequency based analysis at lexical level , emotive words and other lexical clues such as number of first person or sec ond person words carry major role to identify personality associated with the te xt. In this work, we propose a new architecture for the same task using common sen se knowledge with associated sentiment polarity and affective labels. To extract the common sense knowledge with sentiment polarity scores and affect iv labels we used Senticnet which is one of the most useful resources for opinion mining and sentiment analysis. In particular, we combined comm n sense knowledge based features with phycho-linguistic features and freque ncy based features and later the features were employed in supervised classifier . We designed five SMO based supervised classifiers for five personality t raits. We observe that the use of common sense knowledge with affective and sentim ent information enhances the accuracy of the existing frameworks which use o nly psycho-linguistic features and frequency based analysis at lexical le ve .

Sentiment analysis has long been a hot topic in natural language processing. With the development of social network, sentiment analysis on social media such as Facebook, Twitter and Weibo becomes a new trend in recent years. Many different methods have been proposed for sentiment analysis, including traditional methods (SVM and NB) and deep learning methods (RNN and CNN). In addition, the latter always outperform the former. However, most of existing methods only focus on local text information and ignore the user personality and content characteristics. In this paper, we propose an improved LSTM model with considering the user-based features and content-based features. We first analysis the training dataset to extract artificial features which consists of user-based and content-based. Then we construct a hierarchical LSTM model, named LSTM-MF (a hierarchical LSTM model with multiple features), and introduce the features into the model to generate sentence and document representations. The experimental results show that our model achieves significant and consistent improvements compared to all state-of-the-art methods.

Opinion mining is a challenging task to identify the opinions or sentiments underlying user generated contents, such as online product reviews, blogs, discussion forums, etc. Previous studies that adopt machine learning algorithms mainly focus on designing effective features for this complex task. This paper presents our approach based on tree kernels for opinion mining of online product reviews. Tree kernels alleviate the complexity of feature selection and generate effective features to satisfy the special requirements in opinion mining. In this paper, we define several tree kernels for sentiment expression extraction and sentiment classification, which are subtasks of opinion mining. Our proposed tree kernels encode not only syntactic structure information, but also sentiment related information, such as sentiment boundary and sentiment polarity, which are important features to opinion mining. Experimental results on a benchmark data set indicate that tree kernels can significantly improve the performance of both sentiment expression extraction and sentiment classification. Besides, a linear combination of our proposed tree kernels and traditional feature vector kernel achieves the best performances using the benchmark data set.

We tackle Attitude Detection, which we define as the task of extracting the replier's attitude, i.e., a target-polarity pair, from a given one-round conversation. While previous studies considered Target Extraction and Polarity Classification separately, we regard them as subtasks of Attitude Detection. Our experimental results show that treating the two subtasks independently is not the optimal solution for Attitude Detection, as achieving high performance in each subtask is not sufficient for obtaining correct target-polarity pairs. Our jointly trained model AD-NET substantially outperforms the separately trained models by alleviating the target-polarity mismatch problem. Moreover, we proposed a method utilising the attitude detection model to improve retrieval-based chatbots by re-ranking the response candidates with attitude features. Human evaluation indicates that with attitude detection integrated, the new responses to the sampled queries from are statistically significantly more consistent, coherent, engaging and informative than the original ones obtained from a commercial chatbot.

A FUZZY LOGIC-BASED TEXT CLASSIFICATION METHOD FOR SOCIAL MEDIA

Phishing e-mails are considered as spam e-mails, which aim to collect sensitive personal information about the users via network. Since the main purpose of this behavior is mostly to harm users financially, it is vital to detect these phishing or spam e-mails immediately to prevent unauthorized access to users' vital information. To detect phishing e-mails, using a quicker and robust classification method is important. Considering the billions of e-mails on the Internet, this classification process is supposed to be done in a limited time to analyze the results. In this work, we present some of the early results on the classification of spam email using deep learning and machine methods. We utilize word2vec to represent emails instead of using the popular keyword or other rule-based methods. Vector representations are then fed into a neural network to create a learning model. We have tested our method on an open dataset and found over 96% accuracy levels with the deep learning classification methods in comparison to the standard machine learning algorithms.

Text-based deception detection is currently emerging as a vital multidisciplinary field due to its indisputable theoretical and practical value (police, security, and customs, including predatory communications, such as Internet scams). A very important issue associated with deception detection is designing valid text corpora. Most research has been carried out using texts produced in laboratory settings. There is a lack of "real" deceptive texts written when the stakes for deception are high as they are obviously difficult to collect and label. In addition, studies in text-based deception detection have mostly been performed for Romance and Germanic languages. There are few studies dealing with deception detection in Slavic languages. In this paper one can find an overview of available text corpora used for studying text-based deception detection as well as the description of how the first corpus of "real" deceptive texts for Slavic languages was collected and labeled. We expect this corpus once finished to be a handy tool for developing and testing new deception detection techniques and for carrying out related cross-cultural studies.


Question: A deep semantic frame-based method for analyzing deceptive opinion spam.

This paper introduces a method to combat web spam, utilizing a system called TrustRank. Web spammers employ various tactics to manipulate search engine rankings, often resulting in undeservedly high placements. Manual identification of spam pages by experts, while effective, is impractical due to the sheer volume of web content. To address this, we propose a semiautomated approach to distinguish between high-quality, reputable pages and spam. Initially, a small group of seed pages is selected for expert evaluation. Once the reputable seed pages are identified, we leverage the web's link structure to locate other pages that are likely to be trustworthy. The paper discusses various strategies for seed selection and the discovery of reputable pages. We present the outcomes of experiments conducted on the World Wide Web, using data indexed by AltaVista, to evaluate the efficacy of our techniques. Our findings reveal that a well-chosen seed set of fewer than 200 sites enables us to effectively filter out a substantial portion of web spam.

The proliferation of misleading information in everyday access media outlets such as social media feeds, news blogs, and online newspapers have made it challenging to identify trustworthy news sources, thus increasing the need for computational tools able to provide insights into the reliability of online content. In this paper, we focus on the automatic identification of fake content in online news. Our contribution is twofold. First, we introduce two novel datasets for the task of fake news detection, covering seven different news domains. We describe the collection, annotation, and validation process in detail and present several exploratory analyses on the identification of linguistic differences in fake and legitimate news content. Second, we conduct a set of learning experiments to build accurate fake news detectors, and show that we can achieve accuracies of up to 76%. In addition, we provide comparative analyses of the automatic and manual identification of fake news.

In the field of text mining the sentiment analysis is one of the current research topics. Sentiment analysis is the best solution for opinions and sentiment mining from natural language. Sentiment analysis gives significant information for decision making in various domains. The sentiments include ratings, reviews and emoticons. Several sentiment detection approaches are available which affect the quality of result. This paper finding the sentiments of people related to the services of E-shopping websites. The central goal is to recommend the products to users which are posted in E-shopping website and analyzing which one is the best product. The proposed system use stochastic learning algorithm which analyze various feedbacks related to the services and then the sentiments are classified as negative, positive and neutral. The pre-processing of the data is greatly affecting the quality of detected sentiments so that the fake reviews in the website are analyzed. Finally the classification technique can be used to analysis the products which are posted in E-shopping website. The proposed system find out fake reviews made by posting fake comments about a product by identifying the MAC address along with review posting patterns. In order to find out the review is fake or genuine the proposed system find out the MAC address of the user if the system observes fake review send by the same MAC address many a times it will inform the admin to remove that review from the system. The proposed system helps the user to find out correct review of the product.

The main task we address in our research is classification of text using fine-grained attitude labels. The developed @AM system relies on the compositionality principle and a novel approach based on the rules elaborated for semantically distinct verb classes. The evaluation of our method on 1000 sentences, that describe personal experiences, showed promising results: average accuracy on the finegrained level (14 labels) was 62%, on the middle level (7 labels) – 71%, and on the top level (3 labels) – 88%.

How can web services that depend on user generated content discern fraudulent input by spammers from legitimate input? In this paper we focus on the social network Facebook and the problem of discerning ill-gotten Page Likes, made by spammers hoping to turn a profit, from legitimate Page Likes. Our method, which we refer to as CopyCatch, detects lockstep Page Like patterns on Facebook by analyzing only the social graph between users and Pages and the times at which the edges in the graph (the Likes) were created. We offer the following contributions: (1) We give a novel problem formulation, with a simple concrete definition of suspicious behavior in terms of graph structure and edge constraints. (2) We offer two algorithms to find such suspicious lockstep behavior - one provably-convergent iterative algorithm and one approximate, scalable MapReduce implementation. (3) We show that our method severely limits "greedy attacks" and analyze the bounds from the application of the Zarankiewicz problem to our setting. Finally, we demonstrate and discuss the effectiveness of CopyCatch at Facebook and on synthetic data, as well as potential extensions to anomaly detection problems in other domains. CopyCatch is actively in use at Facebook, searching for attacks on Facebook's social graph of over a billion users, many millions of Pages, and billions of Page Likes.

Humans modify facial expressions in order to mislead observers regarding their true emotional states. Being able to recognize the authenticity of emotional displays is notoriously difficult for human observers. Evidence in experimental psychology shows that discriminative facial responses are short and subtle. This suggests that such behavior would be easier to distinguish when captured in high resolution at an increased frame rate. We are proposing SASE-FE, the first dataset of genuine and deceptive facial expressions of emotions for automatic recognition1 . We show that overall the problem of recognizing deceptive facial expressions can be successfully addressed by learning spatio-temporal representations of the data. For this purpose, we propose a method that aggregates features along fiducial trajectories in a deeply learnt feature space. Interesting additional results show that on average it is easier to distinguish among genuine expressions than deceptive ones and that certain emotion pairs are more difficult to distinguish than others.

The Web has become an excellent source for gathering consumer opinions (more specifically, consumer reviews) about products. Consumer reviews are essential for retailers and product manufacturers to understand the general responses of customers to their products and improve their marketing campaigns or products accordingly. In addition, consumer reviews enable retailers to recognize the specific preferences of each customer, which facilitates effective marketing decisions. As the number of consumer reviews expands, it is essential and desirable to develop an efficient and effective sentiment analysis technique that is capable of extracting product features stated in consumer reviews (i.e., product feature extraction) and determining the sentiments (positive or negative semantic orientations) of consumers for these product features (i.e., opinion orientation identification). Product feature extraction is critical to sentiment analysis, because its effectiveness significantly affects the performance of opinion orientation identification, as well as the ultimate effectiveness of sentiment analysis. Therefore, this study concentrates on product feature extraction from consumer reviews. C.-P. Wei (&) Institute of Service Science, College of Technology Management, National Tsing Hua University, Hsinchu, Taiwan, ROC e-mail: cpwei@mx.nthu.edu.tw Y.-M. Chen Department of Information Management, College of Management, National Sun Yat-Sen University, Kaohsiung, Taiwan, ROC C.-S. Yang Department of Information Management, College of Informatics, Yuan-Ze University, Chung-Li, Taiwan, ROC e-mail: csyang@saturn.yzu.edu.tw C. C. Yang College of Information Science and Technology, Drexel University, Philadelphia, PA, USA e-mail: chris.yang@ischool.drexel.edu 123 Inf Syst E-Bus Manage (2010) 8:149–167 DOI 10.1007/s10257-009-0113-9

Argumentation mining aims to automatically identify structured argument data from unstructured natural language text. This challenging, multifaceted task is recently gaining a growing attention, especially due to its many potential applications. One particularly important aspect of argumentation mining is claim identification. Most of the current approaches are engineered to address specific domains. However, argumentative sentences are often characterized by common rhetorical structures, independently of the domain. We thus propose a method that exploits structured parsing information to detect claims without resorting to contextual information, and yet achieve a performance comparable to that of state-of-the-art methods that heavily rely on the context.

In natural-language discourse, related events tend to appear near each other to describe a larger scenario. Such structures can be formalized by the notion of a frame (a.k.a. template), which comprises a set of related events and prototypical participants and event transitions. Identifying frames is a prerequisite for information extraction and natural language generation, and is usually done manually. Methods for inducing frames have been proposed recently, but they typically use ad hoc procedures and are difficult to diagnose or extend. In this paper, we propose the first probabilistic approach to frame induction, which incorporates frames, events, participants as latent topics and learns those frame and event transitions that best explain the text. The number of frames is inferred by a novel application of a split-merge method from syntactic parsing. In end-to-end evaluations from text to induced frames and extracted facts, our method produced state-of-the-art results while substantially reducing engineering effort.

Identifying trustworthy information in the presence of noisy data contributed by numerous unvetted sources from online social media (e.g., Twitter, Facebook, and Instagram) has been a crucial task in the era of big data. This task, referred to as truth discovery, targets at identifying the reliability of the sources and the truthfulness of claims they make without knowing either a priori. In this work, we identified three important challenges that have not been well addressed in the current truth discovery literature. The first one is “misinformation spread” where a significant number of sources are contributing to false claims, making the identification of truthful claims difficult. For example, on Twitter, rumors, scams, and influence bots are common examples of sources colluding, either intentionally or unintentionally, to spread misinformation and obscure the truth. The second challenge is “data sparsity” or the “long-tail phenomenon” where a majority of sources only contribute a small number of claims, providing insufficient evidence to determine those sources’ trustworthiness. For example, in the Twitter datasets that we collected during real-world events, more than 90% of sources only contributed to a single claim. Third, many current solutions are not scalable to large-scale social sensing events because of the centralized nature of their truth discovery algorithms. In this paper, we develop a Scalable and Robust Truth Discovery (SRTD) scheme to address the above three challenges. In particular, the SRTD scheme jointly quantifies both the reliability of sources and the credibility of claims using a principled approach. We further develop a distributed framework to implement the proposed truth discovery scheme using Work Queue in an HTCondor system. The evaluation results on three real-world datasets show that the SRTD scheme significantly outperforms the state-of-the-art truth discovery methods in terms of both effectiveness and efficiency.

Opinion mining is an interested area of research, which epitomize the customer reviews of a product or service and express whether the opinions are positive or negative. Various methods have been proposed as classifiers for opinion mining such as Naïve Bayesian, and Support vector machine, these methods classify opinion without giving us the reasons about why the instance opinion is classified to certain class. Therefore, in our work, we investigate opinion mining of Arabic text at the document level, by applying decision trees classification classifier to have clear, understandable rule, also we apply parallel decision trees classifiers to have efficient results. We applied parallel decision trees on two Arabic corpus of text documents by using parallel implementation of RapidMiner tools. In case of applying parallel decision tree family on OCA we get the best results of accuracy (93.83%), f-measure (93.22) and consumed time 42 Sec at thread 4, one of the resulted rule is Urdu language lines. In case of applying parallel decision tree family on BHA we get the best results of accuracy (90.63%), f-measure (82.29) and consumed time 219 Sec at thread 4, one of the resulted rule is Urdu language lines.


Question: A deep semantic frame approach to deceptive opinion spam analysis.

This research paper delves into the development of a comprehensive strategy for detecting deceptive opinion spam online, a critical issue given the significant impact user-generated reviews have on consumer purchasing decisions. The prevalence of deceptive opinion spam, characterized by fabricated reviews designed to appear genuine and mislead readers, has escalated concerns. To address this, we propose a novel methodology grounded in a meticulously curated gold standard dataset, encompassing three distinct domains: Hotel, Restaurant, and Doctor. Each domain features three review categories: genuine customer feedback, deceptive reviews crafted by Turkers, and deceptive reviews authored by domain experts (employees). Our analytical approach aims to discern the linguistic nuances distinguishing deceptive from authentic reviews. This distinction is crucial for empowering consumers to make informed purchase decisions and enabling review platform operators, such as TripAdvisor and Yelp, to detect and address potential fraud on their sites effectively.

**Question**: Understanding the Role of Social Media in Promoting Mental Health Awareness

**Documents**: Social Media as a Tool for Mental Health Awareness and Support: A Systematic Review
Social media platforms have become increasingly popular as tools for mental health awareness and support. This systematic review aims to explore the role of social media in promoting mental health awareness and providing support to individuals with mental health issues. We searched electronic databases for studies published between 2000 and 2020 that examined the use of social media for mental health purposes. A total of 45 studies met our inclusion criteria and were included in the review. The results showed that social media can be an effective tool for mental health awareness and support. Social media platforms can provide a space for individuals to share their experiences, seek and offer support, and access information about mental health. However, there are also potential risks associated with the use of social media for mental health, such as cyberbullying, misinformation, and the potential for social isolation. Overall, the findings suggest that social media can play a positive role in mental health awareness and support, but more research is needed to understand how to maximize its benefits while minimizing its risks.

**Similar Documents**: This systematic review investigates the utilization of social media as a pivotal resource for enhancing mental health awareness and delivering support to individuals grappling with mental health challenges. Given the escalating popularity of social media platforms, their potential in this context is noteworthy. The review encompasses a comprehensive analysis of studies published from 2000 to 2020, specifically focusing on the application of social media for mental health-related objectives. After a rigorous screening process, 45 studies were deemed eligible for inclusion. The

In this paper, we apply reinforcement learning for automatically learning cooperative persuasive dialogue system policies using framing, the use of emotionally charged statements common in persuasive dialogue between humans. In order to apply reinforcement learning, we describe a method to construct user simulators and reward functions specifically tailored to persuasive dialogue based on a corpus of persuasive dialogues between human interlocutors. Then, we evaluate the learned policy and the effect of framing through experiments both with a user simulator and with real users. The experimental evaluation indicates that applying reinforcement learning is effective for construction of cooperative persuasive dialogue systems which use framing.

In this paper, we propose a system for aspect-based sentiment analysis (ABSA) by incorporating the concepts of multi-objective optimization (MOO), distributional thesaurus (DT) and unsupervised lexical induction. The task can be thought of as a sequence of processes such as aspect term extraction, opinion target expression identification and sentiment classification. We use MOO for selecting the most relevant features, and demonstrate that classification with the resulting feature set can improve classification accuracy on many datasets. As base learning algorithms we make use of Support Vector Machines (SVM) for sentiment classification and Conditional Random Fields (CRF) for aspect term and opinion target expression extraction tasks. Distributional thesaurus and unsupervised DT prove to be effective with enhanced performance. Experiments on benchmark setups of SemEval-2014 and SemEval-2016 shared tasks show that we achieve the state of the art on aspect-based sentiment analysis for several languages.

We present a classifier to predict contextual polarity of subjective phrases in a sentence. Our approach features lexical scoring derived from the Dictionary of Affect in Language (DAL) and extended through WordNet, allowing us to automatically score the vast majority of words in our input avoiding the need for manual labeling. We augment lexical scoring with n-gram analysis to capture the effect of context. We combine DAL scores with syntactic constituents and then extract ngrams of constituents from all sentences. We also use the polarity of all syntactic constituents within the sentence as features. Our results show significant improvement over a majority class baseline as well as a more difficult baseline consisting of lexical n-grams.

Our daily lives have been immersed in widespread location-based social networks (LBSNs). As an open platform, LBSNs typically allow all kinds of users to register accounts. Malicious attackers can easily join and post misleading information, often with the intention of influencing users' decisions in urban computing environments. To provide reliable information and improve the experience for legitimate users, we design and implement DeepScan, a malicious account detection system for LBSNs. Different from existing approaches, DeepScan leverages emerging deep learning technologies to learn users' dynamic behavior. In particular, we introduce the long short-term memory (LSTM) neural network to conduct time series analysis of user activities. DeepScan combines newly introduced time series features and a set of conventional features extracted from user activities, and exploits a supervised machine-learning-based model for detection. Using real traces collected from Dianping, a representative LBSN, we demonstrate that DeepScan can achieve excellent prediction performance with an F1-score of 0.964. We also find that the time series features play a critical role in the detection system.

Nonparametric Bayesian Models for Joint Analysis of Imagery and Text

Identifying individual’s political ideology from their speeches and written texts is important for analyzing political opinions and user behavior on social media. Traditional opinion mining methods rely on bag-of-words representations to classify texts into different ideology categories. Such methods are too coarse for understanding political ideologies. The key to identify different ideologies is to recognize different opinions expressed toward a specific topic. To model this insight, we classify ideologies based on the distribution of opinions expressed towards real-world entities or topics. Specifically, we propose a novel approach to political ideology detection that makes predictions based on an opinion-aware knowledge graph. We show how to construct such graph by integrating the opinions and targeted entities extracted from text into an existing structured knowledge base, and show how to perform ideology inference by information propagation on the graph. Experimental results demonstrate that our method achieves high accuracy in detecting ideologies compared to baselines including LR, SVM and RNN.

Since Internet has become an excellent source of consumer reviews, the area of sentiment analysis (also called sentiment extraction, opinion mining, opinion extraction, and sentiment mining) has seen a large increase in academic interest over the last few years. Sentiment analysis mines opinions at word, sentence, and document levels, and gives sentiment polarities and strengths of articles. As known, the opinions of consumers are expressed in sentiment Chinese phrases. But due to the fuzziness of Chinese characters, traditional machine learning techniques can not represent the opinion of articles very well. In this paper, we propose a multi-strategy sentiment analysis method with semantic fuzziness to solve the problem. The results show that this hybrid sentiment analysis method can achieve a good level of effectiveness.

This paper proposes method for aspect identification and sentiment analysis. This paper deals with the problem of sentence level. We use semantic syntax tree to extract NP nodes, a method based on slack to identify aspect and mini-support function to narrow aspect; to analyze the sentiment for the aspect we use dictionary-based method. Experimental results show that the proposed approach achieves the state-of-the-art.

This paper gives a short overview of the state-of-the-art and goals of argumentation mining and it provides ideas for further research. Its content is based on two invited lectures on argumentation mining respectively at the FIRE 2013 conference at the India International Center in New Delhi, India and a lecture given as SICSA distinguished visitor at the University of Dundee, UK in the summer of 2014.

Sentiment analysis (SA) has become one of the most active and progressively popular areas in information retrieval and text mining due to the expansion of the World Wide Web (WWW). SA deals with the computational treatment or the classification of user’s sentiments, opinions and emotions hidden within the text. Aspect extraction is the most vital and extensively explored phase of SA to carry out the classification of sentiments in precise manners. During the last decade, enormous number of research has focused on identifying and extracting aspects. Therefore, in this survey, a comprehensive overview has been attempted for different aspect extraction techniques and approaches. These techniques have been categorized in accordance with the adopted approach. Despite being a traditional survey, a comprehensive comparative analysis is conducted among different approaches of aspect extraction, which not only elaborates the performance of any technique but also guides the reader to compare the accuracy with other state-of-the-art and most recent approaches.


Question: A deep semantic frame-based method for analyzing deceptive opinion spam.

This study delves into the methodologies potentially employed by Yelp's fake review filter, a critical mechanism in maintaining the integrity of online review platforms. Given the prevalence and impact of deceptive opinion spam, the need for effective detection systems is paramount. Yelp stands out as a platform that actively filters fake reviews, albeit the specifics of its algorithm remain a trade secret. Our investigation aims to uncover insights into Yelp's filtering strategies by scrutinizing the characteristics of the reviews it flags. We explore two primary filtering methodologies: supervised and unsupervised learning. Supervised learning, which we adopt for this study, leverages Yelp's filtered reviews for training purposes. Prior research in supervised learning has predominantly utilized pseudo fake reviews rather than actual reviews filtered by a commercial website. Notably, recent studies have highlighted the efficacy of linguistic n-gram features in identifying crowdsourced fake reviews created through platforms like Amazon Mechanical Turk (AMT), achieving accuracy rates of approximately 90%. However, when applying these established methods to Yelp's real-world data, we observe an unexpected trend: while behavioral features prove highly effective, linguistic features fall short in performance. To explore this discrepancy, we introduce an innovative information-theoretic analysis designed to pinpoint the psycholinguistic distinctions between AMT-generated reviews and those found on Yelp. Our findings reveal a significant difference in the nature of crowdsourced versus commercially-filtered fake reviews. These insights, coupled with our experimental outcomes, lead us to hypothesize that Yelp's filtering algorithm is indeed effective, targeting suspicious patterns of spamming behavior with notable precision.

People share their opinions about things like products, movies and services using social media channels. The analysis of these textual contents for sentiments is a gold mine for marketing experts, thus automatic sentiment analysis is a popular area of applied artificial intelligence. We propose a latent syntactic structure-based approach for sentiment analysis which requires only sentence-level polarity labels for training. Our experiments on three domains (movie, IT products, restaurant) show that a sentiment analyzer that exploits syntactic parses and has access only to sentence-level polarity annotation for in-domain sentences can outperform state-of-the-art models that were trained on out-domain parse trees with sentiment annotation for each node of the trees. In practice, millions of sentence-level polarity annotations are usually available for a particular domain thus our approach is applicable for training a sentiment analyzer for a new domain while it can exploit the syntactic structure of sentences as well.

The popularity of social media in recent years has increased a lot and created new opportunities to study the interactions of different groups of people. Because of this it comes to possible to study the user‟s opinion. Two popular topics in the study of social networks are community detection and finding trends. Trend Analysis is an ongoing field of research in Data mining field, May referred as an “Opinion Mining” by some researchers. Trend Analysis is the computational treatment of opinions, sentiments and subjectivity of text. The related field to Trend Analysis contains finding emotions in a statement which attracted researchers recently are discussed. The main target of this survey is to give nearly full image of Trend Analysis along with the community detection and their related fields with brief details. The main contributions of this paper include the overall analysis to bridge the concept between trend analyses with community detection. To find out the anomaly detection in a community with using the concept of trend analysis is a difficult task.

The publication and spread of misleading content is a problem of increasing magnitude, complexity and consequences in a world that is increasingly relying on user-generated content for news sourcing. To this end, multimedia analysis techniques could serve as an assisting tool to spot and debunk misleading content on the Web. In this paper, we tackle the problem of misleading multimedia content detection on Twitter in real time. We propose a number of new features and a new semi-supervised learning event adaptation approach that helps generalize the detection capabilities of trained models to unseen content, even when the event of interest is of different nature compared to that used for training. Combined with bagging, the proposed approach manages to outperform previous systems by a significant margin in terms of accuracy. Moreover, in order to communicate the verification process to end users, we develop a web-based application for visualizing the results.

Web 2.0 technologies have enabled more and more people to freely comment on different kinds of entities (e.g. sellers, products, services). The large scale of information poses the need and challenge of automatic summarization. In many cases, each of the user-generated short comments comes with an overall rating. In this paper, we study the problem of generating a ``rated aspect summary'' of short comments, which is a decomposed view of the overall ratings for the major aspects so that a user could gain different perspectives towards the target entity. We formally define the problem and decompose the solution into three steps. We demonstrate the effectiveness of our methods by using eBay sellers' feedback comments. We also quantitatively evaluate each step of our methods and study how well human agree on such a summarization task. The proposed methods are quite general and can be used to generate rated aspect summary automatically given any collection of short comments each associated with an overall rating.

The ability to analyze the adequacy of supporting information is necessary for determining the strength of an argument.1 This is especially the case for online user comments, which often consist of arguments lacking proper substantiation and reasoning. Thus, we develop a framework for automatically classifying each proposition as UNVERIFIABLE, VERIFIABLE NONEXPERIENTIAL, or VERIFIABLE EXPERIENTIAL2, where the appropriate type of support is reason, evidence, and optional evidence, respectively3. Once the existing support for propositions are identified, this classification can provide an estimate of how adequately the arguments have been supported. We build a goldstandard dataset of 9,476 sentences and clauses from 1,047 comments submitted to an eRulemaking platform and find that Support Vector Machine (SVM) classifiers trained with n-grams and additional features capturing the verifiability and experientiality exhibit statistically significant improvement over the unigram baseline, achieving a macro-averaged F1 of 68.99%.

Sentiment Analysis of tweets is a complex task, because these short messages employ unconventional language to increase the expressiveness. This task becomes even more difficult when people use figurative language (e.g. irony, sarcasm and metaphors) because it causes a mismatch between the literal meaning and the actual expressed sentiment. In this paper, we describe a sentiment analysis system designed for handling ironic and sarcastic tweets. Features grounded on several linguistic levels are proposed and used to classify the tweets in a 11-scale range, using a decision tree. The system is evaluated on the dataset released by the organizers of the SemEval 2015, task 11. The results show that our method largely outperforms the systems proposed by the participants of the task on ironic and sarcastic tweets.

The main task we address in our research is classification of text using fine-grained attitude labels. The developed @AM system relies on the compositionality principle and a novel approach based on the rules elaborated for semantically distinct verb classes. The evaluation of our method on 1000 sentences, that describe personal experiences, showed promising results: average accuracy on the finegrained level (14 labels) was 62%, on the middle level (7 labels) – 71%, and on the top level (3 labels) – 88%.

This paper introduces an approach to sentiment analysis which uses support vector machines (SVMs) to bring together diverse sources of potentially pertinent information, including several favorability measures for phrases and adjectives and, where available, knowledge of the topic of the text. Models using the features introduced are further combined with unigram models which have been shown to be effective in the past (Pang et al., 2002) and lemmatized versions of the unigram models. Experiments on movie review data from Epinions.com demonstrate that hybrid SVMs which combine unigram-style feature-based SVMs with those based on real-valued favorability measures obtain superior performance, producing the best results yet published using this data. Further experiments using a feature set enriched with topic information on a smaller dataset of music reviews handannotated for topic are also reported, the results of which suggest that incorporating topic information into such models may also yield improvement.

In this paper, we apply reinforcement learning for automatically learning cooperative persuasive dialogue system policies using framing, the use of emotionally charged statements common in persuasive dialogue between humans. In order to apply reinforcement learning, we describe a method to construct user simulators and reward functions specifically tailored to persuasive dialogue based on a corpus of persuasive dialogues between human interlocutors. Then, we evaluate the learned policy and the effect of framing through experiments both with a user simulator and with real users. The experimental evaluation indicates that applying reinforcement learning is effective for construction of cooperative persuasive dialogue systems which use framing.

We propose a novel data augmentation approach to enhance computational behavioral analysis using social media text. In particular, we collect a Twitter corpus of the descriptions of annoying behaviors using the #petpeeve hashtags. In the qualitative analysis, we study the language use in these tweets, with a special focus on the fine-grained categories and the geographic variation of the language. In quantitative analysis, we show that lexical and syntactic features are useful for automatic categorization of annoying behaviors, and frame-semantic features further boost the performance; that leveraging large lexical embeddings to create additional training instances significantly improves the lexical model; and incorporating frame-semantic embedding achieves the best overall performance.