Question: An overview of user needs for biometric verification on mobile devices.

An Exploration of Coefficient Alpha: Insights into Its Significance and Application in Psychological Research

Psychological studies that involve the creation of scales have been significantly impacted by a pervasive misunderstanding of coefficient alpha and the broader reliability theory. This paper aims to clarify the assumptions and implications of coefficient alpha, providing a deeper understanding of its role in research. We delve into the impact of test length and dimensionality on alpha through a series of calculations for hypothetical tests. These calculations vary the number of items, the number of orthogonal dimensions, and the average intercorrelation between items. Our findings offer guidelines for the appropriate use of coefficient alpha in psychological research, emphasizing the importance of considering these factors to ensure accurate and reliable results.

**Question**: An analysis of the impact of social media on mental health among teenagers

**Documents**: The Role of Social Support in the Mental Health of Teenagers: A Longitudinal Study
This study investigated the role of social support in the mental health of teenagers over a period of two years. Data were collected from 300 teenagers aged between 13 and 17 years, using a self-report questionnaire. The results showed that social support, particularly from family and friends, played a crucial role in maintaining good mental health among teenagers. Teenagers who reported higher levels of social support also reported lower levels of depression and anxiety.

**Similar Documents**: A Longitudinal Examination of Social Support's Influence on Adolescent Mental Health

This research explored the significance of social support in shaping the mental health of adolescents over a two-year timeframe. A sample of 300 teenagers, aged 13 to 17, participated in the study, completing a self-report questionnaire. The findings highlighted the critical importance of social support, especially from family and friends, in fostering mental well-being in this age group. Notably, adolescents who indicated strong social support networks also reported experiencing less depression and anxiety, underscoring the protective role of social connections in adolescent mental health.

This is the second of a series of papers describing the results of a project whose goal was to identify five remote identity proofing solutions that can be used as alternatives to knowledge-based verification. This paper describes the second solution, which makes use of a rich credential adapted for use on a blockchain and backed by a blockchain PKI. A rich credential, also used in Solution 1, allows the subject to identify him/herself to a remote verifier with which the subject has no prior relationship by presenting verification factors including possession of a private key, knowledge of a password, and possession of one or more biometric features, with selective disclosure of attributes and selective presentation of verification factors. In Solution 2 the issuer is a bank and the biometric verification factor is speaker recognition, which can be combined with face recognition to defeat voice morphing. The paper describes in detail the concept of a blockchain PKI, and shows that it has remarkable advantages over a traditional PKI, notably the fact that revocation checking is performed on the verifier’s local copy of the blockchain without requiring CRLs or OCSP.

Users have begun downloading an increasingly large number of mobile phone applications in response to advancements in handsets and wireless networks. The increased number of applications results in a greater chance of installing Trojans and similar malware. In this paper, we propose the Kirin security service for Android, which performs lightweight certification of applications to mitigate malware at install time. Kirin certification uses security rules, which are templates designed to conservatively match undesirable properties in security configuration bundled with applications. We use a variant of security requirements engineering techniques to perform an in-depth security analysis of Android to produce a set of rules that match malware characteristics. In a sample of 311 of the most popular applications downloaded from the official Android Market, Kirin and our rules found 5 applications that implement dangerous functionality and therefore should be installed with extreme caution. Upon close inspection, another five applications asserted dangerous rights, but were within the scope of reasonable functional needs. These results indicate that security configuration bundled with Android applications provides practical means of detecting malware.

Mobile devices are used more and more in everyday life. They are our cameras, wallets, and keys. Basically, they embed most of our private information in our pocket. For this and other reasons, mobile devices, and in particular the software that runs on them, are considered first-class citizens in the software-vulnerabilities landscape. Several studies investigated the software-vulnerabilities phenomenon in the context of mobile apps and, more in general, mobile devices. Most of these studies focused on vulnerabilities that could affect mobile apps, while just few investigated vulnerabilities affecting the underlying platform on which mobile apps run: the Operating System (OS). Also, these studies have been run on a very limited set of vulnerabilities. In this paper we present the largest study at date investigating Android-related vulnerabilities, with a specific focus on the ones affecting the Android OS. In particular, we (i) define a detailed taxonomy of the types of Android-related vulnerability, (ii) investigate the layers and subsystems from the Android OS affected by vulnerabilities, and (iii) study the survivability of vulnerabilities (i.e., the number of days between the vulnerability introduction and its fixing). Our findings could help OS and apps developers in focusing their verification & validation activities, and researchers in building vulnerability detection tools tailored for the mobile world.

In the past, most studies used the technology acceptance model (TAM) to survey the subjective perception of users in using information technology. The usability test was also used to assess the ease of use of user interfaces. This study introduces a conceptual framework to explore the relationship between user’s beliefs of TAM and usability testing attributes. Usability testing was conducted on an eCampus learning system with a mobile device. TAM data was collected from the participants for analyzing a possible relationship. The findings of this study reveal that TAM results contradict the usability test results in certain areas. The focus of our proposed research model is supported from the causality between perceived ease of use and usability; however, the correlation between perceived usefulness and usability remains unclear.

Security is a key concern in today’s mobile devices and a number of hardware implementations of security primitives have been proposed, including true random number generators, differential power attack avoidance, and chip-ID generators [1-4]. Recently, physically unclonable functions (PUFs) were proposed as a secure method for chip authentication in unsecure environments [5-7]. A PUF is a function that maps an input code (“challenge”) to an output code (“response”) in a manner that is unique for every chip. PUFs are increasingly used for IC authentication to offer protection against identity theft, cloning, and counterfeit components [2-4].

At the start of the second decade of 21th century, the time has come to make the Smart Houses a reallity for regular use. The different parts of a Smart House are researched but there are still distances from an applicable system, using the modern technology. In this paper we present an overview of the Smart House subsystems necessary for controlling the house using a mobile application efficiently and securely. The sequence diagram of the mobile application connectiing to the server application and also the usecases possible are presented. The challenges faced in designing the mobile application and illustrating the updated house top plane view in that application, are discussed and soloutions are adapted for it. Finally the designed mobile application was implemented and the important sections of it were described, such as the interactive house top view map which indicates the status of the devices using predefined icons. The facilities to manage the scheduled tasks and defined rules are also implemented in this mobile application that was developed for use in Windows Mobile platform. This application has the capability of connecting to the main server using GPRS mobile internet and SMS. This system is expected to be an important step towards a unified system structure that can be used efficiently in near future regular houses. Keywords—Smart House, Mobile Application, Remote Control, Automated Home, Windows Mobile.

The paper presents the setup and results of a mobile application usability test using mobile eye-tracking glasses. Quasi-real conditions of the natural environment were assured for the testing in order to avoid incidental disorders possible in natural conditions and the weaknesses of a fully lab experiment. The experiment methodology was developed and used to carry out a usability test of the mobile part of a Sale Force Automation (SFA) application in terms of its basic functionality. The paper's outlines are the following: the methodology used, arrangement and realisation of the usability testing experiment, test results and answers to the research questions. A list of general conclusions rounds off the report.

Manual voting system has been deployed for many yea rs in our country. However in many parts of our cou ntry people cannot attend the voting because of several r asons. In order to solve these problems there is a need of online election voting system in addition to manual voting system. The long-term goal of this project is to g reatly reduce the cost and complexity of running elections and increase th e accuracy of results by removing the direct involv ement of humans in gathering and counting of votes. The voters will be a le to give their votes at any field areas by usi ng the system if they prefer online voting In this system, the user votes using android applic ation which can be downloaded over internet. The us er has to login before giving his vote. Authentication will b e done using SMS confirmation and face recognition. Users can view parties, enter votes, can verify vote via SMS or email and can view result after result is declared.

Over the past few years, the evaluation of Electrocardio-graphic (ECG) signals as a prospective biometric modality has revealed promising results. Given the vital and continuous nature of this information source, ECG signals offer several advantages to the field of biometrics; yet, several challenges currently prevent the ECG from being adopted as a biometric modality in operational settings. These arise partially due to ECG signal's clinical tradition and intru-siveness, but also from the lack of evidence on the permanence of the ECG templates over time. The problem of in-trusiveness has been recently overcome with the “off-the-person” approach for capturing ECG signals. In this paper we provide an evaluation of the permanence of ECG signals collected at the fingers, with respect to the biometric authentication performance. Our experimental results on a small dataset suggest that further research is necessary to account for and understand sources of variability found in some subjects. Despite these limitations, “off-the-person” ECG appears to be a viable trait for multi-biometric or standalone biometrics, low user throughput, real-world scenarios.

With capacitive fingerprint readers being increasingly used for access control as well as for smartphone unlock and payments, there is a growing interest among metrology agencies (e.g., the National Institute of Standards and Technology) to develop standard artifacts (targets) and procedures for repeatable evaluation of capacitive readers. We present our design and fabrication procedures to create conductive 3D targets (gold fingers) for capacitive readers. Wearable 3D targets with known feature markings (e.g., fingerprint ridge flow and ridge spacing) are first fabricated using a high-resolution 3D printer. A sputter coating process is subsequently used to deposit a thin layer (~300 nm) of conductive materials (titanium and gold) on 3D printed targets. The wearable gold finger targets are used to evaluate a PIV-certified single-finger capacitive reader as well as small-area capacitive readers embedded in smartphones and access control terminals. In additional, we show that a simple procedure to create 3D printed spoofs with conductive carbon coating is able to successfully spoof a PIV-certified single-finger capacitive reader as well as a capacitive reader embedded in an access control terminal.


Question: An overview of user needs for biometric verification on mobile devices.

This study investigates the user experience and performance metrics associated with various biometric authentication methods on mobile devices, including voice, face, and gesture recognition, as well as traditional password entry. Through a controlled laboratory experiment, we observed user behaviors, strategies, and responses to different authentication techniques. The findings revealed that face and voice biometrics were quicker than password entry, with voice PINs being the fastest for biometric sample collection. However, short-term memory recall was more effective in the face verification scenario. Interestingly, combining two biometric methods for entry reduced the time required for biometric sample acquisition compared to using them separately. Despite this efficiency, combined methods were rated poorly by users and had high error rates in memory tasks. These empirical results highlight the distinct cognitive and motor demands of various biometric authentication modalities, providing valuable insights for policymakers when choosing appropriate authentication mechanisms.

One of the potential vulnerabilities in a biometric system is the leakage of biometric template information, which may lead to serious security and privacy threats. Most of the available template protection techniques fail to meet all the desired requirements of a practical biometric system like revocability, security, privacy, and high matching accuracy. In particular, protecting the fingerprint templates has been a difficult problem due to large intra-user variations (e.g., rotation, translation, nonlinear deformation, and partial prints). There are two fundamental challenges in any fingerprint template protection scheme. First, we need to select an appropriate representation scheme that captures most of the discriminatory information, but is sufficiently invariant to changes in finger placement and can be secured using available template protection algorithms. Secondly, we need to automatically align or register the fingerprints obtained during enrollment and matching without using any information that could reveal the features, which uniquely characterize a fingerprint. This chapter analyzes how these two challenges are being addressed in practice and how the design choices affect the trade-off between the security and matching accuracy. Though much progress has been made over the last decade, we believe that fingerprint template protection algorithms are still not sufficiently robust to be incorporated into practical fingerprint recognition systems. Anil K. Jain Department of Computer Science & Engineering, Michigan State University, East Lansing, MI 48824, USA and Department of Brain & Cognitive Engineering, Korea University, Seoul. e-mail: jain@cse.msu.edu, Karthik Nandakumar Institute for Infocomm Research, A*STAR, Fusionopolis, Singapore. e-mail: knandakumar@i2r.astar.edu.sg, Abhishek Nagar Department of Computer Science & Engineering, Michigan State University, East Lansing, MI 48824, USA. e-mail: nagarabh@cse.msu.edu

The objective of this paper is to design a bank locker security system which is using Face Recognition, Iris Scanner and Palm Vein Technology (PVR) for securing valuable belongings. A face recognition system is a system which identifies and authenticates the image of the authorized user by using MATLAB software. The images of a person entering a unrestricted zone are taken by the camera and software compares the image with an existing database of valid users. Iris Recognition system uses generous characteristics present in human body. This technology is employed for biometric authentication in ATM’s, Immigration & border control, public safety, hospitality and tourism etc. This paper serves techniques so that capability of palm vein recognition system can be improved using modified vascular pattern thinning algorithm. Palm Vein Recognition (PVR) is a technology which recognizes palm vein pattern of an individual and matches it with the data stored in database for authentication. This is a very reliable technique and has a very good accuracy and is considered as the most secure and better technique for security purposes.

Over the last ten years or so, mobile devices technology has changed significantly, with these devices and operating systems becoming more sophisticated. These developments have led to a huge variety of mobile applications designed for mobile operating systems. These mobile applications are typically harder to design and build because of several factors such as screen size and limited processing power and so forth. Therefore, it is important to clearly identify the characteristics of mobile application development and the issues and challenges related to it, as well as, the key features that characterize a great mobile application which make them valuable and useful. This paper has reviewed existing literature of the challenge and best practices of mobile application development. This study contributes towards a great understanding of the characteristics of mobile application development process, examines real challenges faced and explores the best practices that can be effectively applied to improve the development of mobile application.

Biometric systems are quite common in our everyday life. Despite the higher difficulty to circumvent them, nowadays criminals are developing techniques to accurately simulate physical, physiological, and behavioral traits of valid users, process known as spoofing attack. In this context, robust countermeasure methods must be developed and integrated with the traditional biometric applications in order to prevent such frauds. Despite face being a promising trait due to its convenience and acceptability, face recognition systems can be easily fooled with common printed photographs. Most of state-of-the-art antispoofing techniques for face recognition applications extract handcrafted texture features from images, mainly based on the efficient local binary patterns (LBP) descriptor, to characterize them. However, recent results indicate that high-level (deep) features are more robust for such complex tasks. In this brief, a novel approach for face spoofing detection that extracts deep texture features from images by integrating the LBP descriptor to a modified convolutional neural network is proposed. Experiments on the NUAA spoofing database indicate that such deep neural network (called LBPnet) and an extended version of it (n-LBPnet) outperform other state-of-the-art techniques, presenting great results in terms of attack detection.

Biometric recognition is an integral component of modern identity management and access control systems. Due to the strong and permanent link between individuals and their biometric traits, exposure of enrolled users' biometric information to adversaries can seriously compromise biometric system security and user privacy. Numerous techniques have been proposed for biometric template protection over the last 20 years. While these techniques are theoretically sound, they seldom guarantee the desired noninvertibility, revocability, and nonlinkability properties without significantly degrading the recognition performance. The objective of this work is to analyze the factors contributing to this performance divide and highlight promising research directions to bridge this gap. The design of invariant biometric representations remains a fundamental problem, despite recent attempts to address this issue through feature adaptation schemes. The difficulty in estimating the statistical distribution of biometric features not only hinders the development of better template protection algorithms but also diminishes the ability to quantify the noninvertibility and nonlinkability of existing algorithms. Finally, achieving nonlinkability without the use of external secrets (e.g., passwords) continues to be a challenging proposition. Further research on the above issues is required to cross the chasm between theory and practice in biometric template protection.

Mobile cloud computing is gaining popularity among mobile users. The ABI Research predicts that the number of mobile cloud computing subscribers is expected to grow from 42.8 million (1.1% of total mobile users) in 2008 to 998 million (19% of total mobile users) in 2014. Despite the hype achieved by mobile cloud computing, the growth of mobile cloud computing subscribers is still below expectations. According to the recent survey conducted by the International Data Corporation, most IT Executives and CEOs are not interested in adopting such services due to the risks associatedwith security and privacy. The security threats have become a hurdle in the rapid adaptability of the mobile cloud computing paradigm. Significant efforts have been devoted in research organizations and academia to build securemobile cloud computing environments and infrastructures. In spite of the efforts, there are a number of loopholes and challenges that still exist in the security policies of mobile cloud computing. This literature review: (a)highlights the current state of the artwork proposed to securemobile cloud computing infrastructures, (b) identifies the potential problems, and (c) provides a taxonomy of the state of the art. © 2012 Elsevier B.V. All rights reserved.

Mobile security has become more and more important due to the boom of mobile commerce (m-commerce). However, the development of m-commerce is facing many challenges regarding data security problems. Recently, blockchain has been introduced as an effective security solution deployed successfully in many applications in practice, such as, Bitcoin, cloud computing, and Internet-of-Things. In this paper, we introduce a new m-commerce application using blockchain technology, namely, MobiChain, to secure transactions in the m-commerce. Especially, in the MobiChain application, the mining processes can be executed efficiently on mobile devices using our proposed Android core module. Through real experiments, we evaluate the performance of the proposed model and show that blockchain will be an efficient security solution for future m-commerce.

Merchants have developed apps for the smartphone that assist consumers through the buying process, from when they gather information to when they decide to purchase. Sharing information, such as location, shopping preferences and financial data, can enhance consumers’ experience. However, they may be reluctant to disclose these details unless they perceive that the benefits gained are more than the risk of privacy loss. This privacy calculus is added to the unified theory of acceptance and use of technology (UTAUT2) in order to explain consumers’ willingness to exchange the disclosure of personal information for additional value. Sharing information makes mobile commerce more convenient by saving time and effort. Companies are able to send offers that are tailored to a specific customer. Payments are processed faster because the merchant already has the financial data on hand. UTAUT2 is further extended with the Theory of Convenience. Results from a survey of over 300 consumers show that perceived value and perceived convenience are influencing variables and that perceived value mediates the influence of perceived convenience on intention to use.

Despite on-going efforts over the past decade, mobile payments are yet to take off successfully. Repeated failures show that mobile payment platforms are complex to launch. The objective of this paper is to unveil factors explaining the failure of past mobile payment platforms. The use of a multi-level framework aims at enriching the variety of issues investigated. In order to explore the reasons of failure, we selected four cases from different countries happening at different times. Our results show that these cases share many of the same failure factors: lack of collaboration between the stakeholders, no technology standard, and low value-added for consumers and merchants compared to existing payment solutions. Even though the list is not exhaustive, the uncovered factors in our study seem to be necessary (but not sufficient) for the further developments of mobile payment platforms.

QR (Quick Response) codes are two-dimensional barcodes with the ability to encode different types of information. Because of their high information density and robustness, QR codes have gained popularity in various fields of application. Even though they offer a broad range of advantages, QR codes pose significant security risks. Attackers can encode malicious links that lead e.g. to phishing sites. Such malicious QR codes can be printed on small stickers and replace benign ones on billboard advertisements. Although many real world examples of QR code based attacks have been reported in the media, only little research has been conducted in this field and almost no attention has been paid on the interplay of security and human-computer interaction. In this work, we describe the manifold use cases of QR codes. Furthermore, we analyze the most significant attack scenarios with respect to the specific use cases. Additionally, we systemize the research that has already been conducted and identified usable security and security awareness as the main research challenges. Finally we propose design requirements with respect to the QR code itself, the reader application and usability aspects in order to support further research into to making QR code processing both secure and usable.


Question: A system for 3D visualization and control in an immersive environment utilizing a wireless bimanual gesture interface.

This paper introduces an innovative theoretical perspective on the asymmetry observed in human skilled bimanual actions. It highlights the common oversight in experimental studies of human manual lateralization, which often neglect the distinct roles played by each hand in most manual tasks. To address this gap, the concept of lateral preference is proposed as a more comprehensive alternative to the traditional notions of manual preference and manual superiority, which are primarily applicable to unimanual actions. Lateral preference encompasses the preference for one of the two possible role assignments to the two hands. A novel model is proposed to explain the favored intermanual division of labor in humans, characterized by the following principles: 1) The two hands are considered as two motors, with the complexity of individual hand movements being simplified in this context. 2) In humans, these two manual motors collaborate as if they are connected in series, forming a kinematic chain. This model posits that, in a right-handed individual acting according to their lateral preferences, the motion generated by the right hand integrates smoothly with that of the left hand. The kinematic chain model is posited as a tool to elucidate the adaptive benefits of human manual specialization, offering a fresh perspective on the evolutionary advantages of bimanual asymmetry in skilled actions.

Recently, there has been a substantial increase in efforts to develop 3-D visualization technologies that can provide the viewers with a realistic 3-D visual experience. Various terms such as “reality communication” have been used to categorize these efforts. In order to provide the viewers with a complete and realistic visual sensation, the display or visualization system and the displayed content need to match the physiological 3-D information sensing capabilities of the human visual system which can be quite complex. These may include spatial and temporal resolutions, depth perception, dynamic range, spectral contents, nonlinear effects, and vergence accommodation effects. In this paper, first we present an overview of some of the 3-D display research efforts which have been extensively pursued in Asia, Europe, and North America among other areas. Based on the limitations and comfort-based requirements of the human visual system when viewing a nonnatural visual input from 3-D displays, we present an analytical framework that combines main perception and human visual requirements with analytical tools and principles used in related disciplines such as optics, computer graphics, computational imaging, and signal processing. Building on the widely used notion of light fields, we define a notion of perceivable light fields to account for the human visual system physiological requirements, and propagate it back to the display device to determine the display device specifications. This helps us clarify the fundamental and practical requirements of the 3-D display devices for reality viewing communication. In view of the proposed analytical framework, we overview various methods that can be applied to overcome the extensive information needed to be displayed in order to meet the requirements imposed by the human visual system.

In this paper we present a markerless augmented reality (MAR) approach based on real-time 3D reconstruction using a low-cost depth camera, the Kinect. We intend to use this MAR approach for 3D medical visualization. In the actual state of developing, our approach is based on the KinectFusion algorithm with some minor adaptations, as long as our goal is not reconstruct scenes but use the reconstructed model as a reference to MAR. Keywords-Augmented Reality; 3D Reconstruction; Kinect;

In this work, we designed stylus type haptic interfaces interacting with a touch screen. There are two kinds of haptic styli termed wUbi-Pen. The type I has functions of providing vibration, impact, texture and sound and it is a stand alone system including own battery and wireless communication module. The type II does not include the texture display module and it is a miniaturized version. We present a new interaction scheme on the Windows graphical user interface based on pointer movement haptic feedback events such as clicking, drag, drop, moving and etc. In addition, a simple interactive digital sketchbook has been implemented, which is providing haptic and auditory feedback while drawing and touching objects. We also introduce a tactile image display method on a touch screen with the wUbi-Pen and a simple fitting puzzle utilizing haptic feedback events.

This paper presents a proposal for the use of Spatial Augmented Reality as a tool for 3D data visualizations. The use of traditional display technologies such as LCD monitors provides a fish tank metaphor for the user, i.e. the information is behind a plane of glass. The use of VR technologies allows the user to be immersed in the 3D volume and remove this fish tank problem, but can limit the set of techniques that allow users to interact with spatial data. Spatial Augmented Reality employed in conjunction with a highresolution monitor provides an elegant blend of spatial reasoning, tangible interaction, and detailed information viewing. This paper purposes a range of usages for SAR in 3D visualizations.

This paper contributes to the nascent body of literature on pointing performance in Virtual Environments (VEs), comparing gaze- and hand-based pointing. Contrary to previous findings, preliminary results indicate that gaze-based pointing is slower than hand-based pointing for distant objects.

This paper presents a new design of a wireless sensor glove developed for American Sign Language finger spelling gesture recognition. Five contact sensors are installed on the glove, in addition to five flex sensors on the fingers and a 3D accelerometer on the back of the hand. Each pair of flex and contact sensors are combined into the same input channel on the BSN node in order to save the number of channels and the installation area. After which, the signal is analyzed and separated back into flex and contact features by software. With electrical contacts and wirings made of conductive fabric and threads, the glove design has become thinner and more flexible. For validation, ASL finger spelling gesture recognition experiments have been performed on signals collected from six speech-impaired subjects and a normal subject. With the new sensor glove design, the experimental results have shown a significant increase in classification accuracy.

We describe a new set of interaction techniques that allow users to interact with physical objects through augmented reality (AR). Previously, to operate a smart device, physical touch is generally needed and a graphical interface is normally involved. These become limitations and prevent the user from operating a device out of reach or operating multiple devices at once. Ubii (Ubiquitous interface and interaction) is an integrated interface system that connects a network of smart devices together, and allows users to interact with the physical objects using hand gestures. The user wears a smart glass which displays the user interface in an augmented reality view. Hand gestures are captured by the smart glass, and upon recognizing the right gesture input, Ubii will communicate with the connected smart devices to complete the designated operations. Ubii supports common inter-device operations such as file transfer, printing, projecting, as well as device pairing. To improve the overall performance of the system, we implement computation offloading to perform the image processing computation. Our user test shows that Ubii is easy to use and more intuitive than traditional user interfaces. Ubii shortens the operation time on various tasks involving operating physical devices. The novel interaction paradigm attains a seamless interaction between the physical and digital worlds.

A major challenge for the realization of intelligent robots is to supply them with cognitive abilities in order to allow ordinary users to program them easily and intuitively. One way of such programming is teaching work tasks by interactive demonstration. To make this effective and convenient for the user, the machine must be capable to establish a common focus of attention and be able to use and integrate spoken instructions, visual perceptions, and non-verbal clues like gestural commands. We report progress in building a hybrid architecture that combines statistical methods, neural networks, and finite state machines into an integrated system for instructing grasping tasks by man-machine interaction. The system combines the GRAVIS-robot for visual attention and gestural instruction with an intelligent interface for speech recognition and linguistic interpretation, and an modality fusion module to allow multi-modal task-oriented man-machine communication with respect to dextrous robot manipulation of objects.

We describe the design and evaluation of a geometric interaction technique for bare-hand mid-air virtual pottery. We model the shaping of a pot as a gradual and progressive convergence of the potprofile to the shape of the user’s hand represented as a point-cloud (PCL). Our pottery-inspired application served as a platform for systematically revealing how users use their hands to express the intent of deformation during a pot shaping process. Through our approach, we address two specific problems: (a) determining start and end of deformation without explicit clutching and declutching, and (b) identifying user’s intent by characterizing grasp and motion of the hand on the pot. We evaluated our approach’s performance in terms of intent classification, users’ behavior, and users’ perception of controllability. We found that the expressive capability of hand articulation can be effectively harnessed for controllable shaping by organizing the deformation process in broad classes of intended operations such as pulling, pushing and fairing. After minimal practice with the pottery application, users could figure out their own strategy for reaching, grasping and deforming the pot. Further, the use of PCL as mid-air input allows for using common physical objects as tools for pot deformation. Users particularly enjoyed this aspect of our method for shaping pots.

Hand gestures are an important type of natural language used in many research areas such as human-computer interaction and computer vision. Hand gestures recognition requires the prior determination of the hand position through detection and tracking. One of the most efficient strategies for hand tracking is to use 2D visual information such as color and shape. However, visual-sensor-based hand tracking methods are very sensitive when tracking is performed under variable light conditions. Also, as hand movements are made in 3D space, the recognition performance of hand gestures using 2D information is inherently limited. In this article, we propose a novel real-time 3D hand tracking method in depth space using a 3D depth sensor and employing Kalman filter. We detect hand candidates using motion clusters and predefined wave motion, and track hand locations using Kalman filter. To verify the effectiveness of the proposed method, we compare the performance of the proposed method with the visual-based method. Experimental results show that the performance of the proposed method out performs visual-based method.


Question: A system for 3D visualization and control in an immersive environment utilizing an unconnected bimanual gesture interface.

This paper presents the Condensation algorithm, a groundbreaking approach to visual tracking in complex, cluttered environments. Traditional methods, such as Kalman filtering, often fall short in these scenarios due to their reliance on unimodal Gaussian densities, which cannot effectively represent multiple competing hypotheses. In contrast, Condensation employs a technique known as "factored sampling," initially developed for the analysis of static images. This method involves the creation of a randomly generated set to represent the probability distribution of various possible interpretations. By integrating learned dynamical models with visual observations, Condensation is capable of evolving this random set over time, resulting in remarkably resilient tracking of fast-moving objects. Despite the use of stochastic methods, the algorithm achieves near real-time performance, making it a highly practical solution for tracking agile motion in visually crowded scenes.

**Question**: A hybrid algorithm for solving the economic dispatch problem with valve-point effect

**Documents**: A Hybrid Genetic Algorithm and Tabu Search Approach to the Multiple-Choice Multidimensional Knapsack Problem
This paper presents a hybrid genetic algorithm and tabu search (GA-TS) approach to solve the multiple-choice multidimensional knapsack problem (MMKP). The GA-TS algorithm uses a genetic algorithm with a special mutation operator and a tabu search procedure to explore the solution space more effectively. The mutation operator is designed to select a random solution from the neighborhood of the current solution, while the tabu search procedure is used to avoid cycling and to escape from local optima. The GA-TS algorithm is tested on a set of benchmark problems and compared with other algorithms from the literature. The results show that the GA-TS algorithm is able to find high-quality solutions for the MMKP in a reasonable amount of time.

**Similar Documents**: This study introduces a novel hybrid algorithm that combines a genetic algorithm (GA) with tabu search (TS) to tackle the multiple-choice multidimensional knapsack problem (MMKP). The proposed GA-TS approach features a specialized mutation operator within the GA framework, along with an integrated TS procedure. The mutation operator is crafted to select a solution from the vicinity of the current solution, enhancing the exploration of the solution space. The TS component, on the other hand, is employed to prevent cycling and facilitate the algorithm's escape from local optima, ensuring a more thorough search. The effectiveness of the GA-TS algorithm is demonstrated through its application to a series of benchmark problems, with the results indicating its capability to locate high-quality solutions to the MMKP efficiently. Comparative analysis against other established algorithms

In the future, it will become more common for humans to team up with robotic systems to perform tasks that humans cannot realistically accomplish alone. Even for autonomous and semiautonomous systems, teleoperation will be an important default mode. However, teleoperation can be a challenging task because the operator is remotely located. As a result, the operator's situation awareness of the remote environment can be compromised and the mission effectiveness can suffer. This paper presents a detailed examination of more than 150 papers covering human performance issues and suggested mitigation solutions. The paper summarizes the performance decrements caused by video images bandwidth, time lags, frame rates, lack of proprioception, frame of reference, two-dimensional views, attention switches, and motion effects. Suggested solutions and their limitations include stereoscopic displays, synthetic overlay, multimodal interfaces, and various predicative and decision support systems.

Recently, there has been a substantial increase in efforts to develop 3-D visualization technologies that can provide the viewers with a realistic 3-D visual experience. Various terms such as “reality communication” have been used to categorize these efforts. In order to provide the viewers with a complete and realistic visual sensation, the display or visualization system and the displayed content need to match the physiological 3-D information sensing capabilities of the human visual system which can be quite complex. These may include spatial and temporal resolutions, depth perception, dynamic range, spectral contents, nonlinear effects, and vergence accommodation effects. In this paper, first we present an overview of some of the 3-D display research efforts which have been extensively pursued in Asia, Europe, and North America among other areas. Based on the limitations and comfort-based requirements of the human visual system when viewing a nonnatural visual input from 3-D displays, we present an analytical framework that combines main perception and human visual requirements with analytical tools and principles used in related disciplines such as optics, computer graphics, computational imaging, and signal processing. Building on the widely used notion of light fields, we define a notion of perceivable light fields to account for the human visual system physiological requirements, and propagate it back to the display device to determine the display device specifications. This helps us clarify the fundamental and practical requirements of the 3-D display devices for reality viewing communication. In view of the proposed analytical framework, we overview various methods that can be applied to overcome the extensive information needed to be displayed in order to meet the requirements imposed by the human visual system.

The fundamental idea behind the three-dimensional display is to present the user with a perspective image which changes as he moves. The retinal image of the real objects which we see is, after all, only two-dimensional. Thus if we can place suitable two-dimensional images on the observer's retinas, we can create the illusion that he is seeing a three-dimensional object. Although stereo presentation is important to the three-dimensional illusion, it is less important than the change that takes place in the image when the observer moves his head. The image presented by the three-dimensional display must change in exactly the way that the image of a real object would change for similar motions of the user's head. Psychologists have long known that moving perspective images appear strikingly three-dimensional even without stereo presentation; the three-dimensional display described in this paper depends heavily on this "kinetic depth effect."

The Tele-immersive Dance Environment (TED) is a geographically distributed, real-time 3-D virtual room where multiple participants interact independent of physical distance. TED, a highly interactive collaborative environment, offers digital options with multiple viewpoints, enhancing the creative movement composition involved with dance choreography. A symbiotic relationship for creativity and design exists between dance artists and computer scientists as the tele-immersive environment is analyzed as a creativity and learning tool. We introduce the advancements of the interactive digital options, new interface developments, user study results, and the possibility of a computational model for human creativity through Laban Movement Analysis.

We describe OctoPocus, an example of a dynamic guide that combines on-screen feedforward and feedback to help users learn, execute and remember gesture sets. OctoPocus can be applied to a wide range of single-stroke gestures and recognition algorithms and helps users progress smoothly from novice to expert performance. We provide an analysis of the design space and describe the results of two experi-ments that show that OctoPocus is significantly faster and improves learning of arbitrary gestures, compared to con-ventional Help menus. It can also be adapted to a mark-based gesture set, significantly improving input time compared to a two-level, four-item Hierarchical Marking menu.

Virtual reality systems would benefit from a compelling force sensory substitute when workspace or stability limitations prevent the use of kinesthetic force feedback systems. We present a wearable fingertip haptic device with the ability to make and break contact in addition to rendering both shear and normal skin deformation to the fingerpad. A delta mechanism with novel bias spring and tether actuator relocation method enables the use of high-end motors and encoders, allowing precise device control: 10 Hz bandwidth and 0.255 mm RMS tracking error were achieved during testing. In the first of two experiments, participants determined the orientation of a stiff region in a surrounding compliant virtual surface with an average angular error of 7.6 degree, similar to that found in previous studies using traditional force feedback. In the second experiment, we evaluated participants’ ability to interpret differences in friction. The Just Noticeable Difference (JND) of surface friction coefficient discrimination using our skin deformation device was 0.20, corresponding with a reference friction coefficient of 0.5. While higher than that found using kinesthetic feedback, this demonstrates that users can perceive differences in surface friction without world-grounded kinesthetic forces. These experiments show that three DoF skin deformation enables both stiffness and friction discrimination capability in the absence of kinesthetic force feedback.

This paper proposes a Myo device that has electromyography (EMG) sensors for detecting electrical activities from different parts of the forearm muscles; it also has a gyroscope and an accelerometer. EMG sensors detect and provide very clear and important data from muscles compared with other types of sensors. The Myo armband sends data from EMG, gyroscope, and accelerometer sensors to a computer via Bluetooth and uses these data to control a virtual robotic arm which was built in Unity 3D. Virtual robotic arms based on EMG, gyroscope, and accelerometer sensors have different features. A robotic arm based on EMG is controlled by using the tension and relaxation of muscles. Consequently, a virtual robotic arm based on EMG is preferred for a hand amputee to a virtual robotic arm based on a gyroscope and an accelerometer

Driven by technological advancements, gesture interfaces have recently found their way into vehicular prototypes of various kind. Unfortunately, their application is less than perfect and detailed information about preferred gesture execution regions, spatial extent, and time behavior are not available yet. Providing car (interior) manufacturer with gesture characteristics would allow them to design future in-vehicle concepts in a way to not interfere with gestural interaction. To tackle the problem, this research aims as preliminary work for a later standardization of the diverse properties of gestures and gesture classes similarly to what is already standardized in norms such as ISO 3958/4040 for placement and reachability of traditional controls and indicators. We have set up a real driving experiment recording trajectories and time behavior of gestures related to car and media control tasks. Data evaluation reveals that most of the subjects perform gestures in the same region (bounded by a "triangle" steering wheel, rear mirror, and gearshift) and with similar spatial extent (on average below 2 sec.). The generated density plots can be further used for an initial discussion about gesture execution in the passenger compartment. The final aim is to propose a new standard on permitted gesture properties (time, space) in the car.

The 3D tracking of the articulation of human hands is a theoretically interesting and challenging problem with numerous and diverse applications [1]. Several attempts have been made to address the problem by considering markerless visual data. Existing approaches can be categorized into modeland appearance-based [1]. In this work, we propose a model-based approach to the problem (Fig. 1). We formulate it as an optimization problem that minimizes the discrepancy between the 3D structure and appearance of hypothesized 3D hand model instances, and actual visual observations. Optimization is performed with a variant of an existing stochastic optimization method (Particle Swarm Optimization PSO) [2]. The most computationally demanding parts of the process have been implemented to run efficiently on a GPU [3]. The resulting system performs hand articulations tracking at a rate of 15Hz.

This paper provides a first look at a telepresence system offering room-sized, fully dynamic real-time 3D scene capture and continuous-viewpoint head-tracked display on a life-sized tiled display wall. The system is an expansion of a previous system, based an array of commodity depth sensors. We describe adjustments and improvements made to camera calibration, sensor data processing, data merger, rendering, and display, as required to scale the earlier system to room-sized.


Question: A scheme for 3D visualization and handling in an immersive setting utilizing an untethered bimanual gestural interface.

This paper delves into the concept and realization of the CAVE Automatic Virtual Environment, a groundbreaking virtual reality and scientific visualization system. It highlights that while various systems meet some criteria of virtual reality (VR), few achieve the full spectrum. Flight simulators, for instance, offer vehicle tracking but not head tracking, and often lack binocular stereo capability. Omnimax theaters, known for their expansive field of view, sometimes in stereo, lack interactivity. Head-tracked monitors provide most VR elements except for a broad angle of view. Head-mounted displays (HMDs) and BOOMs, which move the display screens, can achieve VR by our definition. However, this paper focuses on the potential of large screen projection to create a VR experience. It elaborates on the CAVE system, demonstrating that projection technology, when applied to VR objectives, can match the quality of workstation screens in terms of resolution, color, and flicker-free stereo. The paper also addresses how this format minimizes the impact of common tracking and system latency issues. The off-axis perspective projection techniques are presented as simple and direct. The strategies for multi-screen stereo vision are outlined, along with past and current design hurdles. The paper discusses the pros and cons of the projection paradigm, analyzing the effect of tracking noise and delay on the user. It introduces successive refinement as an essential tool for scientific visualization in the VR context. The CAVE's use as a one-to-many presentation device for computational science data at SIGGRAPH '92 and Supercomputing '92 is also highlighted. The paper traces the history of VR, starting with Sutherland's concept of the "Ultimate Display" in 1965, followed by Jim Clark's development of a system that allowed wireframe graphics VR through a head-mounted BOOM-type display for his dissertation at the University of Utah in the late 1960s. The prevalent VR devices today, the HMD and BOOM, are mentioned, along with other notable systems like the monitor-based system showcased by Lipscomb at SIGGRAPH '91 and the Virtual Portal, a closet-sized three-wall projection-based system demonstrated by Deering at SIGGRAPH '92. The CAVE, a projection-based VR display, was also introduced at SIGGRAPH '92. The Virtual Portal and CAVE share a similar vision but employ distinct implementation methods.

We present a system for interactively browsing and exploring large unstructured collections of photographs of a scene using a novel 3D interface. Our system consists of an image-based modeling front end that automatically computes the viewpoint of each photograph as well as a sparse 3D model of the scene and image to model correspondences. Our photo explorer uses image-based rendering techniques to smoothly transition between photographs, while also enabling full 3D navigation and exploration of the set of images and world geometry, along with auxiliary information such as overhead maps. Our system also makes it easy to construct photo tours of scenic or historic locations, and to annotate image details, which are automatically transferred to other relevant images. We demonstrate our system on several large personal photo collections as well as images gathered from Internet photo sharing sites.

Children have great facility in the physical world, and can skillfully model in clay and draw expressive illustrations. Traditional digital modeling tools have focused on mouse, keyboard and stylus input. These tools may be complicated and difficult for young users to easily and quickly create exciting designs. We seek to bring physical interaction to digital modeling, to allow users to use existing physical objects as tangible building blocks for new designs. We introduce KidCAD a digital clay interface for children to remix toys. KidCAD allows children to imprint 2.5D shapes from physical objects into their digital models by deforming a malleable gel input device, deForm. Users can mashup existing objects, edit and sculpt or draw new designs on a 2.5D canvas using physical objects, hands and tools as well as 2D touch gestures. We report on a preliminary user study with 13 children, ages 7 to 10, which provides feedback for our design and helps guide future work in tangible modeling for children.

The task of managing and accessing large information spaces is a problem in large scale cognition. Emerging technologies for 3D visualization and interactive animation offer potential solutions to this problem, especially when the structure of the information can be visualized. We describe one of these Information Visualization techniques, called the Cone Tree, which is used for visualizing hierarchical information structures. The hierarchy is presented in 3D to maximize effective use of available screen space and enable visualization of the whole structure. Interactive animation is used to shift some of the user's cognitive load to the human perceptual system.

This paper presents an augmented reality based system for hand rehabilitation. In this system, digital data gloves are used to detect the movements of the patients' hands and collect the physical information from the patients. Using augmented reality technology, a highly controllable environment with tasks of different levels of difficulty is provided to the patients for them to perform the rehabilitation exercises gradually. The targets of the exercises can be adjusted dynamically with respect to the physical conditions and progress of the patients. Multimodal feedbacks are provided to facilitate and encourage the patients during the rehabilitation sessions.

tures throughout our history. The paintings at Niaux, Altamira, and Lascaux (Clottes, 1995; Ruspoli, 1986), for example, are known to be about 14,000 years old, but with the recently discovered paintings in the Grotte Chauvet, the origin of representational art appears to have been pushed back even further (Chauvet, Brunel Deschamps, & Hillaire, 1995; Clottes, 1996), to 20,000 years ago if not longer.1 Thus, these paintings date from about the time at which homo sapiens sapiens first appeared in Europe (Nougier, 1969). We should remember these paintings in the context of virtual reality; our fascination with pictures is by no means recent. My intent is threefold: first, to discuss our perception of the cluttered layout, or space, that we normally find around us; second, to discuss the development of representational art up to our current appreciation of it; and third, to apply this knowledge to virtual reality systems. The first discussion focuses on the use of multiple sources of information specifying ordinal depth relations, within the theoretical framework that I have called directed perception (Cutting, 1986, 1991). The second discussion is embedded within the first, but it is steeped in neither history nor art history; instead, it is offered through the peculiar eyes of optics and psychology, particularly psychophysics. The third is addressed to one of the pressing problems of graphics and of virtual reality—how we perceive the layout of the environments that we simulate.

Recently, there has been a substantial increase in efforts to develop 3-D visualization technologies that can provide the viewers with a realistic 3-D visual experience. Various terms such as “reality communication” have been used to categorize these efforts. In order to provide the viewers with a complete and realistic visual sensation, the display or visualization system and the displayed content need to match the physiological 3-D information sensing capabilities of the human visual system which can be quite complex. These may include spatial and temporal resolutions, depth perception, dynamic range, spectral contents, nonlinear effects, and vergence accommodation effects. In this paper, first we present an overview of some of the 3-D display research efforts which have been extensively pursued in Asia, Europe, and North America among other areas. Based on the limitations and comfort-based requirements of the human visual system when viewing a nonnatural visual input from 3-D displays, we present an analytical framework that combines main perception and human visual requirements with analytical tools and principles used in related disciplines such as optics, computer graphics, computational imaging, and signal processing. Building on the widely used notion of light fields, we define a notion of perceivable light fields to account for the human visual system physiological requirements, and propagate it back to the display device to determine the display device specifications. This helps us clarify the fundamental and practical requirements of the 3-D display devices for reality viewing communication. In view of the proposed analytical framework, we overview various methods that can be applied to overcome the extensive information needed to be displayed in order to meet the requirements imposed by the human visual system.

Actuated Interfaces are receiving a great deal of interest from the research community. The field can now present a range of point designs, illustrating the potential design space of Actuated Interfaces. However, despite the increasing interest in Actuated Interfaces, the research carried out is nevertheless primarily preoccupied with the technical challenges and potential application areas, rather than how users actually approach, experience, interpret and understand Actuated Interfaces. Based on three case studies, investigating how people experience Actuated Interfaces, we point to; magic, movement and ambiguity as fruitful perspectives for understanding users' experiences with Actuated Interfaces. The three perspectives are employed to reflect upon opportunities and challenges, as well as point to open questions and relevant areas for future research for Actuated Interfaces.

The fundamental idea behind the three-dimensional display is to present the user with a perspective image which changes as he moves. The retinal image of the real objects which we see is, after all, only two-dimensional. Thus if we can place suitable two-dimensional images on the observer's retinas, we can create the illusion that he is seeing a three-dimensional object. Although stereo presentation is important to the three-dimensional illusion, it is less important than the change that takes place in the image when the observer moves his head. The image presented by the three-dimensional display must change in exactly the way that the image of a real object would change for similar motions of the user's head. Psychologists have long known that moving perspective images appear strikingly three-dimensional even without stereo presentation; the three-dimensional display described in this paper depends heavily on this "kinetic depth effect."

Gesture recognition is a well-establish topic of research that is widely adopted for a broad range of applications. For instance, it can be exploited for the command of a smart environment without any remote control unit or even for the recognition of human activities from a set of video cameras deployed in strategic position. Many researchers working on assistive smart home, such as our team, believe that the intrusiveness of that technology will prevent the future adoption and commercialization of smart homes. In this paper, we propose a novel gesture recognition algorithm that is solely based on passive RFID technology. This technology enables the localization of small tags that can be embedded in everyday life objects (a cup or a book, for instance) while remaining non intrusive. However, until now, this technology has been largely ignored by researchers on gesture recognition, mostly because it is easily disturbed by noise (metal, human, etc.) and offer limited precision. Despite these issues, the localization algorithms have improved over the years, and our recent efforts resulted in a real-time tracking algorithm with a precision approaching 14cm. With this, we developed a gesture recognition algorithm able to perform segmentation of gestures and prediction on a spatio-temporal data series. Our new model, exploiting works on qualitative spatial reasoning, achieves recognition of 91%. Our goal is to ultimately use that knowledge for both human activity recognition and errors detection.

This thesis evaluates how a head mounted display (HMD) can be used to increase usability compared to existing computer programs that are used during maintenance work on vehicles. Problems identified during a case study in a vehicle workshop are first described. As an attempt to solve some of the identified problems a prototype application using a HMD was developed. The prototype application aids the user during troubleshooting of systems on the vehicle by leading the mechanic with textual information and augmented reality (AR). Assessment of the prototype application was done by comparing it to the existing computer program and measuring error rate and time to completion for a predefined task. Usability was also measured using the System Usability Scale. The assessment showed that HMDs can provide higher usability in terms of efficiency and satisfaction. Furthermore, the thesis describes and discusses other possibilities and limitations that usage of HMDs and AR can lead to that were identified both from theory and during implementation.


Question: A scheme for 3D visualization and handling in a submersed environment utilizing a wireless bimanual gestural interface.

In this work, we address the challenge of identifying and locating a described visual object within a real-world photograph. Central to our approach is the development of a detailed representation scheme for visual structures and the formulation of a matching metric to evaluate the accuracy of detection or correspondence. This metric serves as the foundation for assessing how well a given description aligns with the actual visual content in the photograph. By focusing on the representation and matching of pictorial structures, we aim to enhance the precision and reliability of object detection in photographic images. This involves not only defining a comprehensive framework for describing visual objects but also devising an effective method to quantitatively measure the similarity between these descriptions and the objects found in photographs. Our goal is to establish a robust system that can accurately locate and identify objects based on their visual characteristics, thereby improving the overall performance of detection algorithms in real-world scenarios.

The synthesis of human hand motion and grasping of arbitrary shaped objects is a very complex problem. Therefore high-level control is needed to perform these actions. In order to satisfy the kinematic and physical constraints associated with the human hand and to reduce the enormous search space associated with the problem of grasping objects, a knowledge based approach is used. A three-phased scheme is presented which incorporates the role of the hand, the object, the environment and the animator. The implementation of a hand simulation system HANDS is discussed.

During the usage of conventional stereo microscope, repetitive head movement are often inevitable for the users to retrieve information away from the microscope. Moreover, risks of surgeons loosing focus and increasing fatigue could also arise during microsurgeries. Therefore, Scope+, a stereoscopic video see-through augmented reality system was created not only to solve the problems mentioned above, but also to improve users stereo microscopic experience.

We describe OctoPocus, an example of a dynamic guide that combines on-screen feedforward and feedback to help users learn, execute and remember gesture sets. OctoPocus can be applied to a wide range of single-stroke gestures and recognition algorithms and helps users progress smoothly from novice to expert performance. We provide an analysis of the design space and describe the results of two experi-ments that show that OctoPocus is significantly faster and improves learning of arbitrary gestures, compared to con-ventional Help menus. It can also be adapted to a mark-based gesture set, significantly improving input time compared to a two-level, four-item Hierarchical Marking menu.

Short-range continuous-wave Doppler radar sensors have been mainly used for noncontact detection of various motions. In this paper, we investigate the feasibility to implement the function of a remote mouse, an input device of a computer, by recognizing human gestures based on a dual-channel Doppler radar sensor. Direct conversion architecture, symmetric subcarrier modulation, and bandpass sampling techniques are used to obtain a cost-effective solution. An arcsine algorithm and a motion imaging algorithm are proposed to linearly reconstruct the hand and finger motions in a 2-D plane from the demodulated Doppler phase shifts. Experimental results verified the effectiveness of the proposed architecture and algorithms. Different from the frequency-domain “micro-Doppler” approach, the proposed remote gesture recognition based on linear motion reconstruction is able to recognize definitive signatures for the corresponding motions, exhibiting promising potential in practical applications of human-computer interaction.

This paper introduces a new class of interactive interfaces that can be moved around to appear on ordinary objects and surfaces anywhere in a space. By dynamically adapting the form, function, and location of an interface to suit the context of the user, such steerable interfaces have the potential to offer radically new and powerful styles of interaction in intelligent pervasive computing spaces. We propose defining characteristics of steerable interfaces and present the first steerable interface system that combines projection, gesture recognition, user tracking, environment modeling and geometric reasoning components within a system architecture. Our work suggests that there is great promise and rich potential for further research on steerable interfaces.

Several common systems satisfy some but not all of the VR definition above. Flight simulators provide vehicle tracking, not head tracking, and do not generally operate in binocular stereo. Omnimax theaters give a large angle of view [8], occasionally in stereo, but are not interactive. Head-tracked monitors [4][6] provide all but a large angle of view. Head-mounted displays (HMD) [7][13] and BOOMs [9] use motion of the actual display screens to achieve VR by our definition. Correct projection of the imagery on large screens can also create a VR experience, this being the subject of this paper. This paper describes the CAVE (CAVE Automatic Virtual Environment) virtual reality/scientific visualization system in detail and demonstrates that projection technology applied to virtual-reality goals achieves a system that matches the quality of workstation screens in terms of resolution, color, and flicker-free stereo. In addition, this format helps reduce the effect of common tracking and system latency errors. The off-axis perspective projection techniques we use are shown to be simple and straightforward. Our techniques for doing multi-screen stereo vision are enumerated, and design barriers, past and current, are described. Advantages and disadvantages of the projection paradigm are discussed, with an analysis of the effect of tracking noise and delay on the user. Successive refinement, a necessary tool for scientific visualization, is developed in the virtual reality context. The use of the CAVE as a one-to-many presentation device at SIGGRAPH '92 and Supercomputing '92 for computational science data is also mentioned. Previous work in the VR area dates back to Sutherland [12], who in 1965 wrote about the “Ultimate Display.” Later in the decade at the University of Utah, Jim Clark developed a system that allowed wireframe graphics VR to be seen through a headmounted, BOOM-type display for his dissertation. The common VR devices today are the HMD and the BOOM. Lipscomb [4] showed a monitor-based system in the IBM booth at SIGGRAPH '91 and Deering [6] demonstrated the Virtual Portal, a closets ized three-wal l project ion-based system, in the Sun Microsystems' booth at SIGGRAPH '92. The CAVE, our projection-based VR display [3], also premiered at SIGGRAPH '92. The Virtual Portal and CAVE have similar intent, but different implementation schemes.

This paper reports on a method to perform robust visual relocalization between temporally separated sets of underwater images gathered by a robot. The place recognition and relocalization problem is more challenging in the underwater environment mainly due to three factors: 1) changes in illumination; 2) long-term changes in the visual appearance of features because of phenomena like biofouling on man-made structures and growth or movement in natural features; and 3) low density of visually salient features for image matching. To address these challenges, a patch-based feature matching approach is proposed, which uses image segmentation and local intensity contrast to locate salient patches and HOG description to make correspondences between patches. Compared to traditional point-based features that are sensitive to dramatic appearance changes underwater, patch-based features are able to encode higher level information such as shape or structure which tends to persist across years in underwater environments. The algorithm is evaluated on real data, from multiple years, collected by a Hovering Autonomous Underwater Vehicle for ship hull inspection. Results in relocalization performance across missions from different years are compared to other traditional methods.

As part of a continuous research and development of underwater robotics technology at ITB, a visionbased distance measurement system for an Unmanned Underwater vehicle (UUV) has been designed. The proposed system can be used to predict horizontal distance between underwater vehicle and wall in front of vehicle. At the same time, it can be used to predict vertical distance between vehicle and the surface below it as well. A camera and a single laser pointer are used to obtain data needed by our algorithm. The vision-based navigation consists of two main processes which are the detection of a laser spot using image processing and the calculation of the distance based on laser spot position on the image.

In this demo, we present LLAP, a hand tracking system that uses ultrasound to localize the hand of the user to enable device-free gesture inputs. LLAP utilizes speakers and microphones on Commercial-Off-The-Shelf (COTS) mobile devices to play and record sound waves that are inaudible to humans. By measuring the phase of the sound signal reflected by the hands or fingers of the user, we can accurately measure the gesture movements. With a single pair of speaker/microphone, LLAP can track hand movement with accuracy of 3.5 mm. For devices with two microphones, LLAP enables drawing-in-the air capability with tracking accuracy of 4.6 mm. Moreover, the latency for LLAP is smaller than 15 ms for both the Android and the iOS platforms so that LLAP can be used for real-time applications.

Demands on capturing dynamic scenes of underwater environments are rapidly growing. Passive stereo is applicable to capture dynamic scenes, however the shape with textureless surfaces or irregular reﬂections cannot be recovered by the technique. In our system, we add a pattern projector to the stereo camera pair so that artiﬁcial textures are augmented on the objects. To use the system at underwater environments, several problems should be compensated, i.e., refraction, disturbance by ﬂuctuation and bubbles. Further, since surface of the objects are interfered by the bubbles, projected patterns, etc., those noises and patterns should be removed from captured images to recover original texture. To solve these problems, we propose three approaches; a depth-dependent calibration, Convolutional Neural Network(CNN)-stereo method and CNN-based texture recovery method. A depth-dependent calibration I sour analysis to ﬁnd the acceptable depth range for approximation by center projection to ﬁnd the certain target depth for calibration. In terms of CNN stereo, unlike common CNN based stereo methods which do not consider strong disturbances like refraction or bubbles, we designed a novel CNN architecture for stereo matching using multi-scale information, which is intended to be robust against such disturbances. Finally, we propose a multi-scale method for bubble and a projected-pattern removal method using CNNs to recover original textures. Experimental results are shown to prove the effectiveness of our method compared with the state of the art techniques. Furthermore, reconstruction of a live swimming ﬁsh is demonstrated to conﬁrm the feasibility of our techniques.


Question: A system for 3D visualization and control in an immersive environment utilizing a wireless bimanual gesture interface.

In response to a request from the National Science Foundation's (NSF) Interactive Systems Program, an invitational workshop was convened on March 23-24, 1992, at the University of North Carolina at Chapel Hill. The workshop aimed to delineate and propose future research trajectories in the field of virtual environments (VE). Approximately 18 leading experts, alongside four NSF representatives, from various academic, industrial, and technical backgrounds participated. The workshop's schedule was structured to include full-group discussions as well as focused sessions on five core areas: Perception, Human-Machine Software Interface, Software, Hardware, and Applications. These areas served as the foundation for the subsequent recommendations. Additionally, two attendees developed a VE applications taxonomy, which is appended to the report. Key recommendations include:

1. The establishment of collaborative science-technology development initiatives across multiple sites in the U.S. to foster tighter integration between VE developers and scientists.
2. A focus on theoretical research to develop performance and task demand metrics within VE environments.
3. The identification of paradigmatic applications and theoretical inquiries that highlight the synergy between science and technology.

In the realm of audition and spatial sound, research should concentrate on individual differences in Head-Related Transfer Functions (HRTFs), critical cues for distance and externalization, and spectral cues for elevation enhancement and cone-of-confusion resolution. Head-motion and intersensory interaction and adaptation are also vital areas for investigation, with artificial enhancement of localization cues being a promising research direction. Simultaneously, there is a need for joint basic and applied research to develop perceptually-effective methods for simplifying sound synthesis techniques, enhancing algorithm efficiency for complex room modeling. Efforts should also continue toward creating more realistic acoustic environment models, leveraging more powerful hardware.

Regarding nonspeech audio, theoretical research should investigate lower-level sensory and higher-level cognitive factors in acoustic perceptual organization, emphasizing how acoustic parameters interact to facilitate the identification, segregation, and localization of multiple concurrent sources. Technological advancements should concentrate on hardware and software systems designed for real-time generation and control of acoustic information displays.

In the area of haptics, development should be encouraged for a range of computer-controlled mechanical devices for both scientific exploration of the human haptic system and as haptic interfaces for virtual environments and teleoperation. Collaborative research programs should be initiated to bridge the gap between engineers capable of constructing high-precision robotic devices and scientists proficient in conducting biomechanical and perceptual experiments with these devices. These recommendations are designed to guide future research and development efforts in virtual

WAVES, a Wearable Asymmetric Vibration Excitation System, is a novel wearable haptic device for presenting three dimensions of translation and rotation guidance cues. In contrast to traditional vibration feedback, which usually requires that users learn to interpret a binary cue, asymmetric vibrations have been shown to induce a pulling sensation in a desired direction. When attached to the fingers, a single voicecoil actuator presents a translation guidance cue and a pair of voicecoil actuators presents a rotation guidance cue. The directionality of mechanoreceptors in the skin led to our choice of the location and orientation of the actuators in order to elicit very strong sensations in certain directions. For example, users distinguished a "left" cue versus a "right" cue 94.5% of the time. When presented with one of six possible direction cues, users on average correctly identified the direction of translation cues 86.1% of the time and rotation cues 69.0% of the time.

We present FinDroidHR, a novel gesture input technique for off-the-shelf smartwatches. Our technique is designed to detect 10 hand gestures on the hand wearing a smartwatch. The technique is enabled by analysing features of the Photoplethysmography (PPG) signal that optical heart-rate sensors capture. In a study with 20 participants, we show that FinDroidHR achieves 90.55% accuracy and 90.73% recall. Our work is the first study to explore the feasibility of using optical sensors on the off-the-shelf wearable devices to recognise gestures. Without requiring bespoke hardware, FinDroidHR can be readily used on existing smartwatches.

This paper provides a first look at a telepresence system offering room-sized, fully dynamic real-time 3D scene capture and continuous-viewpoint head-tracked display on a life-sized tiled display wall. The system is an expansion of a previous system, based an array of commodity depth sensors. We describe adjustments and improvements made to camera calibration, sensor data processing, data merger, rendering, and display, as required to scale the earlier system to room-sized.

We present an approach for real-time, robust and accurate hand pose estimation from moving egocentric RGB-D cameras in cluttered real environments. Existing methods typically fail for hand-object interactions in cluttered scenes imaged from egocentric viewpoints—common for virtual or augmented reality applications. Our approach uses two subsequently applied Convolutional Neural Networks (CNNs) to localize the hand and regress 3D joint locations. Hand localization is achieved by using a CNN to estimate the 2D position of the hand center in the input, even in the presence of clutter and occlusions. The localized hand position, together with the corresponding input depth value, is used to generate a normalized cropped image that is fed into a second CNN to regress relative 3D hand joint locations in real time. For added accuracy, robustness and temporal stability, we refine the pose estimates using a kinematic pose tracking energy. To train the CNNs, we introduce a new photorealistic dataset that uses a merged reality approach to capture and synthesize large amounts of annotated data of natural hand interaction in cluttered scenes. Through quantitative and qualitative evaluation, we show that our method is robust to self-occlusion and occlusions by objects, particularly in moving egocentric perspectives.

Manipulation in micro or nanoscale with robotic manipulators under observation of electron microscopes is a widely used strategy for fabrication of nanodevices and nanoscale material property characterization. These types of manipulation systems can handle the relatively larger scale of objects. However, the complexity of manipulation increases highly for 3D manipulation. Since the manipulation system consists of multiple components including manipulator, microscope, and also some end-effector tools, a proper offline visualization of the system is necessary for operation. Therefore, we propose a web-based virtual interface between the user and the actual manipulator operated under digital microscope initially. It gives the operator 3D positional feedback from the virtual model by mapping data read during remote operation. The same interface is used for remote operation of the manipulator within the SEM chamber and a manipulation task is performed.

We present Gazture, a light-weight gaze based real-time gesture control system on commercial tablets. Unlike existing approaches that require dedicated hardware (e.g., high resolution camera), high computation overhead (powerful CPU) or specific user behavior (keeping head steady), Gazture provides gesture recognition based on easy-to-control user gaze input with a small overhead. To achieve this goal, Gazture incorporates a two-layer structure: The first layer focuses on real-time gaze estimation with acceptable tracking accuracy while incurring a small overhead. The second layer implements a robust gesture recognition algorithm while compensating gaze estimation error. To address user posture change while using mobile device, we design a online transfer function based method to convert current eye features into corresponding eye features in reference posture, which then facilitates efficient gaze position estimation. We implement Gazture on Lenovo Tab3 8 Plus tablet with Android 6.0.1, and evaluate its performance in different scenarios. The evaluation results show that Gazture can achieve a high accuracy in gesture recognition while incurring a low overhead.

The Kinect has primarily been used as a gesture-driven device for motion-based controls. To date, Kinect-based research has predominantly focused on improving tracking and gesture recognition across a wide base of users. In this paper, we propose to use the Kinect for biometrics; rather than accommodating a wide range of users we exploit each user's uniqueness in terms of gestures. Unlike pure biometrics, such as iris scanners, face detectors, and fingerprint recognition which depend on irrevocable biometric data, the Kinect can provide additional revocable gesture information. We propose a dynamic time-warping (DTW) based framework applied to the Kinect's skeletal information for user access control. Our approach is validated in two scenarios: user identification, and user authentication on a dataset of 20 individuals performing 8 unique gestures. We obtain an overall 4.14%, and 1.89% Equal Error Rate (EER) in user identification, and user authentication, respectively, for a gesture and consistently outperform related work on this dataset. Given the natural noise present in the real-time depth sensor this yields promising results.

This thesis presents WiTrack, a system that tracks the 3D motion of a user from the radio signals reflected off her body. It works even if the person is occluded from the WiTrack device or in a different room. WiTrack does not require the user to carry any wireless device, yet its accuracy exceeds current RF localization systems, which require the user to hold a transceiver. Empirical measurements with a WiTrack prototype show that, on average, it localizes the center of a human body to within a median of 10 to 13 cm in the x and y dimensions, and 21 cm in the z dimension. It also provides coarse tracking of body parts, identifying the direction of a pointing hand with a median of 11.20. WiTrack bridges a gap between RF-based localization systems which locate a user through walls and occlusions, and human-computer interaction systems like Kinect, which can track a user without instrumenting her body, but require the user to stay within the direct line of sight of the device. Thesis Supervisor: Dina Katabi Title: Professor of Computer Science and Engineering

The use of the commercial video games as rehabilitation tools, such as the Nintendo WiiFit, has recently gained much interest in the physical therapy arena. Motion tracking controllers such as the Nintendo Wiimote are not sensitive enough to accurately measure performance in all components of balance. Additionally, users can figure out how to "cheat" inaccurate trackers by performing minimal movement (e.g. wrist twisting a Wiimote instead of a full arm swing). Physical rehabilitation requires accurate and appropriate tracking and feedback of performance. To this end, we are developing applications that leverage recent advances in commercial video game technology to provide full-body control of animated virtual characters. A key component of our approach is the use of newly available low cost depth sensing camera technology that provides markerless full-body tracking on a conventional PC. The aim of this research was to develop and assess an interactive game-based rehabilitation tool for balance training of adults with neurological injury.

The Polhemus Isotrak is often used as an orientation and position tracking device in virtual reality environments. When it is used to dynamically determine the user’s viewpoint and line of sight ( e.g. in the case of a head mounted display) the noise and delay in its measurement data causes temporal-spatial distortion, perceived by the user as jittering of images and lag between head movement and visual feedback. To tackle this problem, we first examined the major cause of the distortion, and found that the lag felt by the user is mainly due to the delay in orientation data, and the jittering of images is caused mostly by the noise in position data. Based on these observations, a predictive Kalman filter was designed to compensate for the delay in orientation data, and an anisotropic low pass filter was devised to reduce the noise in position data. The effectiveness and limitations of both approaches were then studied, and the results shown to be satisfactory.