Question: Regarding the differential privacy of Thompson Sampling when used with a Gaussian prior.

In the realm of evolutionary computation and artificial intelligence, the concept of adaptation is paramount, as it underpins the survival and evolutionary success of organisms in nature. This foundational principle has been a focal point in the development of AI systems designed to emulate biological optimization processes. These systems aim to replicate the evolutionary strategies observed in nature, such as survival of the fittest, balancing exploration and exploitation, niche finding, and the robust adaptation to changing environments. The ability to self-regulate, self-repair, and self-reproduce are additional critical features that AI seeks to emulate.

Within the context of artificial intelligence, adaptation and modification are considered the core of intelligence. This perspective has led to the emergence of various AI branches beyond genetic algorithms (GAs). For instance, expert systems, which are rule-based, have been developed to mimic human decision-making processes. These systems aim to capture and apply human knowledge in specific domains, demonstrating AI's capability to perform tasks that humans excel at. The pursuit of giving computers common sense and enabling them to make simple decisions without explicit instructions is another key area of focus. This reflects the ongoing challenge in AI to create systems that can operate autonomously and make decisions based on a deep understanding of their environment.

In essence, the adaptation and modification capabilities that are fundamental to evolution in nature are being harnessed and integrated into AI systems. This approach not only enhances the performance and versatility of these systems but also brings them closer to achieving the level of intelligence and adaptability observed in living organisms. The quest to make computers do what they do in the movies or to replicate human cognitive abilities is driving the continuous evolution of AI technologies.

The energy industry has recently begun using smart meters to take fine-grained readings of energy usage. These smart meters enable flexible timeof-use billing, forecasting, and demand response, but they also raise serious user privacy concerns. We propose a novel technique for provably hiding sensitive power consumption information in the overall power consumption stream. Our technique relies on a rechargeable battery that is connected to the household’s power supply. This battery is used to modify the household’s power consumption by adding or subtracting noise (i.e., increasing or decreasing power consumption), in order to establish strong privacy guarantees in the sense of differential privacy. To achieve these privacy guarantees in realistic settings, we first investigate the influence of, and the interplay between, capacity and throughput bounds that batteries face in reality. We then propose an integrated method based on noise cascading that allows for recharging the battery on-the-fly so that differential privacy is retained, while adhering to capacity and throughput constraints, and while keeping the additional consumption of energy induced by our technique to a minimum.

Hardware enclaves such as Intel SGX are a promising technology to increase the security of databases outsourced to the cloud. These enclaves provide an execution environment isolated from the hypervisor/OS, and encryption of data in memory. However, for applications that use large amounts of memory—including most realistic databases—enclaves do not protect against access pattern leaks, where an attacker observes which locations in memory are accessed. The näıve way to address this issue, using Oblivious RAM (ORAM) primitives, adds prohibitive overhead. In this paper, we propose ObliDB, a database that co-designs both its data structures (e.g., oblivious B+ trees) and physical operators to accelerate oblivious relational queries, giving up to 329× speedup over näıve ORAM. On analytics workloads, ObliDB ranges from competitive to 19× faster than previous oblivious systems designed only for analytics, such as Opaque, and comes within 2.6× of Spark SQL. Moreover, ObliDB also supports point queries, insertions, and deletions with latencies of 3–10ms, which is 7–22× faster than previously published oblivious data structures, and makes ObliDB suitable for transactional workloads too. To our knowledge, ObliDB is the first general-purpose oblivious database to approach practical performance. PVLDB Reference Format: Saba Eskandarian and Matei Zaharia. An Efficient Oblivious Database for the Public Cloud. PVLDB, 11 (4): xxxx-yyyy, 2017. DOI: https://doi.org/TBD

Cheap ubiquitous computing enables the collection of massive amounts of personal data in a wide variety of domains. Many organizations aim to share such data while obscuring features that could disclose personally identifiable information. Much of this data exhibits weak structure (e.g., text), such that machine learning approaches have been developed to detect and remove identifiers from it. While learning is never perfect, and relying on such approaches to sanitize data can leak sensitive information, a small risk is often acceptable. Our goal is to balance the value of published data and the risk of an adversary discovering leaked identifiers. We model data sanitization as a game between 1) a publisher who chooses a set of classifiers to apply to data and publishes only instances predicted as non-sensitive and 2) an attacker who combines machine learning and manual inspection to uncover leaked identifying information. We introduce a fast iterative greedy algorithm for the publisher that ensures a low utility for a resource-limited adversary. Moreover, using five text data sets we illustrate that our algorithm leaves virtually no automatically identifiable sensitive instances for a state-of-the-art learning algorithm, while sharing over 93 percent of the original data, and completes after at most five iterations.

The contextual bandit literature has traditionally focused on algorithms that address the explorationexploitation tradeoff. In particular, greedy algorithms that exploit current estimates without any exploration may be sub-optimal in general. However, exploration-free greedy algorithms are desirable in practical settings where exploration may be costly or unethical (e.g., clinical trials). Surprisingly, we find that a simple greedy algorithm can be rate-optimal (achieves asymptotically optimal regret) if there is sufficient randomness in the observed contexts (covariates). We prove that this is always the case for a two-armed bandit under a general class of context distributions that satisfy a condition we term covariate diversity. Furthermore, even absent this condition, we show that a greedy algorithm can be rate-optimal with nonzero probability. Thus, standard bandit algorithms may unnecessarily explore. Motivated by these results, we introduce Greedy-First, a new algorithm that uses only observed contexts and rewards to determine whether to follow a greedy algorithm or to explore. We prove that this algorithm is rate-optimal without any additional assumptions on the context distribution or the number of arms. Extensive simulations demonstrate that Greedy-First successfully reduces experimentation and outperforms existing (exploration-based) contextual bandit algorithms such as Thompson sampling or UCB.

One of the major obstacles to using Bayesian methods for pattern recognition has been its computational expense. This thesis presents an approximation technique that can perform Bayesian inference faster and more accurately than previously possible. This method, "Expectation Propagation," unifies and generalizes two previous techniques: assumeddensity filtering, an extension of the Kalman filter, and loopy belief propagation, an extension of belief propagation in Bayesian networks. The unification shows how both of these algorithms can be viewed as approximating the true posterior distribution with a simpler distribution, which is close in the sense of KL-divergence. Expectation Propagation exploits the best of both algorithms: the generality of assumed-density filtering and the accuracy of loopy belief propagation. Loopy belief propagation, because it propagates exact belief states, is useful for limited types of belief networks, such as purely discrete networks. Expectation Propagation approximates the belief states with expectations, such as means and variances, giving it much wider scope. Expectation Propagation also extends belief propagation in the opposite direction-propagating richer belief states which incorporate correlations between variables. This framework is demonstrated in a variety of statistical models using synthetic and real-world data. On Gaussian mixture problems, Expectation Propagation is found, for the same amount of computation, to be convincingly better than rival approximation techniques: Monte Carlo, Laplace's method, and variational Bayes. For pattern recognition, Expectation Propagation provides an algorithm for training Bayes Point Machine classifiers that is faster and more accurate than any previously known. The resulting classifiers outperform Support Vector Machines on several standard datasets, in addition to having a comparable training time. Expectation Propagation can also be used to choose an appropriate feature set for classification, via Bayesian model selection. Thesis Supervisor: Rosalind Picard Title: Associate Professor of Media Arts and Sciences

We give concentration bounds for martingales that are uniform over finite times and extend classical Hoeffding and Bernstein inequalities. We also demonstrate our concentration bounds to be optimal with a matching anti-concentration inequality, proved using the same method. Together these constitute a finite-time version of the law of the iterated logarithm, and shed light on the relationship between it and the central limit theorem.

One main concern for individuals to participate in the data collection of personal location history records is the disclosure of their location and related information when a user queries for statistical or pattern mining results derived from these records. In this paper, we investigate how the privacy goal that the inclusion of one's location history in a statistical database with location pattern mining capabilities does not substantially increase one's privacy risk. In particular, we propose a differentially private pattern mining algorithm for interesting geographic location discovery using a region quadtree spatial decomposition to preprocess the location points followed by applying a density-based clustering algorithm. A differentially private region quadtree is used for both de-noising the spatial domain and identifying the likely geographic regions containing the interesting locations. Then, a differential privacy mechanism is applied to the algorithm outputs, namely: the interesting regions and their corresponding stay point counts. The quadtree spatial decomposition enables one to obtain a localized reduced sensitivity to achieve the differential privacy goal and accurate outputs. Experimental results on synthetic datasets are used to show the feasibility of the proposed privacy preserving location pattern mining algorithm.

We consider the problem of estimating the mean and covariance of a distribution from i.i.d. samples in the presence of a fraction of malicious noise. This is in contrast to much recent work where the noise itself is assumed to be from a distribution of known type. The agnostic problem includes many interesting special cases, e.g., learning the parameters of a single Gaussian (or finding the best-fit Gaussian) when a fraction of data is adversarially corrupted, agnostically learning mixtures, agnostic ICA, etc. We present polynomial-time algorithms to estimate the mean and covariance with error guarantees in terms of information-theoretic lower bounds. As a corollary, we also obtain an agnostic algorithm for Singular Value Decomposition.

During our digital social life, we share terabytes of information that can potentially reveal private facts and personality traits to unexpected strangers. Despite the research efforts aiming at providing efficient solutions for the anonymization of huge databases (including networked data), in online social networks the most powerful privacy protection is in the hands of the users. However, most users are not aware of the risks derived by the indiscriminate disclosure of their personal data. With the aim of fostering their awareness on private data leakage risk, some measures have been proposed that quantify the privacy risk of each user. However, these measures do not capture the objective risk of users since they assume that all user’s direct social connections are close (thus trustworthy) friends. Since this assumption is too strong, in this paper we propose an alternative approach: each user decides which friends are allowed to see each profile item/post and our privacy score is defined accordingly. We show that it can be easily computed with minimal user intervention by leveraging an active learning approach. Finally, we validate our measure on a set of real Facebook users.

This paper considers the optimal adaptive allocation of measurement effort for identifying the best among a finite set of options or designs. An experimenter sequentially chooses designs to measure and observes noisy signals of their quality with the goal of confidently identifying the best design after a small number of measurements. I propose three simple Bayesian algorithms for adaptively allocating measurement effort. One is Top-Two Probability sampling, which computes the two designs with the highest posterior probability of being optimal, and then randomizes to select among these two. One is a variant a top-two sampling which considers not only the probability a design is optimal, but the expected amount by which its quality exceeds that of other designs. The final algorithm is a modified version of Thompson sampling that is tailored for identifying the best design. I prove that these simple algorithms satisfy a strong optimality property. In a frequestist setting where the true quality of the designs is fixed, one hopes the posterior definitively identifies the optimal design, in the sense that that the posterior probability assigned to the event that some other design is optimal converges to zero as measurements are collected. I show that under the proposed algorithms this convergence occurs at an exponential rate, and the corresponding exponent is the best possible among all allocation rules.


Question: "Mask-Bot 2i": An interactive, customizable robotic head featuring swappable facial components.

The "Mask-Bot 2i" Project: Designing an Adaptive Robotic Head with Interchangeable Facial Features

In an effort to advance the capabilities of humanoid robots in social interaction, physical embodiment, and the integration of diverse sensory and motor systems, a cutting-edge project has been initiated, known as "Mask-Bot 2i." This innovative robot features an upper-torso design with an active, customisable robotic head that supports twenty-one degrees of freedom. A distinctive aspect of this robotic head is its ability to adopt interchangeable facial features, allowing for a dynamic range of expressions and appearances. This chapter delves into the methodology and research objectives that have driven the project, detailing its current status and long-term aspirations.

The "Mask-Bot 2i" has been equipped with an array of sensory systems, including visual, auditory, kinesthetic, and tactile inputs, to facilitate its interaction with the environment. The project highlights the implementation of sophisticated visual-motor routines, such as smooth-pursuit tracking, saccades, binocular vergence, and reflexes that mimic human vestibular-ocular and opto-kinetic responses. The robot also demonstrates advanced orientation behaviors and motor control techniques, which are essential for its interaction with the world around it.

In terms of social interaction, the "Mask-Bot 2i" has been programmed to engage in a variety of behaviors, such as pointing to visual targets, recognizing and responding to joint attention through face and eye finding, imitating head nods, and using expressive feedback to regulate interactions. These capabilities are crucial for enhancing the robot's social intelligence and its ability to communicate effectively with humans.

Looking ahead, the project envisions further research and development to refine the "Mask-Bot 2i's" embodied system, aiming to create a more comprehensive and sophisticated humanoid robot. This includes improving its cognitive processes, enhancing its social and emotional intelligence, and expanding its physical dexterity and adaptability. The ultimate goal is to create a robot that can seamlessly integrate into human environments, facilitating social and educational interactions, and contributing to the advancement of robotics technology in the realm of human-robot interaction.

Dante II is a unique walking robot that provides important insight into high-mobility robotic locomotion and remote robotic exploration. Dante II’s uniqueness stems from its combined legged and rappelling mobility system, its scanning-laser rangefinder, and its multilevel control scheme. In 1994 Dante II was deployed and successfully tested in a remote Alaskan volcano, as a demonstration of the fieldworthiness of these technologies. For more than five days the robot explored alone in the volcano crater using a combination of supervised autonomous control and teleoperated control. Human operators were located 120 km distant during the mission. This article first describes in detail the robot, support systems, control techniques, and user interfaces. We then describe results from the battery of field tests leading up to and including the volcanic mission. Finally, we put forth important lessons which comprise the legacy of this project. We show that framewalkers are appropriate for rappelling in severe terrain, though tether systems have limitations. We also discuss the importance of future “autonomous” systems to realize when they require human support rather than relying on humans for constant oversight. KEY WORDS—rappelling, walking robot, supervised autonomy, behavior-based control, terrain visualization, volcano exploration

This paper summarizes recent activities carried out for the development of an innovative anthropomorphic robotic hand called the DEXMART Hand. The main goal of this research is to face the problems that affect current robotic hands by introducing suitable design solutions aimed at achieving simplification and cost reduction while possibly enhancing robustness and performance. While certain aspects of the DEXMART Hand development have been presented in previous papers, this paper is the first to give a comprehensive description of the final hand version and its use to replicate humanlike grasping. In this paper, particular emphasis is placed on the kinematics of the fingers and of the thumb, the wrist architecture, the dimensioning of the actuation system, and the final implementation of the position, force and tactile sensors. The paper focuses also on how these solutions have been integrated into the mechanical structure of this innovative robotic hand to enable precise force and displacement control of the whole system. Another important aspect is the lack of suitable control tools that severely limits the development of robotic hand applications. To address this issue, a new method for the observation of human hand behavior during interaction with common day-to-day objects by means of a 3D computer vision system is presented in this work together with a strategy for mapping human hand postures to the robotic hand. A simple control strategy based on postural synergies has been used to reduce the complexity of the grasp planning problem. As a preliminary evaluation of the DEXMART Hand’s capabilities, this approach has been adopted in this paper to simplify and speed up the transfer of human actions to the robotic hand, showing its effectiveness in reproducing human-like grasping.

ARTICLE INFO In this system an electro-mechanical robot is designed and controlled using hand gesture in real time. The system is designed on microcontroller platform using Keil and MPLAB tools. Hand gesture recognition is done on the principle of resistance change sensed through flex sensor. These sensors are integrated in a hand gloves from which input to the system is given. The designed system is divided into two sections as transmitter and receiver. The transmitter section will be in hand gloves from which the data is sensed and processed through PIC16F7487 and send serially to the receiver section. RF technology is used to transmit the data at the receiver section at the frequency of 2.4 GHz. ARM 7 (LPC2148) processor is used to receive the data. Here from the received data, the character is predicted and matched with the closet character from which the character is identified and displayed on LCD. The various case studies is prepared for the designed system and tested in real time. The proposed system can be used for the various applications such as in unmanned machines, industries, handicapped personnel etc. Keywords— a Sensor gloves, flex sensors, PIC controller, ARM processor etc Article History Received:20 th September 2015 Received in revised form :

We present an ample description of a socially compliant mobile robotic platform, which is developed in the EU-funded project SPENCER. The purpose of this robot is to assist, inform and guide passengers in large and busy airports. One particular aim is to bring travellers of connecting flights conveniently and efficiently from their arrival gate to the passport control. The uniqueness of the project stems from the strong demand of service robots for this application with a large potential impact for the aviation industry on one side, and on the other side from the scientific advancements in social robotics, brought forward and achieved in SPENCER. The main contributions of SPENCER are novel methods to perceive, learn, and model human social behavior and to use this knowledge to plan appropriate actions in realtime for mobile platforms. In this paper, we describe how the project advances the fields of detection and tracking of individuals and groups, recognition of human social relations and activities, normative human behavior learning, socially-aware task and motion planning, learning socially annotated maps, and conducting empirical experiments to assess socio-psychological effects of normative robot behaviors.

BodyAvatar is a Kinect-based interactive system that allows users without professional skills to create freeform 3D avatars using body gestures. Unlike existing gesture-based 3D modeling tools, BodyAvatar centers around a first-person "you're the avatar" metaphor, where the user treats their own body as a physical proxy of the virtual avatar. Based on an intuitive body-centric mapping, the user performs gestures to their own body as if wanting to modify it, which in turn results in corresponding modifications to the avatar. BodyAvatar provides an intuitive, immersive, and playful creation experience for the user. We present a formative study that leads to the design of BodyAvatar, the system's interactions and underlying algorithms, and results from initial user trials.

Physiological responses during human-robots interaction are useful alternatives to subjective measures of uncanny feelings for nearly humanlike robots (uncanny valley) and comparable emotional responses between humans and robots (media equation). However, no studies have employed the easily accessible measure of pupillometry to confirm the uncanny valley and media equation hypotheses, evidence in favor of the existence of these hypotheses in interaction with emotional robots is scarce, and previous studies have not controlled for low level image statistics across robot appearances. We therefore recorded pupil size of 40 participants that viewed and rated pictures of robotic and human faces that expressed a variety of basic emotions. The robotic faces varied along the dimension of human likeness from cartoonish to humanlike. We strictly controlled for confounding factors by removing backgrounds, hair, and color, and by equalizing low level image statistics. After the presentation phase, participants indicated to what extent the robots appeared uncanny and humanlike, and whether they could imagine social interaction with the robots in real life situations. The results show that robots rated as nearly humanlike scored higher on uncanniness, scored lower on imagined social interaction, evoked weaker pupil dilations, and their emotional expressions were more difficult to recognize. Pupils dilated most strongly to negative expressions and the pattern of pupil responses across emotions was highly similar between robot and human stimuli. These results highlight the usefulness of pupillometry in emotion studies and robot design by confirming the uncanny valley and media equation hypotheses.

The Face Expression Based Emoticon Identification (FEBEI) system is an open source extension to the Tracker.js framework which converts a human facial expression to the best matching emoticon. The contribution of this project was to build this robust classifier which can identify facial expression in real time without any reliance on an external server or computation node. An entirely client-side JavaScript implementation has clear privacy benefits as well as the avoidance of any lag inherent in uploading and downloading images. We accomplished this by utilizing several computationally efficient methods. Tracking.js provided a Viola Jones based face detector which we used to pass facial images to our own implementation of an eigenemotion detection system which was trained to distinguish between happy and angry faces. We have implemented a similar eigenface classifier in python and have trained a Convoluted Neural Network (CNN) to classify emotions to provide a comparative view of its advantages. We aim to make FEBEI easily extendable so that the developer community will be able to add classifiers for more emoticon.

The paper describes experience with applying a user-centric design methodology in developing systems for human-robot teaming in Urban Search & Rescue. A human-robot team consists of several semi-autonomous robots (rovers/UGVs, microcopter/UAVs), several humans at an off-site command post (mission commander, UGV operators) and one on-site human (UAV operator). This system has been developed in close cooperation with several rescue organizations, and has been deployed in a real-life tunnel accident use case. The human-robot team jointly explores an accident site, communicating using a multi-modal team interface, and spoken dialogue. The paper describes the development of this complex socio-technical system per se, as well as recent experience in evaluating the performance of this system.

This paper introduces Dynamic 3D Printing, a fast and reconstructable shape formation system. Dynamic 3D Printing can assemble an arbitrary three-dimensional shape from a large number of small physical elements. Also, it can disassemble the shape back to elements and reconstruct a new shape. Dynamic 3D Printing combines the capabilities of 3D printers and shape displays: Like conventional 3D printing, it can generate arbitrary and graspable three-dimensional shapes, while allowing shapes to be rapidly formed and reformed as in a shape display. To demonstrate the idea, we describe the design and implementation of Dynablock, a working prototype of a dynamic 3D printer. Dynablock can form a three-dimensional shape in seconds by assembling 3,000 9 mm blocks, leveraging a 24 x 16 pin-based shape display as a parallel assembler. Dynamic 3D printing is a step toward achieving our long-term vision in which 3D printing becomes an interactive medium, rather than the means for fabrication that it is today. In this paper, we explore possibilities for this vision by illustrating application scenarios that are difficult to achieve with conventional 3D printing or shape display systems.

Based on the autonomous field robot BoniRob, which was developed for plant phenotyping, the authors have created an enhanced platform BoniRob (V2). The design of BoniRob (V2) considers two main improvements: At first the robustness for continuous outdoor use is increased and secondly the robot now makes available a reusable platform that can serve for multiple agricultural purposes. The combination of the platform BoniRob (V2) and application modules (“BoniRob-Apps”) is comparable to the classical combination of a tractor and an implement. In order to demonstrate this new concept, the authors have implemented three BoniRob-Apps, which can be integrated in BoniRob (V2) via defined mechanical, electrical and logical interfaces. The research project RemoteFarming.1 goes even beyond the accomplishment of a specific task by a robot and demonstrates the integration of the BoniRob (V2) platform into agricultural processes. For that purpose innovative agricultural engineering is combined with webbased communication technologies. The target is to develop a robotic mechanical weed control system which integrates a human user as remote worker in the process. The robotic weed control system is used for intra row weed treatment in carrots at BBCH-scales 10 to 20 in organic farming. This task is today conducted by hand. Introduction In recent years a variety of field robot prototypes have been proposed. They cover several applications, such as scouting and sampling, precise variable rate application of pesticides and fertilizers as well as autonomous weeding [1][2]. However, until now there is no commercial application of autonomous field operation in agriculture available. This is partly due to open issues regarding safety and legal requirements of autonomous operations [3].Another problem of the presented prototypes for autonomous field robots is their conception as single-purpose robots, despite there being several applications with market potential [6]. As the single-purpose concept leads typically would lead high costs due to a low workload over the year (specific tasks are required over only a short period of time), the systems are not economically feasible yet. To overcome this barrier a new concept for a multipurpose field robot is shown here. Multipurpose field robot platform BoniRob (V2) The BoniRob (V2) platform has been designed as an improved version of the BoniRob crop scout, which has been announced by the authors [5]. The BoniRob (V2) is shown in Fig 1. Fig.1: BoniRob (V2) during a RemoteFarming.1 field trial It was reengineered with focus on robustness and reusability. This manifests in increases of power supply by the electrical generator, continuous and peak torque at the wheel drives as well as chassis clearance (Table 1). Table 1: Technical figures of BoniRob (V2) BoniRob crop scout BoniRob (V2) platform Power of electric generator 2.0 kW 2.6 kW Battery’s capacity 110 Ah 170 Ah Continuous drive torque (per wheel) 55 Nm 70 Nm Peak drive torque (per wheel) 110 Nm 240 Nm As it can be seen in Fig.1, the height adjustment of the entire robot body was replaced by vertically fixed arms for each wheel, which can be rotated in the horizontal plane for adjustment of track width and centration of the robot body over the row to be treated. The hydraulic height adjustment was replaced by a manual, mechanical height-adjustment module frame. Furthermore, all other hydraulic drives were replaced by electrical systems. The BoniRob-App concept Using the multipurpose field robot platform BoniRob (V2) as a carrier, supplier and base for multiple BoniRob-Apps can be compared to the traditional combination of a single tractor with multiple implements. BoniRob-Apps can be integrated into the platform using defined mechanical, electrical and logical interfaces. The module frame of 0.77 m x 0.75 m in the centre of the robot allows mechanically attaching the BoniRob-App (Fig. 2). The mounting height can be adjusted in a range of 0.4 m. The electrical connection is set up using a custom plug. This provides energy supply to the BoniRob-App and different voltage levels (230V AC, 24 V DC, 12 V DC, 5 V DC). Moreover, it allows linking the emergency stop infrastructure of platform and App, such that even the App can stop the entire robot in case of emergency. Also, the custom plug sets up a Gigabit-Ethernet connection to the host-computer on BoniRob (V2). The entire communication infrastructure on BoniRob (V2) and between platform and the Apps is designed using Ethernet and ROS [4]. A sequence controller with 2 levels opens the way for distributed drive control of the robot. In the row the App (level 2 sequence controller) can access the drive control. On the headland the platform (level 1 sequence controller) holds the access to the drive control and runs the turn procedure of the robot. This Fig. 2: Multiple BoniRob-Apps can be used in the same platform allows switching the access to the drive depending on the state including letting the App control the drive train of the robot in properly defined situations. Again, this can be compared to a tractor-implement combination. In recent years in the field of Tractor Implement Management advances have been made towards control of the tractor by the implement in specific situations, because the implement and its sensors are closer to the process and more specifically adapted to it. This also shows that BoniRob-Apps are more than just tools to be used with the BoniRob (V2) platform but smart combinations of mechanical and electrical systems together with processand application-specific intelligence. BoniRob-Apps: Demonstrating examples In order to demonstrate the new concept, the authors implemented three BoniRob-Apps:


Question: "Mask-Bot 2i": An interactive, customizable robotic head featuring swappable facial components.

This paper outlines the development of an advanced 3D surface acquisition system utilizing structured light and readily available components. The system is centered around a novel technique known as line shift processing, which allows for the use of a standard multimedia projector as the projection device. A key benefit of this approach is that the projector does not need to be stable over time or undergo calibration, making it highly adaptable and cost-effective. With the ability to control the illumination source, the system can also capture registered color information alongside the 3D data, using standard monochrome cameras. The proposed method is capable of acquiring highly detailed 3D models (comprising several hundred thousand triangles) of large objects within seconds. The paper includes a description of the system's design, practical examples of its application, and details on the calibration process. This innovative system demonstrates the potential for using off-the-shelf components to create sophisticated 3D scanning technology.

This review investigates two recent developments in artificial intelligence and neural computation: learning from imitation and the development of humanoid robots. It is postulated that the study of imitation learning offers a promising route to gain new insights into mechanisms of perceptual motor control that could ultimately lead to the creation of autonomous humanoid robots. Imitation learning focuses on three important issues: efficient motor learning, the connection between action and perception, and modular motor control in the form of movement primitives. It is reviewed here how research on representations of, and functional connections between, action and perception have contributed to our understanding of motor acts of other beings. The recent discovery that some areas in the primate brain are active during both movement perception and execution has provided a hypothetical neural basis of imitation. Computational approaches to imitation learning are also described, initially from the perspective of traditional AI and robotics, but also from the perspective of neural network models and statistical-learning research. Parallels and differences between biological and computational approaches to imitation are highlighted and an overview of current projects that actually employ imitation learning for humanoid robots is given.

How will naive users program domestic robots? This paper describes the design of a practical system that uses natural language to teach a vision-based robot how to navigate in a miniature town. To enable unconstrained speech the robot is provided with a set of primitive procedures derived from a corpus of route instructions. When the user refers to a route that is not known to the robot, the system will learn it by combining primitives as instructed by the user. This paper describes the components of the Instruction Based Learning architecture and discusses issues of knowledge representation, the selection of primitives and the conversion of natural language into robot-understandable procedures.

The demands on both delivery methodology and content of control curriculum continue to evolve as more applications in our world incorporate control theory and the need for continuing education increases. Not only has the content evolved, but the application of social-behavioral science research has resulted in cooperative and active learning practices by faculty. In response to these shifts in education, an open-source inverted pendulum robot was used in a linear systems theory (LST) class taught as part of a professional master's program (PMP). The robot had to have a low cost, and enough capability to enable the students to explore and test the ideas presented in lecture and to engage in collaborative learning experiences. This paper discusses the robot, describes the key control theory experiments, and reviews the lessons learned from this experience.

This paper presents the EACare project, an ambitious multidisciplinary collaboration with the aim to develop an embodied system, capable of carrying out neuropsychological tests to detect early signs of dementia, e.g., due to Alzheimer’s disease. The system will use methods from Machine Learning and Social Robotics, and be trained with examples of recorded clinician-patient interactions. The interaction will be developed using a participatory design approach. We describe the scope and method of the project, and report on a first Wizard of Oz prototype.

Swarm systems consist of large numbers of robots that collaborate autonomously. With an appropriate level of human control, swarm systems could be applied in a variety of contexts ranging from search-and-rescue situations to Cyber defence. The two decision making cycles of swarms and humans operate on two different time-scales, where the former is normally orders of magnitude faster than the latter. Closing the loop at the intersection of these two cycles will create fast and adaptive humanswarm teaming networks. This paper brings desperate pieces of the ground work in this research area together to review this multidisciplinary literature. We conclude with a framework to synthesize the findings and summarize the multi-modal indicators needed for closed-loop human-swarm adaptive systems.

We present an ample description of a socially compliant mobile robotic platform, which is developed in the EU-funded project SPENCER. The purpose of this robot is to assist, inform and guide passengers in large and busy airports. One particular aim is to bring travellers of connecting flights conveniently and efficiently from their arrival gate to the passport control. The uniqueness of the project stems from the strong demand of service robots for this application with a large potential impact for the aviation industry on one side, and on the other side from the scientific advancements in social robotics, brought forward and achieved in SPENCER. The main contributions of SPENCER are novel methods to perceive, learn, and model human social behavior and to use this knowledge to plan appropriate actions in realtime for mobile platforms. In this paper, we describe how the project advances the fields of detection and tracking of individuals and groups, recognition of human social relations and activities, normative human behavior learning, socially-aware task and motion planning, learning socially annotated maps, and conducting empirical experiments to assess socio-psychological effects of normative robot behaviors.

Designing systems able to interact with students in a natural manner is a complex and far from solved problem. A key aspect of natural interaction is the ability to understand and appropriately respond to human emotions. This paper details our response to the continuous Audio/Visual Emotion Challenge (AVEC'12) whose goal is to predict four affective signals describing human emotions. The proposed method uses Fourier spectra to extract multi-scale dynamic descriptions of signals characterizing face appearance, head movements and voice. We perform a kernel regression with very few representative samples selected via a supervised weighted-distance-based clustering, that leads to a high generalization power. We also propose a particularly fast regressor-level fusion framework to merge systems based on different modalities. Experiments have proven the efficiency of each key point of the proposed method and our results on challenge data were the highest among 10 international research teams. Key-words Affective computing, Dynamic features, Multimodal fusion, Feature selection, Facial expressions. ACM Classification

Believable agents are autonomous agents that exhibit rich personalities. Interactive dramas take place in virtual worlds inhabited by characters (believable agents) with whom an audience interacts. In the course of this interaction, the audience experiences a story (lives a plot arc). This report presents the research philosophy behind the Oz Project, a research group at CMU that has spent the last ten years studying believable agents and interactive drama. The report then surveys current work from an Oz perspective. This research was partially supported by a grant from Intel Corporation. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of Intel Corporation.

This paper presents the concept for a test setup to prototype control algorithms for a multi-finger gripper. The human-robot interface has to provide enough degrees of freedom (DOF) to intuitively control advanced gripper and to accomplish this task a simple sensor glove equipped with flex and force sensors has been prepared for this project. The software architecture has to support both real hardware and simulation, as well as flexible communication standards, therefore, ROS architecture was employed in this matter. Paper presents some preliminary results for using sensor glove and simulated model of the three finger gripper.

We present a real-time face tracker in this paper The system has achieved a rate of 30% frameshecond using an HP-9000 workstation with a framegrabber and a Canon VC-CI camera. It can track a person 'sface while the person moves freely (e.g., walks, jumps, sits down and stands up) in a room. Three types of models have been employed in developing the system. First, we present a stochastic model to characterize skin-color distributions of human faces. The information provided by the model is sufJicient for tracking a human face in various poses and views. This model is adaptable to different people and different lighting conditions in real-time. Second, a motion model e's used to estimate image motion and to predict search window. Third, a camera model is used toprediet and to compensate for camera motion. The system can be applied to tele-conferencing and many HCI applications including lip-reading and gaze tracking. The principle in developing this system can be extended to other tracking problems such as tracking the human hand.


Question: "Mask-Bot 2i": An interactive, customizable robotic head featuring replaceable facial components.

In commemoration of the centennial of Albert Einstein's special relativity theory, KAIST HUBO and Hanson Robotics collaborated to create the Android type humanoid robot, Albert HUBO, which may be the first walking biped robot with a highly expressive human face. The design of Albert HUBO amalgamates KAIST's expertise in HUBO body mechanics with Hanson Robotics' proficiency in creating realistic android heads. The robot stands at 137 cm tall and weighs 57 Kg, equipped with a total of 66 degrees of freedom (DOFs) for dynamic movement; 31 DOFs are dedicated to head motions, while 35 are for body motions. The head is crafted with 'fubber', a type of smooth artificial skin, to achieve a lifelike appearance. To generate a wide range of facial expressions, including laughter, sadness, anger, and surprise, 28 servo motors control the face movements, and 3 additional servo motors manage neck motions. The body of Albert HUBO is an enhanced version of the HUBO model (KHR-3) from 2004, adapted to accommodate the android head and fitted with 35 DC motors to mimic a variety of human-like body actions. This integration of advanced technologies results in a humanoid robot capable of nuanced expressions and movements, bridging the gap between human and machine interaction.

Physiological sensors are gaining the attention of manufacturers and users. As denoted by devices such as smartwatches or the newly released Kinect 2 – which can covertly measure heartbeats – or by the popularity of smartphone apps that track heart rate during fitness activities. Soon, physiological monitoring could become widely accessible and transparent to users. We demonstrate how one could take advantage of this situation to increase users’ engagement and enhance user experience in human-agent interaction. We created an experimental protocol involving embodied agents – “virtual avatars”. Those agents were displayed alongside a beating heart. We compared a condition in which this feedback was simply duplicating the heart rates of users to another condition in which it was set to an average heart rate. Results suggest a superior social presence of agents when they display feedback similar to users’ internal state. This physiological “similarity-attraction” effect may lead, with little effort, to a better acceptance of agents and robots by the general public.

Functional near infrared spectroscopy (NIRS) is a promising new tool for research in human-robot interaction (HRI). The use of NIRS in HRI has already been demonstrated both as a means for investigating brain activity during human-robot interactions, as well as in the development of brain-robot interfaces that passively monitor a person’s cognitive state to adapt robots’ behaviors accordingly. In this paper, we survey the utility of NIRS and its range of applications in HRI. We discuss both some exemplary applications as well as several pressing challenges to the deployment of NIRS in more realistic settings.

This paper describes the mechanical and electrical design of a new lattice based self-reconfigurable robot, called the ATRON. The ATRON system consists of several fully self-contained robot modules, each having their own processing power, power supply, sensors and actuators. The ATRON modules are roughly spheres with equatorial rotation. Each module can be connected to up to eight neighbors through four male and four female connectors. In this paper, we describe the realization of the design, both the mechanics and the electronics. Details on power sharing and power consumption is given. Finally, this paper includes a brief outline of our future work on the ATRON system.

We present the design and construction of a handheld mirror simulation device. The perception of the world reflected through a mirror depends on the viewer's position with respect to the mirror and the 3-D geometry of the world. In order to simulate a real mirror on a computer screen, images of the observed world, consistent with the viewer's position, must be synthesized and displayed in realtime. Our system is build around a LCD screen manipulated by the user, a single camera fixed on the screen, and a tracking device. The continuous input video stream and tracker data is used to synthesize, in real-time, a continuous video stream displayed on the LCD screen. The synthesized video stream is a close approximation of what the user would see on the screen surface if it were a real mirror. Our system provides a generic interface for applications involving rich, first-person interaction, such as the Virtual Daguerreotype.

The creation of effective autonomous agents (bots) for combat scenarios has long been a goal of the gaming industry. However, a secondary consideration is whether the autonomous bots behave like human players; this is especially important for simulation/training applications which aim to instruct participants in real-world tasks. Bots often compensate for a lack of combat acumen with advantages such as accurate targeting, predefined navigational networks, and perfect world knowledge, which makes them challenging but often predictable opponents. In this paper, we examine the problem of teaching a bot to play like a human in first-person shooter game combat scenarios. Our bot learns attack, exploration and targeting policies from data collected from expert human player demonstrations in Unreal Tournament. We hypothesize that one key difference between human players and autonomous bots lies in the relative valuation of game states. To capture the internal model used by expert human players to evaluate the benefits of different actions, we use inverse reinforcement learning to learn rewards for different game states. We report the results of a human subjects’ study evaluating the performance of bot policies learned from human demonstration against a set of standard bot policies. Our study reveals that human players found our bots to be significantly more human-like than the standard bots during play. Our technique represents a promising stepping-stone toward addressing challenges such as the Bot Turing Test (the CIG Bot 2K Competition).

Glove-TaikII is a system which translates hand gestures-· to speech through an adaptive interface. Hand gestures are mapped continuously to 10 control parameters of a parallel formant speech synthesizer. The mapping allows the hand to act as an artificial vocal tract that produces speech in real time. This gives an unlimited vocabulary, multiple languages in addition to direct control of fundamental frequency and volume. Currently, the best version of Glove-TaikII uses several input devices (including a Cyberglove, a ContactGlove, a polhemus sensor, and a foot-pedal), a parallel formant speech synthesizer and 3 neural networks. The gestureto-speech task is divided into vowel and consonant production by using a gating network to weight the outputs of a vowel and a consonant neural network. The gating network and the consonant network are trained with examples from the user. The vowel network implements a fixed, user-defined relationship between hand-position and vowel sound and does not require any training examples from the user. Volume, fundamental frequency and stop consonants are produced with a fixed mapping from the input devices. One subject has trained for about 100 hours to speak intelligibly with Glove-TalkII. He passed through eight distinct stages while learning to speak. He speaks slowly with speech quality similar to a text-to-speech synthesizer but with far more natural-sounding pitch variations.

Real-time functional MRI (rtfMRI) has been used as a basis for brain-computer interface (BCI) due to its ability to characterize region-specific brain activity in real-time. As an extension of BCI, we present an rtfMRI-based brain-machine interface (BMI) whereby 2-dimensional movement of a robotic arm was controlled by the regulation (and concurrent detection) of regional cortical activations in the primary motor areas. To do so, the subjects were engaged in the right- and/or left-hand motor imagery tasks. The blood oxygenation level dependent (BOLD) signal originating from the corresponding hand motor areas was then translated into horizontal or vertical robotic arm movement. The movement was broadcasted visually back to the subject as a feedback. We demonstrated that real-time control of the robotic arm only through the subjects' thought processes was possible using the rtfMRI-based BMI trials.

The presence of robots in the medical and healthcare fields is increasing. Commercially available medical and healthcare robots are in use by hospitals and nursing homes. Many more assistive, rehabilitation, and surgical robots are being developed at research institutions. In this paper, we examine the awareness of medical and healthcare professionals with respect to robotic applications and the social and psychological impacts of human-robot interaction.

The facial performance of an individual is inherently rich in subtle deformation and timing details. Although these subtleties make the performance realistic and compelling, they often elude both motion capture and hand animation. We present a technique for adding fine-scale details and expressiveness to low-resolution art-directed facial performances, such as those created manually using a rig, via marker-based capture, by fitting a morphable model to a video, or through Kinect reconstruction using recent faceshift technology. We employ a high-resolution facial performance capture system to acquire a representative performance of an individual in which he or she explores the full range of facial expressiveness. From the captured data, our system extracts an expressiveness model that encodes subtle spatial and temporal deformation details specific to that particular individual. Once this model has been built, these details can be transferred to low-resolution art-directed performances. We demonstrate results on various forms of input; after our enhancement, the resulting animations exhibit the same nuances and fine spatial details as the captured performance, with optional temporal enhancement to match the dynamics of the actor. Finally, we show that our technique outperforms the current state-of-the-art in example-based facial animation.

In this paper, the development of humanoid robot HRP-3 is presented. HRP-3, which stands for Humanoid Robotics Platform-3, is a human-size humanoid robot developed as the succeeding model of HRP-2. One of features of HRP-3 is that its main mechanical and structural components are designed to prevent the penetration of dust or spray. Another is that its wrist and hand are newly designed to improve manipulation. Software for a humanoid robot in a real environment is also improved. We also include information on mechanical features of HRP-3 and together with the newly developed hand. Also included are the technologies implemented in HRP-3 prototype. Electrical features and some experimental results using HRP-3 are also presented.


Question: "Mask-Bot 2i": An interactive, customizable robotic head featuring swappable facial components.

This document outlines the creation and deployment of a sophisticated binocular, foveated active vision system, developed as part of the Cog project at the MIT Artificial Intelligence Laboratory. The system is equipped with a 3 degree of freedom mechanical platform that accommodates four color cameras, enabling it to perform complex visual tasks. It also features a motion control system and a network of digital signal processors for parallel image processing, enhancing its capability to handle real-time visual data. To showcase the system's functionalities, the report includes case studies of four distinct visual-motor tasks, illustrating the system's potential applications. Financial backing for this project is derived from a National Defense Science and Engineering Graduate Fellowship, and additional support is provided through an ONR/ARPA Vision MURI Grant (No. N00014-95-1-0600), underscoring the project's significance and the interest it has garnered from multiple funding sources.

The report emphasizes the system's advanced features and its potential to revolutionize the field of robotics and artificial intelligence, particularly in tasks requiring precise visual recognition and motor coordination. The integration of multiple color cameras, motion control systems, and digital signal processors in a 3 degree of freedom platform represents a significant advancement in active vision systems, enabling the system to perform tasks that mimic human visual capabilities. The detailed analysis of the system's performance in various visual-motor tasks further highlights its robustness and adaptability, making it a promising tool for future research and development in the realm of robotics and AI.

Today, most industrial robots are interfaced using text-based programming languages. These languages offer the possibility to declare robotic-specific data types, to specify simple motions, and to interact with tools and sensors via I/O operations. While tailored to the underlying robot controller, they usually only offer a fixed and controller-specific set of possible instructions. The specification of complex motions, the synchronization of cooperating robots and the advanced use of sensors is often very difficult or not even feasible. To overcome these limitations, this paper presents a generic and extensible interface for industrial robots, the Realtime Primitives Interface, as part of a larger software architecture. It allows a flexible specification of complex control instructions and can facilitate the development of sustainable robot controllers. The advantages of this approach are illustrated with several examples.

In this paper, we introduce a real-time face recognition (and announcement) system targeted at aiding the blind and low-vision people. The system uses a Microsoft Kinect sensor as a wearable device, performs face detection, and uses temporal coherence along with a simple biometric procedure to generate a sound associated with the identified person, virtualized at his/her estimated 3-D location. Our approach uses a variation of the K-nearest neighbors algorithm over histogram of oriented gradient descriptors dimensionally reduced by principal component analysis. The results show that our approach, on average, outperforms traditional face recognition methods while requiring much less computational resources (memory, processing power, and battery life) when compared with existing techniques in the literature, deeming it suitable for the wearable hardware constraints. We also show the performance of the system in the dark, using depth-only information acquired with Kinect's infrared camera. The validation uses a new dataset available for download, with 600 videos of 30 people, containing variation of illumination, background, and movement patterns. Experiments with existing datasets in the literature are also considered. Finally, we conducted user experience evaluations on both blindfolded and visually impaired users, showing encouraging results.

The objective of this work is to enhance the mobility of small mobile robots by enabling them to link into a train configuration capable of crossing relatively large obstacles. In particular, we are building on Millibots, semiautonomous, tracked mobile sensing/communication platforms at the 5-cm scale previously developed at Carnegie Mellon University. TheMillibot Train concept provides couplers that allow the Millibot modules to engage/disengage under computer control and joint actuators that allow lifting of one module by another and control of the whole train shape in two dimensions. A manually configurable train prototype demonstrated the ability to climb standard stairs and vertical steps nearly half the train length. A fully functional module with powered joints has been developed and several have been built and tested. Construction of a set of six modules is well underway and will allow testing of the complete train in the near future. This paper focuses on the development, design, and construction of the electromechanical hardware for the Millibot Train.

We introduce AutoMoDe: a novel approach to the automatic design of control software for robot swarms. The core idea in AutoMoDe recalls the approach commonly adopted in machine learning for dealing with the bias–variance tradedoff: to obtain suitably general solutions with low variance, an appropriate design bias is injected. AutoMoDe produces robot control software by selecting, instantiating, and combining preexisting parametric modules—the injected bias. The resulting control software is a probabilistic finite state machine in which the topology, the transition rules and the values of the parameters are obtained automatically via an optimization process that maximizes a task-specific objective function. As a proof of concept, we define AutoMoDe-Vanilla, which is a specialization of AutoMoDe for the e-puck robot. We use AutoMoDe-Vanilla to design the robot control software for two different tasks: aggregation and foraging. The results show that the control software produced by AutoMoDe-Vanilla (i) yields good results, (ii) appears to be robust to the so called reality gap, and (iii) is naturally human-readable.

The advancement in technology indicates that there is an opportunity to enhance Human-Computer Interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users’ self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as defined by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.

A development of humanoid robot HRP-3P, which is a humanoid robot HRP-3 prototype, is presented in this paper. HRP-3 is under development as the succession humanoid to HRP-2, which we developed in phase two of HRP (humanoid robotics project). One of futures of HRP-3P is that its main mechanical and structural components are designed for protection against dust and water. Another is that node controllers are developed for realization of distributed control system. Real-time communication on Ethernet is also newly developed for communication between main controller and node controllers. In this paper, mechanical features and electrical features are presented with specifications of HRP-3P

In order to successfully perform object manipulation, humanoid robots must be equipped with tactile sensors. However, the limited space that is available in robotic fingers imposes severe design constraints. In [1] we presented a small prototype fingertip which incorporates a capacitive pressure system. This paper shows an improved version, which has been integrated on the hand of the humanoid robot iCub. The fingertip is 14.5 mm long and 13 mm wide. The capacitive pressure sensor system has 12 sensitive zones and includes the electronics to send the 12 measurements over a serial bus with only 4 wires. Each synthetic fingertip is shaped approximately like a human fingertip. Furthermore, an integral part of the capacitive sensor is soft silicone foam, and therefore the fingertip is compliant. We describe the structure of the fingertip, their integration on the humanoid robot iCub and present test results to show the characteristics of the sensor.

ARTICLE INFO In this system an electro-mechanical robot is designed and controlled using hand gesture in real time. The system is designed on microcontroller platform using Keil and MPLAB tools. Hand gesture recognition is done on the principle of resistance change sensed through flex sensor. These sensors are integrated in a hand gloves from which input to the system is given. The designed system is divided into two sections as transmitter and receiver. The transmitter section will be in hand gloves from which the data is sensed and processed through PIC16F7487 and send serially to the receiver section. RF technology is used to transmit the data at the receiver section at the frequency of 2.4 GHz. ARM 7 (LPC2148) processor is used to receive the data. Here from the received data, the character is predicted and matched with the closet character from which the character is identified and displayed on LCD. The various case studies is prepared for the designed system and tested in real time. The proposed system can be used for the various applications such as in unmanned machines, industries, handicapped personnel etc. Keywords— a Sensor gloves, flex sensors, PIC controller, ARM processor etc Article History Received:20 th September 2015 Received in revised form :

Care issues and costs associated with an increasing elderly population are becoming a major concern for many countries. The use of assistive robots in “smart-home” environments has been suggested as a possible partial solution to these concerns. A challenge is the personalization of the robot to meet the changing needs of the elderly person over time. One approach is to allow the elderly person, or their carers or relatives, to make the robot learn activities in the smart home and teach it to carry out behaviors in response to these activities. The overriding premise being that such teaching is both intuitive and “nontechnical.” To evaluate these issues, a commercially available autonomous robot has been deployed in a fully sensorized but otherwise ordinary suburban house. We describe the design approach to the teaching, learning, robot, and smart home systems as an integrated unit and present results from an evaluation of the teaching component with 20 participants and a preliminary evaluation of the learning component with three participants in a human-robot interaction experiment. Participants reported findings using a system usability scale and ad-hoc Likert questionnaires. Results indicated that participants thought that this approach to robot personalization was easy to use, useful, and that they would be capable of using it in real-life situations both for themselves and for others.

This paper presents our work towards a decentralized reconfiguration strategy for self-reconfiguring modular robots, assembling furniture-like structures from Roombots (RB) metamodules. We explore how reconfiguration by locomotion from a configuration A to a configuration B can be controlled in a distributed fashion. This is done using Roombots metamodules—two Roombots modules connected serially—that use broadcast signals, lookup tables of their movement space, assumptions about their neighborhood, and connections to a structured surface to collectively build desired structures without the need of a centralized planner.


Question: "Mask-Bot 2i": An interactive, customizable robotic head featuring swappable facial components.

This paper delves into an investigation focused on human-robot interaction, specifically concerning the design and perception of humanoid robot heads. Our primary objective was to ascertain the specific features and dimensions of a robot's face that significantly impact an individual's perception of its human-like qualities. To address this query, we embarked on a comprehensive analysis of 48 distinct robotic models and conducted surveys to gauge public perception regarding the human-like characteristics of each robot. Our research unveiled that the presence of certain facial features, the scale of the head, and the overall number of facial attributes play pivotal roles in shaping perceptions of a robot's human likeness. This paper encapsulates our discoveries and offers preliminary design guidelines for the creation of humanoid robot heads that effectively resonate with human observers.

To enable situated human–robot interaction (HRI), an autonomous robot must both understand and control proxemics—the social use of space—to employ natural communication mechanisms analogous to those used by humans. This work presents a computational framework of proxemics based on data-driven probabilistic models of how social signals (speech and gesture) are produced (by a human) and perceived (by a robot). The framework and modelswere implemented as autonomous proxemic behavior systems for sociable robots, including: (1) a sampling-based method for robot proxemic goal state estimation with respect to human–robot distance and orientation parameters, (2) a reactive proxemic controller for goal state realization, and (3) a cost-based trajectory planner for maximizing automated robot speech and gesture recognition rates along a path to the goal state. Evaluation results indicate that the goal state estimation and realization significantly improve upon past work in human–robot proxemics with respect to “interaction potential”—predicted automated speech and gesture recognition rates as the robot enters into and engages in faceto-face social encounters with a human user—illustrating their efficacy to support richer robot perception and autonomy in HRI. Electronic supplementary material The online version of this article (doi:10.1007/s10514-016-9572-2) contains supplementary material, which is available to authorized users. B Ross Mead rossmead@usc.edu 1 3710 McClintock Avenue, RTH 423, Los Angeles, CA 90089-0781, USA 2 3710 McClintock Avenue, RTH 407, Los Angeles, CA 90089-0781, USA

Swarm systems consist of large numbers of robots that collaborate autonomously. With an appropriate level of human control, swarm systems could be applied in a variety of contexts ranging from search-and-rescue situations to Cyber defence. The two decision making cycles of swarms and humans operate on two different time-scales, where the former is normally orders of magnitude faster than the latter. Closing the loop at the intersection of these two cycles will create fast and adaptive humanswarm teaming networks. This paper brings desperate pieces of the ground work in this research area together to review this multidisciplinary literature. We conclude with a framework to synthesize the findings and summarize the multi-modal indicators needed for closed-loop human-swarm adaptive systems.

Children have great facility in the physical world, and can skillfully model in clay and draw expressive illustrations. Traditional digital modeling tools have focused on mouse, keyboard and stylus input. These tools may be complicated and difficult for young users to easily and quickly create exciting designs. We seek to bring physical interaction to digital modeling, to allow users to use existing physical objects as tangible building blocks for new designs. We introduce KidCAD a digital clay interface for children to remix toys. KidCAD allows children to imprint 2.5D shapes from physical objects into their digital models by deforming a malleable gel input device, deForm. Users can mashup existing objects, edit and sculpt or draw new designs on a 2.5D canvas using physical objects, hands and tools as well as 2D touch gestures. We report on a preliminary user study with 13 children, ages 7 to 10, which provides feedback for our design and helps guide future work in tangible modeling for children.

BodyAvatar is a Kinect-based interactive system that allows users without professional skills to create freeform 3D avatars using body gestures. Unlike existing gesture-based 3D modeling tools, BodyAvatar centers around a first-person "you're the avatar" metaphor, where the user treats their own body as a physical proxy of the virtual avatar. Based on an intuitive body-centric mapping, the user performs gestures to their own body as if wanting to modify it, which in turn results in corresponding modifications to the avatar. BodyAvatar provides an intuitive, immersive, and playful creation experience for the user. We present a formative study that leads to the design of BodyAvatar, the system's interactions and underlying algorithms, and results from initial user trials.

Robotic surgery systems are highly complex and expensive pieces of equipment. Demands for lower cost of care can be met if these systems are employable in a flexible manner for a large variety of procedures. To protect the initial investment the capabilities of a robotic system need to be expandable as new tasks arise.

Recent approaches in robot perception follow the insight that perception is facilitated by interaction with the environment. These approaches are subsumed under the term Interactive Perception (IP). This view of perception provides the following benefits. First, interaction with the environment creates a rich sensory signal that would otherwise not be present. Second, knowledge of the regularity in the combined space of sensory data and action parameters facilitates the prediction and interpretation of the sensory signal. In this survey, we postulate this as a principle for robot perception and collect evidence in its support by analyzing and categorizing existing work in this area. We also provide an overview of the most important applications of IP. We close this survey by discussing remaining open questions. With this survey, we hope to help define the field of Interactive Perception and to provide a valuable resource for future research.

We outline a design for a FACS-based facial expression recognition system and describe in more detail the implementation of two of its main components. Firstly we look at how features that are useful from a pattern analysis point of view can be extracted from a raw input image. We show that good results can be obtained by using the method of local binary patterns (LPB) to generate a large number of candidate features and then selecting from them using fast correlation-based filtering (FCBF). Secondly we show how Platt scaling can be used to improve the performance of an error-correcting output code (ECOC) classifier.

The Effects of Intervention Using a Robot on the Levels of Compliance of Four Children with Autism to Requests Given by Their Mothers Holly J. Nelson Department of Communication Disorders Master of Science The current study presents the use of a humanoid robot to facilitate compliant behaviors to two types of directives in four children with autism. The children participated in a three-month intervention program that incorporated core components of the SCERTS model in order to facilitate social communication (Prizant, Wetherby, Rubin, & Laurent, 2003). Treatment sessions were comprised of 40 minutes of traditional treatment and 10 minutes of interaction with a humanoid robot. Preand post-intervention assessment were conducted, each involving a 5 minute interaction with the child’s mother, in which they were presented with directives in the form of physical manipulation and verbal requests accompanied by a gesture or model. These preand post-intervention sessions were recorded, analyzed, and coded for compliant and noncompliant behavior to the directives, as well as any eye contact, language, or reciprocal action that accompanied their behavior. The overall results were variable, revealing that two participants made notable gains, one child remained consistent, and another participant showed a decrease in compliant behavior in the post-intervention sessions. Further research should be conducted to include a longer period of baseline and intervention, more systematic identification of the most effective probes for the child, and documentation of the child’s physical and emotional state.

The Face Expression Based Emoticon Identification (FEBEI) system is an open source extension to the Tracker.js framework which converts a human facial expression to the best matching emoticon. The contribution of this project was to build this robust classifier which can identify facial expression in real time without any reliance on an external server or computation node. An entirely client-side JavaScript implementation has clear privacy benefits as well as the avoidance of any lag inherent in uploading and downloading images. We accomplished this by utilizing several computationally efficient methods. Tracking.js provided a Viola Jones based face detector which we used to pass facial images to our own implementation of an eigenemotion detection system which was trained to distinguish between happy and angry faces. We have implemented a similar eigenface classifier in python and have trained a Convoluted Neural Network (CNN) to classify emotions to provide a comparative view of its advantages. We aim to make FEBEI easily extendable so that the developer community will be able to add classifiers for more emoticon.

Research has begun to explore the use of virtual humans (VHs) in clinical interviews (Bickmore, Gruber, & Picard, 2005). When designed as supportive and ‘‘safe’’ interaction partners, VHs may improve such screenings by increasing willingness to disclose information (Gratch, Wang, Gerten, & Fast, 2007). In health and mental health contexts, patients are often reluctant to respond honestly. In the context of health-screening interviews, we report a study in which participants interacted with a VH interviewer and were led to believe that the VH was controlled by either humans or automation. As predicted, compared to those who believed they were interacting with a human operator, participants who believed they were interacting with a computer reported lower fear of self-disclosure, lower impression management, displayed their sadness more intensely, and were rated by observers as more willing to disclose. These results suggest that automated VHs can help overcome a significant barrier to obtaining truthful patient information. 2014 Elsevier Ltd. All rights reserved.


Question: A visual tracking method using transfer learning and Gaussian processes regression.

This paper introduces a novel approach for integrating semi-supervised learning and active learning through the use of Gaussian random fields and harmonic functions. The method leverages a weighted graph representation where vertices correspond to labeled and unlabeled data points, and edge weights encode similarity between instances. The semi-supervised learning problem is formulated within the context of a Gaussian random field on this graph, with the mean characterized by harmonic functions. Active learning is performed by selecting queries from the unlabeled data that minimize the estimated expected classification error or risk. In the Gaussian random field model, the risk can be efficiently calculated using matrix methods. Experimental results on synthetic data, handwritten digit recognition, and text classification tasks demonstrate that the proposed active learning scheme requires significantly fewer queries to achieve high accuracy compared to random selection. The method combines the strengths of semi-supervised learning and active learning by utilizing unlabeled data to enhance the learning process and by allowing the algorithm to strategically choose which instances to label, leading to a more effective learning scheme. The framework allows for the efficient estimation of the expected generalization error after querying a point, which results in a more refined query selection criterion than simply selecting the point with the highest label ambiguity. After queries are selected and added to the labeled dataset, the classifier is trained using both labeled and remaining unlabeled data, resulting in improved performance on tasks such as synthetic data analysis, text classification, and image classification. The semi-supervised learning algorithm is based on a relaxation of the binary label requirement, allowing for continuous labels on unlabeled nodes, which simplifies and makes the algorithm more tractable. The energy function is defined to encourage smoothness in the label assignment across the graph, and the minimum energy function is harmonic, satisfying a specific property that ensures the value at each unlabeled node is the average of its neighbors. The harmonic energy minimizing function, which is unique and satisfies specific conditions, is the mean of the Gaussian random field and is computed using matrix methods. The proposed method offers several advantages over existing active learning and semi-supervised learning approaches, including the ability to efficiently estimate the risk of the classifier and to select queries that minimize this risk, leading to improved performance with fewer labeled instances.

We introduce a new computer vision based system for robust traffic sign recognition and tracking. Such a system presents a vital support for driver assistance in an intelligent automotive. Firstly, a color based segmentation method is applied to generate traffic sign candidate regions. Secondly, the HoG features are extracted to encode the detected traffic signs and then generating the feature vector. This vector is used as an input to an SVM classifier to identify the traffic sign class. Finally, a tracking method based on optical flow is performed to ensure a continuous capture of the recognized traffic sign while accelerating the execution time. Our method affords high precision rates under different challenging conditions.

This paper presents a monocular model-based 3D tracking approach for textureless objects. Instead of explicitly searching for 3D-2D correspondences as previous methods, which unavoidably generates individual outlier matches, we aim to minimize the holistic distance between the predicted object contour and the query image edges. We propose a method that can directly solve 3D pose parameters in unsegmented edge distance field. We derive the differentials of edge matching distance with respect to the pose parameters, and search the optimal 3D pose parameters using standard gradient-based non-linear optimization techniques. To avoid being trapped in local minima and to deal with potential large inter-frame motions, a particle filtering process with a first order autoregressive state dynamics is exploited. Occlusions are handled by a robust estimator. The effectiveness of our approach is demonstrated using comparative experiments on real image sequences with occlusions, large motions and cluttered backgrounds.

In this paper, a deep learning inspired solution is proposed and developed to enable timeliness and practicality of the pre-existing ground stereo vision guidance system for flxed-wing UAVs' safe landing. Since the ground guidance prototype was restricted within applications due to its untimeliness, eventually the vision-based detection less than 15 fps (frame per second). Under such circumstances, we employ a regression based deep learning algorithm into automatic detection on the flying aircraft in the landing sequential images. The system architecture is upgraded so as to be compatible with the novel deep learning requests, and furthermore, annotated datasets are conducted to support training and testing of the regression-based learning detection algorithm. Experimental results validate that the detection attaches 100 fps or more while the localization accuracy is kept in the same level.

Autonomous vehicles, surveillance systems, face detection systems lead to the development of accurate object detection system [1]. These systems recognize, classify and localize every object in an image by drawing bounding boxes around the object [2]. These systems use existing classification models as backbone for Object Detection purpose. Object detection is the process of finding instances of real-world objects such as human faces, animals and vehicles etc., in pictures, images or in videos. An Object detection algorithm uses extracted features and learning techniques to recognize the objects in an image. In this paper, various Object Detection techniques have been studied and some of them are implemented. As a part of this paper, three algorithms for object detection in an image were implemented and their results were compared. The algorithms are “Object Detection using Deep Learning Framework by OpenCV”, “Object Detection using Tensorflow” and “Object Detection using Keras models”.

A major challenge in scaling object detection is the difficulty of obtaining labeled images for large numbers of categories. Recently, deep convolutional neural networks (CNNs) have emerged as clear winners on object classification benchmarks, in part due to training with 1.2M+ labeled classification images. Unfortunately, only a small fraction of those labels are available for the detection task. It is much cheaper and easier to collect large quantities of image-level labels from search engines than it is to collect detection data and label it with precise bounding boxes. In this paper, we propose Large Scale Detection through Adaptation (LSDA), an algorithm which learns the difference between the two tasks and transfers this knowledge to classifiers for categories without bounding box annotated data, turning them into detectors. Our method has the potential to enable detection for the tens of thousands of categories that lack bounding box annotations, yet have plenty of classification data. Evaluation on the ImageNet LSVRC-2013 detection challenge demonstrates the efficacy of our approach. This algorithm enables us to produce a >7.6K detector by using available classification data from leaf nodes in the ImageNet tree. We additionally demonstrate how to modify our architecture to produce a fast detector (running at 2fps for the 7.6K detector). Models and software are available at lsda.berkeleyvision.org.

Motivated by the increasing industry trends towards autonomous driving, vehicles, and transportation we focus on developing a traffic analysis framework for the automatic exploitation of a large pool of available data relative to traffic applications. We propose a cooperative detection and tracking algorithm for the retrieval of vehicle trajectories in video surveillance footage based on deep CNN features that is ultimately used for two separate traffic analysis modalities: (a) vehicle speed estimation based on a state of the art fully automatic camera calibration algorithm and (b) the detection of possibly abnormal events in the scene using robust optical flow descriptors of the detected vehicles and Fisher vector representations of spatiotemporal visual volumes. Finally we measure the performance of our proposed methods in the NVIDIA AI CITY challenge evaluation dataset.

The objective of this paper is to propose a new homography-based approach to image-based visual tracking and servoing. The visual tracking algorithm proposed in the paper is based on a new efficient second-order minimization method. Theoretical analysis and comparative experiments with other tracking approaches show that the proposed method has a higher convergence rate than standard first-order minimization techniques. Therefore, it is well adapted to real-time robotic applications. The output of the visual tracking is a homography linking the current and the reference image of a planar target. Using the homography, a task function isomorphic to the camera pose has been designed. A new image-based control law is proposed which does not need any measure of the 3D structure of the observed target (e.g. the normal to the plane). The theoretical proof of the existence of the isomorphism between the task function and the camera pose and the theoretical proof of the stability of the control law are provided. The experimental results, obtained with a 6 d.o.f. robot, show the advantages of the proposed method with respect to the existing approaches. KEY WORDS—visual tracking, visual servoing, efficient second-order minimization, homography-based control law

Multi-object tracking is still a challenging task in computer vision. We propose a robust approach to realize multi-object tracking using multi-camera networks. Detection algorithms are utilized to detect object regions with confidence scores for initialization of individual particle filters. Since data association is the key issue in Tracking-by-Detection mechanism, we present an efficient greedy matching algorithm considering multiple judgments based on likelihood functions. Furthermore, tracking in single cameras is realized by a greedy matching method. Afterwards, 3D geometry positions are obtained from the triangulation relationship between cameras. Corresponding objects are tracked in multiple cameras to take the advantages of multi-camera based tracking. Our algorithm performs online and does not need any information about the scene, no restrictions of enter-and-exit zones, no assumption of areas where objects are moving on and can be extended to any class of object tracking. Experimental results show the benefits of using multiple cameras by the higher accuracy and precision rates.

Simultaneous tracking of multiple persons in real-world environments is an active research field and several approaches have been proposed, based on a variety of features and algorithms. Recently, there has been a growing interest in organizing systematic evaluations to compare the various techniques. Unfortunately, the lack of common metrics for measuring the performance of multiple object trackers still makes it hard to compare their results. In this work, we introduce two intuitive and general metrics to allow for objective comparison of tracker characteristics, focusing on their precision in estimating object locations, their accuracy in recognizing object configurations and their ability to consistently label objects over time. These metrics have been extensively used in two large-scale international evaluations, the 2006 and 2007 CLEAR evaluations, to measure and compare the performance of multiple object trackers for a wide variety of tracking tasks. Selected performance results are presented and the advantages and drawbacks of the presented metrics are discussed based on the experience gained during the evaluations.

Reliable real-time detection of traffic lights is a major concern for the task of autonomous driving. As deep convolutional networks have proven to be a powerful tool in visual object detection, we propose DeepTLR, a camera-based system for real-time detection and classification of traffic lights. Detection and state classification are realized using a single deep convolutional network. DeepTLR does not use any prior knowledge about traffic light locations. Also the detection is executed frame by frame without using temporal information. It is able to detect traffic lights on the whole camera image without any presegmentation. This is achieved by classifying each fine-grained pixel region of the input image and performing a bounding box regression on regions of each class. We show that our algorithm is able to run on frame-rates required for real-time applications while reaching notable results.