Question: An enhanced home energy management system incorporating renewable energy sources and storage resources for optimization.

This paper delves into the relationship between effective optimization algorithms and the problems they address, presenting a framework to explore this dynamic. Several 'No Free Lunch' (NFL) theorems are introduced, illustrating that any algorithm's superior performance on one set of problems is invariably offset by inferior performance on another set. This leads to a geometric interpretation of what constitutes an algorithm's suitability for a specific optimization problem. The paper further examines the implications of the NFL theorems on the information-theoretic dimensions of optimization and benchmark measures of performance. Additional topics covered include the challenges of time-varying optimization problems and the a priori minimax distinctions between optimization algorithms, which can exist despite the NFL theorems' enforcement of a certain uniformity across all algorithms.

**Question**: A Comparative Study of Metaheuristic Algorithms for the Travelling Salesman Problem

**Documents**: A simple genetic algorithm for bin packing
The one dimensional bin packing problem is a classic NP complete problem in combinatorial optimization In this paper we introduce a simple genetic algorithm for solving the bin packing problem The algorithm uses an initial population of randomly generated solutions and an evolutionary approach based on the survival of the fittest The crossover and mutation operators are designed to maintain the feasibility of the offspring and to explore the search space efficiently The algorithm is tested on a set of benchmark instances and the results are compared with those obtained by other heuristic and exact algorithms The genetic algorithm is shown to be competitive with the best known algorithms for the bin packing problem

**Similar Documents**: This paper presents a straightforward genetic algorithm for tackling the one-dimensional bin packing problem, a well-known NP-complete issue in the field of combinatorial optimization. The algorithm employs a starting population of randomly generated solutions, applying an evolutionary strategy that embodies the principle of 'survival of the fittest'. To ensure the feasibility of the offspring and to effectively explore the search space, the crossover and mutation operations are meticulously crafted. The algorithm's performance is assessed against a collection of benchmark instances, with the outcomes compared to those from other heuristic and exact algorithms. The results demonstrate that the genetic algorithm can rival the performance of the most effective algorithms currently available for solving the bin packing problem.

Optimal placement and sizing of DG in distribution network is an optimization problem with continuous and discrete variables. Many researchers have used evolutionary methods for finding the optimal DG placement and sizing. This paper proposes a hybrid algorithm PSO&HBMO for optimal placement and sizing of distributed generation (DG) in radial distribution system to minimize the total power loss and improve the voltage profile. The proposed method is tested on a standard 13 bus radial distribution system and simulation results carried out using MATLAB software. The simulation results indicate that PSO&HBMO method can obtain better results than the simple heuristic search method and PSO algorithm. The method has a potential to be a tool for identifying the best location and rating of a DG to be installed for improving voltage profile and line losses reduction in an electrical power system. Moreover, current reduction is obtained in distribution system.

One essential requirement for a modern Smart Grid is continuous monitoring of the grid variables. This task is efficiently achieved using self-powered electronic devices deployed throughout the grid network. Within the vicinity of the grid power lines, the high voltages of these lines can provide a source of energy for such loads using capacitive coupling. This is known as Electric-Field Energy Harvesting (EFEH). However, the reported strategies that use this principle have not explored how to maximise their power transfer, which is essential for practical systems since the available energy from EFEH is invariably quite small. In this paper, a novel approach for optimum energy harvesting is proposed using the parasitic capacitance of medium-voltage power line insulators. A follow-on power electronic conversion system is then used to maintain the load conditions that maximise the extracted energy. The system performance has been analysed using theoretical studies and computer simulations to identify the operating conditions that maximise the extracted energy. Results obtained so far suggest that up to 100 mW can be harvested from 22 kV grid feeders, using only the capacitive coupling that exists between the line insulator harvester and the feeder conductor.

Intelligent appliances with multimedia capability have been emerging into our daily life, thanks to the fast advance of computing technology and the wide use of the Interne. Smart home is one of the most prominent areas of intelligent appliances. Kitchen is one of the places where such intelligent appliances have been used. Since modern life style is driving people spending less time on cooking healthy food at home, an enjoyable and healthy life style can be assisted with an intelligent kitchenware such as a smart fridge. In this paper we introduce a novel idea of the smart fridge which can enable better nutrition and enhance health. It is designed for managing items stored in it and advising its users with cooking methods depending on what kind of food is stored. More importantly, it can perform other actions such as dietary control, nutrition monitoring, eating habit analysis, etc. The characteristics, functions and design details of the smart fridge are presented in details. We are confident that such a smart fridge will be an important component in future smart home.

Integrating daylight and energy performance with optimization into the design process has always been a challenge for designers. Most of the building environmental performance simulation tools require a considerable amount of time and iterations for achieving accurate results. Moreover the combination of daylight and energy performances has always been an issue, as different software packages are needed to perform detailed calculations. A simplified method to overcome both issues using recent advances in software integration is explored here. As a case study; the optimization of external shadings in a typical office space in Australia is presented. Results are compared against common solutions adopted as industry standard practices. Visual comfort and energy efficiency are analysed in an integrated approach. The DIVA (Design, Iterate, Validate and Adapt) plug-in for Rhinoceros/Grasshopper software is used as the main tool, given its ability to effectively calculate daylight metrics (using the Radiance/Daysim engine) and energy consumption (using the EnergyPlus engine). The optimization process is carried out parametrically controlling the shadings’ geometries. Genetic Algorithms (GA) embedded in the evolutionary solver Galapagos are adopted in order to achieve close to optimum results by controlling iteration parameters. The optimized result, in comparison with conventional design techniques, reveals significant enhancement of comfort levels and energy efficiency. Benefits and drawbacks of the proposed strategy are then discussed. OPEN ACCESS Buildings 2015, 5 561

The application of batteries and ultracapacitors in electric energy storage units for battery powered (EV) and charge sustaining and plug-in hybrid-electric (HEV and PHEV) vehicles have been studied in detail. The use of IC engines and hydrogen fuel cells as the primary energy converters for the hybrid vehicles was considered. The study focused on the use of lithium-ion batteries and carbon/carbon ultracapacitors as the energy storage technologies most likely to be used in future vehicles. The key findings of the study are as follows. 1) The energy density and power density characteristics of both battery and ultracapacitor technologies are sufficient for the design of attractive EVs, HEVs, and PHEVs. 2) Charge sustaining, engine powered hybrid-electric vehicles (HEVs) can be designed using either batteries or ultracapacitors with fuel economy improvements of 50% and greater. 3) Plug-in hybrids (PHEVs) can be designed with effective all-electric ranges of 30-60 km using lithium-ion batteries that are relatively small. The effective fuel economy of the PHEVs can be very high (greater than 100 mpg) for long daily driving ranges (80-150 km) resulting in a large fraction (greater than 75%) of the energy to power the vehicle being grid electricity. 4) Mild hybrid-electric vehicles (MHEVs) can be designed using ultracapacitors having an energy storage capacity of 75-150 Wh. The fuel economy improvement with the ultracapacitors is 10%-15% higher than with the same weight of batteries due to the higher efficiency of the ultracapacitors and more efficient engine operation. 5) Hybrid-electric vehicles powered by hydrogen fuel cells can use either batteries or ultracapacitors for energy storage. Simulation results indicate the equivalent fuel economy of the fuel cell powered vehicles is 2-3 times higher than that of a gasoline fueled IC vehicle of the same weight and road load. Compared to an engine-powered HEV, the equivalent fuel economy of the hydrogen fuel cell vehicle would be 1.66-2.0 times higher

This article presents a review of the state of the art of the Wind Farm Design and Optimization (WFDO) problem. The WFDO problem refers to a set of advanced planning actions needed to extremize the performance of wind farms, which may be composed of a few individual Wind Turbines (WTs) up to thousands of WTs. The WFDO problem has been investigated in different scenarios, with substantial differences in main objectives, modelling assumptions, constraints, and numerical solution methods. The aim of this paper is: (1) to present an exhaustive survey of the literature covering the full span of the subject, an analysis of the state-of-the-art models describing the performance of wind farms as well as its extensions, and the numerical approaches used to solve the problem; (2) to provide an overview of the available knowledge and recent progress in the application of such strategies to real onshore and offshore wind farms; and (3) to propose a comprehensive agenda for future research. OPEN ACCESS Energies 2014, 7 6931

In recent years advances in energy e cient design and wireless technologies have enabled exciting new applications for wireless devices These applications span a wide range including real time and streaming video and audio delivery remote monitoring using networked microsensors personal medical monitoring and home networking of everyday appliances While these applications require high performance from the network they su er from resource constraints that do not appear in more traditional wired computing environments In particular wireless spectrum is scarce often limiting the bandwidth available to applications and making the channel error prone and the nodes are battery operated often limiting available energy My thesis is that this harsh environment with severe resource constraints requires an application speci c protocol architecture rather than the traditional layered approach to obtain the best pos sible performance This dissertation supports this claim using detailed case studies on microsensor networks and wireless video delivery The rst study develops LEACH Low Energy Adaptive Clustering Hierarchy an architecture for remote microsensor networks that combines the ideas of energy e cient cluster based routing and media access together with application speci c data aggregation to achieve good performance in terms of system lifetime latency and application perceived quality This approach improves system lifetime by an order of magnitude compared to general purpose approaches when the node energy is limited The second study develops an unequal error protection scheme for MPEG compressed video delivery that adapts the level of protection applied to portions of a packet to the degree of importance of the corresponding bits This approach obtains better application perceived performance than current approaches for the same amount of transmission bandwidth These two systems show that application speci c protocol architectures achieve the energy and latency e ciency and error robustness needed for wireless networks Thesis Supervisor Anantha P Chandrakasan Title Associate Professor Thesis Supervisor Hari Balakrishnan Title Assistant Professor This thesis is dedicated to the memory of Robert H Rabiner The rst Rabiner at the Tute

A novel integrated power unit realizes both energy harvesting and energy storage by a textile triboelectric nanogenerator (TENG)-cloth and a flexible lithium-ion battery (LIB) belt, respectively. The mechanical energy of daily human motion is converted into electricity by the TENG-cloth, sustaining the energy of the LIB belt to power wearable smart electronics.

Demand-side management technology is a key element of the proposed smart grid, which will help utilities make more efficient use of their generation assets by reducing consumers' energy demand during peak load periods. However, although some modern appliances can respond to price signals from the utility companies, there is a vast stock of older appliances that cannot. For such appliances, utilities must infer what appliances are operating in a home, given only the power signals on the main feeder to the home (i.e., the home's power consumption must be disaggregated into individual appliances). We report on an in-depth investigation of multi-label classification algorithms for disaggregating appliances in a power signal. A systematic review of this research topic shows that this class of algorithms has received little attention in the literature, even though it is arguably a more natural fit to the disaggregation problem than the traditional single-label classifiers used to date. We examine a multi-label meta-classification framework (RAkEL), and a bespoke multi-label classification algorithm (MLkNN), employing both time-domain and wavelet-domain feature sets. We test these classifiers on two real houses from the Reference Energy Disaggregation Dataset. We found that the multilabel algorithms are effective and competitive with published results on the datasets.

Demand side management (DSM) is one of the important functions in a smart grid that allows customers to make informed decisions regarding their energy consumption, and helps the energy providers reduce the peak load demand and reshape the load profile. This results in increased sustainability of the smart grid, as well as reduced overall operational cost and carbon emission levels. Most of the existing demand side management strategies used in traditional energy management systems employ system specific techniques and algorithms. In addition, the existing strategies handle only a limited number of controllable loads of limited types. This paper presents a demand side management strategy based on load shifting technique for demand side management of future smart grids with a large number of devices of several types. The day-ahead load shifting technique proposed in this paper is mathematically formulated as a minimization problem. A heuristic-based Evolutionary Algorithm (EA) that easily adapts heuristics in the problem was developed for solving this minimization problem. Simulations were carried out on a smart grid which contains a variety of loads in three service areas, one with residential customers, another with commercial customers, and the third one with industrial customers. The simulation results show that the proposed demand side management strategy achieves substantial savings, while reducing the peak load demand of the smart grid.


Question: An enhanced home energy management system incorporating renewable energy sources and storage resources for optimized performance.

This paper presents an enhanced approach for the scheduling of residential Distributed Energy Resources (DER) to optimize smart home energy services. The proposed method empowers homeowners to prioritize their energy needs and strategically schedule their available DER to maximize overall benefits. Particle Swarm Optimization (PSO), chosen for its ease of implementation and proven effectiveness in generating near-optimal schedules in a timely manner, is at the core of this enhanced decision-support tool. To refine the standard cooperative PSO algorithm, we introduce a stochastic repulsion mechanism among particles, leading to more efficient DER scheduling. The resultant schedules are then utilized to assess the additional value that coordinated DER scheduling can offer to consumers. This is achieved by comparing the costs incurred by homeowners when the enhanced algorithm schedules all DER concurrently, versus the scenario where each DER is scheduled independently. This comparison allows homeowners to ascertain whether their specific energy service requirements, DER mix, and electricity tariff structures justify the complexity of coordinated scheduling, or if a simplified, segmented optimization process would suffice.

**Question**: A Comparative Analysis of Different Techniques for Predicting Solar Power Generation

**Documents**: A Hybrid Prediction Model for Solar Power Generation Based on Empirical Mode Decomposition and Support Vector Machine
The prediction of solar power generation is crucial for the effective integration of renewable energy into the power grid. In this paper, a hybrid prediction model combining Empirical Mode Decomposition (EMD) and Support Vector Machine (SVM) is proposed to improve the accuracy of solar power prediction. The EMD method is first used to decompose the original solar power data into a set of Intrinsic Mode Functions (IMFs) and a residual trend. Then, an SVM model is trained separately for each IMF and the residual trend. The final prediction is obtained by aggregating the predictions of all IMFs and the residual trend. The proposed model is validated using real-world solar power data, and its performance is compared with that of traditional SVM and other prediction models. The results demonstrate the superior prediction accuracy of the proposed hybrid model.

**Similar Documents**: This paper introduces a novel hybrid prediction model for solar power generation, integrating Empirical Mode Decomposition (EMD) with Support Vector Machine (SVM) to enhance forecasting precision. Recognizing the importance of accurate solar power prediction for the seamless integration of renewable energy into the power grid, this study proposes a method that initially applies EMD to decompose the raw solar power data into several Intrinsic Mode Functions (IMFs) and a residual component. Subsequently, SVM models are individually trained for each IMF and the residual trend. The ultimate

The functional scope of today's software-defined data centers (SDDC) has expanded to such an extent that servers face a growing amount of critical background operational tasks like load monitoring, logging, migration, and duplication, etc. These ancillary operations, which we refer to as management operations, often nibble the stringent data center power envelope and exert a tremendous amount of pressure on front-end user tasks. However, existing power capping, peak shaving, and time shifting mechanisms mainly focus on managing data center power demand at the "macro level" -- they do not distinguish ancillary background services from user tasks, and therefore often incur significant performance degradation and energy overhead.
 In this study we explore "micro-level" power management in SDDC: tuning a specific set of critical loads for the sake of overall system efficiency and performance. Specifically, we look at management operations that can often lead to resource contention and energy overhead in an IaaS SDDC. We assess the feasibility of this new power management paradigm by characterizing the resource and power impact of various management operations. We propose HOPE, a new system optimization framework for eliminating the potential efficiency bottleneck caused by the management operations in the SDDC. HOPE is implemented on a customized OpenStack cloud environment with heavily instrumented power infrastructure. We thoroughly validate HOPE models and optimization efficacy under various user workload scenarios. Our deployment experiences show that the proposed technique allows SDDC to reduce energy consumption by 19%, reduce management operation execution time by 25.4%, and in the meantime improve workload performance by 30%.

We first review some of the suggested methods for energy disaggregation. We then provide a real-life data set for researchers to help developing novel energy disaggregation algorithms. We also present a publicly available experimental data set captured from a typical building on the University of California, Berkeley campus.

Distribution feeders and substations need to provide additional capacity to serve the growing electrical demand of customers without compromising the reliability of the electrical networks. Also, more control devices, such as DG (Distributed Generation) units are being integrated into distribution feeders. Distribution networks were not planned to host these intermittent generation units before construction of the systems. Therefore, additional distribution facilities are needed to be planned and prepared for the future growth of the electrical demand as well as the increase of network hosting capacity by DG units. This paper presents a multiobjective optimization algorithm for the MDEP (Multi-Stage Distribution Expansion Planning) in the presence of DGs using nonlinear formulations. The objective functions of the MDEP consist of minimization of costs, END (Energy-Not-Distributed), active power losses and voltage stability index based on SCC (Short Circuit Capacity). A MPSO (modified Particle Swarm Optimization) algorithm is developed and used for this multiobjective MDEP optimization. In the proposed MPSO algorithm, a new mutation method is implemented to improve the global searching ability and restrain the premature convergence to local minima. The effectiveness of the proposed method is tested on a typical 33-bus test system and results are presented.

This paper presents an improved dc-link voltage fast control strategy based on energy balance for a pulse width modulation (PWM) rectifier-inverter system to reduce the fluctuation of dc-link voltage. A conclusion is drawn that the energy in dc-link capacitor cannot be kept constant in dynamic process when the system operates in rectifier mode, even in ideal case. Meanwhile, the minimum dc-link voltage deviation is analyzed. Accordingly, a predictive dc-link voltage control scheme based on energy balance is proposed, while the grid current is regulated by deadbeat predictive control. A prediction method for output power is also introduced. Furthermore, a small-signal model with the control delay is adopted to analyze the stability and robustness of the control strategy. The simulation and experimental results demonstrate both good dynamic and steady-state performances of the rectifier-inverter system with the proposed control scheme.

At the start of the second decade of 21th century, the time has come to make the Smart Houses a reallity for regular use. The different parts of a Smart House are researched but there are still distances from an applicable system, using the modern technology. In this paper we present an overview of the Smart House subsystems necessary for controlling the house using a mobile application efficiently and securely. The sequence diagram of the mobile application connectiing to the server application and also the usecases possible are presented. The challenges faced in designing the mobile application and illustrating the updated house top plane view in that application, are discussed and soloutions are adapted for it. Finally the designed mobile application was implemented and the important sections of it were described, such as the interactive house top view map which indicates the status of the devices using predefined icons. The facilities to manage the scheduled tasks and defined rules are also implemented in this mobile application that was developed for use in Windows Mobile platform. This application has the capability of connecting to the main server using GPRS mobile internet and SMS. This system is expected to be an important step towards a unified system structure that can be used efficiently in near future regular houses. Keywords—Smart House, Mobile Application, Remote Control, Automated Home, Windows Mobile.

The telecommunication sector plays a significant role in shaping the global economy and the way people share information and knowledge. At present, the telecommunication sector is liable for its energy consumption and the amount of emissions it emits in the environment. In the context of off-grid telecommunication applications, offgrid base stations (BSs) are commonly used due to their ability to provide radio coverage over a wide geographic area. However, in the past, the off-grid BSs usually relied on emission-intensive power supply solutions such as diesel generators. In this review paper, various types of solutions (including, in particular, the sustainable solutions) for powering BSs are discussed. The key aspects in designing an ideal power supply solution are reviewed, and these mainly include the pre-feasibility study and the thermal management of BSs, which comprise heating and cooling of the BS shelter/cabinets and BS electronic equipment and power supply components. The sizing and optimization approaches used to design the BSs’ power supply systems as well as the operational and control strategies adopted to manage the power supply systems are also reviewed in this paper.

Battery management system (BMS) emerges a decisive system component in battery-powered applications, such as (hybrid) electric vehicles and portable devices. However, due to the inaccurate parameter estimation of aged battery cells and multi-cell batteries, current BMSs cannot control batteries optimally, and therefore affect the usability of products. In this paper, we proposed a smart management system for multi-cell batteries, and discussed the development of our research study in three directions: i) improving the effectiveness of battery monitoring and current sensing, ii) modeling the battery aging process, and iii) designing a self-healing circuit system to compensate performance variations due to aging and other variations.

The Internet of Things envisions integration, coordination, communication, and collaboration of real-world objects in order to perform daily tasks in a more intelligent and efficient manner. To comprehend this vision, this article studies the design of a large-scale IoT system for smart grid application, which constitutes a large number of home users and has the requirement of fast response time. In particular, we focus on the messaging protocol of a universal IoT home gateway, where our cloud enabled system consists of a back-end server, a unified home gateway (UHG) at the end users, and a user interface for mobile devices. We discuss the features of such an IoT system to support a large-scale deployment with a UHG and real-time residential smart grid applications. Based on the requirements, we design an IoT system using XMPP and implemented in a testbed for energy management applications. To show the effectiveness of the designed testbed, we present some results using the proposed IoT architecture.

Optimal placement and sizing of DG in distribution network is an optimization problem with continuous and discrete variables. Many researchers have used evolutionary methods for finding the optimal DG placement and sizing. This paper proposes a hybrid algorithm PSO&HBMO for optimal placement and sizing of distributed generation (DG) in radial distribution system to minimize the total power loss and improve the voltage profile. The proposed method is tested on a standard 13 bus radial distribution system and simulation results carried out using MATLAB software. The simulation results indicate that PSO&HBMO method can obtain better results than the simple heuristic search method and PSO algorithm. The method has a potential to be a tool for identifying the best location and rating of a DG to be installed for improving voltage profile and line losses reduction in an electrical power system. Moreover, current reduction is obtained in distribution system.

Energy and environmental sustainability can benefit a lot from advances in data mining and machine learning techniques. However, those advances rely on the availability of relevant datasets required to develop, improve and validate new techniques. Only recently the first datasets were made publicly available for the energy and sustainability research community. In this paper we present a freely available dataset containing power usage and related information from 50 homes. Here we describe our dataset, the hardware and software setups used when collecting the data and how others can access it. We then discuss potential uses of this data in the future of energy ecofeedback and demand side management research.


Question: Identifying ascending staircases through the utilization of stereo vision technology.

This paper introduces methods for accurately detecting and localizing curbs and stairways to facilitate autonomous navigation. Our approach integrates brightness information, manifested as edgels, with three-dimensional data acquired from a commercially available stereo system. The comprehensive system, encompassing stereo processing, operates at approximately 4 Hz on a 1 GHz laptop. We present empirical outcomes and critically evaluate the strengths and limitations of our methodology.

In our work, we address the challenge of identifying and precisely locating curbs and stairways using stereo vision. Our algorithms effectively combine brightness data, represented as edge information, with depth data obtained from a commercial stereo vision system. The entire system, inclusive of stereo computation, achieves a processing rate of around 4 Hz on a 1 GHz laptop. We demonstrate the effectiveness of our approach through experimental results and provide a detailed analysis of its benefits and potential drawbacks.

I ncreasingly powerful integrated circuits are making an entire range of new applications possible. Complementary metal-oxide semiconductor (CMOS) sensors, for example, have made the digital camera a commonplace consumer item. These light-sensitive chips, positioned where film would normally be, capture images as reusable digital files that users can upload to their computer, manipulate with software, and distribute electronically. Recent technological advances are enabling a new generation of smart cameras that represent a quantum leap in sophistication. While today's digital cameras capture images, smart cameras capture high-level descriptions of the scene and analyze what they see. These devices could support a wide variety of applications including human and animal detection, surveillance, motion analysis, and facial identification. Video processing has an insatiable demand for real-time performance. Fortunately, Moore's law provides an increasing pool of available computing power to apply to real-time analysis. Smart cameras leverage very large-scale integration (VLSI) to provide such analysis in a low-cost, low-power system with substantial memory. Moving well beyond pixel processing and compression, these systems run a wide range of algorithms to extract meaning from streaming video. Because they push the design space in so many dimensions, smart cameras are a leading-edge application for embedded system research. has developed a first-generation smart camera system that can detect people and analyze their movement in real time. Although there are many approaches to real-time video analysis, we chose to focus initially on human gesture recognition—identifying whether a subject is walking, standing, waving his arms, and so on. Because much work remains to be done on this problem, we sought to design an embedded system that can incorporate future algorithms as well as use those we created exclusively for this application. As Figure 1 shows, our algorithms use both low-level and high-level processing. The low-level component identifies different body parts and categorizes their movement in simple terms. The high-level component, which is application-dependent, uses this information to recognize each body part's action and the person's overall activity based on scenario parameters. The system captures images from the video input, which can be either uncompressed or compressed (MPEG and motion JPEG), and applies four different algorithms to detect and identify human body parts. Region extraction. The first algorithm transforms the pixels of an image, like that shown in Figure 2a, into an M × N bitmap and eliminates the background. It then detects the body part's skin area using a YUV …

Tracking, or camera pose determination, is the main technical challenge in creating augmented realities. Constraining the degree to which the environment may be altered to support tracking heightens the challenge. This paper describes several years of work at the USC Computer Graphics and Immersive Technologies (CGIT) laboratory to develop self-contained, minimally intrusive tracking systems for use in both indoor and outdoor settings. These hybrid-technology tracking systems combine vision and inertial sensing with research in fiducial design, feature detection, motion estimation, recursive filters, and pragmatic engineering to satisfy realistic application requirements.

Detecting human beings accurately in a visual surveillance system is crucial for diverse application areas including abnormal event detection, human gait characterization, congestion analysis, person identification, gender classification and fall detection for elderly people. The first step of the detection process is to detect an object which is in motion. Object detection could be performed using background subtraction, optical flow and spatio-temporal filtering techniques. Once detected, a moving object could be classified as a human being using shape-based, texture-based or motion-based features. A comprehensive review with comparisons on available techniques for detecting human beings in surveillance videos is presented in this paper. The characteristics of few benchmark datasets as well as the future research directions on human detection have also been discussed.

Stereo matching, a key element towards extracting depth information from stereo images, is widely used in several embedded consumer electronic and multimedia systems. Such systems demand high processing performance and accurate depth perception, while their deployment in embedded and mobile environments implies that cost, energy and memory overheads need to be minimized. Hardware acceleration has been demonstrated in efficient embedded stereo vision systems. To this end, this paper presents the design and implementation of a hardware-based stereo matching system able to provide high accuracy and concurrently high performance for embedded vision devices, which are associated with limited hardware and power budget. We first implemented a compact and efficient design of the guided image filter, an edge-preserving filter, which reduces the hardware complexity of the implemented stereo algorithm, while at the same time maintains high-quality results. The guided filter design is used in two parts of the stereo matching pipeline, showing that it can simplify the hardware complexity of the Adaptive Support Weight aggregation step, and efficiently enable a powerful disparity refinement unit, which improves matching accuracy, even though cost aggregation is based on simple, fixed support strategies. We implemented several variants of our design on a Kintex-7 FPGA board, which was able to process HD video (1,280 × 720) in real-time (60 fps), using ~57.5k and ~71k of the FPGA's logic (CLB) and register resources, respectively. Additionally, the proposed stereo matching design delivers leading accuracy when compared to state-of-the-art hardware implementations based on the Middlebury evaluation metrics (at least 1.5 percent less bad matching pixels).

When rescue robots navigate in a rough terrain, significant vibration of the video occurs unavoidably and a video stabilization system is proposed in this paper to reduce the disturbance on the visual system of the present rescue robots. The Kalman filter is applied to estimate the motion vector of the robot and the false estimation occurrence can be thus greatly reduced by applying analysis of correlation and variance of the motion vector estimation. With a hierarchical searching algorithm implemented on the TI DSP 6437, the frame rate can be improved from 10 fps to 28 fps to realize the real-time video stabilization and furthermore, the cooperative mission of multiple rescue robots has been achieved by applying the developed visual servo technique. The proposed virtual bounded motion control algorithm further leads the robots approaching the target precisely with cooperation of multiple rescue robots. The demo video is available at: http://lab816.cn.nctu.edu.tw/DemoVideo/. video stabilization

This paper describes performance evaluation of a wearable augmented reality system for natural outdoor environments. Applied Research Associates (ARA), as prime integrator on the DARPA ULTRA-Vis (Urban Leader Tactical, Response, Awareness, and Visualization) program, is developing a soldier-worn system to provide intuitive ‘heads-up’ visualization of tactically-relevant geo-registered icons. Our system combines a novel pose estimation capability, a helmet-mounted see-through display, and a wearable processing unit to accurately overlay geo-registered iconography (e.g., navigation waypoints, sensor points of interest, blue forces, aircraft) on the soldier’s view of reality. We achieve accurate pose estimation through fusion of inertial, magnetic, GPS, terrain data, and computer-vision inputs. We leverage a helmet-mounted camera and custom computer vision algorithms to provide terrain-based measurements of absolute orientation (i.e., orientation of the helmet with respect to the earth). These orientation measurements, which leverage mountainous terrain horizon geometry and mission planning landmarks, enable our system to operate robustly in the presence of external and body-worn magnetic disturbances. Current field testing activities across a variety of mountainous environments indicate that we can achieve high icon geo-registration accuracy (<10mrad) using these vision-based methods.

Pedestrian detection in low resolution videos can be challenging. In outdoor surveillance scenarios, the size of pedestrians in the images is often very small (around 20 pixels tall). The most common and successful approaches for single frame pedestrian detection use gradient-based features and a support vector machine classifier. We propose an extension of these ideas, and develop a new algorithm that extracts gradient features from a spatiotemporal volume, consisting of a short sequence of images (about one second in duration). The additional information provided by the motion of the person compensates for the loss of resolution. On standard datasets (PETS2001, VIRAT) we show a significant improvement in performance over single-frame detection.

This paper describes the vehicle guidance system of an autonomous vehicle. This system is part of the control structure of the vehicle and consists of a path generator, a motion planning algorithm and a sensor fusion module. A stereo vision sensor and a DGPS sensor are used as position sensors. The trajectory for the vehicle motion is generated in a rst step by using only information from a digital map. Object-detecting sensors such as the stereo vision sensor, three laserscanner and a radar sensor observe the vehicle environment and report detected objects to the sensor fusion module. This information is used to dynamically update the planned vehicle trajectory to the nal vehicle motion.

We present a real-time stereoscopic video-see-through augmented reality (AR) system applied to the medical procedure known as ultrasound-guided needle biopsy of the breast. The AR system was used by a physician during procedures on breast models and during non-invasive examinations of human subjects. The system merges rendered live ultrasound data and geometric elements with stereo images of the patient acquired through head-mounted video cameras and presents these merged images to the physician in a head-mounted display. The physician sees a volume visualization of the ultrasound data directly under the ultrasound probe, properly registered within the patient and with the biopsy needle. Using this system, a physician successfully guided a needle into an artificial tumor within a training phantom of a human breast. We discuss the construction of the AR system and the issues and decisions which led to the system architecture and the design of the video see-through head-mounted display. We designed methods to properly resolve occlusion of the real and synthetic image elements. We developed techniques for realtime volume visualization of timeand position-varying ultrasound data. We devised a hybrid tracking system which achieves improved registration of synthetic and real imagery and we improved on previous techniques for calibration of a magnetic tracker. CR

This paper describes the implementation of a LCD stripe projection system, based on a new processing scheme called line shift processing. One advantage of the method is, that the projection device can, but is not required to be stable over time nor does it have to be calibrated. The new method therefore allows us to use an off-the-shelf multimedia projector to acquire dense 3-D data (several 100K Triangles) of large objects in a matter of seconds. Since we are able to control our source of illumination, we can also acquire registered color information using standard monochrome cameras. Besides the description of the system some practical examples are given. The calibration of the system is also addressed.


Question: Identifying ascending staircases through the application of stereo vision techniques.

This document introduces the semiglobal matching (SGM) technique for stereo vision processing, which is particularly effective in handling radiometric discrepancies between input images. The method utilizes a pixelwise mutual information (MI)-based matching cost to address these differences. Unlike traditional methods that rely on a global cost function to enforce smoothness constraints, SGM offers a faster alternative through pathwise optimization from multiple directions. The paper also delves into strategies for occlusion detection, subpixel accuracy enhancement, and multibaseline matching. Additional postprocessing steps are discussed for outlier removal, problem resolution in structured environments, and gap interpolation. The document further outlines methods for efficiently processing large images and the fusion of disparity images through orthographic projection. Comparative analysis on standard stereo images reveals that SGM ranks among the leading algorithms, especially when subpixel accuracy is a requirement. The algorithm's complexity is linear with respect to the number of pixels and the disparity range, leading to processing times of just 1-2 seconds on typical test images. An extensive evaluation of the MI-based matching cost highlights its robustness against various radiometric transformations. The document concludes with real-world examples demonstrating the effectiveness of the proposed techniques in reconstructing aerial frame and pushbroom images, showcasing their practical applicability in large-scale scenarios.

We describe the architecture, algorithms, and experiments with HERB, an autonomous mobile manipulator that performs useful manipulation tasks in the home. We present new algorithms for searching for objects, learning to navigate in cluttered dynamic indoor scenes, recognizing and registering objects accurately in high clutter using vision, manipulating doors and other constrained objects using caging grasps, grasp planning and execution in clutter, and manipulation on pose and torque constraint manifolds. S.S. Srinivasa ( ) · D. Ferguson · C.J. Helfrich Intel Research Pittsburgh, 4720 Forbes Avenue, Suite 410, Pittsburgh, PA 15213, USA e-mail: siddhartha.srinivasa@intel.com C.J. Helfrich e-mail: casey.j.helfrich@intel.com D. Berenson · A. Collet · R. Diankov · G. Gallagher · G. Hollinger · J. Kuffner · M.V. Weghe The Robotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA D. Berenson e-mail: dberenso@ri.cmu.edu A. Collet e-mail: acollet@ri.cmu.edu R. Diankov e-mail: rdiankov@ri.cmu.edu G. Gallagher e-mail: ggallagh@ri.cmu.edu G. Hollinger e-mail: gholling@ri.cmu.edu J. Kuffner e-mail: kuffner@ri.cmu.edu M.V. Weghe e-mail: vandeweg@ri.cmu.edu We also present numerous severe real-world test results from the integration of these algorithms into a single mobile manipulator.

In this paper, we first review the approaches to recover 3D shape and related movements of a human and then we present an easy and reliable approach to recover a 3D model using just one image or monocular video sequence. A simplification of the perspective camera model is required, due to the absence of stereo view. The human figure is reconstructed in a skeleton form and to improve the visual quality, a pre-defined human model is also fitted to the recovered 3D data.

An application of insect visuomotor behavior to automatic control of landing is explored. Insects, being perhaps more reliant on image motion cues than mammals or higher vertebrates, are proving to be an excellent organism in which to investigate how information on optic flow is exploited to guide locomotion and navigation. We have observed how bees perform grazing landings on a flat surface and have deduced the algorithmic basis for the behavior. A smooth landing is achieved by a surprisingly simple and elegant strategy: image velocity is held constant as the surface is approached, thus automatically ensuring that flight speed is close to zero at touchdown. No explicit knowledge of flight speed or height above the ground is necessary. The feasibility of this landing strategy was tested by implementation in a robotic gantry. We also outline our current efforts at exploring the applicability of this and related techniques to the guidance of uninhabited airborne vehicles (UAVs). Aspects of the algorithm were tested on a small UAV using real imagery to control

This paper presents the vision-based road detection system currently installed onto the MOB-LAB land vehicle. Based on a geometrical transform and on a fast morphological processing, the system is capable to detect road markings even in extremely severe shadow conditions on at and structured roads. The use of a special-purpose massively architecture (PAPRICA) allows to achieve a processing rate of about 17 Hz.

No feature-based vision system can work unless good features can be identi ed and tracked from frame to frame. Although tracking itself is by and large a solved problem, selecting features that can be tracked well and correspond to physical points in the world is still hard. We propose a feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world. These methods are based on a new tracking algorithm that extends previous Newton-Raphson style search methods to work under a ne image transformations. We test performance with several simulations and experiments.

The present study investigated the ability to inter actively locate points in a three dimensional computer environment using a six degree of freedom input device. Four different visu al feedback modes were tested: fixed viewpoint monoscopic persp ective, fixed viewpoint stereoscopic perspective, head-tracked mo noscopic perspective and head-tracked stereoscopic perspecti ve. Targets were located at plus and minus 10cm along the X, Y or Z axes, from a fixed starting location. Data about the time to complete the task and positioning accuracy (error) are gathered for each trial. In addition, subjective feedback regarding the apparat us, visual mode and task difficulty was solicited from subject s. The results indicate that stereoscopic performance is superior to monoscopic performance and that asymmetries exist both across and within axes. Head tracking had no appreciable effect upon performance. Subjective feedback regarding performance is usuall y consistent with objective measures, but some inconsistencies a re present.

This paper relates first experiences using a state-of-the-art, time-of-flight sensor that is able to deliver 3D images. The properties and capabilities of the sensor make it a potential powerful tool for applications within mobile robotics especially for real-time tasks, as the sensor features a frame rate of up to 30 frames per second. Its capabilities in terms of basic obstacle avoidance and local path-planning are evaluated and compared to the performance of a standard laser scanner.

Autonomous navigation of indoor unmanned aircraft systems (UAS) requires accurate pose estimations usually obtained from indirect measurements. Navigation based on inertial measurement units (IMU) is known to be affected by high drift rates. The incorporation of cameras provides complementary information due to the different underlying measurement principle. The scale ambiguity problem for monocular cameras is avoided when a light-weight stereo camera setup is used. However, also frame-to-frame stereo visual odometry (VO) approaches are known to accumulate pose estimation errors over time. Several valuable real-time capable techniques for outlier detection and drift reduction in frame-to-frame VO, for example robust relative orientation estimation using random sample consensus (RANSAC) and bundle adjustment, are available. This study addresses the problem of choosing appropriate VO components. We propose a frame-to-frame stereo VO method based on carefully selected components and parameters. This method is evaluated regarding the impact and value of different outlier detection and drift-reduction strategies, for example keyframe selection and sparse bundle adjustment (SBA), using reference benchmark data as well as own real stereo data. The experimental results demonstrate that our VO method is able to estimate quite accurate trajectories. Feature bucketing and keyframe selection are simple but effective strategies which further improve the VO results. Furthermore, introducing the stereo baseline constraint in pose graph optimization (PGO) leads to significant improvements. * Corresponding author

This paper presents a novel approach to the problem of computing the matching-cost for stereo vision. The approach is based upon a Convolutional Neural Network that is used to compute the similarity of input patches from stereo image pairs. In combination with state-ofthe-art stereo pipeline steps, the method achieves top results in major stereo benchmarks. The paper introduces the problem of stereo matching, discusses the proposed method and shows results from recent stereo datasets.

This paper introduces a system for estimating the attention of a driver wearing a first person view camera using salient objects to improve gaze estimation. A challenging data set of pedestrians crossing intersections has been captured using Google Glass worn by a driver. A challenge unique to first person view from cars is that the interior of the car can take up a large part of the image. The proposed system automatically filters out the dashboard of the car, along with other parts of the instrumentation. The remaining area is used as a region of interest for a pedestrian detector. Two cameras looking at the driver are used to determine the direction of the driver's gaze, by examining the eye corners and the center of the iris. This coarse gaze estimation is then linked to the detected pedestrians to determine which pedestrian the driver is focused on at any given time.


Question: Identifying ascending staircases through the utilization of stereo vision technology.

This paper introduces a pair of efficient algorithms designed for real-time surface normal estimation from organized point cloud data, specifically optimized for high-speed processing. The methodologies presented herein leverage integral images to facilitate accelerated smoothing and covariance estimation, both of which are crucial for accurate surface normal calculation. By employing a depth-dependent and border-aware approach, these algorithms ensure robustness and reliability in the presence of large, complex point clouds. The demonstrated ability to handle such data at high frame rates makes these techniques particularly suitable for integration into real-time computer vision systems that process Kinect-like sensor data, enabling advanced applications in various fields.

A study by Slater, et al., [1995] indicated that naive subjects in an immersive virtual environment experience a higher subjective sense of presence when they locomote by walking-in-place (virtual walking) than when they push-button-fly (along the floor plane). We replicated their study, adding real walking as a third condition. Our study confirmed their findings. We also found that real walking is significantly better than both virtual walking and flying in ease (simplicity, straightforwardness, naturalness) as a mode of locomotion. The greatest difference in subjective presence was between flyers and both kinds of walkers. In addition, subjective presence was higher for real walkers than virtual walkers, but the difference was statistically significant only in some models. Follow-on studies show virtual walking can be substantially improved by detecting footfalls with a head accelerometer. As in the Slater study, subjective presence significantly correlated with subjects’ degree of association with their virtual bodies (avatars). This, our strongest statistical result, suggests that substantial potential presence gains can be had from tracking all limbs and customizing avatar appearance. An unexpected by-product was that real walking through our enhanced version of Slater’s visual-cliff virtual environment (Figure 1) yielded a strikingly compelling virtual experience—the strongest we and most of our visitors have yet experienced. The most needed system improvement is the substitution of wireless technology for all links to the user. CR

Underlying principles of stereoscopic direct-view displays, binocular head-mounted displays, and autostereoscopic direct-view displays are explained and some early work as well as the state of the art in those technologies are reviewed. Stereoscopic displays require eyewear and can be categorized based on the multiplexing scheme as: 1) color multiplexed (old technology but there are some recent developments; low-quality due to color reproduction and crosstalk issues; simple and does not require additional electronics hardware); 2) polarization multiplexed (requires polarized light output and polarization-based passive eyewear; high-resolution and high-quality displays available); and 3) time multiplexed (requires faster display hardware and active glasses synchronized with the display; high-resolution commercial products available). Binocular head-mounted displays can readily provide 3-D, virtual images, immersive experience, and more possibilities for interactive displays. However, the bulk of the optics, matching of the left and right ocular images and obtaining a large field of view make the designs quite challenging. Some of the recent developments using unconventional optical relays allow for thin form factors and open up new possibilities. Autostereoscopic displays are very attractive as they do not require any eyewear. There are many possibilities in this category including: two-view (the simplest implementations are with a parallax barrier or a lenticular screen), multiview, head tracked (requires active optics to redirect the rays to a moving viewer), and super multiview (potentially can solve the accommodation-convergence mismatch problem). Earlier 3-D booms did not last long mainly due to the unavailability of enabling technologies and the content. Current developments in the hardware technologies provide a renewed interest in 3-D displays both from the consumers and the display manufacturers, which is evidenced by the recent commercial products and new research results in this field.

To reduce elderly falling risk in the area where video surveillance system is unavailable due to privacy reason and so on, we need a monitor system which does no recognize what they are doing. This paper presents an elderly-falling detection system using ultrasonic sensors. The ultrasonic technology-based multi sensors are couple of receiver and transmitter together, which are connected to Arduino microcontroller in order to send the elderly person's fall related signal using WiFi to the processing unit. The sensors are positioned as an array on the roof and wall in the room. The signal is analyzed to recognize human by sensing distance, and detect the action such as standing, sitting, and falling by pattern matching with the standard templates of top and side signal. In experiments, the proposed system can recognize accuracy of falling detection is 93% approximately.

Dante II is a unique walking robot that provides important insight into high-mobility robotic locomotion and remote robotic exploration. Dante II’s uniqueness stems from its combined legged and rappelling mobility system, its scanning-laser rangefinder, and its multilevel control scheme. In 1994 Dante II was deployed and successfully tested in a remote Alaskan volcano, as a demonstration of the fieldworthiness of these technologies. For more than five days the robot explored alone in the volcano crater using a combination of supervised autonomous control and teleoperated control. Human operators were located 120 km distant during the mission. This article first describes in detail the robot, support systems, control techniques, and user interfaces. We then describe results from the battery of field tests leading up to and including the volcanic mission. Finally, we put forth important lessons which comprise the legacy of this project. We show that framewalkers are appropriate for rappelling in severe terrain, though tether systems have limitations. We also discuss the importance of future “autonomous” systems to realize when they require human support rather than relying on humans for constant oversight. KEY WORDS—rappelling, walking robot, supervised autonomy, behavior-based control, terrain visualization, volcano exploration

Traditionally, the indoor scene classification problem has been approached from a 2D image recognition point of view. In most visual scene classification systems, a descriptor for the input image is generated to obtain a suitable representation that includes features related to color, shape or spatial information. Techniques based on the use of a spatial pyramid have proven to be adequate to perform this step. In the past years, on the other hand, 3D sensors have become widely available, which allows to include new information sources to the framework previously described. In this work we rely on RGB-D data to extend the spatial pyramid approach, aimed at building descriptors that can lead to a more robust representation against changing lighting conditions. The proposed descriptors are evaluated on the RobotVision@ImageCLEF-2013 benchmark dataset, remarkably outperforming state-of-the-art 3D local and global descriptors.

This paper presents a multi-cue approach to curb recognition in urban traffic. We propose a novel texture-based curb classifier using local receptive field (LRF) features in conjunction with a multi-layer neural network. This classification module operates on both intensity images and on three-dimensional height profile data derived from stereo vision. We integrate the proposed multi-cue curb classifier as an additional measurement module into a state-of-the-art Kaiman filter-based urban lane recognition system. Our experiments involve a challenging real-world dataset captured in urban traffic with manually labeled ground-truth. We quantify the benefit of the proposed multi-cue curb classifier in terms of the improvement in curb localization accuracy of the integrated system. Our results indicate a 25% reduction of the average curb localization error at real-time processing speeds.

To facilitate scene understanding and robot navigation in a modern urban area, we design a multilayer feature graph (MFG) based on two views from an on-board camera. The nodes of an MFG are features such as scale invariant feature transformation (SIFT) feature points, line segments, lines, and planes while edges of the MFG represent different geometric relationships such as adjacency, parallelism, collinearity, and coplanarity. MFG also connects the features in two views and the corresponding 3D coordinate system. Building on SIFT feature points and line segments, MFG is constructed using feature fusion which incrementally, iteratively, and extensively verifies the aforementioned geometric relationships using random sample consensus (RANSAC) framework. Physical experiments show that MFG can be successfully constructed in urban area and the construction method is demonstrated to be very robust in identifying feature correspondence.

An increasing safety and reducing road accidents, thereby saving lives are one of great interest in the context of Advanced Driver Assistance Systems. Apparently, among the complex and challenging tasks of future road vehicles is road lane detection or road boundaries detection. It is based on lane detection (which includes the localization of the road, the determination of the relative position between vehicle and road, and the analysis of the vehiclepsilas heading direction). One of the principal approaches to detect road boundaries and lanes using vision system on the vehicle. However, lane detection is a difficult problem because of the varying road conditions that one can encounter while driving. In this paper, a vision-based lane detection approach capable of reaching real time operation with robustness to lighting change and shadows is presented. The system acquires the front view using a camera mounted on the vehicle then applying few processes in order to detect the lanes. Using a pair of hyperbolas which are fitting to the edges of the lane, those lanes are extracted using Hough transform. The proposed lane detection system can be applied on both painted and unpainted road as well as curved and straight road in different weather conditions. This approach was tested and the experimental results show that the proposed scheme was robust and fast enough for real time requirements. Eventually, a critical overview of the methods were discussed, their potential for future deployment were assist.

It is not known how the flexible protective zone maintained around oneself during locomotion (personal space or PS; see [Gérin-Lajoie M, Richards CL, McFadyen BJ. The negotiation of stationary and moving obstructions during walking: anticipatory locomotor adaptations and preservation of personal space. Motor Control 2005;9:242-69]) is modulated with walking speed, whether both sides of the PS are symmetrical, and whether the circumvention of physical and virtual obstructions elicit the same use of such PS. Personal space was measured in ten adults as they circumvented a cylindrical obstacle that was stationary within their path. Both left and right passes were performed at natural self-selected, slow and fast walking speeds. The same circumvention task was also performed at natural speeds in an immersive virtual environment (VE) replicating the same obstruction scenario. The shape and size of PS were maintained across walking speeds, and a smaller PS was generally observed on the dominant side. The general shape and lateral bias of the PS were preserved in the VE while its size was slightly increased. The systematic behavior across walking speeds and types of environment and the lateral bias suggest that PS is used to control navigation. This study deepens our understanding of normal adaptive walking behavior and has implications for the development of better tools for the assessment and retraining of locomotor capacity in different populations, from people with walking deficits to elite athletes. Since the PS behavior was shown to be robust in the VE used for this study, the virtual reality technology is proposed as a promising platform for the development of such assessment and retraining applications.

This paper describes the environment perception system designed for intelligent vehicle SmartV-II, which won the 2010 Future Challenge. This system utilizes the cooperation of multiple lasers and cameras to realize several necessary functions of autonomous navigation: road curb detection, lane detection and traffic sign recognition. Multiple single scan lasers are integrated to detect the road curb based on Z-variance method. Vision based lane detection is realized by two scans method combining with image model. Haar-like feature based method is applied for traffic sign detection and SURF matching method is used for sign classification. The results of experiments validate the effectiveness of the proposed algorithms and the whole system.


Question: Identifying ascending staircases through the utilization of stereo vision technology.

This document offers an in-depth classification and assessment of dense two-frame stereo correspondence algorithms, a critical area within computer vision research. Despite the proliferation of stereo correspondence algorithms, there has been a notable absence of detailed performance analysis. To address this, we have developed a comprehensive taxonomy of dense, two-frame stereo techniques. This taxonomy is designed to evaluate the distinct elements and design choices in individual stereo algorithms. Utilizing this framework, we conduct a comparative analysis of existing stereo methods and present empirical data assessing the performance of numerous variations. To foster a unified software environment and facilitate straightforward evaluation, we have crafted a standalone, adaptable C++ implementation. This software allows for the assessment of individual components and can be easily expanded to incorporate new algorithms. Additionally, we have generated a series of multi-frame stereo data sets, complete with ground truth data, which are now available online. Alongside the release of our code and data sets, we present a comprehensive evaluation of a wide array of high-performing stereo algorithms currently in use.

In a related context, the question pertains to the detection of ascending stairs using stereo vision. While the provided documents focus on stereo correspondence algorithms and their evaluation, a similar document could delve into the application of stereo vision for specific tasks, such as stair detection. This could involve a review of algorithms tailored for stair detection, their performance metrics, and a comparative analysis similar to the one presented in the original document. The document might also highlight the development of a specialized software tool for evaluating stair detection algorithms, complete with a collection of relevant data sets and ground truth information.

The paper addresses the problem of autonomous navigation of a car-like robot evolving in an urban environment. Such an environment exhibits an heterogeneous geometry and is cluttered with moving obstacles. Furthermore, in this context, motion safety is a critical issue. The proposed approach to the problem lies in the coupling of two crucial robotic capabilities, namely perception and planning. The main contributions of this work are the development and integration of these modules into one single application, considering explicitly the constraints related to the environment and the system

Detecting human beings accurately in a visual surveillance system is crucial for diverse application areas including abnormal event detection, human gait characterization, congestion analysis, person identification, gender classification and fall detection for elderly people. The first step of the detection process is to detect an object which is in motion. Object detection could be performed using background subtraction, optical flow and spatio-temporal filtering techniques. Once detected, a moving object could be classified as a human being using shape-based, texture-based or motion-based features. A comprehensive review with comparisons on available techniques for detecting human beings in surveillance videos is presented in this paper. The characteristics of few benchmark datasets as well as the future research directions on human detection have also been discussed.

Recognizing and localizing queried objects in range images plays an important role for robotic manipulation and navigation. Even though it has been steadily studied, it is still a challenging task for scenes with occlusion and clutter.

People with visual impairment face enormous difficulties in terms of their mobility as they do not have enough information about their location and orientation with respect to traffic and obstacles on their route. Visually impaired people can navigate unknown areas by relying on the assistance of canes, other people, or specially trained guide dogs. The traditional ways of using guide dogs and long-cane only help to avoid obstacles, not to know where they are. The research presented in this paper introduces a mobile assistant navigation prototype to locate and direct blind people indoors. Since most of the existing navigation systems developed so far for blind people employ a complex conjunction of positioning systems, video cameras, locationbased and image processing algorithms, we designed an affordable low-cost prototype navigation system for orienting and tracking the position of blind people in complex environments. The prototype system is based on the inertial navigation system and experiments have been performed on NXT Mindstorms platform.

Several common systems satisfy some but not all of the VR definition above. Flight simulators provide vehicle tracking, not head tracking, and do not generally operate in binocular stereo. Omnimax theaters give a large angle of view [8], occasionally in stereo, but are not interactive. Head-tracked monitors [4][6] provide all but a large angle of view. Head-mounted displays (HMD) [7][13] and BOOMs [9] use motion of the actual display screens to achieve VR by our definition. Correct projection of the imagery on large screens can also create a VR experience, this being the subject of this paper. This paper describes the CAVE (CAVE Automatic Virtual Environment) virtual reality/scientific visualization system in detail and demonstrates that projection technology applied to virtual-reality goals achieves a system that matches the quality of workstation screens in terms of resolution, color, and flicker-free stereo. In addition, this format helps reduce the effect of common tracking and system latency errors. The off-axis perspective projection techniques we use are shown to be simple and straightforward. Our techniques for doing multi-screen stereo vision are enumerated, and design barriers, past and current, are described. Advantages and disadvantages of the projection paradigm are discussed, with an analysis of the effect of tracking noise and delay on the user. Successive refinement, a necessary tool for scientific visualization, is developed in the virtual reality context. The use of the CAVE as a one-to-many presentation device at SIGGRAPH '92 and Supercomputing '92 for computational science data is also mentioned. Previous work in the VR area dates back to Sutherland [12], who in 1965 wrote about the “Ultimate Display.” Later in the decade at the University of Utah, Jim Clark developed a system that allowed wireframe graphics VR to be seen through a headmounted, BOOM-type display for his dissertation. The common VR devices today are the HMD and the BOOM. Lipscomb [4] showed a monitor-based system in the IBM booth at SIGGRAPH '91 and Deering [6] demonstrated the Virtual Portal, a closets ized three-wal l project ion-based system, in the Sun Microsystems' booth at SIGGRAPH '92. The CAVE, our projection-based VR display [3], also premiered at SIGGRAPH '92. The Virtual Portal and CAVE have similar intent, but different implementation schemes.

Model-based recognition and motion tracking depends upon the ability to solve for projection and model parameters that will best fit a 3-D model to matching 2-D image features. This paper extends current methods of parameter solving to handle objects with arbitrary curved surfaces and with any number of internal parameters representing articulations, variable dimensions, or surface deformations. Numerical stabilization methods are developed that take account of inherent inaccuracies in the image measurements and allow useful solutions to be determined even when there are fewer matches than unknown parameters. The LevenbergMarquardt method is used to always ensure convergence of the solution. These techniques allow model-based vision to be used for a much wider class of problems than was possible with previous methods. Their application is demonstrated for tracking the motion of curved, parameterized objects. This paper has been published in IEEE Transactions on Pattern Analysis and Machine Intelligence, 13, 5 (May 1991), pp. 441–450.

We describe a system to detect objects in three-dimensional space using video and inertial sensors (accelerometer and gyrometer), ubiquitous in modern mobile platforms from phones to drones. Inertials afford the ability to impose class-specific scale priors for objects, and provide a global orientation reference. A minimal sufficient representation, the posterior of semantic (identity) and syntactic (pose) attributes of objects in space, can be decomposed into a geometric term, which can be maintained by a localization-and-mapping filter, and a likelihood function, which can be approximated by a discriminatively-trained convolutional neural network The resulting system can process the video stream causally in real time, and provides a representation of objects in the scene that is persistent: Confidence in the presence of objects grows with evidence, and objects previously seen are kept in memory even when temporarily occluded, with their return into view automatically predicted to prime re-detection.

We present an analysis of measuring stride-to-stride gait variability passively, in a home setting using two vision based monitoring techniques: anonymized video data from a system of two web-cameras, and depth imagery from a single Microsoft Kinect. Millions of older adults fall every year. The ability to assess the fall risk of elderly individuals is essential to allowing them to continue living safely in independent settings as they age. Studies have shown that measures of stride-to-stride gait variability are predictive of falls in older adults. For this analysis, a set of participants were asked to perform a number of short walks while being monitored by the two vision based systems, along with a marker based Vicon motion capture system for ground truth. Measures of stride-to-stride gait variability were computed using each of the systems and compared against those obtained from the Vicon.

I n this paper. we present the mechanism, system cor$guration, basic control algorithm and integrated ,functions of the Nonda humanoid robot. Like its human counterpart, this robot lzas the ability to move forward and backward, sideways to the right or the left, as well as diagonally. I n addition, the robot can turn in any direction, walk up and down stairs continuously. Furthermore, due to its unique posture stability control, the robot is able to maintain its balance despite unexpected complications such as uneven ground su faces As a part of its integrated functions, this robot is able to inove on a planned path autonomously and to perform simple operations via wireless tele-operation.

We consider the task of driving a remote control car at high speeds through unstructured outdoor environments. We present an approach in which supervised learning is first used to estimate depths from single monocular images. The learning algorithm can be trained either on real camera images labeled with ground-truth distances to the closest obstacles, or on a training set consisting of synthetic graphics images. The resulting algorithm is able to learn monocular vision cues that accurately estimate the relative depths of obstacles in a scene. Reinforcement learning/policy search is then applied within a simulator that renders synthetic scenes. This learns a control policy that selects a steering direction as a function of the vision system's output. We present results evaluating the predictive ability of the algorithm both on held out test data, and in actual autonomous driving experiments.


Question: Identifying ascending staircases through the application of stereo vision techniques.

This article describes an advanced vision-based road detection system integrated into the MOB-LAB land vehicle, designed to operate effectively in challenging shadow environments. The system employs a geometric transformation and rapid morphological processing to accurately identify road markings, even on textured roads under severe shadow conditions. Leveraging a specialized massively parallel architecture (PAPRICA), the system achieves an impressive processing speed of approximately 17 Hz, ensuring real-time detection capabilities under critical circumstances.

The sheer volume of data generated by depth cameras provides a challenge to process in real time, in particular when used for indoor mobile robot localization and navigation. We introduce the Fast Sampling Plane Filtering (FSPF) algorithm to reduce the volume of the 3D point cloud by sampling points from the depth image, and classifying local grouped sets of points as belonging to planes in 3D (the “plane filtered” points) or points that do not correspond to planes within a specified error margin (the “outlier” points). We then introduce a localization algorithm based on an observation model that down-projects the plane filtered points on to 2D, and assigns correspondences for each point to lines in the 2D map. The full sampled point cloud (consisting of both plane filtered as well as outlier points) is processed for obstacle avoidance for autonomous navigation. All our algorithms process only the depth information, and do not require additional RGB data. The FSPF, localization and obstacle avoidance algorithms run in real time at full camera frame rates (30Hz) with low CPU requirements (16%). We provide experimental results demonstrating the effectiveness of our approach for indoor mobile robot localization and navigation. We further compare the accuracy and robustness in localization using depth cameras with FSPF vs. alternative approaches that simulate laser rangefinder scans from the 3D data.

In this paper a novel reconstruction method is presented that uses the topological relationship of detected image features to create a highly abstract but semantically rich 3D model of the reconstructed scenes. In the first step, a combined image-based reconstruction of points and lines is performed based on the current state of art structure from motion methods. Subsequently, connected planar threedimensional structures are reconstructed by a novel method that uses the topological relationships between the detected image features. The reconstructed 3D models enable a simple extraction of geometric shapes, such as rectangles, in the scene.

Bicycles that share the road with intelligent vehicles present particular challenges for automated perception systems. Bicycle detection is important because bicycles share the road with vehicles and can move at comparable speeds in urban environments. From a computer vision standpoint, bicycle detection is challenging as bicycle's appearance can change dramatically between viewpoints and a person riding on the bicycle is a non-rigid object. In this paper, we present a vision-based framework to detect and track bicycles that takes into account these issues. A mixture model of multiple viewpoints is defined and trained via a Support Vector Machine (SVM) to detect bicycles under a variety of circumstances. Each component of the model uses a part-based representation and known geometric context is used to improve overall detection efficiency. An extended Kalman filter (EKF) is used to estimate the position and velocity of the bicycle in vehicle coordinates. We demonstrate the effectiveness of this approach through a series of experiments run on video data of moving bicycles captured from a vehicle-mounted camera

We present a system that transforms a monocular video of a soccer game into a moving 3D reconstruction, in which the players and field can be rendered interactively with a 3D viewer or through an Augmented Reality device. At the heart of our paper is an approach to estimate the depth map of each player, using a CNN that is trained on 3D player data extracted from soccer video games. We compare with state of the art body pose and depth estimation techniques, and show results on both synthetic ground truth benchmarks, and real YouTube soccer footage.

A novel scene reconstruction technique is presented, different from previous approaches in its ability to cope with large changes in visibility and its modeling of intrinsic scene color and texture information. The method avoids image correspondence problems by working in a discretized scene space whose voxels are traversed in a fixed visibility ordering. This strategy takes full account of occlusions and allows the input cameras to be far apart and widely distributed about the environment. The algorithm identifies a special set of invariant voxels which together form a spatial and photometric reconstruction of the scene, fully consistent with the input images. The approach is evaluated with images from both inward-facing and outward-facing cameras.

This paper describes the vehicle guidance system of an autonomous vehicle. This system is part of the control structure of the vehicle and consists of a path generator, a motion planning algorithm and a sensor fusion module. A stereo vision sensor and a DGPS sensor are used as position sensors. The trajectory for the vehicle motion is generated in a rst step by using only information from a digital map. Object-detecting sensors such as the stereo vision sensor, three laserscanner and a radar sensor observe the vehicle environment and report detected objects to the sensor fusion module. This information is used to dynamically update the planned vehicle trajectory to the nal vehicle motion.

We present a survey of the most significant techniques, used in the last few years, concerning the coded structured light methods employed to get 3D information. In fact, depth perception is one of the most important subjects in computer vision. Stereovision is an attractive and widely used method, but, it is rather limited to make 3D surface maps, due to the correspondence problem. The correspondence problem can be improved using a method based on structured light concept, projecting a given pattern on the measuring surfaces. However, some relations between the projected pattern and the reflected one must be solved. This relationship can be directly found codifying the projected light, so that, each imaged region of the projected pattern carries the needed information to solve the correspondence problem. ( 1998 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved. Pattern projection Correspondence problem Active stereo Depth perception Range data Computer vision.

One challenge in multimodal interface research is the lack of robust subsystems that support multimodal interactions. By focusing on a chair—an object that is involved in virtually all human–computer interactions, the sensing chair project enables an ordinary office chair to become aware of its occupant’s actions and needs. Surface-mounted pressure distribution sensors are placed over the seatpan and backrest of the chair for real time capturing of contact information between the chair and its occupant. Given the similarity between a pressure distribution map and a grayscale image, pattern recognition techniques commonly used in computer and robot vision, such as principal components analysis, have been successfully applied to solving the problem of sitting posture classification. The current static posture classification system operates in real time with an overall classification accuracy of 96% and 79% for familiar (people it had felt before) and unfamiliar users, respectively. Future work is aimed at a dynamic posture tracking system that continuously tracks not only steady-state (static) but transitional (dynamic) sitting postures. Results reported here form important stepping stones toward an intelligent chair that can find applications in many areas including multimodal interfaces, intelligent environment, and safety of automobile operations.

Detecting human beings accurately in a visual surveillance system is crucial for diverse application areas including abnormal event detection, human gait characterization, congestion analysis, person identification, gender classification and fall detection for elderly people. The first step of the detection process is to detect an object which is in motion. Object detection could be performed using background subtraction, optical flow and spatio-temporal filtering techniques. Once detected, a moving object could be classified as a human being using shape-based, texture-based or motion-based features. A comprehensive review with comparisons on available techniques for detecting human beings in surveillance videos is presented in this paper. The characteristics of few benchmark datasets as well as the future research directions on human detection have also been discussed.

This paper presents the vision-based road detection system currently installed onto the MOB-LAB land vehicle. Based on a geometrical transform and on a fast morphological processing, the system is capable to detect road markings even in extremely severe shadow conditions on at and structured roads. The use of a special-purpose massively architecture (PAPRICA) allows to achieve a processing rate of about 17 Hz.


Question: Pattern discovery in the Bitcoin transaction directed hypergraph through exchange analysis.

This paper introduces the concept of "network motifs," recurrent patterns of interconnections that are significantly more prevalent in complex networks than in randomized networks, serving as fundamental building blocks for these intricate systems. Motifs have been identified across a diverse range of networks, spanning biochemical pathways, neural networks, ecological food webs, and engineered systems. Notably, the motifs shared by ecological food webs exhibit distinct characteristics from those found in the genetic regulatory networks of organisms like Escherichia coli and Saccharomyces cerevisiae, as well as those observed in the World Wide Web. Interestingly, similar motifs are present in networks tasked with information processing, regardless of the vastly different nature of their components—ranging from biomolecules within a cell to synaptic connections between neurons in Caenorhabditis elegans. These motifs may serve as universal identifiers, categorizing networks into distinct classes. This approach holds the potential to unravel the foundational elements that underpin the structure of most networks, offering insights into their design principles and functional implications.

Due to the explosion of social networking and the information sharing among their users, the interest in analyzing social networks has increased over the recent years. Two general interests in this kind of studies are community detection and visualization. In the first case, most of the classic algorithms for community detection use only the structural information to identify groups, that is, how clusters are formed according to the topology of the relationships. However, these methods do not take into account any semantic information which could guide the clustering process, and which may add elements to conduct further analyses. In the second case most of the layout algorithms for clustered graphs have been designed to differentiate the groups within the graph, but they are not designed to analyze the interactions between such groups. Identifying these interactions gives an insight into the way different communities exchange messages or information, and allows the social network researcher to identify key actors, roles, and paths from one community to another.
 This article presents a novel model to use, in a conjoint way, the semantic information from the social network and its structural information to, first, find structurally and semantically related groups of nodes, and second, a layout algorithm for clustered graphs which divides the nodes into two types, one for nodes with edges connecting other communities and another with nodes connecting nodes only within their own community. With this division the visualization tool focuses on the connections between groups facilitating deep studies of augmented social networks.

The concept of peer-to-peer applications is not new, nor is the concept of distributed hash tables. What emerged in 2008 with the publication of the Bitcoin white paper was an incentive structure that unified these two software paradigms with a set of economic stimuli to motivate the creation of a dedicated computing network orders of magnitude more powerful than the world’s fastest supercomputers. The purpose of which is the maintenance of a massive distributed database known as the Bitcoin “block chain”. Apart from the digital currency it enables, block chain technology is a fascinating new computing paradigm with broad implications for the future development of the World Wide Web, and by extension, the further growth of Linked Data and the Semantic Web. This work is divided into two main sections, we first demonstrate how block chain technologies can contribute toward the realization of a more robust Semantic Web, and subsequently we provide a framework wherein the Semantic Web is utilized to ameliorate block chain technology itself.

Hyperledger Fabric is a "permissioned" blockchain architecture, providing a consistent distributed ledger, shared by a set of "peers." As with every blockchain architecture, the core principle of Hyperledger Fabric is that all the peers must have the same view of the shared ledger, making it challenging to support private data for the different peers. Extending Hyperledger Fabric to support private data (that can influence transactions) would open the door to many exciting new applications, in areas from healthcare to commerce, insurance, finance, and more. In this work we explored adding private-data support to Hyperledger Fabric using secure multiparty computation (MPC). Specifically, in our solution the peers store on the chain encryption of their private data, and use secure MPC whenever such private data is needed in a transaction. This solution is very general, allowing in principle to base transactions on any combination of public and private data. We created a demo of our solution over Hyperledger Fabric v1.0, implementing a bidding system where sellers can list assets on the ledger with a secret reserve price, and bidders publish their bids on the ledger but keep secret the bidding price itself. We implemented a smart contract (aka "chaincode") that runs the auction on this secret data, using a simple secure-MPC protocol that was built using the EMP-toolkit library. The chaincode itself was written in Go, and we used the SWIG library to make it possible to call our protocol implementation in C++. We identified two basic services that should be added to Hyperledger Fabric to support our solution, and are now working on implementing them.

Contributions to this research made by a member of The Financial Access Initiative.

Sergei Arnautov, Bohdan Trach, Franz Gregor, Thomas Knauth, Andre Martin, Christian Priebe, Joshua Lind, Divya Muthukumaran, Dan O’Keeffe, Mark L Stillwell, David Goltzsche, David Eyers, Rüdiger Kapitza, Peter Pietzuch, and Christof Fetzer Fakultät Informatik, TU Dresden, christof.fetzer@tu-dresden.de Dept. of Computing, Imperial College London, prp@imperial.ac.uk Informatik, TU Braunschweig, rrkapitz@ibr.cs.tu-bs.de Dept. of Computer Science, University of Otago, dme@cs.otago.ac.nz

Stealth address prevents public association of a blockchain transaction’s output with a recipient’s wallet address and hides the actual destination address of a transaction. While stealth address provides an effective privacy-enhancing technology for a cryptocurrency network, it requires blockchain nodes to actively monitor all the transactions and compute the purported destination addresses, which restricts its application for resource-constrained environments like Internet of Things (IoT). In this paper, we propose DKSAP-IoT, a faster dual-key stealth address protocol for blockchain-based IoT systems. DKSAP-IoT utilizes a technique similar to the TLS session resumption to improve the performance and reduce the transaction size at the same time between two communication peers. Our theoretical analysis as well as the extensive experiments on an embedded computing platform demonstrate that DKSAPIoT is able to reduce the computational overhead by at least 50% when compared to the state-of-the-art scheme, thereby paving the way for its application to blockchain-based IoT systems.

In this paper, a method for transforming a relational database to a graph database model is described. In this approach, the dependency graphs for the entities in the system are transformed into star graphs. This star graph model is transformed into a hyper graph model for the relational database, which, in turn, can be used to develop the domain relationship model that can be converted in to a graph database model.

This paper presents efficient off-line anonymous e-cash schemes where a user can withdraw a wallet containing 2 coins each of which she can spend unlinkably. Our first result is a scheme, secure under the strong RSA and the y-DDHI assumptions, where the complexity of the withdrawal and spend operations is O( + k) and the user’s wallet can be stored using O( +k) bits, where k is a security parameter. The best previously known schemes require at least one of these complexities to be O(2 · k). In fact, compared to previous e-cash schemes, our whole wallet of 2 coins has about the same size as one coin in these schemes. Our scheme also offers exculpability of users, that is, the bank can prove to third parties that a user has double-spent. We then extend our scheme to our second result, the first e-cash scheme that provides traceable coins without a trusted third party. That is, once a user has double spent one of the 2 coins in her wallet, all her spendings of these coins can be traced. However, the price for this is that the complexity of the spending and of the withdrawal protocols becomes O( · k) and O( · k + k) bits, respectively, and wallets take O( · k) bits of storage. All our schemes are secure in the random oracle model.

We consider the problem of human-assisted graph search: given a directed acyclic graph with some (unknown) target node(s), we consider the problem of finding the target node(s) by asking an omniscient human questions of the form “Is there a target node that is reachable from the current node?”. This general problem has applications in many domains that can utilize human intelligence, including curation of hierarchies, debugging workflows, image segmentation and categorization, interactive search and filter synthesis. To our knowledge, this work provides the first formal algorithmic study of the optimization of human computation for this problem. We study various dimensions of the problem space, providing algorithms and complexity results. We also compare the performance of our algorithm against other algorithms, for the problem of webpage categorization on a real taxonomy. Our framework and algorithms can be used in the design of an optimizer for crowdsourcing platforms such as Mechanical Turk.

In the modern world that we live in everything is gradually shifting towards a more digitized outlook. From teaching methods to online transactions, every small aspect of our life is given a more technological touch. In such a case “money” is not left behind either. An approach towards digitizing money led to the creation of “bitcoin”. Bitcoin is the most efficient crypto-currency so far in the history of digital currency. Ruling out the interference of any third party which monitors the cash flow, Bitcoin is a decentralized form of online currency and is widely accepted for internet transactions all over the world. The need for digital money would be extensive in the near future.