Question: An ontology-driven method for traffic scene representation, rule-based situational understanding, and decision-making in autonomous vehicles.

This paper revisits the concept of the Semantic Web, exploring its evolution and the role of intelligent agents in its ecosystem. While the Web is replete with specialized bots and agents designed for specific functions, such as shopbots and auction bots, their capabilities are often limited to handling homogenous data and information. Critics of the Semantic Web suggest that its promise of widespread, versatile agent-mediated services has not been fully realized. However, we posit that the true potential of agents is contingent upon the maturity of standards that enable them to interpret and interact with diverse data types. Over the last half-decade, the development of Web standards aimed at facilitating shared meaning has advanced significantly. This progress is essential for nurturing an environment where intelligent agents can thrive, bridging the gap between human and machine interaction across the heterogeneous landscape of the Web. The ongoing refinement of these standards is crucial for unlocking the full potential of the Semantic Web, where agents can autonomously perform complex tasks and make informed decisions on behalf of their human or corporate users, akin to the envisioned scenarios of traffic scene modeling and decision-making for automated vehicles in a future connected world.

The recent DARPA-sponsored automated vehicle "Challenges" have generated strong interest in both the research community and the general public, raising consciousness about the possibilities for vehicle automation. Driverless vehicles make good subjects for the visually-oriented media, and they pose enough interesting research challenges to occupy generations of graduate students. However, automated vehicles also have the potential to help solve a variety of real-world problems. Engineers need to think carefully about which of those problems we are actually solving.A well-engineered system should be designed to satisfy specific needs, and those needs should be reflected in the definition of system requirements. Alternative technological approaches can then be evaluated and traded off based on their ability to meet the requirements. The article describes the rather different needs of the public road transportation system and the military transportation system, and then shows how those needs influence the requirements for automated vehicle systems. These requirements point toward significantly different technical approaches, but it is possible to find some limited areas of technical commonality.

The Robotics Institute at Carnegie Mellon University (CMU) and the Sarnoff Corporation are developing a system for autonomous Video Surveillance and Monitoring. The technical objective is to use multiple, cooperative video sensors to provide continuous coverage of people and vehicles in cluttered environments. This paper presents an overview of the system and significant results achieved to date.

At this moment Automous cars are probably the biggest and most talked about technology in the Robotics Reserach Community. In spite of great technolgical advances over past few years a full fledged autonomous car is still far from reality. This article talks about the existing system and discusses the possibility of a Computer Vision enabled driving being superior than the LiDar based system. A detailed overview of privacy violations that might arise from autonomous driving has been discussed in detail both from a technical as well as legal perspective. It has been proved through evidence and arguments that efficient and accurate estimation and efficient solution of the contraint satisfaction problem adressed in the case of autonomous cars are negatively correlated with the preserving the privacy of the user. It is a very difficult trade-off since both are very important aspects and has to be taken into account. The fact that one cannot compromise with the safety issues of the car makes it inevidable to run into serious privacy concerns that might have adverse social and political effects.

To enable situated human–robot interaction (HRI), an autonomous robot must both understand and control proxemics—the social use of space—to employ natural communication mechanisms analogous to those used by humans. This work presents a computational framework of proxemics based on data-driven probabilistic models of how social signals (speech and gesture) are produced (by a human) and perceived (by a robot). The framework and modelswere implemented as autonomous proxemic behavior systems for sociable robots, including: (1) a sampling-based method for robot proxemic goal state estimation with respect to human–robot distance and orientation parameters, (2) a reactive proxemic controller for goal state realization, and (3) a cost-based trajectory planner for maximizing automated robot speech and gesture recognition rates along a path to the goal state. Evaluation results indicate that the goal state estimation and realization significantly improve upon past work in human–robot proxemics with respect to “interaction potential”—predicted automated speech and gesture recognition rates as the robot enters into and engages in faceto-face social encounters with a human user—illustrating their efficacy to support richer robot perception and autonomy in HRI. Electronic supplementary material The online version of this article (doi:10.1007/s10514-016-9572-2) contains supplementary material, which is available to authorized users. B Ross Mead rossmead@usc.edu 1 3710 McClintock Avenue, RTH 423, Los Angeles, CA 90089-0781, USA 2 3710 McClintock Avenue, RTH 407, Los Angeles, CA 90089-0781, USA

To understand driving environments effectively, it is important to achieve accurate detection and classification of objects detected by sensor-based intelligent vehicle systems, which are significantly important tasks. Object detection is performed for the localization of objects, whereas object classification recognizes object classes from detected object regions. For accurate object detection and classification, fusing multiple sensor information into a key component of the representation and perception processes is necessary. In this paper, we propose a new object-detection and classification method using decision-level fusion. We fuse the classification outputs from independent unary classifiers, such as 3D point clouds and image data using a convolutional neural network (CNN). The unary classifiers for the two sensors are the CNN with five layers, which use more than two pre-trained convolutional layers to consider local to global features as data representation. To represent data using convolutional layers, we apply region of interest (ROI) pooling to the outputs of each layer on the object candidate regions generated using object proposal generation to realize color flattening and semantic grouping for charge-coupled device and Light Detection And Ranging (LiDAR) sensors. We evaluate our proposed method on a KITTI benchmark dataset to detect and classify three object classes: cars, pedestrians and cyclists. The evaluation results show that the proposed method achieves better performance than the previous methods. Our proposed method extracted approximately 500 proposals on a 1226 × 370 image, whereas the original selective search method extracted approximately 10 6 × n proposals. We obtained classification performance with 77.72% mean average precision over the entirety of the classes in the moderate detection level of the KITTI benchmark dataset.

Accurate maps of the static environment are essential for many advanced driver-assistance systems. A new method for the fast computation of occupancy grid maps with laser range-finders and radar sensors is proposed. The approach utilizes the Graphics Processing Unit to overcome the limitations of classical occupancy grid computation in automotive environments. It is possible to generate highly accurate grid maps in just a few milliseconds without the loss of sensor precision. Moreover, in the case of a lower resolution radar sensor it is shown that it is suitable to apply super-resolution algorithms to achieve the accuracy of a higher resolution laser-scanner. Finally, a novel histogram based approach for road boundary detection with lidar and radar sensors is presented.

We address the problem of multi-UAV-based surveillance in complex urban environments with occlusions. The problem consists of controlling the flight of UAVs with on-board cameras so that the coverage and recency of the information about the designated area is maximized. In contrast to the existing work, sensing constraints due to occlusions and UAV motion constraints are modeled realistically and taken into account. We propose a novel \emph{occlusion-aware} surveillance algorithm based on a decomposition of the surveillance problem into a variant of the 3D art gallery problem and an instance of traveling salesman problem for Dubins vehicles. The algorithm is evaluated on the high-fidelity \textsc{AgentFly} UAV simulation testbed which accurately models all constraints and effects involved. The results confirm the importance of occlusion-aware flight path planning, in particular in the case of narrow-street areas and low UAV flight altitudes.

In this paper, a point-based Markov Decision Process (QMDP) algorithm is used for robust single-lane autonomous driving behavior control under uncertainties. Autonomous vehicle decision making is modeled as a Markov Decision Process (MDP), then extended to a QMDP framework. Based on MDP/QMDP, three kinds of uncertainties are taken into account: sensor noise, perception constraints and surrounding vehicles' behavior. In simulation, the QMDP-based reasoning framework makes the autonomous vehicle perform with differing levels of conservativeness corresponding to different perception confidence levels. Road tests also indicate that the proposed algorithm helps the vehicle in avoiding potentially unsafe situations under these uncertainties. In general, the results indicate that the proposed QMDP-based algorithm makes autonomous driving more robust to limited sensing ability and occasional sensor failures.

We study an algorithm that allows a vehicle to autonomously change lanes in a safe but personalized fashion without the driver's explicit initiation (e.g. activating the turn signals). Lane change initiation in autonomous driving is typically based on subjective rules, functions of the positions and relative velocities of surrounding vehicles. This approach is often arbitrary, and not easily adapted to the driving style preferences of an individual driver. Here we propose a data-driven modeling approach to capture the lane change decision behavior of human drivers. We collect data with a test vehicle in typical lane change situations and train classifiers to predict the instant of lane change initiation with respect to the preferences of a particular driver. We integrate this decision logic into a model predictive control (MPC) framework to create a more personalized autonomous lane change experience that satisfies safety and comfort constraints. We show the ability of the decision logic to reproduce and differentiate between two lane changing styles, and demonstrate the safety and effectiveness of the control framework through simulations.

Detection of non-motorized road users, such as cyclists and pedestrians, is a challenging problem in collision warning/collision avoidance (CW/CA) systems as direct information (e.g. location, speed, and class) cannot be obtained from such users. In this paper, we propose a fusion of LIDAR data and a deep learning-based computer vision algorithm, to substantially improve the detection of regions of interest (ROIs) and subsequent identification of road users. Experimental results on the KITTI object detection benchmark quantify the effectiveness of incorporating LIDAR data with region-based deep convolutional networks. Thus our work provides another step towards the goal of designing safe and smart transportation systems of the future.


Question: An ontology-driven approach to traffic scene modeling, situational awareness based on traffic rules, and decision-making for autonomous vehicles.

This report introduces Pellet, a robust and efficient OWL-DL reasoner designed with practicality in mind, offering a blend of high performance, extensive middleware support, and innovative functionalities. Pellet, developed in Java and released under a permissive open-source license, has been adopted in a wide range of projects, spanning academic research to industrial applications. Notably, Pellet is the first sound and complete OWL-DL reasoner to offer comprehensive support for reasoning with individuals, including nominal support and conjunctive query capabilities, user-defined datatypes, and debugging tools for ontologies. The reasoner also supports several extensions to OWL-DL, such as a formalism for combining OWL-DL ontologies, a non-monotonic operator, and initial support for hybrid reasoning involving OWL and rules. Pellet has been proven to be a dependable tool for managing OWL-DL ontologies and exploring OWL extensions. This paper delves into the features, architectural design, and unique capabilities of Pellet, complemented by a performance comparison against other prominent OWL-DL reasoners, showcasing its reliability and efficiency in the field.

1Department of Computer Engineering, Chung-Ang University, Seoul, Korea 2Department of Computer Science, Universidad Autônoma de Madrid, Spain Correspondence Jai E. Jung, Department of Computer Engineering, Chung-Ang University, Korea. Email: j3ung@cau.ac.kr David Camacho, Department of Computer Science, Universidad Autonoma de Madrid, Spain. Email: david.camacho@uam.es Funding information Chung-Ang University Summary Smart traffic light control at intersections is 1 of the major issues in Intelligent Transportation System. In this paper, on the basis of the new emerging technologies of Internet of Things, we introduce a new approach for smart traffic light control at intersection. In particular, we firstly propose a connected intersection system where every objects such as vehicles, sensors, and traffic lights will be connected and sharing information to one another. By this way, the controller is able to collect effectively and mobility traffic flow at intersection in real-time. Secondly, we propose the optimization algorithms for traffic lights by applying algorithmic game theory. Specially, 2 game models (which are Cournot Model and Stackelberg Model) are proposed to deal with difference scenarios of traffic flow. In this regard, based on the density of vehicles, controller will make real-time decisions for the time durations of traffic lights to optimize traffic flow. To evaluate our approach, we have used Netlogo simulator, an agent-based modeling environment for designing and implementing a simple working traffic. The simulation results shows that our approach achieves potential performance with various situations of traffic flow.

Machine Learning, and in particular Deep Learning, are extremely capable tools for solving problems which are difficult, or intractable to tackle analytically. Application areas include pattern recognition, computer vision, speech and natural language processing. With the automotive industry aiming for increasing amount of automation in driving, the problems to solve become increasingly complex, which appeals to the use of supervised learning methods from Machine Learning and Deep Learning. With this approach, solutions to the problems are learned implicitly from training data, and inspecting their correctness is not possible directly. This presents concerns when the resulting systems are used to support safety-critical functions, as is the case with autonomous driving of automotive vehicles. This thesis studies the safety concerns related to learning systems within autonomous driving and applies a safety monitoring approach to a collision avoidance scenario. Experiments are performed using a simulated environment, with a deep learning system supporting perception for vehicle control, and a safety monitor for collision avoidance. The related operational situations and safety constraints are studied for an autonomous driving function, with potential faults in the learning system introduced and examined. Also, an example is considered for a measure that indicates trustworthiness of the learning system during operation.

Rules and ontologies play a key role in the layered architecture of the Semantic Web, as they are used to ascribe meaning to, and to reason about, data on the Web. While the Ontology Layer of the Semantic Web is quite developed, and the Web Ontology Language (OWL) is a W3C recommendation since a couple of years already, the rules layer is far less developed and an active area of research; a number of initiatives and proposals have been made so far, but no standard as been released yet. Many implementations of rule engines are around which deal with Semantic Web data in one or another way. This article gives a comprehensive, although not exhaustive, overview of such systems, describes their supported languages, and sets them in relation with theoretical approaches for combining rules and ontologies as foreseen in the Semantic Web architecture. In the course of this, we identify desired properties and common features of rule languages and evaluate existing systems against their support. Furthermore, we review technical problems underlying the integration of rules and ontologies, and classify representative proposals for theoretical integration approaches into different categories.

For autonomous parking applications to become possible, knowledge about the parking environment is required. Therefore, a real-time algorithm for detecting parked vehicles from radar data is presented. These data are first accumulated in an occupancy grid from which objects are detected by applying techniques borrowed from the computer vision field. Two random forest classifiers are trained to recognize two categories of objects: parallel-parked vehicles and cross-parked vehicles. Performances of the classifiers are evaluated as well as the capacity of the complete system to detect parked vehicles in real world scenarios.

Bicycles that share the road with intelligent vehicles present particular challenges for automated perception systems. Bicycle detection is important because bicycles share the road with vehicles and can move at comparable speeds in urban environments. From a computer vision standpoint, bicycle detection is challenging as bicycle's appearance can change dramatically between viewpoints and a person riding on the bicycle is a non-rigid object. In this paper, we present a vision-based framework to detect and track bicycles that takes into account these issues. A mixture model of multiple viewpoints is defined and trained via a Support Vector Machine (SVM) to detect bicycles under a variety of circumstances. Each component of the model uses a part-based representation and known geometric context is used to improve overall detection efficiency. An extended Kalman filter (EKF) is used to estimate the position and velocity of the bicycle in vehicle coordinates. We demonstrate the effectiveness of this approach through a series of experiments run on video data of moving bicycles captured from a vehicle-mounted camera

In this paper, we propose a computer vision based system for real-time robust traffic sign detection and recognition, especially developed for intelligent vehicle. In detection phase, a color-based segmentation method is used to scan the scene in order to quickly establish regions of interest (ROI). Sign candidates within ROIs are detected by a set of Haar wavelet features obtained from AdaBoost training. Then, the Speeded Up Robust Features (SURF) is applied for the sign recognition. SURF finds local invariant features in a candidate sign and matches these features to the features of template images that exist in data set. The recognition is performed by finding out the template image that gives the maximum number of matches. We have evaluated the proposed system on our intelligent vehicle SmartVII. A recognition accuracy of over 90% in real-time has been achieved.

To understand driving environments effectively, it is important to achieve accurate detection and classification of objects detected by sensor-based intelligent vehicle systems, which are significantly important tasks. Object detection is performed for the localization of objects, whereas object classification recognizes object classes from detected object regions. For accurate object detection and classification, fusing multiple sensor information into a key component of the representation and perception processes is necessary. In this paper, we propose a new object-detection and classification method using decision-level fusion. We fuse the classification outputs from independent unary classifiers, such as 3D point clouds and image data using a convolutional neural network (CNN). The unary classifiers for the two sensors are the CNN with five layers, which use more than two pre-trained convolutional layers to consider local to global features as data representation. To represent data using convolutional layers, we apply region of interest (ROI) pooling to the outputs of each layer on the object candidate regions generated using object proposal generation to realize color flattening and semantic grouping for charge-coupled device and Light Detection And Ranging (LiDAR) sensors. We evaluate our proposed method on a KITTI benchmark dataset to detect and classify three object classes: cars, pedestrians and cyclists. The evaluation results show that the proposed method achieves better performance than the previous methods. Our proposed method extracted approximately 500 proposals on a 1226 × 370 image, whereas the original selective search method extracted approximately 10 6 × n proposals. We obtained classification performance with 77.72% mean average precision over the entirety of the classes in the moderate detection level of the KITTI benchmark dataset.

Detection of traversable road regions, positive and negative obstacles, and water hazards is a fundamental task for autonomous driving vehicles, especially in off-road environment. This paper proposes an efficient method, called Lidar-histogram. It can be used to integrate the detection of traversable road regions, obstacles and water hazards into one single framework. The weak assumption of the Lidar-histogram is that a decent-sized area in front of the vehicle is flat. The Lidar-histogram is derived from an efficient organized map of Lidar point cloud, called Lidar-imagery, to index, describe and store Lidar data. The organized point-cloud map can be easily obtained by indexing the original unordered 3D point cloud to a Lidar-specific 2D coordinate system. In the Lidar-histogram representation, the 3D traversable road plane in front of vehicle can be projected as a straight line segment, and the positive and negative obstacles are projected above and below the line segment, respectively. In this way, the problem of detecting traversable road and obstacles is converted into a simple linear classification task in 2D space. Experiments have been conducted in different kinds of off-road and urban scenes, and we have obtained very promising results.

A vision-based vehicle guidance system for road vehicles can have three main roles: 1) road detection; 2) obstacle detection; and 3) sign recognition. The first two have been studied for many years and with many good results, but traffi c sign recognition is a less-studied field. Traffi c signs provide drivers with very valuable informatio n about the road, in order to make drivin g safer and easier. We think that traffi c signs must play the same role for autonomous vehicles. They are designed to be easily recognized by human driver s mainly because their color and shapes are very different from natural environments. The algorithm described in this paper takes advantage of these features. I t has two main parts. The first one, for the detection, uses color thresholding to segment the image and shape analysis to detect the signs. The second one, for the classification, uses a neural network. Some results from natural scenes are shown. On the other hand, the algorithm is valid to detect other kinds of marks that would tell the mobile robot to perform some task at that place.

The detection of small road hazards, such as lost cargo, is a vital capability for self-driving cars. We tackle this challenging and rarely addressed problem with a vision system that leverages appearance, contextual as well as geometric cues. To utilize the appearance and contextual cues, we propose a new deep learning-based obstacle detection framework. Here a variant of a fully convolutional network is proposed to predict a pixel-wise semantic labeling of (i) free-space, (ii) on-road unexpected obstacles, and (iii) background. The geometric cues are exploited using a state-of-the-art detection approach that predicts obstacles from stereo input images via model-based statistical hypothesis tests. We present a principled Bayesian framework to fuse the semantic and stereo-based detection results. The mid-level Stixel representation is used to describe obstacles in a flexible, compact and robust manner. We evaluate our new obstacle detection system on the Lost and Found dataset, which includes very challenging scenes with obstacles of only 5 cm height. Overall, we report a major improvement over the state-of-the-art, with a performance gain of 27.4%. In particular, we achieve a detection rate of over 90% for distances of up to 50 m. Our system operates at 22 Hz on our self-driving platform.


Question: An ontology-driven method for traffic scene representation, rule-dependent situational understanding, and decision-making in autonomous vehicles.

Introducing RoadGraph: A Comprehensive Sensor Data Fusion Framework for Enhanced Vehicle Environment Modeling

The RoadGraph represents an innovative environmental model designed for advanced driver assistance systems. Its primary function is to amalgamate data from various sources, including digital maps, onboard sensors, and Vehicle-to-Everything (V2X) communication, into a unified, comprehensive model of the vehicle's surroundings. Upon aggregating this information, the RoadGraph performs an initial, function-independent analysis of the current situation.

In this paper, we delve into the methodologies employed for achieving lane-precise map-matching, even under conditions of suboptimal GPS reception, by leveraging diverse information sources. We also critically assess the process of integrating objects derived from multiple origins, taking into account the a-priori knowledge of the street layout. The outcomes of this pioneering approach are illustrated through a series of results, showcasing the effectiveness of the RoadGraph in enhancing situational awareness for automated vehicles.

This research contributes to the field of automated driving by offering a robust framework for real-time environmental perception, crucial for the development of autonomous vehicles capable of understanding and navigating complex traffic scenes. By integrating data from multiple sources, the RoadGraph facilitates a more accurate and detailed representation of the vehicle's environment, which is essential for making informed decisions in accordance with traffic regulations and situational awareness.

This paper presents a new perspective for 3D scene understanding by reasoning object stability and safety using intuitive mechanics. Our approach utilizes a simple observation that, by human design, objects in static scenes should be stable in the gravity field and be safe with respect to various physical disturbances such as human activities. This assumption is applicable to all scene categories and poses useful constraints for the plausible interpretations (parses) in scene understanding. Given a 3D point cloud captured for a static scene by depth cameras, our method consists of three steps: (i) recovering solid 3D volumetric primitives from voxels; (ii) reasoning stability by grouping the unstable primitives to physically stable objects by optimizing the stability and the scene prior; and (iii) reasoning safety by evaluating the physical risks for objects under physical disturbances, such as human activity, wind or earthquakes. We adopt a novel intuitive physics model and represent the energy landscape of each primitive and object in the scene by a disconnectivity graph (DG). We construct a contact graph with nodes being 3D volumetric primitives and edges representing the supporting relations. Then we adopt a Swendson–Wang Cuts algorithm to partition the contact graph into groups, each of which is a stable object. In order to detect unsafe objects in a static scene, our method further infers hidden and situated causes (disturbances) in the scene, and then introduces intuitive physical mechanics to predict possible effects (e.g., falls) as consequences of the disturbances. In experiments, we demonstrate that the algorithm achieves a substantially better performance for (i) object segmentation, (ii) 3D volumetric recovery, and (iii) scene understanding with respect to other state-of-the-art methods. We also compare the safety prediction from the intuitive mechanics model with human judgement.

Recent approaches on visual scene understanding attempt to build a scene graph – a computational representation of objects and their pairwise relationships. Such rich semantic representation is very appealing, yet difficult to obtain from a single image, especially when considering complex spatial arrangements in the scene. Differently, an image sequence conveys useful information using the multi-view geometric relations arising from camera motions. Indeed, object relationships are naturally related to the 3D scene structure. To this end, this paper proposes a system that first computes the geometrical location of objects in a generic scene and then efficiently constructs scene graphs from video by embedding such geometrical reasoning. Such compelling representation is obtained using a new model where geometric and visual features are merged using an RNN framework. We report results on a dataset we created for the task of 3D scene graph generation in multiple views.

This letter presents a novel semantic mapping approach, Recurrent-OctoMap, learned from long-term three-dimensional (3-D) Lidar data. Most existing semantic mapping approaches focus on improving semantic understanding of single frames, rather than 3-D refinement of semantic maps (i.e. fusing semantic observations). The most widely used approach for the 3-D semantic map refinement is “Bayes update,” which fuses the consecutive predictive probabilities following a Markov-chain model. Instead, we propose a learning approach to fuse the semantic features, rather than simply fusing predictions from a classifier. In our approach, we represent and maintain our 3-D map as an OctoMap, and model each cell as a recurrent neural network, to obtain a Recurrent-OctoMap. In this case, the semantic mapping process can be formulated as a sequence-to-sequence encoding–decoding problem. Moreover, in order to extend the duration of observations in our Recurrent-OctoMap, we developed a robust 3-D localization and mapping system for successively mapping a dynamic environment using more than two weeks of data, and the system can be trained and deployed with arbitrary memory length. We validate our approach on the ETH long-term 3-D Lidar dataset. The experimental results show that our proposed approach outperforms the conventional “Bayes update” approach.

Autonomous or semi-autonomous systems are deployed in environments where contact with programmers or technicians is infrequent or undesirable. To operate reliably, such systems should be able to adapt to new circumstances on their own. This paper describes our combined approach for adaptable software architecture and task synthesis from high-level goals, which is based on a three-layer model. In the uppermost layer, reactive plans are generated from goals expressed in a temporal logic. The middle layer is responsible for plan execution and assembling a configuration of domain-specific software components, which reside in the lowest layer. Moreover, the middle layer is responsible for selecting alternative components when the current configuration is no longer viable for the circumstances that have arisen. The implementation demonstrates that the approach enables us to handle non-determinism in the environment and unexpected failures in software components.

Road traffic accident is one of the major causes of death in Bangladesh. In this article, we propose a method that uses road side video data to learn the traffic pattern and uses vision based techniques to track and determine various kinetic features of vehicles. Finally, the proposed method detects anomaly (possibility of accident) and accidents on the road. The proposed method shows approximately 85% accuracy level in detecting special situations.

The detection of small road hazards, such as lost cargo, is a vital capability for self-driving cars. We tackle this challenging and rarely addressed problem with a vision system that leverages appearance, contextual as well as geometric cues. To utilize the appearance and contextual cues, we propose a new deep learning-based obstacle detection framework. Here a variant of a fully convolutional network is proposed to predict a pixel-wise semantic labeling of (i) free-space, (ii) on-road unexpected obstacles, and (iii) background. The geometric cues are exploited using a state-of-the-art detection approach that predicts obstacles from stereo input images via model-based statistical hypothesis tests. We present a principled Bayesian framework to fuse the semantic and stereo-based detection results. The mid-level Stixel representation is used to describe obstacles in a flexible, compact and robust manner. We evaluate our new obstacle detection system on the Lost and Found dataset, which includes very challenging scenes with obstacles of only 5 cm height. Overall, we report a major improvement over the state-of-the-art, with a performance gain of 27.4%. In particular, we achieve a detection rate of over 90% for distances of up to 50 m. Our system operates at 22 Hz on our self-driving platform.

Robust scene understanding of outdoor environments using passive optical sensors is a onerous and essential task for autonomous navigation. The problem is heavily characterized by changing environmental conditions throughout the day and across seasons. Robots should be equipped with models that are impervious to these factors in order to be operable and more importantly to ensure safety in the real-world. In this paper, we propose a novel semantic segmentation architecture and the convoluted mixture of deep experts (CMoDE) fusion technique that enables a multi-stream deep neural network to learn features from complementary modalities and spectra, each of which are specialized in a subset of the input space. Our model adaptively weighs class-specific features of expert networks based on the scene condition and further learns fused representations to yield robust segmentation. We present results from experimentation on three publicly available datasets that contain diverse conditions including rain, summer, winter, dusk, fall, night and sunset, and show that our approach exceeds the state-of-the-art. In addition, we evaluate the performance of autonomously traversing several kilometres of a forested environment using only the segmentation for perception.

Traffic safety is a severe problem around the world. Many road accidents are normally related with the driver's unsafe driving behavior, e.g. eating while driving. In this work, we propose a vision-based solution to recognize the driver's behavior based on convolutional neural networks. Specifically, given an image, skin-like regions are extracted by Gaussian Mixture Model, which are passed to a deep convolutional neural networks model, namely R*CNN, to generate action labels. The skin-like regions are able to provide abundant semantic information with sufficient discriminative capability. Also, R*CNN is able to select the most informative regions from candidates to facilitate the final action recognition. We tested the proposed methods on Southeast University Driving-posture Dataset and achieve mean Average Precision(mAP) of 97.76% on the dataset which prove the proposed method is effective in drivers's action recognition.

Future driver assistance systems are likely to use a multisensor approach with heterogeneous sensors for tracking dynamic objects around the vehicle. The quality and type of data available for a data fusion algorithm depends heavily on the sensors detecting an object. This article presents a general framework which allows the use sensor specific advantages while abstracting the specific details of a sensor. Different tracking models are used depending on the current set of sensors detecting the object. A sensor independent algorithm for classifying objects regarding their current and past movement state is presented. The described architecture and algorithms have been successfully implemented in Tartan racingpsilas autonomous vehicle for the urban grand challenge. Results are presented and discussed.

The last few decades witnessed the birth and growth of a new sensibility to transportation efficiency. In particular, the need for efficient and improved people and goods mobility pushed researchers to address the problem of intelligent transportation systems. This paper surveys the most advanced approaches to the (partial) customization of road following task, using on-board systems based on artificial vision. The functionalities of lane detection, obstacle detection and pedestrian detection are described and classified, and their possible application on future road vehicles is discussed.


Question: SSR-Net: A compact soft stagewise regression network for estimating age.

This paper explores the impact of increasing the depth of convolutional networks on their accuracy in large-scale image recognition tasks. Our key contribution lies in a comprehensive evaluation of networks with varying depths, highlighting that significant enhancements in performance can be achieved by extending the depth to 16-19 weight layers. This insight was central to our submission for the ImageNet Challenge 2014, where our team secured top positions in both the localization and classification categories. Moreover, we demonstrate that our network architectures generalize effectively to other datasets, achieving state-of-the-art results. Notably, we have released our two most successful ConvNet models to the public, fostering further research on the application of deep visual representations in the field of computer vision.

In essence, our study reveals that deeper networks, with 16-19 weight layers, outperform shallower architectures in large-scale image recognition. This finding was instrumental in our team's success at the ImageNet Challenge 2014, where we claimed the top spots in both major categories. Our models have also proven their versatility by delivering exceptional results on additional datasets. By making our leading ConvNet models publicly accessible, we aim to support ongoing research into the integration of deep visual representations within computer vision applications.

A Aesthetics & Attribute Database (AADB) Fusing Attributes and Content for Aesthetics Ranking Demo, code and model can be download through project webpage http://www.ics.uci.edu/~skong2/aesthetics.html References: [8] He, K., Zhang, X., Ren, S., Sun, J., ECCV, 2014 [15] Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J., IEEE Trans. on Multimedia, 2015 [16] Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z., ACMMM, 2014 [17] Lu, X., Lin, Z., Shen, X., Mech, R., Wang, J.Z., ICCV, 2015 [23] Murray, N., Marchesotti, L., Perronnin, F., CVPR, 2012 Acknowledgements: This work was supported by Adobe gift fund, NSF grants DBI-1262547 and IIS1253538. Experimental Results We use Spearman's rho rank correlation ( ) to measure ranking performance . By thresholding the rating scores, we achieve state-of-the-art classification accuracy on AVA despite never training with a classification loss. We first train a simple model with Euclidean loss for numerical rating of photo aesthetics (a) fine-tuning with rank loss Based on the regression net, we apply rank loss to fine-tune the network

Face representation is a crucial step of face recognition systems. An optimal face representation should be discriminative, robust, compact, and very easyto-implement. While numerous hand-crafted and learning-based representations have been proposed, considerable room for improvement is still present. In this paper, we present a very easy-to-implement deep learning framework for face representation. Our method bases on a new structure of deep network (called Pyramid CNN). The proposed Pyramid CNN adopts a greedy-filter-and-down-sample operation, which enables the training procedure to be very fast and computationefficient. In addition, the structure of Pyramid CNN can naturally incorporate feature sharing across multi-scale face representations, increasing the discriminative ability of resulting representation. Our basic network is capable of achieving high recognition accuracy (85.8% on LFW benchmark) with only 8 dimension representation. When extended to feature-sharing Pyramid CNN, our system achieves the state-of-the-art performance (97.3%) on LFW benchmark. We also introduce a new benchmark of realistic face images on social network and validate our proposed representation has a good ability of generalization.

Due to its indefinite evaluation criterion, facial beauty analysis has shown its challenges in pattern recognition and biometric recognition. Plenty of methods have been designed for solving this problem, but there are still various limitations. Most of the available research results in this field are achieved on a small-scale facial beauty database which is difficult in modeling the structure information for facial beauty. And majority of the presented facial beauty prediction algorithm requires burden landmark or expensive optimization procedure. In this paper, we have established a large scale facial beauty database named LSFBD, which is a new database for facial beauty analysis. The LSFBD database contains 20000 labeled images, which has 10000 unconstrained male subjects and 10000 unconstrained female subjects separately, and all images are verified with well-designed rating with average scores and standard deviation. In this database, manually adding the extreme beauty images make the distribution more reasonable and the result more accurate. CRBM or Eigenfaces results are presented in the paper as benchmark for evaluating the task of facial beauty prediction.

In this paper, we propose a new model called deep network cascade (DNC) to gradually upscale low-resolution images layer by layer, each layer with a small scale factor. DNC is a cascade of multiple stacked collaborative local auto-encoders. In each layer of the cascade, non-local self-similarity search is first performed to enhance high-frequency texture details of the partitioned patches in the input image. The enhanced image patches are then input into a collaborative local auto-encoder (CLA) to suppress the noises as well as collaborate the compatibility of the overlapping patches. By closing the loop on non-local self-similarity search and CLA in a cascade layer, we can refine the super-resolution result, which is further fed into next layer until the required image scale. Experiments on image super-resolution demonstrate that the proposed DNC can gradually upscale a low-resolution image with the increase of network layers and achieve more promising results in visual quality as well as quantitative performance.

We propose DeepHand to estimate the 3D pose of a hand using depth data from commercial 3D sensors. We discriminatively train convolutional neural networks to output a low dimensional activation feature given a depth map. This activation feature vector is representative of the global or local joint angle parameters of a hand pose. We efficiently identify 'spatial' nearest neighbors to the activation feature, from a database of features corresponding to synthetic depth maps, and store some 'temporal' neighbors from previous frames. Our matrix completion algorithm uses these 'spatio-temporal' activation features and the corresponding known pose parameter values to estimate the unknown pose parameters of the input feature vector. Our database of activation features supplements large viewpoint coverage and our hierarchical estimation of pose parameters is robust to occlusions. We show that our approach compares favorably to state-of-the-art methods while achieving real time performance (≈ 32 FPS) on a standard computer.

Learned 3D representations of human faces are useful for computer vision problems such as 3D face tracking and reconstruction from images, as well as graphics applications such as character generation and animation. Traditional models learn a latent representation of a face using linear subspaces or higher-order tensor generalizations. Due to this linearity, they can not capture extreme deformations and nonlinear expressions. To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface. We introduce mesh sampling operations that enable a hierarchical mesh representation that captures non-linear variations in shape and expression at multiple scales within the model. In a variational setting, our model samples diverse realistic 3D faces from a multivariate Gaussian distribution. Our training data consists of 20,466 meshes of extreme expressions captured over 12 different subjects. Despite limited training data, our trained model outperforms state-of-the-art face models with 50% lower reconstruction error, while using 75% fewer parameters. We also show that, replacing the expression space of an existing state-ofthe-art face model with our autoencoder, achieves a lower reconstruction error. Our data, model and code are available at http://github.com/ anuragranj/coma.

We are interested in attribute-guided face generation: given a low-res face input image, an attribute vector that can be extracted from a high-res image (attribute image), our new method generates a highres face image for the low-res input that satisfies the given attributes. To address this problem, we condition the CycleGAN and propose conditional CycleGAN, which is designed to 1) handle unpaired training data because the training low/high-res and high-res attribute images may not necessarily align with each other, and to 2) allow easy control of the appearance of the generated face via the input attributes. We demonstrate high-quality results on the attribute-guided conditional CycleGAN, which can synthesize realistic face images with appearance easily controlled by user-supplied attributes (e.g., gender, makeup, hair color, eyeglasses). Using the attribute image as identity to produce the corresponding conditional vector and by incorporating a face verification network, the attribute-guided network becomes the identity-guided conditional CycleGAN which produces high-quality and interesting results on identity transfer. We demonstrate three applications on identityguided conditional CycleGAN: identity-preserving face superresolution, face swapping, and frontal face generation, which consistently show the advantage of our new method.

Deep learning is popular as an end-to-end framework extracting the prominent features and performing the classification also. In this paper, we extensively investigate deep networks as an alternate to feature encoding technique of lowlevel descriptors for emotion recognition on the benchmark EmoDB dataset. Fusion performance with such obtained encoded features with other available features is also investigated. Highest performance to date in the literature is observed.

BACKGROUND
The restoration of a natural volume distribution is a major goal in facial rejuvenation. The aims of this study were to establish a radiographic method enabling effective measurements of the midfacial fat compartments and to compare the anatomy between human cadavers of younger versus older age.

METHODS
Data from computed tomographic scans of 12 nonfixed cadaver heads, divided into two age groups (group 1, 54 to 75 years, n = 6; and group 2, 75 to 104 years, n = 6), were analyzed. For evaluation of the volume distribution within a specific compartment, the sagittal diameter of the upper, middle, and lower thirds of each compartment was determined. For evaluation of a "sagging" of the compartments, the distance between the cephalad border and the infraorbital rim was determined.

RESULTS
Computed tomography enables a reproducible depiction of the facial fat compartments and reveals aging changes. The distance between the fat compartments and the infraorbital rim was higher in group 2 compared with group 1. The sagittal diameter of the lower third of the compartments was higher, and the sagittal diameter of the upper third was smaller in group 2 compared with group 1. The buccal extension of the buccal fat pad was shown to be an independent, separate compartment.

CONCLUSIONS
This study demonstrates an inferior migration of the midfacial fat compartments and an inferior volume shift within the compartments during aging. Additional distinct compartment-specific changes (e.g., volume loss of the deep medial cheek fat and buccal extension of the buccal fat pad) contribute to the appearance of the aged face.

Face recognition is an important embodiment of human-computer interaction, which has been widely used in access control system, monitoring system and identity verification. However, since face images vary with expressions, ages, as well as poses of people and illumination conditions, the face images of the same sample might be different, which makes face recognition difficult. There are two main requirements in face recognition, the high recognition rate and less training time. In this paper, we combine Convolutional Neural Network (CNN) and Support Vector Machine (SVM) to recognize face images. CNN is used as a feature extractor to acquire remarkable features automatically. We first pre-train our CNN by ancillary data to get the updated weights, and then train the CNN by the target dataset to extract more hidden facial features. Finally we use SVM as our classifier instead of CNN to recognize all the classes. With the input of facial features extracted from CNN, SVM will recognize face images more accurately. In our experiments, some face images in the Casia-Webfaces database are used for pre-training, and FERET database is used for training and testing. The results in experiments demonstrate the efficiency with high recognition rate and less training time.


Question: SSR-Net: A compact soft stagewise regression network for estimating age.

LIBSVM: An Advanced Library for Support Vector Machines

This paper presents an enhanced version of LIBSVM, a comprehensive library designed for support vector machines (SVMs). LIBSVM offers a range of functionalities that enable users to efficiently handle classification and regression tasks. The library supports various kernel functions and optimization algorithms, making it a versatile tool for machine learning practitioners. LIBSVM's advanced features include cross-validation, grid search for parameter tuning, and multi-class classification. The library is optimized for performance, ensuring fast computation even with large datasets. Additionally, LIBSVM provides a user-friendly interface, allowing for easy integration into various applications. This updated version of LIBSVM incorporates several improvements, such as better handling of imbalanced datasets and enhanced support for large-scale problems. The library's effectiveness is demonstrated through extensive experiments on real-world datasets, showcasing its robustness and efficiency in solving complex machine learning tasks.

Facial gender and smile classification in unconstrained environment is challenging due to the invertible and large variations of face images. In this paper, we propose a deep model composed of GNet and SNet for these two tasks. We leverage the multi-task learning and the general-to-specific fine-tuning scheme to enhance the performance of our model. Our strategies exploit the inherent correlation between face identity, smile, gender and other face attributes to relieve the problem of over-fitting on small training set and improve the classification performance. We also propose the tasks-aware face cropping scheme to extract attribute-specific regions. The experimental results on the ChaLearn 16 FotW dataset for gender and smile classification demonstrate the effectiveness of our proposed methods.

One key challenge of facial trait recognition is the large non-rigid appearance variations due to some irrelevant real world factors, such as viewpoint and expression changes. In this paper, we explore how the shape information, i.e. facial landmark positions, can be explicitly deployed into the popular Convolutional Neural Network (CNN) architecture to disentangle such irrelevant non-rigid appearance variations. First, instead of using fixed kernels, we propose a kernel adaptation method to dynamically determine the convolutional kernels according to the spatial distribution of facial landmarks, which helps learning more robust features. Second, motivated by the intuition that different local facial regions may demand different adaptation functions, we further propose a tree-structured convolutional architecture to hierarchically fuse multiple local adaptive CNN subnetworks. Comprehensive experiments on WebFace, Morph II and MultiPIE databases well validate the effectiveness of the proposed kernel adaptation method and tree-structured convolutional architecture for facial trait recognition tasks, including identity, age and gender recognition. For all the tasks, the proposed architecture consistently achieves the state-of-the-art performances.

Face recognition based on the analysis of 3D scans has been an active research subject over the last few years. However, the impact of the resolution of 3D scans on the recognition process has not been addressed explicitly yet being of primal importance after the introduction of a new generation of low cost 4D scanning devices. These devices are capable of combined depth/rgb acquisition over time with a low resolution compared to the 3D scanners typically used in 3D face recognition benchmarks. In this paper, we define a super-resolution model for 3D faces by which a sequence of low-resolution 3D scans can be processed to extract a higher resolution 3D face model, namely the superface model. The proposed solution relies on the Scaled ICP procedure to align the low-resolution 3D models with each other and estimate the value of the high-resolution 3D model based on the statistics of values of the lowresolution scans in corresponding points. The approach is validated on a data set that includes, for each subject, one sequence of low-resolution 3D face scans and one ground-truth high-resolution 3D face model acquired through a high-resolution 3D scanner. In this way, results of the super-resolution process are evaluated qualitatively and quantitatively by measuring the error between the superface and the ground-truth.

This paper describes an easy-to-use system to estimate the shape of a human body and his/her clothes. The system uses a Kinect to capture the human's RGB and depth information from different views. Using the depth data, a non-rigid deformation method is devised to compensate motions between different views, thus to align and complete the dressed shape. Given the reconstructed dressed shape, the skin regions are recognized by a skin classifier from the RGB images, and these skin regions are taken as a tight constraints for the body estimation. Subsequently, the body shape is estimated from the skin regions of the dressed shape by leveraging a statistical model of human body. After the body estimation, the body shape is non-rigidly deformed to fit the dressed shape, so as to extract the cloth field of the dressed shape. We demonstrate our system and the therein algorithms by several experiments. The results show the effectiveness of the proposed method. & 2014 Elsevier B.V. All rights reserved.

We present a hierarchical regression framework for estimating hand joint positions from single depth images based on local surface normals. The hierarchical regression follows the tree structured topology of hand from wrist to finger tips. We propose a conditional regression forest, i.e. the Frame Conditioned Regression Forest (FCRF) which uses a new normal difference feature. At each stage of the regression, the frame of reference is established from either the local surface normal or previously estimated hand joints. By making the regression with respect to the local frame, the pose estimation is more robust to rigid transformations. We also introduce a new efficient approximation to estimate surface normals. We verify the effectiveness of our method by conducting experiments on two challenging real-world datasets and show consistent improvements over previous discriminative pose estimation methods.

Recently, deep learning has been widely applied in binocular stereo matching for depth acquisition, which has led to an immense increase of accuracy. However, little attention has been paid to the structured light field. In this letter, a network for structured light is proposed to extract effective matching features for depth acquisition. The proposed network promotes the Siamese network by considering receptive fields of different scales and assigning proper weights to the corresponding features, which is achieved by combining pyramid-pooling structure with the squeeze-and-excitation network into the Siamese network for feature extraction and weight calculations, respectively. For network training and testing, a structured-light dataset with amended ground truths is generated by projecting a random pattern into the existing binocular stereo dataset. Experiments demonstrate that the proposed network is capable of real-time depth acquisition, and it provides superior depth maps using structured light.

We are interested in attribute-guided face generation: given a low-res face input image, an attribute vector that can be extracted from a high-res image (attribute image), our new method generates a highres face image for the low-res input that satisfies the given attributes. To address this problem, we condition the CycleGAN and propose conditional CycleGAN, which is designed to 1) handle unpaired training data because the training low/high-res and high-res attribute images may not necessarily align with each other, and to 2) allow easy control of the appearance of the generated face via the input attributes. We demonstrate high-quality results on the attribute-guided conditional CycleGAN, which can synthesize realistic face images with appearance easily controlled by user-supplied attributes (e.g., gender, makeup, hair color, eyeglasses). Using the attribute image as identity to produce the corresponding conditional vector and by incorporating a face verification network, the attribute-guided network becomes the identity-guided conditional CycleGAN which produces high-quality and interesting results on identity transfer. We demonstrate three applications on identityguided conditional CycleGAN: identity-preserving face superresolution, face swapping, and frontal face generation, which consistently show the advantage of our new method.

Faces constitute a unique and widely used category of stimuli. In spite of their importance, there are few collections of faces for use in research, none of which adequately represent the different ages of faces across the lifespan. This lack of a range of ages has limited the majority of researchers to using predominantly young faces as stimuli even when their hypotheses concern both young and old participants. We describe a database of 575 individual faces ranging from ages 18 to 93. Our database was developed to be more representative of age groups across the lifespan, with a special emphasis on recruiting older adults. The resulting database has faces of 218 adults age 18-29, 76 adults age 30-49, 123 adults age 50-69, and 158 adults age 70 and older. These faces may be acquired for research purposes from http://agingmind.cns.uiuc.edu/facedb/. This will allow researchers interested in using facial stimuli access to a wider age range of adult faces than has previously been available.

The state-of-the-art of face recognition has been significantly advanced by the emergence of deep learning. Very deep neural networks recently achieved great success on general object recognition because of their superb learning capacity. This motivates us to investigate their effectiveness on face recognition. This paper proposes two very deep neural network architectures, referred to as DeepID3, for face recognition. These two architectures are rebuilt from stacked convolution and inception layers proposed in VGG net [10] and GoogLeNet [16] to make them suitable to face recognition. Joint face identification-verification supervisory signals are added to both intermediate and final feature extraction layers during training. An ensemble of the proposed two architectures achieves 99.53% LFW face verification accuracy and 96.0% LFW rank-1 face identification accuracy, respectively. A further discussion of LFW face verification result is given in the end.

In recent years, the triplet loss-based deep neural networks (DNN) are widely used in the task of face recognition and achieve the state-of-the-art performance. However, the complexity of training the triplet loss-based DNN is significantly high due to the difficulty in generating high-quality training samples. In this paper, we propose a novel DNN training framework to accelerate the training process of the triplet loss-based DNN and meanwhile to improve the performance of face recognition. More specifically, the proposed framework contains two stages: 1) The DNN initialization. A deep architecture based on the softmax loss function is designed to initialize the DNN. 2) The adaptive fine-tuning. Based on the trained model, a set of high-quality triplet samples is generated and used to fine-tune the network, where an adaptive triplet loss function is introduced to improve the discriminative ability of DNN. Experimental results show that, the model obtained by the proposed DNN training framework achieves 97.3% accuracy on the LFW benchmark with low training complexity, which verifies the efficiency and effectiveness of the proposed framework.


Question: SSR-Net: A compact soft stagewise regression network for estimating age.

This paper introduces a pioneering Convolutional Neural Network (CNN) framework, ranking-CNN, designed to enhance the accuracy of age estimation from facial images. Despite the recent advancements in age estimation through deep learning on large datasets, the ordinal nature of age, which implies a sequential progression, has been largely overlooked. To address this gap, we propose ranking-CNN, a model that incorporates a series of fundamental CNNs, each trained on ordinal age labels. The binary outputs from these CNNs are then integrated to generate a final age prediction. We derive a significantly tighter error bound for age estimation using our ranking-based approach. Additionally, we provide a rigorous proof that ranking-CNN has a higher likelihood of producing lower estimation errors compared to conventional multi-class classification methods. Through comprehensive experimentation, we demonstrate that ranking-CNN achieves statistically significant improvements over leading age estimation models on established benchmark datasets.

Recently, deep learning has been widely applied in binocular stereo matching for depth acquisition, which has led to an immense increase of accuracy. However, little attention has been paid to the structured light field. In this letter, a network for structured light is proposed to extract effective matching features for depth acquisition. The proposed network promotes the Siamese network by considering receptive fields of different scales and assigning proper weights to the corresponding features, which is achieved by combining pyramid-pooling structure with the squeeze-and-excitation network into the Siamese network for feature extraction and weight calculations, respectively. For network training and testing, a structured-light dataset with amended ground truths is generated by projecting a random pattern into the existing binocular stereo dataset. Experiments demonstrate that the proposed network is capable of real-time depth acquisition, and it provides superior depth maps using structured light.

BACKGROUND
The purpose of this study was to identify the environmental factors that contribute to facial aging in identical twins.

METHODS
During the Twins Day Festival in Twinsburg, Ohio, 186 pairs of identical twins completed a comprehensive questionnaire, and digital images were obtained. A panel reviewed the images independently and recorded the differences in the perceived twins' ages and their facial features. The perceived age differences were then correlated with multiple factors.

RESULTS
Four-point higher body mass index was associated with an older appearance in twins younger than age 40 but resulted in a younger appearance after age 40 (p = 0.0001). Eight-point higher body mass index was associated with an older appearance in twins younger than age 55 but was associated with a younger appearance after age 55 (p = 0.0001). The longer the twins smoked, the older they appeared (p < 0.0001). Increased sun exposure was associated with an older appearance and accelerated with age (p = 0.015), as was a history of outdoor activities and lack of sunscreen use. Twins who used hormone replacement had a younger appearance (p = 0.002). Facial rhytids were more evident in twins with a history of skin cancer (p = 0.05) and in those who smoked (p = 0.005). Dark and patchy skin discoloration was less prevalent in twins with a higher body mass index (p = 0.01) and more common in twins with a history of smoking (p = 0.005) and those with sun exposure (p = 0.005). Hair quantity was better with a higher body mass index (p = 0.01) although worse with a history of skin cancer (p = 0.005) and better with the use of hormones (p = 0.05).

CONCLUSION
This study offers strong statistical evidence to support the role of some of the known factors that govern facial aging.

Face recognition is an important embodiment of human-computer interaction, which has been widely used in access control system, monitoring system and identity verification. However, since face images vary with expressions, ages, as well as poses of people and illumination conditions, the face images of the same sample might be different, which makes face recognition difficult. There are two main requirements in face recognition, the high recognition rate and less training time. In this paper, we combine Convolutional Neural Network (CNN) and Support Vector Machine (SVM) to recognize face images. CNN is used as a feature extractor to acquire remarkable features automatically. We first pre-train our CNN by ancillary data to get the updated weights, and then train the CNN by the target dataset to extract more hidden facial features. Finally we use SVM as our classifier instead of CNN to recognize all the classes. With the input of facial features extracted from CNN, SVM will recognize face images more accurately. In our experiments, some face images in the Casia-Webfaces database are used for pre-training, and FERET database is used for training and testing. The results in experiments demonstrate the efficiency with high recognition rate and less training time.

Relative location prediction in computed tomography (CT) scan images is a challenging problem. In this paper, a regression model based on one-dimensional convolutional neural networks is proposed to determine the relative location of a CT scan image both robustly and precisely. A public dataset is employed to validate the performance of the study’s proposed method using a 5-fold cross validation. Experimental results demonstrate an excellent performance of the proposed model when compared with the state-of-the-art techniques, achieving a median absolute error of 1.04 cm and mean absolute error of 1.69 cm.

Deep residual networks were shown to be able to scale up to thousands of layers and still have improving performance. However, each fraction of a percent of improved accuracy costs nearly doubling the number of layers, and so training very deep residual networks has a problem of diminishing feature reuse, which makes these networks very slow to train. To tackle these problems, in this paper we conduct a detailed experimental study on the architecture of ResNet blocks, based on which we propose a novel architecture where we decrease depth and increase width of residual networks. We call the resulting network structures wide residual networks (WRNs) and show that these are far superior over their commonly used thin and very deep counterparts. For example, we demonstrate that even a simple 16-layer-deep wide residual network outperforms in accuracy and efficiency all previous deep residual networks, including thousand-layerdeep networks, achieving new state-of-the-art results on CIFAR, SVHN, COCO, and significant improvements on ImageNet. Our code and models are available at https: //github.com/szagoruyko/wide-residual-networks.

Growing number of surveillance and biometric applications seek to recognize the face of individuals appearing in the viewpoint of video cameras. Systems for video-based FR can be subjected to challenging operational environments, where the appearance of faces captured with video cameras varies significantly due to changes in pose, illumination, scale, blur, expression, occlusion, etc. In particular, with still-to-video FR, a limited number of high-quality facial images are typically captured for enrollment of an individual to the system, whereas an abundance facial trajectories can be captured using video cameras during operations, under different viewpoints and uncontrolled conditions. This paper presents a deep learning architecture that can learn a robust facial representation for each target individual during enrollment, and then accurately compare the facial regions of interest (ROIs) extracted from a still reference image (of the target individual) with ROIs extracted from live or archived videos. An ensemble of deep convolutional neural networks (DCNNs) named HaarNet is proposed, where a trunk network first extracts features from the global appearance of the facial ROIs (holistic representation). Then, three branch networks effectively embed asymmetrical and complex facial features (local representations) based on Haar-like features. In order to increase the discriminativness of face representations, a novel regularized triplet-loss function is proposed that reduces the intra-class variations, while increasing the inter-class variations. Given the single reference still per target individual, the robustness of the proposed DCNN is further improved by fine-tuning the HaarNet with synthetically-generated facial still ROIs that emulate capture conditions found in operational environments. The proposed system is evaluated on stills and videos from the challenging COX Face and Chokepoint datasets according to accuracy and complexity. Experimental results indicate that the proposed method can significantly improve performance with respect to state-of-the-art systems for video-based FR.

Visual inspection of neuroimagery is susceptible to human eye limitations. Computerized methods have been shown to be equally or more e↵ective than human clinicians in diagnosing dementia from neuroimages. Nevertheless, much of the work involves the use of domain expertise to extract hand–crafted features. The key technique in this paper is the use of cross–domain features to represent MRI data. We used a sparse autoencoder to learn a set of bases from natural images and then applied convolution to extract features from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset. Using this new representation, we classify MRI instances into three categories: Alzheimer’s Disease (AD), Mild Cognitive Impairment (MCI) and Healthy Control (HC). Our approach, in spite of being very simple, achieved high classification performance, which is competitive with or better than other approaches.

In this paper, we propose a simple and general framework for training very tiny CNNs (e.g. VGG with the number of channels reduced to 1 32 ) for object detection. Due to limited representation ability, it is challenging to train very tiny networks for complicated tasks like detection. To the best of our knowledge, our method, called Quantization Mimic, is the first one focusing on very tiny networks. We utilize two types of acceleration methods: mimic and quantization. Mimic improves the performance of a student network by transfering knowledge from a teacher network. Quantization converts a full-precision network to a quantized one without large degradation of performance. If the teacher network is quantized, the search scope of the student network will be smaller. Using this feature of the quantization, we propose Quantization Mimic. It first quantizes the large network, then mimic a quantized small network. The quantization operation can help student network to better match the feature maps from teacher network. To evaluate our approach, we carry out experiments on various popular CNNs including VGG and Resnet, as well as different detection frameworks including Faster R-CNN and R-FCN. Experiments on Pascal VOC and WIDER FACE verify that our Quantization Mimic algorithm can be applied on various settings and outperforms state-of-the-art model acceleration methods given limited computing resouces.

Deep Convolutional Neural Networks (DCNNs) are used extensively in biomedical image segmentation. However, current DCNNs usually use down sampling layers for increasing the receptive field and gaining abstract semantic information. These down sampling layers decrease the spatial dimension of feature maps, which can be detrimental to semantic image segmentation. Atrous convolution is an alternative for the down sampling layer. It increases the receptive field whilst maintains the spatial dimension of feature maps. In this paper, a method for effective atrous rate setting is proposed to achieve the largest and fully-covered receptive field with a minimum number of atrous convolutional layers. Furthermore, different atrous blocks, shortcut connections and normalization methods are explored to select the optimal network structure setting. These lead to a new and full-scale DCNN Atrous Convolutional Neural Network (ACNN), which incorporates cascaded atrous II-blocks, residual learning and Fine Group Normalization (FGN). Application results of the proposed ACNN to Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) image segmentation demonstrate that the proposed ACNN can achieve comparable segmentation Dice Similarity Coefficients (DSCs) to U-Net, optimized U-Net and hybrid network, but with significantly reduced trainable parameters due to the use of fullscale feature maps and therefore computationally is much more efficient for both the training and inference.

We consider the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise. We cast the recognition problem as one of classifying among multiple linear regression models and argue that new theory from sparse signal representation offers the key to addressing this problem. Based on a sparse representation computed by C1-minimization, we propose a general classification algorithm for (image-based) object recognition. This new framework provides new insights into two crucial issues in face recognition: feature extraction and robustness to occlusion. For feature extraction, we show that if sparsity in the recognition problem is properly harnessed, the choice of features is no longer critical. What is critical, however, is whether the number of features is sufficiently large and whether the sparse representation is correctly computed. Unconventional features such as downsampled images and random projections perform just as well as conventional features such as eigenfaces and Laplacianfaces, as long as the dimension of the feature space surpasses certain threshold, predicted by the theory of sparse representation. This framework can handle errors due to occlusion and corruption uniformly by exploiting the fact that these errors are often sparse with respect to the standard (pixel) basis. The theory of sparse representation helps predict how much occlusion the recognition algorithm can handle and how to choose the training images to maximize robustness to occlusion. We conduct extensive experiments on publicly available databases to verify the efficacy of the proposed algorithm and corroborate the above claims.


Question: SSR-Net: A streamlined soft stagewise regression network for estimating age.

The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) represents a significant milestone in the field of computer vision, specifically focusing on the advancement of image recognition algorithms. Initiated in 2010, this annual competition has been instrumental in pushing the boundaries of machine learning and deep learning models, particularly in tasks such as object detection, image classification, and segmentation. The ILSVRC dataset, which is central to the challenge, consists of millions of images annotated with labels from a thousand different categories, providing a rich and diverse resource for training and evaluating visual recognition models. Over the years, the challenge has not only spurred the development of cutting-edge techniques but has also facilitated the benchmarking of these models against each other, thereby accelerating the pace of innovation in the field. As a result, the ILSVRC has become a pivotal platform for researchers and practitioners to showcase their latest research and to compare the efficacy of their algorithms in a standardized setting.

In a similar vein, the SSR-Net: A Compact Soft Stagewise Regression Network for Age Estimation introduces a novel approach to age estimation, a critical yet challenging task in the realm of computer vision. Unlike the vast scale of the ImageNet challenge, which encompasses a wide array of visual recognition tasks, SSR-Net specifically targets the estimation of human age from facial images. The network is designed to be compact and efficient, making it particularly suitable for real-world applications where computational resources might be limited. By employing a soft stagewise regression strategy, SSR-Net is able to iteratively refine its predictions, leading to improved accuracy compared to traditional regression methods. This approach not only enhances the model's performance but also reduces its complexity, allowing for faster inference times. As age estimation plays a crucial role in various applications, including demographic analysis, personalized advertising, and security systems, the development of SSR-Net marks a significant step forward in addressing this challenging problem with greater precision and efficiency.

Compressive Sensing has become one of the standard methods of face recognition within the literature. We show, however, that the sparsity assumption which underpins much of this work is not supported by the data. This lack of sparsity in the data means that compressive sensing approach cannot be guaranteed to recover the exact signal, and therefore that sparse approximations may not deliver the robustness or performance desired. In this vein we show that a simple £2 approach to the face recognition problem is not only significantly more accurate than the state-of-the-art approach, it is also more robust, and much faster. These results are demonstrated on the publicly available YaleB and AR face datasets but have implications for the application of Compressive Sensing more broadly.

With the powerfulness of convolution neural networks (CNN), CNN based face reconstruction has recently shown promising performance in reconstructing detailed face shape from 2D face images. The success of CNN-based methods relies on a large number of labeled data. The state-of-the-art synthesizes such data using a coarse morphable face model, which however has difficulty to generate detailed photo-realistic images of faces (with wrinkles). This paper presents a novel face data generation method. Specifically, we render a large number of photo-realistic face images with different attributes based on inverse rendering. Furthermore, we construct a fine-detailed face image dataset by transferring different scales of details from one image to another. We also construct a large number of video-type adjacent frame pairs by simulating the distribution of real video data. With these nicely constructed datasets, we propose a coarse-to-fine learning framework consisting of three convolutional networks. The networks are trained for real-time detailed 3D face reconstruction from monocular video as well as from a single image. Extensive experimental results demonstrate that our framework can produce high-quality reconstruction but with much less computation time compared to the state-of-the-art. Moreover, our method is robust to pose, expression and lighting due to the diversity of data.

We present a novel discriminative regression based approach for the Constrained Local Models (CLMs) framework, referred to as the Discriminative Response Map Fitting (DRMF) method, which shows impressive performance in the generic face fitting scenario. The motivation behind this approach is that, unlike the holistic texture based features used in the discriminative AAM approaches, the response map can be represented by a small set of parameters and these parameters can be very efficiently used for reconstructing unseen response maps. Furthermore, we show that by adopting very simple off-the-shelf regression techniques, it is possible to learn robust functions from response maps to the shape parameters updates. The experiments, conducted on Multi-PIE, XM2VTS and LFPW database, show that the proposed DRMF method outperforms state-of-the-art algorithms for the task of generic face fitting. Moreover, the DRMF method is computationally very efficient and is real-time capable. The current MATLAB implementation takes 1 second per image. To facilitate future comparisons, we release the MATLAB code and the pre-trained models for research purposes.

Recent research in the deep learning field has produced a plethora of new architectures. At the same time, a growing number of groups are applying deep learning to new applications. Some of these groups are likely to be composed of inexperienced deep learning practitioners who are baffled by the dizzying array of architecture choices and therefore opt to use an older architecture (i.e., Alexnet). Here we attempt to bridge this gap by mining the collective knowledge contained in recent deep learning research to discover underlying principles for designing neural network architectures. In addition, we describe several architectural innovations, including Fractal of FractalNet network, Stagewise Boosting Networks, and Taylor Series Networks (our Caffe code and prototxt files is available at https://github.com/iPhysicist/CNNDesignPatterns). We hope others are inspired to build on our preliminary work.

Face alignment has witnessed substantial progress in the last decade. One of the recent focuses has been aligning a dense 3D face shape to face images with large head poses. The dominant technology used is based on the cascade of regressors, e.g., CNNs, which has shown promising results. Nonetheless, the cascade of CNNs suffers from several drawbacks, e.g., lack of end-to-end training, handcrafted features and slow training speed. To address these issues, we propose a new layer, named visualization layer, which can be integrated into the CNN architecture and enables joint optimization with different loss functions. Extensive evaluation of the proposed method on multiple datasets demonstrates state-of-the-art accuracy, while reducing the training time by more than half compared to the typical cascade of CNNs. In addition, we compare across multiple CNN architectures, all with the visualization layer, to further demonstrate the advantage of its utilization.

Visual inspection of neuroimagery is susceptible to human eye limitations. Computerized methods have been shown to be equally or more e↵ective than human clinicians in diagnosing dementia from neuroimages. Nevertheless, much of the work involves the use of domain expertise to extract hand–crafted features. The key technique in this paper is the use of cross–domain features to represent MRI data. We used a sparse autoencoder to learn a set of bases from natural images and then applied convolution to extract features from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset. Using this new representation, we classify MRI instances into three categories: Alzheimer’s Disease (AD), Mild Cognitive Impairment (MCI) and Healthy Control (HC). Our approach, in spite of being very simple, achieved high classification performance, which is competitive with or better than other approaches.

Face representation is a crucial step of face recognition systems. An optimal face representation should be discriminative, robust, compact, and very easyto-implement. While numerous hand-crafted and learning-based representations have been proposed, considerable room for improvement is still present. In this paper, we present a very easy-to-implement deep learning framework for face representation. Our method bases on a new structure of deep network (called Pyramid CNN). The proposed Pyramid CNN adopts a greedy-filter-and-down-sample operation, which enables the training procedure to be very fast and computationefficient. In addition, the structure of Pyramid CNN can naturally incorporate feature sharing across multi-scale face representations, increasing the discriminative ability of resulting representation. Our basic network is capable of achieving high recognition accuracy (85.8% on LFW benchmark) with only 8 dimension representation. When extended to feature-sharing Pyramid CNN, our system achieves the state-of-the-art performance (97.3%) on LFW benchmark. We also introduce a new benchmark of realistic face images on social network and validate our proposed representation has a good ability of generalization.

In this paper, we propose a deep learning approach for image registration by predicting deformation from image appearance. Since obtaining ground-truth deformation fields for training can be challenging, we design a fully convolutional network that is subject to dual-guidance: (1) Coarse guidance using deformation fields obtained by an existing registration method; and (2) Fine guidance using image similarity. The latter guidance helps avoid overly relying on the supervision from the training deformation fields, which could be inaccurate. For effective training, we further improve the deep convolutional network with gap filling, hierarchical loss, and multi-source strategies. Experiments on a variety of datasets show promising registration accuracy and efficiency compared with state-of-the-art methods.

BACKGROUND
Aging in the lower face leads to lines, wrinkles, depression of the corners of the mouth, and changes in lip volume and lip shape, with increased sagging of the skin of the jawline. Refined, easy-to-use, validated, objective standards assessing the severity of these changes are required in clinical research and practice.

OBJECTIVE
To establish the reliability of eight lower face scales assessing nasolabial folds, marionette lines, upper and lower lip fullness, lip wrinkles (at rest and dynamic), the oral commissure and jawline, aesthetic areas, and the lower face unit.

METHODS AND MATERIALS
Four 5-point rating scales were developed to objectively assess upper and lower lip wrinkles, oral commissures, and the jawline. Twelve experts rated identical lower face photographs of 50 subjects in two separate rating cycles using eight 5-point scales. Inter- and intrarater reliability of responses was assessed.

RESULTS
Interrater reliability was substantial or almost perfect for all lower face scales, aesthetic areas, and the lower face unit. Intrarater reliability was high for all scales, areas and the lower face unit.

CONCLUSION
Our rating scales are reliable tools for valid and reproducible assessment of the aging process in lower face areas.

Large face datasets are important for advancing face recognition research, but they are tedious to build, because a lot of work has to go into cleaning the huge amount of raw data. To facilitate this task, we describe an approach to building face datasets that starts with detecting faces in images returned from searches for public figures on the Internet, followed by discarding those not belonging to each queried person. We formulate the problem of identifying the faces to be removed as a quadratic programming problem, which exploits the observations that faces of the same person should look similar, have the same gender, and normally appear at most once per image. Our results show that this method can reliably clean a large dataset, leading to a considerable reduction in the work needed to build it. Finally, we are releasing the FaceScrub dataset that was created using this approach. It consists of 141,130 faces of 695 public figures and can be obtained from http://vintage.winklerbros.net/facescrub.html.