Question: An investigation into active learning for clinical information extraction, focusing on external knowledge and query strategies.

This study explores the role of information-based objective functions in enhancing the efficiency of active data selection within the context of clinical information extraction. Active learning algorithms can significantly improve learning outcomes by strategically choosing the most informative data points for labeling. Within a Bayesian learning framework, we examine various objective functions designed to quantify the anticipated informativeness of potential data samples. Three distinct criteria for data selection emerge, each reflecting a different perspective on what information is deemed valuable. These criteria are predicated on the assumption that the hypothesis space is accurate, which could potentially be their Achilles' heel, especially when dealing with complex clinical data where model assumptions may not always hold true. Despite this limitation, these information-based objective functions offer a promising avenue for optimizing the active learning process in clinical information extraction tasks, potentially leading to more accurate and efficient models for medical data analysis.

Distant supervision has attracted recent interest for training information extraction systems because it does not require any human annotation but rather employs existing knowledge bases to heuristically label a training corpus. However, previous work has failed to address the problem of false negative training examples mislabeled due to the incompleteness of knowledge bases. To tackle this problem, we propose a simple yet novel framework that combines a passage retrieval model using coarse features into a state-of-the-art relation extractor using multi-instance learning with fine features. We adapt the information retrieval technique of pseudorelevance feedback to expand knowledge bases, assuming entity pairs in top-ranked passages are more likely to express a relation. Our proposed technique significantly improves the quality of distantly supervised relation extraction, boosting recall from 47.7% to 61.2% with a consistently high level of precision of around 93% in the experiments.

We describe the architecture of the AskMSR question answering system and systematically evaluate contributions of different system components to accuracy. The system differs from most question answering systems in its dependency on data redundancy rather than sophisticated linguistic analyses of either questions or candidate answers. Because a wrong answer is often worse than no answer, we also explore strategies for predicting when the question answering system is likely to give an incorrect answer.

We introduce a methodology for automating the maintenance of domain-specific taxonomies based on natural language text understanding. A given ontology is incrementally updated as new concepts are acquired from real-world texts. The acquisition process is centered around the linguistic and conceptual “quality” of various forms of evidence underlying the generation and refinement of concept hypotheses. On the basis of the quality of evidence, concept hypotheses are ranked according to credibility and the most credible ones are selected for assimilation into the domain knowledge base.

Conventional QA systems cannot answer to the questions composed of two or more sentences. Therefore, we aim to construct a QA system that can answer such multiple-sentence questions. As the first stage, we propose a method for classifying multiple-sentence questions into question types. Specifically, we first extract the core sentence from a given question text. We use the core sentence and its question focus in question classification. The result of experiments shows that the proposed method improves F-measure by 8.8% and accuracy by 4.4%.

An information-seeking system is described which combines traditional keyword querying of WWW resources with the ability to browse and query against RDF annotations of those resources. RDF(S) and RDF are used to specify and populate an ontology and the resultant RDF annotations are then indexed along with the full text of the annotated resources. The resultant index allows both keyword querying against the full text of the document and the literal values occurring in the RDF annotations, along with the ability to browse and query the ontology. We motivate our approach as a key enabler for fully exploiting the semantic Web in the area of knowledge management and argue that the ability to combine searching and browsing behaviours more fully supports a typical information-seeking task. The approach is characterised as "low threshold, high ceiling" in the sense that where RDF annotations exist they are exploited for an improved information-seeking experience but where they do not yet exist, a search capability is still available.

The Internet is the largest source of information in the world. Search engines help people navigate the huge space of available data in order to acquire new skills and knowledge. In this paper, we present an in-depth analysis of sessions in which people explicitly search for new knowledge on the Web based on the log files of a popular search engine. We investigate within-session and cross-session developments of expertise, focusing on how the language and search behavior of a user on a topic evolves over time. In this way, we identify those sessions and page visits that appear to significantly boost the learning process. Our experiments demonstrate a strong connection between clicks and several metrics related to expertise. Based on models of the user and their specific context, we present a method capable of automatically predicting, with good accuracy, which clicks will lead to enhanced learning. Our findings provide insight into how search engines might better help users learn as they search.

We present a theoretical analysis of active learning with more realistic interactions with human oracles. Previous empirical studies have shown oracles abstaining on difficult queries until accumulating enough information to make label decisions. We formalize this phenomenon with an “oracle epiphany model” and analyze active learning query complexity under such oracles for both the realizable and the agnostic cases. Our analysis shows that active learning is possible with oracle epiphany, but incurs an additional cost depending on when the epiphany happens. Our results suggest new, principled active learning approaches with realistic oracles.

This volume takes a broad view of information extraction as any method for ltering information from large volumes of text. This includes the retrieval of documents from collections and the tagging of particular terms in text. In this paper we shall use a narrower de nition: the identi cation of instances of a particular class of events or relationships in a natural language text, and the extraction of the relevant arguments of the event or relationship. Information extraction therefore involves the creation of a structured representation (such as a data base) of selected information drawn from the text. The idea of reducing the information in a document to a tabular structure is not new. Its feasibility for sublanguage texts was suggested by Zellig Harris in the 1950's, and an early implementation for medical texts was done at New York University by Naomi Sager[20]. However, the speci c notion of information extraction described here has received wide currency over the last decade through the series of Message Understanding Conferences [1, 2, 3, 4, 14]. We shall discuss these Conferences in more detail a bit later, and shall use simpli ed versions of extraction tasks from these Conferences as examples throughout this paper. Figure 1 shows a simpli ed example from one of the earlier MUC's, involving terrorist events (MUC-3) [1]. For each terrorist event, the system had to determine the type of attack (bombing, arson, etc.), the date, location, perpetrator (if stated), targets, and e ects on targets. Other examples of extraction tasks are international joint ventures (where the arguments included the partners, the new venture, its product or service, etc.) and executive succession (indicating who was hired or red by which company for which position). Information extraction is a more limited task than \full text understanding". In full text understanding, we aspire to represent in a explicit fashion all the information in a text. In contrast, in information extraction we delimit in advance, as part of the speci cation of the task, the semantic range of the output: the relations we will represent, and the allowable llers in each slot of a relation.

We present a new approach to the effective development of menu construction systems that allow to automatically construct a menu that is strongly tailored to the individual requirements and food preferences of a client. In hospitals and other health care institutions dietitians develop diets for clients which need to change their eating habits. Many clients have special needs in regards to their medical conditions, cultural backgrounds, or special levels of nutrient requirements for better recovery from diseases or surgery, etc. Existing computer support for this task is insufficient-many diets are not specifically tailored for the client's needs or require substantial time of a dietitian to be manually developed. Our approach is based on case-based reasoning, an artificial intelligence technique that finds increasing entry into industrial practice. Our approach goes beyond the traditional case-based reasoning (CBR) approach by allowing an incremental improvement of the system's competency during routine use of the system. The improvement of the system takes place through a direct expert user-system interaction while the expert is accomplishing their tasks of constructing a diet for a given client. Whenever the system performs unsatisfactorily, the expert will need to modify the system-produced diet 'manually', i.e. by entering the desired modifications into the system. Our implemented system, menu construction using an incremental knowledge acquisition system (MIKAS), asks the expert for simple explanations for each of the manual actions he/she takes and incorporates the explanations automatically into its knowledge base (KB) so that the system will perform these manually conducted actions automatically at the next occasion. We present MIKAS and discuss the results of our case study. While still being a prototype, the senior clinical dietitian involved in our evaluation studies judges the approach to have considerable potential to improve the daily routine of hospital dietitians as well as to improve the average quality of the dietary advice given to patients within the limited available time for dietary consultations. Our approach opens up a new avenue towards building highly specialised CBR systems in a more cost-effective way. Hence, our approach promises to allow a significantly more widespread development and practical deployment of CBR systems in a large variety of application domains including many medical applications.

Many Knowledge Bases (KBs) are now readily available and encompass colossal quantities of information thanks to either a long-term funding effort (e.g. WordNet, OpenCyc) or a collaborative process (e.g. Freebase, DBpedia). However, each of them is based on a different rigid symbolic framework which makes it hard to use their data in other systems. It is unfortunate because such rich structured knowledge might lead to a huge leap forward in many other areas of AI like natural language processing (word-sense disambiguation, natural language understanding, ...), vision (scene classification, image semantic annotation, ...) or collaborative filtering. In this paper, we present a learning process based on an innovative neural network architecture designed to embed any of these symbolic representations into a more flexible continuous vector space in which the original knowledge is kept and enhanced. These learnt embeddings would allow data from any KB to be easily used in recent machine learning methods for prediction and information retrieval. We illustrate our method on WordNet and Freebase and also present a way to adapt it to knowledge extraction from raw text.


Question: Automated extraction of road networks from unmanned aerial vehicle images in mountainous regions.

This paper outlines a novel methodology for the automatic extraction of road networks from aerial images, specifically focusing on the detection and refinement of identified roads. The proposed technique is a two-phase process: road footprint detection and subsequent refinement. The initial phase, road footprint detection, is achieved by analyzing the shape of homogeneous regions surrounding individual pixels. These regions are delineated by polygons, which we term 'footprints', to identify potential road segments. Road footprints are detected using a spoke wheel operator, and a unique automatic seeding algorithm is introduced, based on the approximation of footprints as rectangles, to initiate the growth of a road tree. A toe-finding algorithm is then applied to classify these footprints for further growth of the road tree structure. The second phase, refinement, employs a Bayes decision model that leverages the area-to-perimeter ratio (A/P ratio) of footprints to eliminate false detections. A lognormal distribution is utilized to characterize the conditional probability of A/P ratios for footprints within the road tree, and an automatic parameter estimation method for the Bayes decision model is proposed. The effectiveness of the method is demonstrated through its application to a variety of aerial images, with results indicating a high level of performance. Specifically, the road tracker exhibits a completeness range of 84% to 94%, correctness above 81%, and overall quality from 82% to 92%, showcasing its capability in accurately extracting road networks from aerial imagery.

In recent years, much progress has been made in outdoor autonomo us navigation. However, safe navigation is still a daunting challenge in terrain containing vegetation. In this paper, we f ocus on the segmentation of ladar data into three classes using local three-dimensional point cloud statistics. The classes are: ”scatter” to represent porous volumes such as grass and tree canopy, ”linear” to capture thin objects like wires or tree branches, and finally ”surface” to capture solid objects like ground surface, rocks or large trunks. We present the details of the proposed method, and the modifications we made to implement it on-board an autonom ous ground vehicle for real-time data processing. Finally, we present results produced from different sta tionary laser sensors and from field tests using an unmanned ground vehicle.

Modern cars are incorporating an increasing number of driver assist features, among which automatic lane keeping. The latter allows the car to properly position itself within the road lanes, which is also crucial for any subsequent lane departure or trajectory planning decision in fully autonomous cars. Traditional lane detection methods rely on a combination of highly-specialized, hand-crafted features and heuristics, usually followed by post-processing techniques, that are computationally expensive and prone to scalability due to road scene variations. More recent approaches leverage deep learning models, trained for pixel-wise lane segmentation, even when no markings are present in the image due to their big receptive field. Despite their advantages, these methods are limited to detecting a pre-defined, fixed number of lanes, e.g. ego-lanes, and can not cope with lane changes. In this paper, we go beyond the aforementioned limitations and propose to cast the lane detection problem as an instance segmentation problem - in which each lane forms its own instance - that can be trained end-to-end. To parametrize the segmented lane instances before fitting the lane, we further propose to apply a learned perspective transformation, conditioned on the image, in contrast to a fixed ”bird’s-eye view” transformation. By doing so, we ensure a lane fitting which is robust against road plane changes, unlike existing approaches that rely on a fixed, predefined transformation. In summary, we propose a fast lane detection algorithm, running at 50 fps, which can handle a variable number of lanes and cope with lane changes. We verify our method on the tuSimple dataset and achieve competitive results.

Detecting and reading text from natural images is a hard computer vision task that is central to a variety of emerging applications. Related problems like document character recognition have been widely studied by computer vision and machine learning researchers and are virtually solved for practical applications like reading handwritten digits. Reliably recognizing characters in more complex scenes like photographs, however, is far more difficult: the best existing methods lag well behind human performance on the same tasks. In this paper we attack the problem of recognizing digits in a real application using unsupervised feature learning methods: reading house numbers from street level photos. To this end, we introduce a new benchmark dataset for research use containing over 600,000 labeled digits cropped from Street View images. We then demonstrate the difficulty of recognizing these digits when the problem is approached with hand-designed features. Finally, we employ variants of two recently proposed unsupervised feature learning methods and find that they are convincingly superior on our benchmarks.

The majority of methods for the automatic surface reconstruction of an environment from an image sequence have two steps: Structure-from-Motion and dense stereo. From the computational standpoint, it would be interesting to avoid dense stereo and to generate a surface directly from the sparse cloud of 3D points and their visibility information provided by Structure-fromMotion. The previous attempts to solve this problem are currently very limited: the surface is non-manifold or has zero genus, the experiments are done on small scenes or objects using a few dozens of images. Our solution does not have these limitations. Furthermore, we experiment with hand-held or helmet-held catadioptric cameras moving in a city and generate 3D models such that the camera trajectory can be longer than one kilometer.

A 3D vehicle path planning method based on heuristic A∗ algorithm is proposed and simulated which makes the UAV's path planning for low altitude penetration more real-time, effective and engineering-oriented. The digital terrain information is processed with the integrated terrain elevation smoothing algorithm at first, and then, the safe flight surface which integrates the maneuverability of vehicle is built, At last, the mathematical models of the threats are established. An improved A∗ algorithm is used to plan 3D vehicle path on the safe surface, and the planned path is optimized and smoothed. Simulation results indicate that the method has a good performance in creating optimal path in 3D environment and what's more, the path planning algorithm is simple, efficient and easily realized in engineering field, thus adapting to the different missions of the UAV.

Robust cross-seasonal localization is one of the major challenges in long-term visual navigation of autonomous vehicles. In this paper, we exploit recent advances in semantic segmentation of images, i.e., where each pixel is assigned a label related to the type of object it represents, to attack the problem of long-term visual localization. We show that semantically labeled 3D point maps of the environment, together with semantically segmented images, can be efficiently used for vehicle localization without the need for detailed feature descriptors (SIFT, SURF, etc.), Thus, instead of depending on hand-crafted feature descriptors, we rely on the training of an image segmenter. The resulting map takes up much less storage space compared to a traditional descriptor based map. A particle filter based semantic localization solution is compared to one based on SIFT-features, and even with large seasonal variations over the year we perform on par with the larger and more descriptive SIFT-features, and are able to localize with an error below 1 m most of the time.

Dempster-Shafer theory provides a sensor fusion framework that autonomously accounts for obstacle occlusion in dynamic, urban environments. However, to discern static and moving obstacles, the Dempster-Shafer approach requires manual tuning of parameters dependent on the situation and sensor types. The proposed methodology utilizes a deep fully convolutional neural network to improve the robust performance of the information fusion algorithm in distinguishing static and moving obstacles from navigable space. The image-like spatial structure of probabilistic occupancy allows a semantic segmentation framework to discern classes for individual grid cells. A subset of the KITTI LIDAR tracking dataset in combination with semantic map data was used for the information fusion task. The probabilistic occupancy grid output of the Dempster-Shafer information fusion algorithm was provided as input to the neural network. The network then learned an offset from the original DST result to improve semantic labeling performance. The proposed framework outperformed the baseline approach in the mean intersection over union metric reaching 0.546 and 0.531 in the validation and test sets respectively. However, little improvement was achieved in discerning moving and static cells due to the limited dataset size. To improve model performance in future work, the dataset will be expanded to facilitate more effective learning, and temporal data will be fed through individual convolutional networks prior to being merged in channels as input to the main network.

Cloud detection plays a major role for remote sensing image processing. Most of the existed cloud detection methods use the low-level feature of the cloud, which often cause error result especially for thin cloud and complex scene. In this paper, a novel cloud detection method based on deep learning framework is proposed. The designed deep Convolutional Neural Networks (CNNs) consists of four convolutional layers and two fully-connected layers, which can mine the deep features of cloud. The image is firstly clustered into superpixels as sub-region through simple linear iterative cluster (SLIC) method. Through the designed network model, the probability of each superpixel that belongs to cloud region is predicted, so that the cloud probability map of the image is generated. Lastly, the cloud region is obtained according to the gradient of the cloud map. Through the proposed method, both thin cloud and thick cloud can be detected well, and the result is insensitive to complex scene. Experimental results indicate that the proposed method is more robust and effective than compared methods.

In this paper we present a perception system for agriculture robotics that enables an unmanned ground vehicle (UGV) equipped with a multi spectral camera to automatically perform the crop/weed detection and classification tasks in real-time. Our approach exploits a pipeline that includes two different convolutional neural networks (CNNs) applied to the input RGB+near infra-red (NIR) images. A lightweight CNN is used to perform a fast and robust, pixelwise, binary image segmentation, in order to extract the pixels that represent projections of 3D points that belong to green vegetation. A deeper CNN is then used to classify the extracted pixels between the crop and weed classes. A further important contribution of this work is a novel unsupervised dataset summarization algorithm that automatically selects from a large dataset the most informative subsets that better describe the original one. This enables to streamline and speed-up the manual dataset labeling process, otherwise extremely time consuming, while preserving good classification performances. Experiments performed on different datasets taken from a real farm robot confirm the effectiveness of our approach.

This paper describes a method for detecting and recognizing road signs in gray-level and color images acquired by a single camera mounted on a moving vehicle. The method works in three stages. First, the search for the road sign is reduced to a suitable region of the image by using some a priori knowledge on the scene or color clues (when available). Secondly, a geometrical analysis of the edges extracted from the image is carried out, which generates candidates to be circular and triangular signs. Thirdly, a recognition stage tests by cross-correlation techniques each candidate which, if validated, is classi ed according to the data-base of signs. An extensive experimentation has shown that the method is robust against low-level noise corrupting edge detection and contour following, and works for images of cluttered urban streets as well as country roads and highways. A further improvement on the detection and recognition scheme has been obtained by means of temporal integration based on Kalman ltering methods of the extracted information. The proposed approach can be very helpful for the development of a system for driving assistance.


Question: Automated extraction of road networks from unmanned aerial vehicle images in mountainous regions.

This paper details a novel real-time road mapping system designed for the acquisition of high-resolution imagery and geometric data of roads, particularly from unmanned aerial vehicles (UAVs). The system encompasses the design of the UAV platform, onboard instrumentation, data transmission mechanisms, processing algorithms, and storage solutions. Unlike conventional mapping methods, UAV platforms offer unparalleled flexibility, quicker deployment, and the ability to generate ultra-high-resolution data. They are cost-effective, operate without air traffic restrictions, and, with the aid of autonomous flight control systems, can follow pre-determined flight paths with precision. In this study, a dual-engine, dual-generator unmanned fixed-wing aircraft is employed, enhancing the platform's reliability and operational capacity. A digital, three-axis stabilization platform is also developed to maintain camera stability and control attitude during flight. To ensure comprehensive data coverage, a data management system is integrated; this system checks data integrity upon reception and archives it immediately. Should any data anomalies be detected, the system is capable of adjusting the mission plan to re-capture the necessary imagery. The paper further discusses the process of extracting vehicle information from image sequences. Experimental results confirm the system's efficacy in real-time road mapping and vehicle data extraction, particularly in complex environments such as mountainous areas.

We provide a novel hardware and software system for micro air vehicles (MAV) that allows high-speed, low-latency onboard image processing. It uses up to four cameras in parallel on a miniature rotary wing platform. The MAV navigates based on onboard processed computer vision in GPS-denied in- and outdoor environments. It can process in parallel images and inertial measurement information from multiple cameras for multiple purposes (localization, pattern recognition, obstacle avoidance) by distributing the images on a central, low-latency image hub. Furthermore the system can utilize low-bandwith radio links for communication and is designed and optimized to scale to swarm use. Experimental results show successful flight with a range of onboard computer vision algorithms, including localization, obstacle avoidance and pattern recognition.

We propose a new method for the localization of a Micro Aerial Vehicle (MAV) with respect to a ground robot. We solve the problem of registering the 3D maps computed by the robots using different sensors: a dense 3D reconstruction from the MAV monocular camera is aligned with the map computed from the depth sensor on the ground robot. Once aligned, the dense reconstruction from the MAV is used to augment the map computed by the ground robot, by extending it with the information conveyed by the aerial views. The overall approach is novel, as it builds on recent developments in live dense reconstruction from moving cameras to address the problem of air-ground localization. The core of our contribution is constituted by a novel algorithm integrating dense reconstructions from monocular views, Monte Carlo localization, and an iterative pose refinement. In spite of the radically different vantage points from which the maps are acquired, the proposed method achieves high accuracy whereas appearance-based, state-of-the-art approaches fail. Experimental validation in indoor and outdoor scenarios reported an accuracy in position estimation of 0.08 meters and real time performance. This demonstrates that our new approach effectively overcomes the limitations imposed by the difference in sensors and vantage points that negatively affect previous techniques relying on matching visual features.

In this article, the three most used object detection approaches, Linear Binary Pattern cascade, Haarlike cascade, and Histogram of Oriented Gradients with Support Vector Machine are applied to automatic runway detection in high resolution satellite imagery and their results are compared. They have been employed predominantly for human feature recognition and this paper tests theirs applicability to runways. The results show that they can be indeed employed for this purpose with LBP and Haar approaches performing better than HOG+SVM.

Accurate maps of the static environment are essential for many advanced driver-assistance systems. A new method for the fast computation of occupancy grid maps with laser range-finders and radar sensors is proposed. The approach utilizes the Graphics Processing Unit to overcome the limitations of classical occupancy grid computation in automotive environments. It is possible to generate highly accurate grid maps in just a few milliseconds without the loss of sensor precision. Moreover, in the case of a lower resolution radar sensor it is shown that it is suitable to apply super-resolution algorithms to achieve the accuracy of a higher resolution laser-scanner. Finally, a novel histogram based approach for road boundary detection with lidar and radar sensors is presented.

This paper presents a set of segmentation methods for various types of 3D point clouds. Segmentation of dense 3D data (e.g. Riegl scans) is optimised via a simple yet efficient voxelisation of the space. Prior ground extraction is empirically shown to significantly improve segmentation performance. Segmentation of sparse 3D data (e.g. Velodyne scans) is addressed using ground models of non-constant resolution either providing a continuous probabilistic surface or a terrain mesh built from the structure of a range image, both representations providing close to real-time performance. All the algorithms are tested on several hand labeled data sets using two novel metrics for segmentation evaluation.

This paper presents the on-going design and implementation of a robust inertial sensor based Simultaneous Localisation And Mapping (SLAM) algorithm for an Unmanned Aerial Vehicle (UAV) using bearing-only observations. A single colour vision camera is used to observe the terrain from which image points corresponding to features in the environment are extracted. The SLAM algorithm estimates the complete 6-DoF motion of the UAV along with the three-dimensional position of the features in the environment. An Extended Kalman Filter (EKF) approach is used where a technique of delayed initialisation is performed to initialise the 3D position of features from bearing-only observations. Data association is achieved using a multi-hypothesis innovation gate based on the spatial uncertainty of each feature. Results are presented by running the algorithm off-line using inertial sensor and vision data collected during a flight test of a UAV.

Inspired by the recent success of deep convolutional neural networks (CNNs) and feature aggregation in the field of computer vision and machine learning, we propose an effective approach to semantic pixel labeling of aerial and satellite imagery using both CNN features and hand-crafted features. Both CNN and hand-crafted features are applied to dense image patches to produce per-pixel class probabilities. Conditional random fields (CRFs) are applied as a postprocessing step. The CRF infers a labeling that smooths regions while respecting the edges present in the imagery. The combination of these factors leads to a semantic labeling framework which outperforms all existing algorithms on the International Society of Photogrammetry and Remote Sensing (ISPRS) two-dimensional Semantic Labeling Challenge dataset. We advance state-of-the-art results by improving the overall accuracy to 88% on the ISPRS Semantic Labeling Contest. In this paper, we also explore the possibility of applying the proposed framework to other types of data. Our experimental results demonstrate the generalization capability of our approach and its ability to produce accurate results.

Selective weed treatment is a critical step in autonomous crop management as related to crop health and yield. However, a key challenge is reliable and accurate weed detection to minimize damage to surrounding plants. In this letter, we present an approach for dense semantic weed classification with multispectral images collected by a micro aerial vehicle (MAV). We use the recently developed encoder–decoder cascaded convolutional neural network, SegNet, that infers dense semantic classes while allowing any number of input image channels and class balancing with our sugar beet and weed datasets. To obtain training datasets, we established an experimental field with varying herbicide levels resulting in field plots containing only either crop or weed, enabling us to use the normalized difference vegetation index as a distinguishable feature for automatic ground truth generation. We train six models with different numbers of input channels and condition (fine tune) it to achieve  $\sim$0.8 F1-score and 0.78 area under the curve classification metrics. For the model deployment, an embedded Graphics Processing Unit (GPU) system (Jetson TX2) is tested for MAV integration. Dataset used in this letter is released to support the community and future work.

This paper describes a data set collected by an autonomous ground vehicle testbed, based upon a modified Ford F-250 pickup truck. The vehicle is outfitted with a professional (Applanix POS-LV) and consumer (Xsens MTi-G) inertial measurement unit (IMU), a Velodyne 3D-lidar scanner, two push-broom forward looking Riegl lidars, and a Point Grey Ladybug3 omnidirectional camera system. Here we present the time-registered data from these sensors mounted on the vehicle, collected while driving the vehicle around the Ford Research campus and downtown Dearborn, Michigan during NovemberDecember 2009. The vehicle path trajectory in these data sets contain several large and small-scale loop closures, which should be useful for testing various state of the art computer vision and simultaneous localization and mapping (SLAM) algorithms. Fig. 1. The modified Ford F-250 pickup truck.

Road detection is a key component of autonomous driving; however, most fully supervised learning road detection methods suffer from either insufficient training data or high costs of manual annotation. To overcome these problems, we propose a semisupervised learning (SSL) road detection method based on generative adversarial networks (GANs) and a weakly supervised learning (WSL) method based on conditional GANs. Specifically, in our SSL method, the generator generates the road detection results of labeled and unlabeled images, and then they are fed into the discriminator, which assigns a label on each input to judge whether it is labeled. Additionally, in WSL method we add another network to predict road shapes of input images and use them in both generator and discriminator to constrain the learning progress. By training under these frameworks, the discriminators can guide a latent annotation process on the unlabeled data; therefore, the networks can learn better representations of road areas and leverage the feature distributions on both labeled and unlabeled data. The experiments are carried out on KITTI ROAD benchmark, and the results show our methods achieve the state-of-the-art performances.


Question: Automated extraction of road networks from unmanned aerial vehicle images in mountainous regions.

This document outlines a computational methodology for identifying edges in digital images, focusing on a systematic approach to edge detection. The effectiveness of this method hinges on establishing a clear set of objectives for the algorithm that calculates edge points. These objectives should be sufficiently precise to outline the intended behavior of the edge detector, minimizing assumptions about the specific solution form. We introduce detection and localization standards for a defined category of edges, formulating these criteria mathematically as functionals based on the operator's impulse response. An additional criterion is introduced to ensure that the detector responds uniquely to each individual edge. These criteria are applied in an optimization framework to develop detectors for various common image features, with a particular emphasis on step edges. Upon focusing the analysis on step edges, it becomes evident that there exists a fundamental trade-off, akin to the uncertainty principle, between detection and localization performance, the two primary goals. Using this principle, we derive a singular operator shape that is optimal at any scale. The optimal detector can be approximated by a straightforward implementation, where edges are identified at the peaks of the gradient magnitude in an image smoothed by a Gaussian filter. To address varying signal-to-noise ratios within images, we enhance this basic detector by utilizing operators of multiple widths. Additionally, we introduce a comprehensive technique, termed feature synthesis, for the hierarchical integration of information from operators operating at different scales, facilitating a transition from fine to coarse levels of detail. Lastly, we demonstrate that the performance of step edge detection is significantly enhanced as the operator's point spread function is elongated along the edge direction. This methodology is particularly relevant for applications requiring precise and robust edge detection in complex images, such as those encountered in automatic road network extraction from unmanned aerial vehicle (UAV) images in mountainous regions.

We introduce a method for following high-level navigation instructions by mapping directly from images, instructions and pose estimates to continuous low-level velocity commands for real-time control. The Grounded Semantic Mapping Network (GSMN) is a fully-differentiable neural network architecture that builds an explicit semantic map in the world reference frame by incorporating a pinhole camera projection model within the network. The information stored in the map is learned from experience, while the local-to-world transformation is computed explicitly. We train the model using DAGGERFM, a modified variant of DAGGER that trades tabular convergence guarantees for improved training speed and memory use. We test GSMN in virtual environments on a realistic quadcopter simulator and show that incorporating an explicit mapping and grounding modules allows GSMN to outperform strong neural baselines and almost reach an expert policy performance. Finally, we analyze the learned map representations and show that using an explicit map leads to an interpretable instruction-following model.

Developing on-board automotive driver assistance systems aiming to alert drivers about driving environments, and possible collision with other vehicles has attracted a lot of attention lately. In these systems, robust and reliable vehicle detection is a critical step. This paper presents a review of recent vision-based on-road vehicle detection systems. Our focus is on systems where the camera is mounted on the vehicle rather than being fixed such as in traffic/driveway monitoring systems. First, we discuss the problem of on-road vehicle detection using optical sensors followed by a brief review of intelligent vehicle research worldwide. Then, we discuss active and passive sensors to set the stage for vision-based vehicle detection. Methods aiming to quickly hypothesize the location of vehicles in an image as well as to verify the hypothesized locations are reviewed next. Integrating detection with tracking is also reviewed to illustrate the benefits of exploiting temporal continuity for vehicle detection. Finally, we present a critical overview of the methods discussed, we assess their potential for future deployment, and we present directions for future research.

In this paper a new approach to lane marker detection problem is introduced as a significant complement for a semi/fully autonomous driver assistance system. The method incorporates advance line detection using multilayer fractional Fourier transform (MLFRFT) and the state-of-the-art advance lane detector (ALD). Experimental results have shown a considerable reduction in computational complexity.

This paper demonstrates a generalised multi-scale feature learning approach to multi-class segmentation, applied to the estimation of fruit yield on treecrops. The learning approach makes the algorithm flexible and adaptable to different classification problems, and hence applicable to a wide variety of tree-crop applications. Extensive experiments were performed on a dataset consisting of 8000 colour images collected in an apple orchard. This paper shows that the algorithm was able to segment apples with different sizes and colours in an outdoor environment with natural lighting conditions, with a single model obtained from images captured using a monocular colour camera. The segmentation results are applied to the problem of fruit counting and the results are compared against manual counting. The results show a squared correlation coefficient of R2 = 0.81.

Accurate maps of the static environment are essential for many advanced driver-assistance systems. A new method for the fast computation of occupancy grid maps with laser range-finders and radar sensors is proposed. The approach utilizes the Graphics Processing Unit to overcome the limitations of classical occupancy grid computation in automotive environments. It is possible to generate highly accurate grid maps in just a few milliseconds without the loss of sensor precision. Moreover, in the case of a lower resolution radar sensor it is shown that it is suitable to apply super-resolution algorithms to achieve the accuracy of a higher resolution laser-scanner. Finally, a novel histogram based approach for road boundary detection with lidar and radar sensors is presented.

Dempster-Shafer theory provides a sensor fusion framework that autonomously accounts for obstacle occlusion in dynamic, urban environments. However, to discern static and moving obstacles, the Dempster-Shafer approach requires manual tuning of parameters dependent on the situation and sensor types. The proposed methodology utilizes a deep fully convolutional neural network to improve the robust performance of the information fusion algorithm in distinguishing static and moving obstacles from navigable space. The image-like spatial structure of probabilistic occupancy allows a semantic segmentation framework to discern classes for individual grid cells. A subset of the KITTI LIDAR tracking dataset in combination with semantic map data was used for the information fusion task. The probabilistic occupancy grid output of the Dempster-Shafer information fusion algorithm was provided as input to the neural network. The network then learned an offset from the original DST result to improve semantic labeling performance. The proposed framework outperformed the baseline approach in the mean intersection over union metric reaching 0.546 and 0.531 in the validation and test sets respectively. However, little improvement was achieved in discerning moving and static cells due to the limited dataset size. To improve model performance in future work, the dataset will be expanded to facilitate more effective learning, and temporal data will be fed through individual convolutional networks prior to being merged in channels as input to the main network.

We use the AdaBoost algorithm to classify 3D aerial lidar scattered height data into four categories: road, grass, buildings, and trees. To do so we use five features: height, height variation, normal variation, lidar return intensity, and image intensity. We also use only lidar-derived features to organize the data into three classes (the road and grass classes are merged). We apply and test our results using ten regions taken from lidar data collected over an area of approximately eight square miles, obtaining higher than 92% accuracy. We also apply our classifier to our entire dataset, and present visual classification results both with and without uncertainty. We implement and experiment with several variations within the AdaBoost family of algorithms. We observe that our results are robust and stable over all the various tests and algorithmic variations. We also investigate features and values that are most critical in distinguishing between the classes. This insight is important in extending the results from one geographic region to another.

In this paper we describe a method for detecting buried objects of interest using a forward looking infrared camera (FLIR) installed on a moving vehicle. Infrared (IR) detection of buried targets is based on the thermal gradient between the object and the surrounding soil. The processing of FILR images consists in a spot-finding procedure that includes edge detection, opening and closing. Each spot is then described using texture features such as histogram of gradients (HOG) and local binary patterns (LBP) and assigned a target confidence using a support vector machine (SVM) classifier. Next, each spot together with its confidence is projected and summed in the UTM space. To validate our approach, we present results obtained on 6 one mile long runs recorded with a long wave IR (LWIR) camera installed on a moving vehicle.

Nowadays, GPS-based car navigation systems mainly use speech and aerial views of simplified road maps to guide drivers to their destination. However, drivers often experience difficulties in linking the simple 2D aerial map with the visual impression that they get from the real environment, which is inherently ground-level based. Therefore, supplying realistically textured 3D city models at ground-level proves very useful for pre-visualizing an upcoming traffic situation. Because this pre-visualization can be rendered from the expected future viewpoints of the driver, the latter will more easily understand the required maneuver. 3D city models can be reconstructed from the imagery recorded by surveying vehicles. The vastness of image material gathered by these vehicles, however, puts extreme demands on vision algorithms to ensure their practical usability. Algorithms need to be as fast as possible and should result in compact, memory efficient 3D city models for future ease of distribution and visualization. For the considered application, these are not contradictory demands. Simplified geometry assumptions can speed up vision algorithms while automatically guaranteeing compact geometry models. We present a novel city modeling framework which builds upon this philosophy to create 3D content at high speed which could allow for pre-visualization of any conceivable traffic situation by car navigation modules.

In most autonomous driving applications, such as parking and commuting, a vehicle follows a previously taken route, or almost the same route. In this paper, we propose a method to localize a vehicle along a previously driven route using images. The proposed method consists of two stages: offline creation of a database, and online localization. In the offline stage, a database is created from images that are captured when the vehicle drives a route for the first time. The database consists of images, 3D positions of feature points estimated by structure-from-motion, and a topological graph. In the online stage, the method first identifies the database image that is most similar to the current image by topometric localization, which considers topological information on a metric scale. The vehicle poses are then estimated from the 3D-2D correspondences of matching feature points between the current image and the identified database image. In an experiment, we estimated vehicle poses using images captured in an indoor parking lot.


Question: Automated extraction of road networks from unmanned aerial vehicle images in mountainous regions.

An Overview of a Mini Unmanned Aerial Vehicle (UAV) System: Image Acquisition and Cultural Heritage Documentation

In recent years, the use of Unmanned Aerial Vehicle (UAV) systems has gained prominence in various fields, including precision agriculture, infrastructure upkeep, and road maintenance. This paper offers a comprehensive view of UAV systems and their role in photogrammetric documentation and recording of cultural heritage sites. Initially, it traces the historical development of UAV systems and defines UAV helicopters, highlighting the advantages of integrating photogrammetric systems with model helicopters compared to conventional aerial and terrestrial photogrammetry. Given their cost-effectiveness and flexibility, UAVs present a viable alternative to other mobile mapping solutions.

A mini UAV system was deployed for photogrammetric image data collection in Palpa, Peru, focusing on a 13th-century settlement believed to have been a mining site. The plan is to create a precise 3D model using the collected image data. To prepare for the flight, an orthophoto and a Digital Elevation Model (DEM) were derived from aerial images at a scale of 1:7,000, facilitating the creation of a flight plan. The designated flight paths were programmed into the flight control system, enabling the model helicopter to autonomously navigate to predetermined waypoints. Flight tests in Switzerland and at the Pinchango Alto site confirmed that the helicopter could accurately reach the pre-programmed GPS coordinates and maintain the planned flight height, utilizing the GPS/INS stabilization features of the flight control system. This accuracy in strip crossings and altitude control is essential for high-quality image acquisition.

This UAV system demonstrates the potential for automated road network extraction from UAV images in mountainous regions, given its precision in navigating and imaging complex terrains. The system's capabilities in capturing detailed images and its ability to perform autonomous flights make it a promising tool for mapping and documenting road networks in challenging environments.

Semantic image segmentation is an essential component of modern autonomous driving systems, as an accurate understanding of the surrounding scene is crucial to navigation and action planning. Current state-of-the-art approaches in semantic image segmentation rely on pre-trained networks that were initially developed for classifying images as a whole. While these networks exhibit outstanding recognition performance (i.e., what is visible?), they lack localization accuracy (i.e., where precisely is something located?). Therefore, additional processing steps have to be performed in order to obtain pixel-accurate segmentation masks at the full image resolution. To alleviate this problem we propose a novel ResNet-like architecture that exhibits strong localization and recognition performance. We combine multi-scale context with pixel-level accuracy by using two processing streams within our network: One stream carries information at the full image resolution, enabling precise adherence to segment boundaries. The other stream undergoes a sequence of pooling operations to obtain robust features for recognition. The two streams are coupled at the full image resolution using residuals. Without additional processing steps and without pre-training, our approach achieves an intersection-over-union score of 71.8% on the Cityscapes dataset.

A new concept for the detection of small objects from modular optoelectronic multispectral scanner (MOMS-02) high spatial resolution panchromatic satellite imagery is presented. We combine supervised shape classification with unsupervised image segmentation in an iterative procedure which allows a target-oriented search for specific object shapes.

One of the key advantages of a low-flying unmanned aircraft system (UAS) is its ability to acquire digital images at an ultrahigh spatial resolution of a few centimeters. Remote sensing of quantitative biochemical and biophysical characteristics of small-sized spatially fragmented vegetation canopies requires, however, not only high spatial, but also high spectral (i.e., hyperspectral) resolution. In this paper, we describe the design, development, airborne operations, calibration, processing, and interpretation of image data collected with a new hyperspectral unmanned aircraft system (HyperUAS). HyperUAS is a remotely controlled multirotor prototype carrying onboard a lightweight pushbroom spectroradiometer coupled with a dual frequency GPS and an inertial movement unit. The prototype was built to remotely acquire imaging spectroscopy data of 324 spectral bands (162 bands in a spectrally binned mode) with bandwidths between 4 and 5 nm at an ultrahigh spatial resolution of 2–5 cm. Three field airborne experiments, conducted over agricultural crops and over natural ecosystems of Antarctic mosses, proved operability of the system in standard field conditions, but also in a remote and harsh, low-temperature environment of East Antarctica. Experimental results demonstrate that HyperUAS is capable of delivering georeferenced maps of quantitative biochemical and biophysical variables of vegetation and of actual vegetation health state at an unprecedented spatial resolution of 5 cm. C © 2014 Wiley Periodicals, Inc.

Ground plane detection plays an important role in stereo vision based obstacle detection methods. Recently, V-Disparity image has been widely used for ground plane detection. The existing approach based on V-Disparity image can detect flat ground successfully but have difficulty in detecting non-flat ground. In this paper, we discuss the representation of non-flat ground in V-Disparity image, based on which we propose a method to detect non-flat ground using V-Disparity image.

This paper introduces an approach for dense 3D reconstruction from unregistered Internet-scale photo collections with about 3 million images within the span of a day on a single PC (“cloudless”). Our method advances image clustering, stereo, stereo fusion and structure from motion to achieve high computational performance. We leverage geometric and appearance constraints to obtain a highly parallel implementation on modern graphics processors and multi-core architectures. This leads to two orders of magnitude higher performance on an order of magnitude larger dataset than competing state-of-the-art approaches.

Most vision-based approaches to mobile robotics suffer from the limitations imposed by stereo obstacle detection, which is short-range and prone to failure. We present a self-supervised learning process for long-range vision that is able to accurately classify complex terrain at distances up to the horizon, thus allowing superior strategic planning. The success of the learning process is due to the self-supervised training data that is generated on every frame: robust, visually consistent labels from a stereo module, normalized wide-context input windows, and a discriminative and concise feature representation. A deep hierarchical network is trained to extract informative and meaningful features from an input image, and the features are used to train a realtime classifier to predict traversability. The trained classifier sees obstacles and paths from 5 to over 100 meters, far beyond the maximum stereo range of 12 meters, and adapts very quickly to new environments. The process was developed and tested on the LAGR mobile robot. Results from a ground truth dataset are given as well as field test results.

We present a robust and real time approach to lane marker detection in urban streets. It is based on generating a top view of the road, filtering using selective oriented Gaussian filters, using RANSAC line fitting to give initial guesses to a new and fast RANSAC algorithm for fitting Bezier Splines, which is then followed by a post-processing step. Our algorithm can detect all lanes in still images of the street in various conditions, while operating at a rate of 50 Hz and achieving comparable results to previous techniques.

Robust scene understanding of outdoor environments using passive optical sensors is a onerous and essential task for autonomous navigation. The problem is heavily characterized by changing environmental conditions throughout the day and across seasons. Robots should be equipped with models that are impervious to these factors in order to be operable and more importantly to ensure safety in the real-world. In this paper, we propose a novel semantic segmentation architecture and the convoluted mixture of deep experts (CMoDE) fusion technique that enables a multi-stream deep neural network to learn features from complementary modalities and spectra, each of which are specialized in a subset of the input space. Our model adaptively weighs class-specific features of expert networks based on the scene condition and further learns fused representations to yield robust segmentation. We present results from experimentation on three publicly available datasets that contain diverse conditions including rain, summer, winter, dusk, fall, night and sunset, and show that our approach exceeds the state-of-the-art. In addition, we evaluate the performance of autonomously traversing several kilometres of a forested environment using only the segmentation for perception.

This paper proposes a method for detecting moving obstacles on roads, by using a vehicle mounted monocular camera. To detect various moving obstacles, such as vehicles and pedestrians, the ego-motion of the vehicle is initially estimated from images captured by the camera. There are two problems in ego-motion estimation. Firstly, a typical road scene contains moving obstacles. This causes false estimation of the ego-motion. Secondly, roads possess fewer features, when compared to the number associated with background structures. This reduces the accuracy of ego-motion estimation. In our approach, the ego-motion is estimated from the correspondences of dispersed feature points extracted from various regions other than those that contain moving obstacles. After estimating the ego-motion, any moving obstacles are detected by tracking the feature points over consecutive frames. In our experiments, it has been shown that the proposed method is able to detect moving obstacles

A new method for extracting planar polygonal rooftops in monocular aerial imagery is proposed. Structural features are extracted and hierarchically related using perceptual grouping techniques. Top-down feature veri cation is used so that features, and links between the features, are veri ed with local information in the image and weighed in a graph. Cycles in the graph correspond to possible building rooftop hypotheses. Virtual features are hypothesized for the perceptual completion of partially occluded rooftops. Extraction of the \best" grouping of features into a building rooftop hypothesis is posed as a graph search problem. The maximally weighted, independent set of cycles in the graph is extracted as the nal set of roof


Question: Automated extraction of road networks from unmanned aerial vehicle images in mountainous regions.

This paper presents an innovative method for road extraction from unmanned aerial vehicle (UAV) images in mountainous areas, utilizing pixel spectral information for classification and image segmentation-derived object features. The approach involves a two-step process: first, a support vector machine (SVM) is applied to classify the image into road and non-road categories, achieving higher accuracy than the Gaussian maximum likelihood (GML) method. In the second step, the road group image is segmented into homogeneous geometric objects using a region growing technique that prioritizes shape factors over spectral criteria. A thresholding technique is then applied to the shape index and density features of these objects to isolate road features, which are further processed through thinning and vectorization to obtain road centerlines. The effectiveness of the proposed approach is demonstrated through experiments on images containing both rural and urban features. Road information is crucial for transportation applications and geographic information systems (GIS), and automated road extraction can significantly reduce the time and labor required for updating road spatial databases. The paper reviews various road extraction methods, including ridge finding, heuristic reasoning, dynamic programming, statistical inference, and map matching, and discusses their respective advantages and limitations. The proposed method, which combines classification and segmentation techniques, is shown to be particularly effective for road extraction from UAV images in mountainous regions, where traditional methods may struggle due to complex terrain and varying lighting conditions. The use of SVM for classification and region growing for segmentation allows for more accurate and robust road feature extraction, even in challenging environments.

Despite the availability of lane departure and lane keeping systems for highway assistance, unmarked and winding rural roads still pose challenges to lane recognition systems. To detect an upcoming curve as soon as possible, the viewing range of image-based lane recognition systems has to be extended. This is done by evaluating 3D information obtained from stereo vision or imaging radar in this paper. Both sensors deliver evidence grids as the basis for road course estimation. Besides known Kalman Filter approaches, Particle Filters have recently gained interest since they offer the possibility to employ cues of a road, which can not be described as measurements needed for a Kalman Filter approach. We propose to combine both principles and their benefits in a Kalman Particle Filter. The comparison between the results gained from this recently published filter scheme and the classical approaches using real world data proves the advantages of the Kalman Particle Filter.

In this paper, a deep learning inspired solution is proposed and developed to enable timeliness and practicality of the pre-existing ground stereo vision guidance system for flxed-wing UAVs' safe landing. Since the ground guidance prototype was restricted within applications due to its untimeliness, eventually the vision-based detection less than 15 fps (frame per second). Under such circumstances, we employ a regression based deep learning algorithm into automatic detection on the flying aircraft in the landing sequential images. The system architecture is upgraded so as to be compatible with the novel deep learning requests, and furthermore, annotated datasets are conducted to support training and testing of the regression-based learning detection algorithm. Experimental results validate that the detection attaches 100 fps or more while the localization accuracy is kept in the same level.

The lack of reliable data in developing countries is a major obstacle to sustainable development, food security, and disaster relief. Poverty data, for example, is typically scarce, sparse in coverage, and labor-intensive to obtain. Remote sensing data such as high-resolution satellite imagery, on the other hand, is becoming increasingly available and inexpensive. Unfortunately, such data is highly unstructured and currently no techniques exist to automatically extract useful insights to inform policy decisions and help direct humanitarian efforts. We propose a novel machine learning approach to extract large-scale socioeconomic indicators from highresolution satellite imagery. The main challenge is that training data is very scarce, making it difficult to apply modern techniques such as Convolutional Neural Networks (CNN). We therefore propose a transfer learning approach where nighttime light intensities are used as a data-rich proxy. We train a fully convolutional CNN model to predict nighttime lights from daytime imagery, simultaneously learning features that are useful for poverty prediction. The model learns filters identifying different terrains and man-made structures, including roads, buildings, and farmlands, without any supervision beyond nighttime lights. We demonstrate that these learned features are highly informative for poverty mapping, even approaching the predictive performance of survey data collected in the field.

A vision-based vehicle guidance system for road vehicles can have three main roles: 1) road detection; 2) obstacle detection; and 3) sign recognition. The first two have been studied for many years and with many good results, but traffi c sign recognition is a less-studied field. Traffi c signs provide drivers with very valuable informatio n about the road, in order to make drivin g safer and easier. We think that traffi c signs must play the same role for autonomous vehicles. They are designed to be easily recognized by human driver s mainly because their color and shapes are very different from natural environments. The algorithm described in this paper takes advantage of these features. I t has two main parts. The first one, for the detection, uses color thresholding to segment the image and shape analysis to detect the signs. The second one, for the classification, uses a neural network. Some results from natural scenes are shown. On the other hand, the algorithm is valid to detect other kinds of marks that would tell the mobile robot to perform some task at that place.

A real-time SLAM (simultaneous localization and mapping) approach to harvester localization and tree map generation in forest environments is presented in this paper. The method combines 2D laser localization and mapping with GPS information to form global tree maps. Building an incremental map while also using it for localization is the only way a mobile robot can navigate in large outdoor environments. Until recently SLAM has only been confined to small-scale, mostly indoor, environments. We try to addresses the issues of scale for practical implementations of SLAM in extensive outdoor environments. Presented algorithms are tested in real outdoor environments using an all-terrain vehicle equipped with the navigation sensors and a DGPS receiver.

Robust scene understanding of outdoor environments using passive optical sensors is a onerous and essential task for autonomous navigation. The problem is heavily characterized by changing environmental conditions throughout the day and across seasons. Robots should be equipped with models that are impervious to these factors in order to be operable and more importantly to ensure safety in the real-world. In this paper, we propose a novel semantic segmentation architecture and the convoluted mixture of deep experts (CMoDE) fusion technique that enables a multi-stream deep neural network to learn features from complementary modalities and spectra, each of which are specialized in a subset of the input space. Our model adaptively weighs class-specific features of expert networks based on the scene condition and further learns fused representations to yield robust segmentation. We present results from experimentation on three publicly available datasets that contain diverse conditions including rain, summer, winter, dusk, fall, night and sunset, and show that our approach exceeds the state-of-the-art. In addition, we evaluate the performance of autonomously traversing several kilometres of a forested environment using only the segmentation for perception.

The process of routing in Vehicular Ad hoc Networks (VANET) is a challenging task in city environments. Finding the shortest end-to-end connected path satisfying delay restriction and minimal overhead is confronted with many constraints and difficulties. Such difficulties are due to the high mobility of vehicles, the frequent path failures, and the various obstructions, which may affect the reliability of the data transmission and routing. Commercial Unmanned Aerial Vehicles (UAVs) or what are commonly referred to as drones can come in handy in dealing with these constraints. In this paper, we study how UAVs operating in ad hoc mode can cooperate with VANET on the ground so as to assist in the routing process and improve the reliability of the data delivery by bridging the communication gap whenever it is possible. In a previous work, we have proposed UVAR – a UAV-Assisted VANETs routing protocol, which improves data routing and connectivity of the vehicles on the ground through the use of UAVs. However, UVAR does not fully exploit UAVs in the sky for data forwarding because it uses UAVs only when the network is poorly dense. In ∗Corresponding author Email addresses: s.oubbati@lagh-univ.dz (Omar Sami Oubbati), alakas@uaeu.ac.ae (Abderrahmane Lakas), fen.zhou@univ-avignon.fr (Fen Zhou), mesut.guenes@ovgu.de (Mesut Güneş), n.lagraa@lagh-univ.dz (Nasreddine Lagraa), m.yagoubi@lagh-univ.dz (Mohamed Bachir Yagoubi) Preprint submitted to Computer Communications April 4, 2017

The use of flying platforms such as unmanned aerial vehicles (UAVs), popularly known as drones, is rapidly growing in a wide range of wireless networking applications. In particular, with their inherent attributes such as mobility, flexibility, and adaptive altitude, UAVs admit several key potential applications in wireless systems. On the one hand, UAVs can be used as aerial base stations to enhance coverage, capacity, reliability, and energy efficiency of wireless networks. For instance, UAVs can be deployed to complement existing cellular systems by providing additional capacity to hotspot areas as well as to provide network coverage in emergency and public safety situations. On the other hand, UAVs can operate as flying mobile terminals within the cellular networks. Such cellular-connected UAVs can enable a wide range of key applications expanding from real-time video streaming to item delivery. Despite the several benefits and practical applications of using UAVs as aerial wireless devices, one must address many technical challenges. In this paper, a comprehensive tutorial on the potential benefits and applications of UAVs in wireless communications is presented. Moreover, the important challenges and the fundamental tradeoffs in UAV-enabled wireless networks are thoroughly investigated. In particular, the key UAV challenges such as three-dimensional (3D) deployment, performance analysis, air-to-ground channel modeling, and energy efficiency are explored along with representative results. Then, fundamental open problems and potential research directions pertaining to wireless communications and networking with UAVs are introduced. To cope with the open research problems, various analytical frameworks and mathematical tools such as optimization theory, machine learning, stochastic geometry, transport theory, and game theory are described. The use of such tools for addressing unique UAV problems is also presented. In a nutshell, this tutorial provides key guidelines on how to analyze, optimize, and design UAV-based wireless communication systems.

During last decade the scientific research on Unmanned Aerial Vehicless (UAVs) increased spectacularly and led to the design of multiple types of aerial platforms. The major challenge today is the development of autonomously operating aerial agents capable of completing missions independently of human interaction. To this extent, visual sensing techniques have been integrated in the control pipeline of the UAVs in order to enhance their navigation and guidance skills. The aim of this article is to present a comprehensive literature review on vision based applications for UAVs focusing mainly on current developments and trends. These applications are sorted in different categories according to the research topics among various research groups. More specifically vision based position-attitude control, pose estimation and mapping, obstacle detection as well as target tracking are the identified components towards autonomous agents. Aerial platforms could reach greater level of autonomy by integrating all these technologies onboard. Additionally, throughout this article the concept of fusion multiple sensors is highlighted, while an overview on the challenges C. Kanellakis ( ) · G. Nikolakopoulos Luleå University of Technology, Luleå, Sweden e-mail: christoforos.kanellakis@ltu.se G. Nikolakopoulos e-mail: geonik@ltu.se addressed and future trends in autonomous agent development will be also provided.

We describe a method of mobile robot monocular visual navigation, which uses multiple visual cues to detect and segment the ground plane in the robot’s field of view. Corner points are tracked through an image sequence and grouped into coplanar regions using a method which we call an H-based tracker. The H-based tracker employs planar homographys and is initialised by 5-point planar projective invariants. This allows us to detect ground plane patches and the colour within such patches is subsequently modelled. These patches are grown by colour classification to give a ground plane segmentation, which is then used as an input to a new variant of the artificial potential field algorithm.


Question: Using convolutional neural networks for texture synthesis.

This research delves into the impact of convolutional network depth on accuracy within the context of large-scale image recognition. Our primary contribution lies in a comprehensive evaluation of networks with increasing depth, demonstrating that a significant enhancement in performance over previous configurations can be achieved by increasing the depth to 16–19 weight layers. This discovery was pivotal in our submission to the ImageNet Challenge 2014, where our team secured the top positions in both the localization and classification categories. Additionally, we illustrate that our representations have excellent generalization capabilities, achieving state-of-the-art outcomes on other datasets. Significantly, we have made our two top-performing ConvNet models accessible to the public, enabling further exploration into the application of deep visual representations within the field of computer vision.

**Question**: A Comparative Study of Different Evolutionary Algorithms for Solving the Traveling Salesman Problem

**Documents**: A comparative study of different evolutionary algorithms for solving the travelling salesman problem
In this paper, a comparative study of different evolutionary algorithms for solving the travelling salesman problem (TSP) is presented. The evolutionary algorithms used are genetic algorithms (GA), evolutionary programming (EP), evolutionary strategies (ES), and differential evolution (DE). The TSP is a well-known combinatorial optimization problem which is NP-hard. The performance of the evolutionary algorithms is compared using a set of benchmark TSP instances. The results show that the DE algorithm performs better than the other evolutionary algorithms for solving the TSP. The results also show that the performance of the evolutionary algorithms depends on the parameters used. The parameters that have the most significant effect on the performance of the evolutionary algorithms are the population size, the mutation rate, and the crossover rate.

**Similar Documents**: This paper undertakes an in-depth comparative analysis of various evolutionary algorithms aimed at addressing the Traveling Salesman Problem (TSP). The evolutionary algorithms scrutinized include Genetic Algorithms (GA), Evolutionary Programming (EP), Evolutionary Strategies (ES), and Differential Evolution (DE). The TSP, being a notorious combinatorial optimization problem, is known for its NP-hard complexity. Through the evaluation of benchmark TSP instances, the performance of these evolutionary algorithms is contrasted. The findings highlight the superiority of the DE algorithm over its counterparts in solving the TSP. Moreover, the analysis reveals that the algorithms' performance is contingent on the specific parameters employed, with population size, mutation rate, and crossover rate exerting the most pronounced impact on efficiency.

Inspired by recent advances in multimodal learning and machine translation, we introduce an encoder-decoder pipeline that learns (a): a multimodal joint embedding space with images and text and (b): a novel language model for decoding distributed representations from our space. Our pipeline effectively unifies joint image-text embedding models with multimodal neural language models. We introduce the structure-content neural language model that disentangles the structure of a sentence to its content, conditioned on representations produced by the encoder. The encoder allows one to rank images and sentences while the decoder can generate novel descriptions from scratch. Using LSTM to encode sentences, we match the state-of-the-art performance on Flickr8K and Flickr30K without using object detections. We also set new best results when using the 19-layer Oxford convolutional network. Furthermore we show that with linear encoders, the learned embedding space captures multimodal regularities in terms of vector space arithmetic e.g. *image of a blue car* "blue" + "red" is near images of red cars. Sample captions generated for 800 images are made available for comparison.

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

Traditionally, convolutional neural networks are trained for semantic segmentation by having an image given as input and the segmented mask as output. In this work, we propose a neural network trained by being given an image and mask pair, with the output being the quality of that pairing. The segmentation is then created afterwards through backpropagation on the mask. This allows enriching training with semi-supervised synthetic variations on the ground-truth. The proposed iterative segmentation technique allows improving an existing segmentation or creating one from scratch. We compare the performance of the proposed methodology with state-of-the-art deep architectures for image segmentation and achieve competitive results, being able to improve their segmentations.

Image-based models for computer graphics lack resolution independence: they cannot be zoomed much beyond the pixel resolution they were sampled at without a degradation of quality. Interpolating images usually results in a blurring of edges and image details. We describe image interpolation algorithms which use a database of training images to create plausible high-frequency details in zoomed images. Image pre-processing steps allow the use of image detail from regions of the training images which may look quite di erent from the image to be processed. These methods preserve ne details, such as edges, generate believable textures, and can give good results even after zooming multiple octaves. This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonpro t educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Information Technology Center America; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Information Technology Center America. All rights reserved. Copyright c Mitsubishi Electric Information Technology Center America, 2001 201 Broadway, Cambridge, Massachusetts 02139 1. First printing, TR2001-30, August, 2001. Egon Pasztor's present address: MIT Media Lab 20 Ames St. Cambridge, MA 02139 Example-based Super-resolution William T. Freeman, Thouis R. Jones, Egon C. Pasztor MitsubishiElectricResearchLaboratories(MERL) 201Broadway Cambridge,MA 02139 (a) (b) Figure 1: An object modelledby traditionalpolygon techniques may lack someof the richnessof real-world objects,but behaves properlyunderzooming,(b).

-This paper presents a texture segmentation algorithm inspired by the multi-channel filtering theory for visual information processing in the early stages of human visual system. The channels are characterized by a bank of Gabor filters that nearly uniformly covers the spatial-frequency domain, and a systematic filter selection scheme is proposed, which is based on reconstruction of the input image from the filtered images. Texture features are obtained by subjecting each (selected) filtered image to a nonlinear transformation and computing a measure of "energy" in a window around each pixel. A square-error clustering algorithm is then used to integrate the feature images and produce a segmentation. A simple procedure to incorporate spatial information in the clustering process is proposed. A relative index is used to estimate the "'true" number of texture categories. Texture segmentation Multi-channel filtering Clustering Clustering index Gabor filters Wavelet transform I . I N T R O D U C T I O N Image segmentation is a difficult yet very important task in many image analysis or computer vision applications. Differences in the mean gray level or in color in small neighborhoods alone are not always sufficient for image segmentation. Rather, one has to rely on differences in the spatial arrangement of gray values of neighboring pixels-that is, on differences in texture. The problem of segmenting an image based on textural cues is referred to as the texture segmentation problem. Texture segmentation involves identifying regions with "uniform" textures in a given image. Appropriate measures of texture are needed in order to decide whether a given region has uniform texture. Sklansky (o has suggested the following definition of texture which is appropriate in the segmentation context: "A region in an image has a constant texture if a set of local statistics or other local properties of the picture are constant, slowly varying, or approximately periodic". Texture, therefore, has both local and global connotations--i t is characterized by invariance of certain local measures or properties over an image region. The diversity of natural and artificial textures makes it impossible to give a universal definition of texture. A large number of techniques for analyzing image texture has been proposed in the past two decades/2,3) In this paper, we focus on a particular approach to texture analysis which is referred to as ° This work was supported in part by the National Science Foundation infrastructure grant CDA-8806599, and by a grant from E. I. Du Pont De Nemours & Company Inc. the multi-channel filtering approach. This approach is inspired by a multi-channel filtering theory for processing visual information in the early stages of the human visual system. First proposed by Campbell and Robson (4) the theory holds that the visual system decomposes the retinal image into a number of filtered images, each of which contains intensity variations over a narrow range of frequency (size) and orientation. The psychophysical experiments that suggested such a decomposition used various grating patterns as stimuli and were based on adaptation techniques. I~l Subsequent psychophysiological experiments provided additional evidence supporting the theory. De Valois et al. ,(5) for example, recorded the response of simple cells in the visual cortex of the Macaque monkey to sinusoidal gratings with different frequencies and orientations. It was observed that each cell responds to a narrow range of frequency and orientation only. Therefore, it appears that there are mechanisms in the visual cortex of mammals that are tuned to combinations of frequency and orientation in a narrow range. These mechanisms are often referred to as channels, and are appropriately interpreted as band-pass filters. The multi-channel filtering approach to texture analysis is intuitively appealing because it allows us to exploit differences in dominant sizes and orientations of different textures. Today, the need for a multi-resolution approach to texture analysis is well recognized. While other approaches to texture analysis have had to be extended to accommodate this paradigm, the multi-channel filtering approach, is inherently multi-resolutional. Another important

Neural networks are a subset of the field of artificial intelligence (AI). The predominant types of neural networks used for multidimensional signal processing are deep convolutional neural networks (CNNs). The term deep refers generically to networks having from a "few" to several dozen or more convolution layers, and deep learning refers to methodologies for training these systems to automatically learn their functional parameters using data representative of a specific problem domain of interest. CNNs are currently being used in a broad spectrum of application areas, all of which share the common objective of being able to automatically learn features from (typically massive) data bases and to generalize their responses to circumstances not encountered during the learning phase. Ultimately, the learned features can be used for tasks such as classifying the types of signals the CNN is expected to process. The purpose of this "Lecture Notes" article is twofold: 1) to introduce the fundamental architecture of CNNs and 2) to illustrate, via a computational example, how CNNs are trained and used in practice to solve a specific class of problems.

A systematic fractal brownian motion approach is proposed for generating coherent noise, aiming at procedurally generating realistic terrain and textures. Two models are tested and compared to Perlin noise method for two-dimensional height map generation. A fractal analysis is performed in order to compare fractal behaviour of generated data to real terrain coastlines from the point of view of fractal dimension. Performance analysis show that one of the described schemes requires half as many primitive operations than Perlin noise while producing data of equivalent quality.

A handwritten digit recognition system was used in a demonstration project to visualize artificial neural networks, in particular Kohonen’s self-organizing feature map. The purpose of this project was to introduce neural networks through a relatively easy-to-understand application to the general public. This paper describes several techniques used for preprocessing the handwritten digits, as well as a number of ways in which neural networks were used for the recognition task. Whereas the main goal was a purely educational one, a moderate recognition rate of 98% was reached on a test set.

This work plans to approach the texture segmentation problem by incorporating genetic algorithm and K-means clustering method within a multiresolution structure. As the algorithm descends the multiresolution structure, the coarse segmentation results are propagated down to the lower levels so as to reduce the inherent class–position uncertainty and to improve the segmentation accuracy. The procedure is described as follows. In the first step, a quad-tree structure of multiple resolutions is constructed. Sampling windows of different sizes are utilized to partition the underlying image into blocks at different resolution levels and texture features are extracted from each block. Based on the texture features, a hybrid genetic algorithm is employed to perform the segmentation. While the select and mutate operators of the traditional genetic algorithm are adopted in this work, the crossover operator is replaced with K-means clustering method. In the final step, the boundaries and the segmentation result of the current resolution level are propagated down to the next level to act as contextual constraints and the initial configuration of the next level, respectively. q 2003 Elsevier B.V. All rights reserved.

The deep generative adversarial networks (GAN) recently have been shown to be promising for different computer vision applications, like image editing, synthesizing high resolution images, generating videos, etc. These networks and the corresponding learning scheme can handle various visual space mappings. We approach GANs with a novel training method and learning objective, to discover multiple object instances for three cases: 1) synthesizing a picture of a specific object within a cluttered scene; 2) localizing different categories in images for weakly supervised object detection; and 3) improving object discovery in object detection pipelines. A crucial advantage of our method is that it learns a new deep similarity metric, to distinguish multiple objects in one image. We demonstrate that the network can act as an encoder-decoder generating parts of an image which contain an object, or as a modified deep CNN to represent images for object detection in supervised and weakly supervised scheme. Our ranking GAN offers a novel way to search through images for object specific patterns. We have conducted experiments for different scenarios and demonstrate the method performance for object synthesizing and weakly supervised object detection and classification using the MS-COCO and PASCAL VOC datasets.