Question: Worldwide suburban routing utilizing OpenStreetMap

This paper introduces a novel method for enriching current maps with detailed segmentation features, including parking spaces, sidewalks, and the count and position of road lanes. To achieve this, we propose an effective technique that performs concurrent inference on both monocular aerial imagery and ground-level images captured by a stereo camera pair mounted on a vehicle. A critical aspect of our method is the accurate alignment of these two types of images, as even when acquired using advanced GPS+IMU systems, the alignment can be insufficient. We validate the efficacy of our approach on a new dataset that augments the KITTI collection with aerial footage gathered by a camera mounted on an aircraft flying over Karlsruhe, Germany. This dual-image analysis allows for a more comprehensive and accurate map enhancement, surpassing the limitations of single-image processing methods. By combining aerial and ground perspectives, our technique provides a holistic view of the urban environment, making it particularly useful for detailed mapping applications such as autonomous driving and urban planning.

Open Shortest Path First (OSPF), a link state routing protocol, is a popular interior gateway protocol (IGP) in the Internet. Wide spread deployment and years of experience running the protocol have motivated continuous improvements in its operation as the nature and demands of the routing infrastructures have changed. Modern routing domains need to maintain a very high level of service availability. Hence, OSPF needs to achieve fast convergence to topology changes. Also, the ever-growing size of routing domains, and possible presence of wireless mobile adhoc network (MANET) components, requires highly scalable operation on part of OSPF to avoid routing instability. Recent years have seen significant efforts aimed at improving OSPF's convergence speed as well as scalability and extending OSPF to achieve seamless integration of mobile adhoc networks with conventional wired networks. In this paper, we present a comprehensive survey of these efforts.

This paper solves what appears to be a 30 years old problem dealing with the discovery of most effi algorithms possible to compute all-to-one shortest paths in discrete dynamic networks. This problem lies at th of efficient solution approaches to dynamic network models that arise in dynamic transportation systems, suc Intelligent Transportation Systems (ITS) applications. While the main objective of this paper is the study of the one dynamic shortest paths problem, one-to-all fastest paths problems are studied as well. Early results are and new properties are established. We establish the exact complexity of these problems and develop optima run time sense, solution algorithms. A new and simple solution algorithm is proposed for all-to-one all departur intervals shortest path problems. It is proved, theoretically, that the new solution algorithm has an optimal run complexity that equals the complexity of the problem. Computer implementations and experimental evaluatio various solution algorithms support the theoretical findings and demonstrate the efficiency of the proposed so algorithm. We expect our findings to be of major benefit to research and development activities in the field of dynamic, in particular real-time, management and control of large-scale Intelligent Transportation Systems (I

The route planning method based on artificial potential field is an important branch and the hot research focus of assignment planning. The traditional artificial potential field method is easy to fall into the local minimum, and the paper analyzed the causes, and then did some research on the improved artificial potential field method, and then proposed a method modifying the repulsion direction based on the improved artificial potential field to solve the three typical local minimum problem. The simulation results show that the proposed method is effective.

Several domain-specific languages (DSLs) for parallel graph analytics have been proposed recently. In this paper, we argue that existing DSLs can be implemented on top of a general-purpose infrastructure that (i) supports very fine-grain tasks, (ii) implements autonomous, speculative execution of these tasks, and (iii) allows application-specific control of task scheduling policies. To support this claim, we describe such an implementation called the Galois system.
 We demonstrate the capabilities of this infrastructure in three ways. First, we implement more sophisticated algorithms for some of the graph analytics problems tackled by previous DSLs and show that end-to-end performance can be improved by orders of magnitude even on power-law graphs, thanks to the better algorithms facilitated by a more general programming model. Second, we show that, even when an algorithm can be expressed in existing DSLs, the implementation of that algorithm in the more general system can be orders of magnitude faster when the input graphs are road networks and similar graphs with high diameter, thanks to more sophisticated scheduling. Third, we implement the APIs of three existing graph DSLs on top of the common infrastructure in a few hundred lines of code and show that even for power-law graphs, the performance of the resulting implementations often exceeds that of the original DSL systems, thanks to the lightweight infrastructure.

In spatial crowdsourcing, requesters submit their task-related locations and increase the demand of a local area. The platform prices these tasks and assigns spatial workers to serve if the prices are accepted by requesters. There exist mature pricing strategies which specialize in tackling the imbalance between supply and demand in a local market. However, in global optimization, the platform should consider the mobility of workers; that is, any single worker can be the potential supply for several areas, while it can only be the true supply of one area when assigned by the platform. The hardness lies in the uncertainty of the true supply of each area, hence the existing pricing strategies do not work. In the paper, we formally define this Global Dynamic Pricing(GDP) problem in spatial crowdsourcing. And since the objective is concerned with how the platform matches the supply to areas, we let the matching algorithm guide us how to price. We propose a MAtching-based Pricing Strategy (MAPS) with guaranteed bound. Extensive experiments conducted on the synthetic and real datasets demonstrate the effectiveness of MAPS.

Crowdsourcing has inspired a variety of novel mobile applications. However, identifying common practices across different applications is still challenging. In this paper, we use smart parking as a case study to investigate features of crowdsourcing that may apply to other mobile applications. Based on this we derive principles for efficiently harnessing crowdsourcing. We draw three key guidelines: First, we suggest that that the organizer can play an important role in coordinating participants', a key factor to successful crowdsourcing experience. Second, we suggest that the expected participation rate is a key factor when designing the crowdsourcing system: a system with a lower expected participation rate will place a higher burden in individual participants (e.g., through more complex interfaces that aim to improve the accuracy of the collected data). Finally, we suggest that not only above certain threshold of contributors, a crowdsourcing-based system is resilient to freeriding but, surprisingly, that including freeriders (i.e., actors that do not participate in system effort but share its benefits in terms of coordination) benefits the entire system.

There exists a number of network measurement tools and web-based services for end users. Almost all of them focus on network bandwidth and possibly latency, and simply report the results to the user. This paper presents Netradar, a somewhat different measurement service. We analyze network bandwidth and latency but also over ten other parameters. We dwell into the data and show various coverage maps and statistics on our web site for the benefit of the end users. Our long term goal is to be able to answer the simple question: why did I get the indicated result at a given time and place?

It is widely believed that identifying communities in an ad hoc mobile communications system, such as a pocket switched network, can reduce the amount of traffic created when forwarding messages, but there has not been any empirical evidence available to support this assumption to date. In this paper, we show through use of real experimental human mobility data, how using a small label, identifying users according to their affiliation, can bring a large improvement in forwarding performance, in term of both delivery ratio and cost.

We present a new major release of the OpenSubtitles collection of parallel corpora. The release is compiled from a large database of movie and TV subtitles and includes a total of 1689 bitexts spanning 2.6 billion sentences across 60 languages. The release also incorporates a number of enhancements in the preprocessing and alignment of the subtitles, such as the automatic correction of OCR errors and the use of meta-data to estimate the quality of each subtitle and score subtitle pairs.

We describe an approach to simultaneous localization and mapping, SLAM. This approach has the highly desirable property of robustness to data association errors. Another important advantage of our algorithm is that non-linearities are computed exactly, so that global constraints can be imposed even if they result in large shifts to the map. We represent the map as a graph and use the graph to find an efficient map update algorithm. We also show how topological consistency can be imposed on the map, such as, closing a loop. The algorithm has been implemented on an outdoor robot and we have experimental validation of our ideas. We also explain how the graph can be simplified leading to linear approximations of sections of the map. This reduction gives us a natural way to connect local map patches into a much larger global map.


Question: Worldwide suburban routing utilizing OpenStreetMap

OpenStreetMap: A Collaborative Platform for User-Generated Geospatial Data
OpenStreetMap (OSM) represents a pioneering knowledge collective that enables users to generate and contribute street maps. Emulating the collaborative model that birthed Wikipedia, OSM's objective is to produce a comprehensive set of map data that is freely accessible, modifiable, and released under innovative copyright arrangements. A substantial community of contributors actively collaborates on the global map through OSM's technological framework. A dedicated core of roughly 40 volunteers invests significant effort into enhancing OSM's infrastructure, encompassing server maintenance, core software development for server interactions, and the creation of cartographic outputs. Additionally, there is a burgeoning developer community focused on creating software utilities that facilitate the integration of OSM data across diverse application domains, software environments, and hardware platforms. The central hub for all OSM activities is the official OSM website.

We present an interactive system for synthesizing urban layouts by example. Our method simultaneously performs both a structure-based synthesis and an image-based synthesis to generate a complete urban layout with a plausible street network and with aerial-view imagery. Our approach uses the structure and image data of real-world urban areas and a synthesis algorithm to provide several high-level operations to easily and interactively generate complex layouts by example. The user can create new urban layouts by a sequence of operations such as join, expand, and blend without being concerned about low-level structural details. Further, the ability to blend example urban layout fragments provides a powerful way to generate new synthetic content. We demonstrate our system by creating urban layouts using example fragments from several real-world cities, each ranging from hundreds to thousands of city blocks and parcels.

Surveying a suite of algorithms that offer a solution to managing large document archives.

Road information is useful in many fields. In this paper, a real-time mapping system is presented to acquire the image and determine the geometry of the road. These include designs of platform and instruments, data transmission, processing and archiving. Compare with the traditional platforms, the UAV (Unmanned Aerial Vehicle) platforms offer greater flexibility, shorter response time and is able to generate very high resolution data. It is inexpensive to operate, and its operation is not limited by air traffic constraints. With autonomous flight control system, it is easy to navigate the aircraft along the planned lines strictly. In this system, an unmanned fixed-wing aircraft with double engine and double generator is used, which helps to improve the reliability and capability of the platform. A digital three-axis stabilized platform is developed, which performs an important role in camera attitude controlling. To ensure there is no coverage hole, a data manage system is adopted: after an instantaneous inspection of data integrity, the data will be archived as they are received. If any anomaly is detected, the mission planning will be adapted to re-image the concerned area. Vehicle information extraction from image sequence is also described in this paper. The experiment showed the presented system worked well in real-time road mapping and vehicle information extraction.

The Automatic Dependent Surveillance - Broadcast (ADS-B) protocol is one of the key components of the next generation air transportation system. Since ADS-B will become mandatory by 2017 in the European airspace, it is crucial that aspects such as its security and privacy are promptly investigated by the research community. However, as expensive specialized equipment was previously necessary to collect real-world data on a large scale, such data has not been freely accessible until now. To enable researchers around the world to conduct experimental studies based on real air traffic data, we have created OpenSky, a participatory sensor network for air traffic research. In this paper, we describe the setup and capabilities of OpenSky, and detail some of the research into air traffic security that we have conducted using OpenSky.

Geofencing mechanisms allow for timely message delivery to the visitors of predefined target areas. However, conventional geofencing approaches poorly support mobile data collection scenarios in which experts need in situ assistance. In this paper, we propose crowd geofencing environments, in which a large number of crowdworkers generate geofences to support mobile experts. As a first step to open up the possibilities of crowd geofencing, we have tested its feasibility by collecting more than one thousand geofences in an unfamiliar city prior to the visit to look into urban water and air quality issues. Our experience has revealed the strengths and weaknesses of crowd geofencing in terms of geofence quality and crowd-powered situated actions.

Roadside Unit (RSU) is an essential unit in a vehicular ad-hoc network (VANET) for collecting and analyzing traffic data given from smart vehicles. Furthermore, RSUs can take part in controlling traffic flow for vehicle's secure driving by broadcasting locally analyzed data, forwarding some important messages, and communicating with other RSUs, and soon. In order to maximize the availability of RSUs in the VANET, RSUs need to be fully distributed over an entire area. Thus, RSUs can make the best use of all traffic data gathered from every intersection. In this paper, we provide intersection-priority based RSU placement methods to find the optimal number and positions of RSUs for the full distribution providing with a maximal connectivity between RSUs while minimizing RSU setup costs. We propose three optimal algorithms: greedy, dynamic and hybrid algorithms. Finally, we provide simulated analyses of our algorithms using real urban roadmaps of JungGu/Jongrogu, YongsanGu, and GangnamGu in Seoul, each of which has a characteristic road style different than each other. We analyze how our algorithms work in such different types of roadways with real traffic data, and find the optimal number and positions of RSUs in these areas.

In this paper a fast review and comparative study of the adhoc routing protocols used in VANET will be presented. The research filed in Vehicular adhoc network (VANET) is developing very fast. An extensive variety of utilizations has been served under various situation (Highway, urban, and Cities). Many protocols have been adopted to serve different topology and scenarios, these protocols faced various challenges. VANETs provide communication between vehicles moving on the roads. The routing protocols in VANET are affected by the vehicle high speed which leads to frequently link breaks between the communicated vehicles, so the adhoc routing protocols are adapted with the VANET characteristics to deliver the data between vehicles in short time. The main goal of VANET is to assemble a data system among vehicles that are moving on the roads, which enables the vehicles to communicate with each other for the safety manners. In this paper, an endeavor has been made to compare six surely understood protocols AODV, DSR, OLSR, DSDV, ZRP and DSDV. Used in VANET from the (packet deliver ratio, end to end delay, throughput, routing algorithm load, received packets, routing packets, dropped packets, ratio of packet loss, Average Jitter) point of view.

The main contribution of our work is in combining the methods for parametric urban design of highly specialized software such as CityEngine and general-purpose parametric modeling platform such as Grasshopper. Our work facilitates and prompts the use of parametric tools by architects and planners for urban design. In this paper we present a custom grasshopper component for street network generation and block subdivision. The component was developed in C# using the RhinoCommon SDK. We used Grasshopper for the development of an urban design proposal at a teaching exercise. To meet the requirements of the urban design project, additional functionalities had to be added to the range of existing Grasshopper components. In particular, we needed components for street network generation and block subdivision. To develop the component we implemented the street expansion strategies described in (Weber et al., 2009) and the methods for block subdivision described in (Vanegas et al., 2009). Additionally, we adapted and enhanced the strategies to meet the NURBS modeling capabilities of Rhinoceros.

Powered by WebGL, some renderers have recently become available for the visualization of point cloud data over the web, for example Plasio or Potree. We have extended Potree to be able to visualize massive point clouds and we have successfully used it with the second national Lidar survey of the Netherlands, AHN2, with 640 billion points. In addition to the visualization, the publicly available service at http://ahn2.pointclouds.nl/ also features a multi-resolution download tool, a geographic name search bar, a measurement toolkit, a 2D orientation map with field of view depiction, a demo mode and the tuning of the visualization parameters. Potree relies on reorganizing the point cloud data into an multi-resolution octree data structure. However, this reorganization is very time consuming for massive data sets. Hence, we have used a divide and conquer approach to decrease the octree creation time. To achieve such performance improvement we divided the entire space into smaller cells, generated an octree for each of them in a distributed manner and then we merged them into a single massive octree. The merging is possible because the extent of all the nodes of the octrees is known and fixed. All the developed tools are free and open-source (FOSS) and they can be used to visualize over the web other massive point clouds.

Open communications over the Internet pose serious threats to countries with repressive regimes, leading them to develop and deploy censorship mechanisms within their networks. Unfortunately, existing censorship circumvention systems do not provide high availability guarantees to their users, as censors can easily identify, hence disrupt, the traffic belonging to these systems using today’s advanced censorship technologies. In this paper, we propose Serving the Web by Exploiting Email Tunnels SWEET, a highly available censorship-resistant infrastructure. SWEET works by encapsulating a censored user’s traffic inside email messages that are carried over public email services like Gmail and Yahoo Mail. As the operation of SWEET is not bound to any specific email provider, we argue that a censor will need to block email communications all together in order to disrupt SWEET, which is unlikely as email constitutes an important part of today’s Internet. Through experiments with a prototype of our system, we find that SWEET’s performance is sufficient for Web browsing. In particular, regular Websites are downloaded within couple of seconds.


Question: Worldwide suburban routing utilizing OpenStreetMap

This paper introduces a groundbreaking method for accurate metric localization, a critical aspect in the field of mobile robotics, that diverges from traditional mapping approaches. Instead of constructing a map using the robot itself, our method leverages geo-tagged panoramas sourced from Google Street View as a reference for global positioning. The localization problem is framed as a two-phase non-linear least squares estimation process. The initial phase involves estimating the three-dimensional positions of feature points from a series of monocular camera images. The subsequent phase calculates the rigid body transformation between the Google Street View panoramas and the previously estimated points. Notably, this method requires only a stream of monocular camera images and odometry estimates as inputs. To assess the accuracy of our approach, we conducted experiments using a robotic platform in a parking lot environment, utilizing visual fiducials for ground truth validation. Furthermore, we extended the application of this method to personal localization in a real urban setting, employing data from a Google Tango tablet. This innovative approach to metric localization using Google Street View panoramas as a global positioning reference offers a novel solution to the challenges faced in mobile robotics and personal navigation scenarios.

In this paper, we present an infrastructure-free mapping and localization framework for rail vehicles using only a lidar sensor. Our method is designed to handle the pathological environment found in modern underground tunnels: narrow, parallel, and relatively smooth concrete walls with very little infrastructure to break up the empty spaces in the tunnel. By using an RQE-based, point-cloud alignment approach, we are able to implement a sliding-window algorithm, used for both mapping and localization. We demonstrate the proposed method with datasets gathered on a subway train travelling at high speeds (up to 70 km/h) in an underground tunnel for a total of 20 km across 6 runs. Our method is capable of mapping the tunnel with less than 0.6% error over the total length of the generated map. It is capable of continuously localizing, relative to the generated map, to within 10 cm in stations and at crossovers, and 1.8 m in pathological sections of tunnel. This method improves rail-based localization in a tunnel, which can be used to increase capacity on existing railways and for automated trains.

Recognizing high-level human behavior and decisions from their digital traces are critical issues in pervasive computing systems. In this paper, we develop a novel methodology to reveal cabdrivers’ operation patterns by analyzing their continuous digital traces. For the first time, we systematically study large scale cabdrivers’ behavior in a real and complex city context through their daily digital traces. We identify a set of valuable features, which are simple and effective to classify cabdrivers, delineate cabdrivers’ operation patterns and compare the different cabdrivers’ behavior. The methodology and steps could spatially and temporally quantify, visualize, and examine different cabdrivers’ operation patterns. Drivers were categorized into top drivers and ordinary drivers by their daily income. We use the daily operations of 3000 cabdrivers in over 48 million of trips and 240 million kilometers to uncover: (1) spatial selection behavior, (2) context-aware spatio-temporal operation behavior, (3) route choice behavior, and (4) operation tactics. Though we focused on cabdriver operation patterns analysis from their digital traces, the methodology is a general empirical and analytical methodology for any GPS-like trace analysis. Our work demonstrates the great potential to utilize the massive pervasive data sets to understand human behavior and high-level intelligence. Published by Elsevier Ltd.

Traffic engineering consists in improving the performance of the teleco-munication networks which is evaluated by a large number of criteria. The ultimate objective is to avoid congestion in the network by keeping its links from being overloaded. In large Ethernet networks with thousands of servers, such as data centers, improving the performance of the traditional switching protocols is a crucial but very challenging task due to an exploration in the size of solution space and the complexity. Thus, exact methods are inappropriate even for reasonable size networks. Local Search (LS) is a powerful method for solving computational optimization problems such as the Vertex Cover, Traveling Salesman, or Boolean Satisfiability. The advantage of LS for these problems is its ability to find an intelligent path from a low quality solution to a high quality one in a huge search space. In this thesis, we propose different approximate methods based on Local Search for solving the class of traffic engineering problems in data center networks that implement Spanning Tree Protocol and Multiple Spanning Tree Protocol. First, we tackle the minimization of the maximal link utilization in the Ethernet networks with one spanning tree. Next, we cope with data center networks containing many spanning trees. We then deal with the minimization of service disruption and the worst-case maximal link utilization in data center networks with many spanning trees. Last, we develop a novel design of multi-objective algorithms for solving the traffic engineering problems in large data centers by taking into account three objectives to be minimized: maximal link utilization, network total load and number of used links. Our schemes reduce significantly the size of the search space by releasing the dependence of the solutions from the link cost computation iii iv in order to obtain an intended spanning tree. We propose efficient in-cremental techniques to speed up the computation of objective values. Furthermore, our approaches show good results on the credible data sets and are evaluated by the strong assessment methods. ACKNOWLEDGEMENTS First of all, I would like to thank Yves Deville and Olivier Bonaventure for the four years they spent helping me. I am very lucky to have two great advisors of two different promoting styles. Yves Deville is a very demanding supervisor who wants every smallest detail to be clear and intuitive. His ideas and his criticisms sometimes made me nervous or even angry but I could not deny the truth …

The simultaneous localisation and map building (SLAM) problem asks if it is possible for an autonomous vehicle to start in an unknown location in an unknown environment and then to incrementally build a map of this environment while simultaneously using this map to compute absolute vehicle location. Starting from the estimation-theoretic foundations of this problem developed in [1], [2], [3], this paper proves that a solution to the SLAM problem is indeed possible. The underlying structure of the SLAM problem is first elucidated. A proof that the estimated map converges monotonically to a relative map with zero uncertainty is then developed. It is then shown that the absolute accuracy of the map and the vehicle location reach a lower bound defined only by the initial vehicle uncertainty. Together, these results show that it is possible for an autonomous vehicle to start in an unknown location in an unknown environment and, using relative observations only, incrementally build a perfect map of the world and to compute simultaneously a bounded estimate of vehicle location. This paper also describes a substantial implementation of the SLAM algorithm on a vehicle operating in an outdoor environment using millimeter-wave (MMW) radar to provide relative map observations. This implementation is used to demonstrate how some key issues such as map management and data association can be handled in a practical environment. The results obtained are cross-compared with absolute locations of the map landmarks obtained by surveying. In conclusion, this paper discusses a number of key issues raised by the solution to the SLAM problem including sub-optimal map-building algorithms and map management.

In this paper we present an approach to enhance existing maps with fine grained segmentation categories such as parking spots and sidewalk, as well as the number and location of road lanes. Towards this goal, we propose an efficient approach that is able to estimate these fine grained categories by doing joint inference over both, monocular aerial imagery, as well as ground images taken from a stereo camera pair mounted on top of a car. Important to this is reasoning about the alignment between the two types of imagery, as even when the measurements are taken with sophisticated GPS+IMU systems, this alignment is not sufficiently accurate. We demonstrate the effectiveness of our approach on a new dataset which enhances KITTI [8] with aerial images taken with a camera mounted on an airplane and flying around the city of Karlsruhe, Germany.

To reduce the amount of CO2 produced from the personal cars in the world, the parking lot guidance systems are considered to be the solution in shopping malls or department stores in many countries. However, most of the current parking lot systems are located in the parking area of each shopping mall and they cannot show the parking lots information for the driver that are driving on the road. So, The drivers can see the parking lot information if and only if they arrive at the shopping mall area. This is the fact that the CO2 are still produced to the world although we use the parking lot guidance system. From the reason as mentioned, we propose the parking lots guidance software to share a parking lot information to a large number of clients in real-time based on MQTT Protocol. The proposed software can share the real-time parking lot information to the mobile devices of the drivers in any location or when they are driving on the road. The results show that the proposed software can share the parking lot information in real-time for at least 1,000 sessions simultaneously with the high average score of usability, design and the benefits.

Open-source software, usually created by volunteer programmers dispersed worldwide, now competes with that developed by software firms. This achievement is particularly impressive as open-source programmers rarely meet. They rely heavily on electronic media, which preclude the benefits of face-to-face contact that programmers enjoy within firms. In this paper, we describe findings that address this paradox based on observation, interviews and quantitative analyses of two open-source projects. The findings suggest that spontaneous work coordinated afterward is effective, rational organizational culture helps achieve agreement among members and communications media moderately support spontaneous work. These findings can imply a new model of dispersed collaboration.

This article briefly describes the advantages and disadvantages of crowdsourcing applications applied to disaster relief coordination. It also discusses several challenges that must be addressed to make crowdsourcing a useful tool that can effectively facilitate the relief progress in coordination, accuracy, and security.

Thanks to both the rapid deployment of the Internet and advances in satellite technology, the market for broadband satellite services is poised for substantial growth in the coming decade. Current communications satellite systems have generally been designed to provide either voice or data transaction (low data rate) services through small terminals, or trunking (high data rate, or broadband) services through large terminals. However, technological advances are enabling new systems that combine broadband data rates with small terminals, thereby providing more affordable " last-mile " network access to home and small business users worldwide. In particular, two types of broadband satellite systems are under development: high-power satellites deployed at traditional geostationary (GEO) orbits, and large constellations of satellites deployed at much lower (LEO) orbits. In this thesis, we explore research problems that have arisen from this shift in satellite network architectures. When using GEO satellites to provide Internet access service, the performance of the Internet's Transmission Control Protocol (TCP) is degraded by the high latency and high degree of bandwidth asymmetry present in such systems. We therefore undertook a comprehensive study of TCP performance in the context of broadband satellite systems used for network access. We first studied whether TCP's congestion avoidance algorithm can be adjusted to provide better fairness when satellite connections are forced to share bottleneck links with other (shorter delay) connections. Our data suggests that adjustments of the policy used in that algorithm may yield substantial fairness benefits without compromizing utilization. We next demonstrated how minor variations in TCP implementations can have drastic performance implications when used over satellite links (such as a reduction in file transfer throughput by over half), and from our observations constructed a satellite-optimized TCP implementation using standardized options. We explored the performance of TCP for short data transfers such as Web traffic, and found that two experimental options relating to how TCP starts a connection, when used together, could reduce the user-perceived latency by a factor of two to three. However, because not all of these options are likely to be deployed on a wide scale, and because even the best satellite-optimized TCP implementation is vulnerable to the fairness problems identified above, we explored the performance benefits of splitting a TCP connection at a protocol gateway within the satellite network, and found that such an approach can allow the performance of the satellite connection to approach that of a non-satellite connection. Carrying this …

It is widely believed that identifying communities in an ad hoc mobile communications system, such as a pocket switched network, can reduce the amount of traffic created when forwarding messages, but there has not been any empirical evidence available to support this assumption to date. In this paper, we show through use of real experimental human mobility data, how using a small label, identifying users according to their affiliation, can bring a large improvement in forwarding performance, in term of both delivery ratio and cost.


Question: Worldwide suburban routing using the OpenStreetMap database.

This paper discusses an advanced navigation system designed for autonomous robots operating in bustling urban and pedestrian zones, areas that have received relatively less attention in comparison to indoor, non-urban outdoor, and road environments. Urban environments, particularly city centers, pose significant challenges for autonomous robots due to their unstructured and dynamic nature. Here, we introduce a comprehensive navigation system tailored for mobile robots navigating crowded cityscapes. This system encompasses various critical components: a Simultaneous Localization and Mapping (SLAM) module adept at handling extensive city center maps; a planning component that not only determines feasible paths but also considers terrain traversability and type; a localization module optimized for accuracy in dynamic environments; and mechanisms for platform calibration and monitoring. The system's effectiveness has been demonstrated through multiple large-scale field trials, where a real-world robot successfully navigated autonomously over several kilometers in a complex urban setting. Notably, the system's capabilities were showcased during a public demonstration, where the robot autonomously traversed a route exceeding three kilometers in length through the heart of Freiburg, Germany. These trials highlight the system's robustness and reliability in challenging urban environments.

Road information is useful in many fields. In this paper, a real-time mapping system is presented to acquire the image and determine the geometry of the road. These include designs of platform and instruments, data transmission, processing and archiving. Compare with the traditional platforms, the UAV (Unmanned Aerial Vehicle) platforms offer greater flexibility, shorter response time and is able to generate very high resolution data. It is inexpensive to operate, and its operation is not limited by air traffic constraints. With autonomous flight control system, it is easy to navigate the aircraft along the planned lines strictly. In this system, an unmanned fixed-wing aircraft with double engine and double generator is used, which helps to improve the reliability and capability of the platform. A digital three-axis stabilized platform is developed, which performs an important role in camera attitude controlling. To ensure there is no coverage hole, a data manage system is adopted: after an instantaneous inspection of data integrity, the data will be archived as they are received. If any anomaly is detected, the mission planning will be adapted to re-image the concerned area. Vehicle information extraction from image sequence is also described in this paper. The experiment showed the presented system worked well in real-time road mapping and vehicle information extraction.

In this paper, a new two-step approach (detecting and pruning) for automatic extraction of road networks from aerial images is presented. The road detection step is based on shape classification of a local homogeneous region around a pixel. The local homogeneous region is enclosed by a polygon, called the footprint of the pixel. This step involves detecting road footprints, tracking roads, and growing a road tree. We use a spoke wheel operator to obtain the road footprint. We propose an automatic road seeding method based on rectangular approximations to road footprints and a toe-finding algorithm to classify footprints for growing a road tree. The road tree pruning step makes use of a Bayes decision model based on the area-to-perimeter ratio (the A/P ratio) of the footprint to prune the paths that leak into the surroundings. We introduce a lognormal distribution to characterize the conditional probability of A/P ratios of the footprints in the road tree and present an automatic method to estimate the parameters that are related to the Bayes decision model. Results are presented for various aerial images. Evaluation of the extracted road networks using representative aerial images shows that the completeness of our road tracker ranges from 84% to 94%, correctness is above 81%, and quality is from 82% to 92%.

It is challenging to map large spatial flow data due to the problem of occlusion and cluttered display, where hundreds of thousands of flows overlap and intersect each other. Existing flow mapping approaches often aggregate flows using predetermined high-level geographic units (e.g. states) or bundling partial flow lines that are close in space, both of which cause a significant loss or distortion of information and may miss major patterns. In this research, we developed a flow clustering method that extracts clusters of similar flows to avoid the cluttering problem, reveal abstracted flow patterns, and meanwhile preserves data resolution as much as possible. Specifically, our method extends the traditional hierarchical clustering method to aggregate and map large flow data. The new method considers both origins and destinations in determining the similarity of two flows, which ensures that a flow cluster represents flows from similar origins to similar destinations and thus minimizes information loss during aggregation. With the spatial index and search algorithm, the new method is scalable to large flow data sets. As a hierarchical method, it generalizes flows to different hierarchical levels and has the potential to support multi-resolution flow mapping. Different distance definitions can be incorporated to adapt to uneven spatial distribution of flows and detect flow clusters of different densities. To assess the quality and fidelity of flow clusters and flow maps, we carry out a case study to analyze a data set of 243,850 taxi trips within an urban area.

Algorithms for Continuous Queries: A Geometric

Surveying a suite of algorithms that offer a solution to managing large document archives.

This paper studies the name lookup issue with longest prefix matching, which is widely used in URL filtering, content routing/switching, etc. Recently Content-Centric Networking (CCN) has been proposed as a clean slate future Internet architecture to naturally fit the contentcentric property of today’s Internet usage: instead of addressing end hosts, the Internet should operate based on the identity/name of contents. A core challenge and enabling technique in implementing CCN is exactly to perform name lookup for packet forwarding at wire speed. In CCN, routing tables can be orders of magnitude larger than current IP routing tables, and content names are much longer and more complex than IP addresses. In pursuit of conquering this challenge, we conduct an implementation-based case study on wire speed name lookup, exploiting GPU’s massive parallel processing power. Extensive experiments demonstrate that our GPU-based name lookup engine can achieve 63.52M searches per second lookup throughput on large-scale name tables containing millions of name entries with a strict constraint of no more than the telecommunication level 100μs per-packet lookup latency. Our solution can be applied to contexts beyond CCN, such as search engines, content filtering, and intrusion prevention/detection. c ⃝Prof. Qunfeng Dong (qunfeng.dong@gmail.com) and Prof. Bin Liu (lmyujie@gmail.com), placed in alphabetic order, are the correspondence authors of the paper. Yi Wang and Yuan Zu, placed in alphabetic order, are the lead student authors of Tsinghua University and University of Science and Technology of China, respectively. This paper is supported by 863 project (2013AA013502), NSFC (61073171, 61073184), Tsinghua University Initiative Scientific Research Program(20121080068), the Specialized Research Fund for the Doctoral Program of Higher Education of China(20100002110051), the Ministry of Education (MOE) Program for New Century Excellent Talents (NCET) in University, the Science and Technological Fund of Anhui Province for Outstanding Youth (10040606Y05), by the Fundamental Research Funds for the Central Universities (WK0110000007, WK0110000019), and Jiangsu Provincial Science Foundation (BK2011360).

Taxi drivers always look for strategies to locate passengers quickly and therefore increase their profit margin. In reality, the passenger seeking strategies are mostly empirical and substantially vary among taxi drivers. From the history taxi data, the top performing taxi drivers can earn 25% more than the ones with mediocre seeking strategy in the same period of time. A better strategy not only helps taxi drivers earn more with less effort, but also reduce fuel consumption and carbon emissions. It is interesting to examine the influential factors in passenger seeking strategies and find algorithms to guide taxi drivers to passenger hotspots with the right timing. With the abundant availability of history taxicab traces, the existing methods of doing taxi business have been radically changed. This paper focuses on the problem of mining efficient operation strategies from a large-scale history taxi traces collected over one year. Our approach presents generic insights into the dynamics of taxicab services with the objective of maximizing the profit margins for the concerned parties. We propose important metrics, such as trip frequency, hot spots, and taxi mileage, and provide valuable insights toward more efficient operation strategies. We analyze these metrics using techniques, such as Newton’s polynomial interpolation and Gamma distribution, to understand their dynamics. Our strategies use the real taxicab traces from the city of Changsha (P.R.China), may predict the taxi rides at different times by 90.68% per day, and increase the taxi drivers income levels up to 19.38% by controlling appropriate mileage per trip and following the route across more urban hot spots.

Navigating information spaces is an essential part of our everyday lives, and in order to design efficient and user-friendly information systems, it is important to understand how humans navigate and find the information they are looking for. We perform a large-scale study of human wayfinding, in which, given a network of links between the concepts of Wikipedia, people play a game of finding a short path from a given start to a given target concept by following hyperlinks. What distinguishes our setup from other studies of human Web-browsing behavior is that in our case people navigate a graph of connections between concepts, and that the exact goal of the navigation is known ahead of time. We study more than 30,000 goal-directed human search paths and identify strategies people use when navigating information spaces. We find that human wayfinding, while mostly very efficient, differs from shortest paths in characteristic ways. Most subjects navigate through high-degree hubs in the early phase, while their search is guided by content features thereafter. We also observe a trade-off between simplicity and efficiency: conceptually simple solutions are more common but tend to be less efficient than more complex ones. Finally, we consider the task of predicting the target a user is trying to reach. We design a model and an efficient learning algorithm. Such predictive models of human wayfinding can be applied in intelligent browsing interfaces.

The culture of sharing instead of ownership is sharply increasing in individuals behaviors. Particularly in transportation, concepts of sharing a ride in either carpooling or ridesharing have been recently adopted. An efficient optimization approach to match passengers in real-time is the core of any ridesharing system. In this paper, we model ridesharing as an online matching problem on general graphs such that passengers do not drive private cars and use shared taxis. We propose an optimization algorithm to solve it. The outlined algorithm calculates the optimal waiting time when a passenger arrives. This leads to a matching with minimal overall overheads while maximizing the number of partnerships. To evaluate the behavior of our algorithm, we used NYC taxi real-life data set. Results represent a substantial reduction in overall overheads.

Powered by WebGL, some renderers have recently become available for the visualization of point cloud data over the web, for example Plasio or Potree. We have extended Potree to be able to visualize massive point clouds and we have successfully used it with the second national Lidar survey of the Netherlands, AHN2, with 640 billion points. In addition to the visualization, the publicly available service at http://ahn2.pointclouds.nl/ also features a multi-resolution download tool, a geographic name search bar, a measurement toolkit, a 2D orientation map with field of view depiction, a demo mode and the tuning of the visualization parameters. Potree relies on reorganizing the point cloud data into an multi-resolution octree data structure. However, this reorganization is very time consuming for massive data sets. Hence, we have used a divide and conquer approach to decrease the octree creation time. To achieve such performance improvement we divided the entire space into smaller cells, generated an octree for each of them in a distributed manner and then we merged them into a single massive octree. The merging is possible because the extent of all the nodes of the octrees is known and fixed. All the developed tools are free and open-source (FOSS) and they can be used to visualize over the web other massive point clouds.


Question: Worldwide suburban routing utilizing OpenStreetMap

This paper introduces efficient algorithms for processing a category of tasks that entail transforming a cost function on a grid by incorporating spatial data. These tasks can be seen as an extension of the conventional distance transforms used in binary images, where the binary image is substituted with a continuous function on a grid. Alternatively, these operations can be understood through the lens of minimum convolution of two functions, a critical operation in grayscale morphology. As a byproduct of our methodologies, we present a straightforward and expedient technique for calculating the Euclidean distance transform of a binary image. The algorithms we propose are also applicable to various other domains, such as Viterbi decoding, belief propagation, and optimal control problems. The classification of our work under ACM is F.2.1 and I.4, while under AMS it falls under 68T45 and 68W40.

Despite the detailed focus on algorithmic advancements in grid-based cost function transformations, the question at hand pertains to global outer-urban navigation using OpenStreetMap. While the algorithms discussed could potentially contribute to the optimization of navigation systems by improving the processing of spatial data, they do not directly address the complexities and challenges of navigating in outer-urban areas with OpenStreetMap. Such navigation challenges may include the integration of real-time traffic data, the handling of large-scale map data, and the adaptation of routing algorithms to diverse and changing urban environments. To effectively address global outer-urban navigation, a comprehensive approach that combines robust spatial data processing with advanced map data management and dynamic routing strategies is required.

The transportation mode such as walking, cycling or on a train denotes an important characteristic of the mobile user's context. In this paper, we propose an approach to inferring a user's mode of transportation based on the GPS sensor on her mobile device and knowledge of the underlying transportation network. The transportation network information considered includes real time bus locations, spatial rail and spatial bus stop information. We identify and derive the relevant features related to transportation network information to improve classification effectiveness. This approach can achieve over 93.5% accuracy for inferring various transportation modes including: car, bus, aboveground train, walking, bike, and stationary. Our approach improves the accuracy of detection by 17% in comparison with the GPS only approach, and 9% in comparison with GPS with GIS models. The proposed approach is the first to distinguish between motorized transportation modes such as bus, car and aboveground train with such high accuracy. Additionally, if a user is travelling by bus, we provide further information about which particular bus the user is riding. Five different inference models including Bayesian Net, Decision Tree, Random Forest, Naïve Bayesian and Multilayer Perceptron, are tested in the experiments. The final classification system is deployed and available to the public.

Recognizing high-level human behavior and decisions from their digital traces are critical issues in pervasive computing systems. In this paper, we develop a novel methodology to reveal cabdrivers’ operation patterns by analyzing their continuous digital traces. For the first time, we systematically study large scale cabdrivers’ behavior in a real and complex city context through their daily digital traces. We identify a set of valuable features, which are simple and effective to classify cabdrivers, delineate cabdrivers’ operation patterns and compare the different cabdrivers’ behavior. The methodology and steps could spatially and temporally quantify, visualize, and examine different cabdrivers’ operation patterns. Drivers were categorized into top drivers and ordinary drivers by their daily income. We use the daily operations of 3000 cabdrivers in over 48 million of trips and 240 million kilometers to uncover: (1) spatial selection behavior, (2) context-aware spatio-temporal operation behavior, (3) route choice behavior, and (4) operation tactics. Though we focused on cabdriver operation patterns analysis from their digital traces, the methodology is a general empirical and analytical methodology for any GPS-like trace analysis. Our work demonstrates the great potential to utilize the massive pervasive data sets to understand human behavior and high-level intelligence. Published by Elsevier Ltd.

Training and simulation applications in virtual worlds require significant amounts of urban environments. Procedural generation is an efficient way to create such models. Existing approaches for procedural modelling of cities aim at facilitating the work of urban planners and artists, but either require expert knowledge or external input data to generate results that resemble real-life cities, or they have long computation times, and thus are unsuitable for non-experts such as training instructors. We propose a method that procedurally creates layouts of structurally plausible cities from high-level, intuitive user input such as city size, location and historic background. The resulting layouts consist of different kinds of city districts which are arranged using constraints derived from established models of urban land use. Our approach avoids the need for external expert engagement in the creation process, and allows for the generation of large city layouts in seconds, making it significantly faster than comparable agent-based software and thus supporting the needs of non-expert creators of virtual cities for many applications.

Resolving location expressions in text to the correct physical location, also known as geocoding or grounding, is complicated by the fact that so many places around the world share the same name. Correct resolution is made even more difficult when there is little context to determine which place is intended, as in a 140-character Twitter message, or when location cues from different sources conflict, as may be the case among different metadata fields of a Twitter message. We used supervised machine learning to weigh the different fields of the Twitter message and the features of a world gazetteer to create a model that will prefer the correct gazetteer candidate to resolve the extracted expression. We evaluated our model using the F1 measure and compared it to similar algorithms. Our method achieved results higher than state-of-the-art competitors.

Large-Scale Scientific Information Systems, Jacobs University, Bremen, Germany; Rasdaman GmbH, Bremen, Germany; CNR-IIA, National Research Council of Italy, Institute of Atmospheric Pollution Research, Florence, Italy; EOX IT Services GmbH, Vienna, Austria; Consorzio COMETA, Catania, Italy; Division of Catania, Italian National Institute for Nuclear Physics, 15 Catania, Italy; Department of Physics and Astronomy, University of Catania, Catania, Italy; MEEO S.r.l., Ferrara, Italy; Plymouth Marine Laboratory, Plymouth, UK; Fraunhofer IGD, Darmstadt, Germany; Athena Research and Innovation Center in Information Communication & Knowledge Technologies, Athens, Greece; British Geological Survey, Edinburgh, UK; Software Engineering Italia S.r.l., Catania, Italy; British Geological Survey, Keyworth, UK; NASA Ames 20 Research Center, Moffett Field, CA, USA

Roadside Unit (RSU) is an essential unit in a vehicular ad-hoc network (VANET) for collecting and analyzing traffic data given from smart vehicles. Furthermore, RSUs can take part in controlling traffic flow for vehicle's secure driving by broadcasting locally analyzed data, forwarding some important messages, and communicating with other RSUs, and soon. In order to maximize the availability of RSUs in the VANET, RSUs need to be fully distributed over an entire area. Thus, RSUs can make the best use of all traffic data gathered from every intersection. In this paper, we provide intersection-priority based RSU placement methods to find the optimal number and positions of RSUs for the full distribution providing with a maximal connectivity between RSUs while minimizing RSU setup costs. We propose three optimal algorithms: greedy, dynamic and hybrid algorithms. Finally, we provide simulated analyses of our algorithms using real urban roadmaps of JungGu/Jongrogu, YongsanGu, and GangnamGu in Seoul, each of which has a characteristic road style different than each other. We analyze how our algorithms work in such different types of roadways with real traffic data, and find the optimal number and positions of RSUs in these areas.

Virtual reality applications prefer real walking to provide highly immersive presence than other locomotive methods. Mapping-based techniques are very effective for supporting real walking in small physical workspaces while exploring large virtual scenes. However, the existing methods for computing real walking maps suffer from poor quality due to distortion. In this paper, we present a novel divide-and-conquer method, called Smooth Assembly Mapping (SAM), to compute real walking maps with low isometric distortion for large-scale virtual scenes. First, the input virtual scene is decomposed into a set of smaller local patches. Then, a group of local patches is mapped together into a real workspace by minimizing a low isometric distortion energy with smoothness constraints between the adjacent patches. All local patches are mapped and assembled one by one to obtain a complete map. Finally, a global optimization is adopted to further reduce the distortion throughout the entire map. Our method easily handles teleportation technique by computing maps of individual regions and assembling them with teleporter conformity constraints. A large number of experiments, including formative user studies and comparisons, have shown that our method succeeds in generating high-quality real walking maps from large-scale virtual scenes to small real workspaces and is demonstrably superior to state-of-the-art methods.

In the past several years, social media (e.g., Twitter and Facebook) has been experiencing a spectacular rise and popularity, and becoming a ubiquitous discourse for content sharing and social networking. With the widespread of mobile devices and location-based services, social media typically allows users to share whereabouts of daily activities (e.g., check-ins and taking photos), and thus strengthens the roles of social media as a proxy to understand human behaviors and complex social dynamics in geographic spaces. Unlike conventional spatiotemporal data, this new modality of data is dynamic, massive, and typically represented in stream of unstructured media (e.g., texts and photos), which pose fundamental representation, modeling and computational challenges to conventional spatiotemporal analysis and geographic information science. In this paper, we describe a scalable computational framework to harness massive location-based social media data for efficient and systematic spatiotemporal data analysis. Within this framework, the concept of space-time trajectories (or paths) is applied to represent activity profiles of social media users. A hierarchical spatiotemporal data model, namely a spatiotemporal data cube model, is developed based on collections of space-time trajectories to represent the collective dynamics of social media users across aggregation boundaries at multiple spatiotemporal scales. The framework is implemented based upon a public data stream of Twitter feeds posted on the continent of North America. To demonstrate the advantages and performance of this framework, an interactive flow mapping interface (including both single-source and multiple-source flow mapping) is developed to allow real-time, and interactive visual exploration of movement dynamics in massive location-based social media at multiple scales. ∗Corresponding author Preprint submitted to Computers, Enviroment and Urban Systems September 10, 2014 ar X iv :1 40 9. 28 26 v1 [ cs .S I] 8 S ep 2 01 4

Updating road markings is one of the routine tasks of transportation agencies. Compared with traditional road inventory mapping techniques, vehicle-borne mobile light detection and ranging (LiDAR) systems can undertake the job safely and efficiently. However, current hurdles include software and computing challenges when handling huge volumes of highly dense and irregularly distributed 3-D mobile LiDAR point clouds. This paper presents the development and implementation aspects of an automated object extraction strategy for rapid and accurate road marking inventory. The proposed road marking extraction method is based on 2-D georeferenced feature (GRF) images, which are interpolated from 3-D road surface points through a modified inverse distance weighted (IDW) interpolation. Weighted neighboring difference histogram (WNDH)-based dynamic thresholding and multiscale tensor voting (MSTV) are proposed to segment and extract road markings from the noisy corrupted GRF images. The results obtained using 3-D point clouds acquired by a RIEGL VMX-450 mobile LiDAR system in a subtropical urban environment are encouraging.

For Open Science to be widely adopted, a strong institutional support for scientists will be essential. Bielefeld University and the associated Center of Excellence Cognitive Interaction Technology (CITEC) have developed a platform that enables researchers to manage their publications and the underlying research data in an easy and efficient way. Following a Linked Data approach we integrate this data into a unified linked data store and interlink it with additional data sources from inside the university and outside sources like DBpedia. Based on the existing platform, a concrete case study from the domain of biology is implemented that releases optical motion tracking data of stick insect locomotion. We investigate the cost and usefulness of such a detailed, domain-specific semantic enrichment in order to evaluate whether this approach might be considered for large-scale deployment.


Question: RBFOpt: an open-source library for handling black-box optimization problems with expensive function evaluations.

This paper revisits direct search methods, a class of optimization techniques that gained prominence in the 1960s for their ability to tackle problems without relying on derivative information. Despite facing criticism and decline in popularity among mathematical optimization experts by the early 1970s due to a lack of rigorous analysis, direct search methods maintained a strong user base, particularly due to their ease of implementation and reliability in certain scenarios. The resurgence of interest in these methods over the past two decades can be attributed to several factors, including the development of mathematical frameworks that support their convergence properties, as well as the growing demand for algorithms that can be effectively parallelized or distributed. This review starts by outlining the historical context of direct search methods and identifies the types of problems for which they are particularly effective. The core of the paper is dedicated to presenting a comprehensive framework that unifies various direct search methods, facilitating a more in-depth understanding of their convergence characteristics. This unified approach also allows for the adaptation of these methods to handle bound constraints and linear constraints, thereby broadening their applicability. Additionally, the paper explores the potential for extending these methods to cope with nonlinear constraints, further enhancing their versatility in the optimization landscape.

In the context of black-box optimization where function evaluations are expensive, the paper introduces RBFOpt, an open-source library designed to address such challenges. RBFOpt is specifically tailored to optimize problems where the cost of evaluating the objective function is high, making it a valuable tool in scenarios where traditional optimization methods become impractical due to computational or time constraints. The library employs a combination of modeling techniques and search strategies to efficiently explore the solution space, aiming to minimize the number of function evaluations required to find a satisfactory solution. This is particularly relevant in fields such as engineering design, where simulations can be computationally intensive, or in financial modeling, where predictive models might involve complex calculations. RBFOpt's design philosophy emphasizes flexibility and ease of use, making it accessible to a wide range of users, from academic researchers to industry practitioners. By providing a robust and adaptable framework, RBFOpt contributes to the advancement of optimization techniques for costly function evaluations, facilitating more efficient and effective problem-solving in various domains.

The package REGULARIZATION TOOLS consists of 54 Matlab routines for analysis and solution of discrete ill-posed problems, i.e., systems of linear equations whose coefficient matrix has the properties that its condition number is very large, and its singular values decay gradually to zero. Such problems typically arise in connection with discretization of Fredholm integral equations of the first kind, and similar ill-posed problems. Some form of regularization is always required in order to compute a stabilized solution to discrete ill-posed problems. The purpose of REGULARIZATION TOOLS is to provide the user with easy-to-use routines, based on numerical robust and efficient algorithms, for doing experiments with regularization of discrete ill-posed problems. By means of this package, the user can experiment with different regularization strategies, compare them, and draw conclusions from these experiments that would otherwise require a major programming effert. For discrete ill-posed problems, which are indeed difficult to treat numerically, such an approach is certainly superior to a single black-box routine. This paper describes the underlying theory gives an overview of the package; a complete manual is also available.

Design and Optimization of OpenFOAM-based CFD Applications for Modern Hybrid and Heterogeneous HPC Platforms Amani AlOnazi The progress of high performance computing platforms is dramatic, and most of the simulations carried out on these platforms result in improvements on one level, yet expose shortcomings of current CFD packages capabilities. Therefore, hardwareaware design and optimizations are crucial towards exploiting modern computing resources. This thesis proposes optimizations aimed at accelerating numerical simulations, which are illustrated in OpenFOAM solvers. A hybrid MPI and GPGPU parallel conjugate gradient linear solver has been designed and implemented to solve the sparse linear algebraic kernel that derives from two CFD solver: icoFoam, which is an incompressible flow solver, and laplacianFoam, which solves the Poisson equation, for e.g., thermal diffusion. A load-balancing step is applied using heterogeneous decomposition, which decomposes the computations taking into account the performance of each computing device and seeking to minimize communication. In addition, we implemented the recently developed pipeline conjugate gradient as an algorithmic improvement, and parallelized it using MPI, GPGPU, and a hybrid technique. While many questions of ultimately attainable per node performance and multi-node scaling remain, the experimental results show that the hybrid implementation of both solvers significantly outperforms state-of-the-art implementations of a widely used open source package.

We apply Stochastic Meta-Descent (SMD), a stochastic gradient optimization method with gain vector adaptation, to the training of Conditional Random Fields (CRFs). On several large data sets, the resulting optimizer converges to the same quality of solution over an order of magnitude faster than limited-memory BFGS, the leading method reported to date. We report results for both exact and inexact inference techniques.

Various meta-heuristic optimization approaches have been recently created and applied in different areas. Many of these approaches are inspired by swarm behaviors in the nature. This paper studies the solving optimization problems using Black Hole Algorithm (BHA) which is a population-based algorithm. Since the performance of this algorithm was not tested in mathematical functions, we have studied this issue using some standard functions. The results of the BHA are compared with the results of GA and PSO algorithms which indicate that the performance of BHA is better than the other two mentioned algorithms.

Python is a powerful programming language for handling engineering and scientific computational tasks efficiently [1]-[5]. Used by companies such as Google and Intel and by organizations including NASA and Los Alamos National Laboratory, it offers an extremely wide selection of tools for tasks such as scientific computing, signal processing, Web site construction, database programming, and graphical user interface (GUI) design. The language is platform independent with most programs running on Linux, Microsoft Windows, or MAC OS virtually unchanged. Python has been distributed as a part of Open Source Initiative, and most versions are General Public License (GPL) compatible. In microwave and radio frequency (RF) engineering, it can be used for numerical programming, automated RF testing, automated monolithic microwave integrated-circuit (MMIC) layout generation, automated netlist generation and simulator sequencing, and other tasks. In this article, we are going to provide examples demonstrating the application areas of Python in microwave and RF engineering. We are also going to give a brief introduction to the language highlighting its salient features.

In this contribution, calculation machinery and code to compute different performance indices, which are necessary for the parametric optimization, are proposed. Moreover, a short review about parametric optimization is carried out. It is highlighted that normally only the ISE (Integral Square Error) performance index is used for control system design. One reason for that is the existence of several numerical algorithms to compute it along the infinite trajectory at once. Finally, an optimal design of a PI controller is used to illustrate and compare the new algorithms.

We introduce a new problem of continuous, coverage-aware trajectory optimization under localization and sensing uncertainty. In this problem, the goal is to plan a path from a start state to a goal state that maximizes the coverage of a user-specified region while minimizing the control costs of the robot and the probability of collision with the environment. We present a principled method for quantifying the coverage sensing uncertainty of the robot. We use this sensing uncertainty along with the uncertainty in robot localization to develop C-OPT, a coverage-optimization algorithm which optimizes trajectories over belief-space to find locally optimal coverage paths. We highlight the applicability of our approach in multiple simulated scenarios inspired by surveillance, UAV crop analysis, and search-and-rescue tasks. We also present a case study on a physical, differential-drive robot. We also provide quantitative and qualitative analysis of the paths generated by our approach.

Hessian-Free optimization (HF) is an approach for unconstrained minimization of real-valued smooth objective functions. Like standard Newton’s method, it uses local quadratic approximations to generate update proposals. It belongs to the broad class of approximate Newton methods that are practical for problems of very high dimensionality, such as the training objectives of large neural networks. Different algorithms that use many of the same key principles have appeared in the literatures of various communities under different names such as Newton-CG, CG-Steihaug, Newton-Lanczos, and Truncated Newton [27, 28], but applications to machine learning and especially neural networks, have been limited or non-existent until recently. With the work of Martens [22] and later Martens and Sutskever [23] it has been demonstrated that such an approach, if carefully designed and implemented, can work very well for optimizing nonconvex functions such as the training objective for deep neural networks and recurrent neural networks (RNNs), given sensible random initializations. This was significant because gradient descent methods have been observed to be very slow and sometimes completely ineffective [16, 4, 17] on these problems, unless special non-random initializations schemes like layer-wise pre-training [16, 15, 3] are used. HF, which is a general optimization-based approach, can be used in conjunction with or as an alternative to existing pre-training methods and is more widely applicable, since it relies on fewer assumptions about the specific structure of the network. In this report we will first describe the basic HF approach, and then examine well-known general purpose performance-improving techniques as well as others that are specific to HF (versus other Truncated-Newton type approaches) or to neural networks. We will also provide practical tips for creating efficient and correct implementations, and discuss the pitfalls which may arise when designing and using an HF-based approach in a particular application.

In this paper, we present an empirical comparison of some Differential Evolution variants to solve global optimization problems. The aim is to identify which one of them is more suitable to solve an optimization problem, depending on the problem's features and also to identify the variant with the best performance, regardless of the features of the problem to be solved. Eight variants were implemented and tested on 13 benchmark problems taken from the specialized literature. These variants vary in the type of recombination operator used and also in the way in which the mutation is computed. A set of statistical tests were performed in order to obtain more confidence on the validity of the results and to reinforce our discussion. The main aim is that this study can help both researchers and practitioners interested in using differential evolution as a global optimizer, since we expect that our conclusions can provide some insights regarding the advantages or limitations of each of the variants studied.

This talk presents a new method for optimizing and automating component sizing in CMOS analog circuits. It is shown that a wide variety of circuit performance measures have a special form, i.e., they are posynomial functions of the design variables. As a result, circuit design problems can be posed as geometric programs, a special type of convex optimization problem for which very efficient global optimization methods exist. The synthesis method is therefore fast, and determines the globally optimal design; in particular, the final solution is completely independent of the starting point, and infeasible specifications are unambiguously detected. Also, because the method is highly efficient, in practice it can be used to carry out robust designs and quickly explore the design space. We show that despite the restricted form of geometric programs, a large variety of circuit design problems can be posed as geometric programs.


Question: RBFOpt: an open-source library for handling black-box optimization with expensive function evaluations.

In recent years, the imperative to address environmental challenges has gained significant traction, leading to an urgent need for sustainable solutions for nuclear waste management. The disposal of nuclear waste is intricately linked to the radioactivity level and the half-life of the isotopes involved. After extensive research, notably spearheaded by ANDRA, deep geological disposal has emerged as the leading strategy for the containment of high-activity and long-lived radioactive waste. To ensure the efficacy of this approach, simulations are indispensable for assessing the potential migration of radioactive elements within the subterranean environment. These simulations aim to quantify the potential environmental impact resulting from the movement of radioelements. To accurately model fluid flow in the complex, porous geological structures surrounding the waste repository, it is essential to determine the pertinent physical properties of these strata. Parameters such as porosity and permeability are not easily quantifiable through direct measurement techniques, necessitating the formulation and resolution of an inverse problem. This problem-solving approach enables the estimation of these crucial characteristics, thereby facilitating a more comprehensive understanding of the subsurface environment and the behavior of radioactive materials within it.

Reinforcement Learning (RL) algorithms allow artificial agents to improve their selection of actions so as to increase rewarding experiences in their environments. The learning can become intractably slow as the state space of the environment grows. This has motivated methods that learn internal representations of the agents state by a function approximator. Impressive results have been produced by using deep artificial neural networks to learn the state representations. However, deep reinforcement learning algorithms require solving a non-convex and nonlinear unconstrained optimization problem. Methods for solving the optimization problems in deep RL are restricted to the class of first-order algorithms, like stochastic gradient descent (SGD). The major drawback of the SGD methods is that they have the undesirable effect of not escaping saddle points. Furthermore, these methods require exhaustive trial and error to fine-tune many learning parameters. Using second derivative information can result in improved convergence properties, but computing the Hessian matrix for large-scale problems is not practical. QuasiNewton methods, like SGD, require only first-order gradient information, but they can result in superlinear convergence, which makes them attractive alternatives. The limitedmemory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) approach is one of the most popular quasi-Newton methods that construct positive definite Hessian approximations. In this paper, we introduce an efficient optimization method, based on the limited memory BFGS quasi-Newton method using line search strategy – as an alternative to SGD methods. Our method bridges the disparity between first order methods and second order methods by continuing to use gradient information to calculate a low-rank Hessian approximations. We provide empirical results on variety of the classic ATARI 2600 games. Our results show a robust convergence with preferred generalization characteristics, as well as fast training time and no need for the experience replaying mechanism.

This paper presents Open Accelerator Research Compiler (OpenARC): an open-source framework that supports the full feature set of OpenACC V1.0 and performs source-to-source transformations, targeting heterogeneous devices, such as NVIDIA GPUs. Combined with its high-level, extensible Intermediate Representation (IR) and rich semantic annotations, OpenARC serves as a powerful research vehicle for prototyping optimization, source-to-source transformations, and instrumentation for debugging, performance analysis, and autotuning. In fact, OpenARC is equipped with various capabilities for advanced analyses and transformations, as well as built-in performance and debugging tools. We explain the overall design and implementation of OpenARC, and we present key analysis techniques necessary to efficiently port OpenACC applications. Porting various OpenACC applications to CUDA GPUs using OpenARC demonstrates that OpenARC performs similarly to a commercial compiler, while serving as a general research framework.

In the Bacteria Foraging Optimization Algorithm (BFAO), the chemotactic process is randomly set, imposing that the bacteria swarm together and keep a safe distance from each other. In hybrid bacteria foraging optimization algorithm and particle swarm optimization (hBFOA-PSO) algorithm the principle of swarming is introduced in the framework of BFAO. The hBFOA-PSO algorithm is based on the adjustment of each bacterium position according to the neighborhood environment. In this paper, the effectiveness of the hBFOA-PSO algorithm has been tested for Automatic Generation Control (AGC) of an interconnected power system. A widely used linear model of two area non-reheat thermal system equipped with Proportional-Integral (PI) controller is considered initially for the design and analysis purpose. At first, a conventional Integral Time multiply Absolute Error (ITAE) based objective function is considered and the performance of hBFOA-PSO algorithm is compared with PSO, BFOA and GA. Further a modified objective function using ITAE, damping ratio of dominant eigenvalues and settling time with appropriate weight coefficients is proposed to increase the performance of the controller. Further, robustness analysis is carried out by varying the operating load condition and time constants of speed governor, turbine, tieline power in the range of +50% to -50% as well as size and position of step load perturbation to demonstrate the robustness of the proposed hBFOA-PSO optimized PI controller. The proposed approach is also extended to a nonlinear power system model by considering the effect of governor dead band non-linearity and the superiority of the proposed approach is shown by comparing the results of Craziness based Particle Swarm Optimization (CRAZYPSO) approach for the identical interconnected power system. Finally, the study is extended to a three area system considering both thermal and hydro units with different PI coefficients and comparison between ANFIS and proposed approach has been provided.

We introduce EvalAI, an open source platform for evaluating and comparing machine learning (ML) and artificial intelligence algorithms (AI) at scale. EvalAI is built to provide a scalable solution to the research community to fulfill the critical need of evaluating machine learning models and agents acting in an environment against annotations or with a human-in-the-loop. This will help researchers, students, and data scientists to create, collaborate, and participate in AI challenges organized around the globe. By simplifying and standardizing the process of benchmarking these models, EvalAI seeks to lower the barrier to entry for participating in the global scientific effort to push the frontiers of machine learning and artificial intelligence, thereby increasing the rate of measurable progress in this domain. Our code is available here.

We describe in this paper a new approach to parallelize branchand-bound on a certain number of processors. We propose to split the optimization of the original problem into the optimization of several subproblems that can be optimized separately with the goal that the amount of work that each processor carries out is balanced between the processors, while achieving interesting speedups. The main innovation of our approach consists in the use of machine learning to create a function able to estimate the difficulty (number of nodes) of a subproblem of the original problem. We also present a set of features that we developed in order to characterize the encountered subproblems. These features are used as input of the function learned with machine learning in order to estimate the difficulty of a subproblem. The estimates of the numbers of nodes are then used to decide how to partition the original optimization tree into a given number of subproblems, and to decide how to distribute them among the available processors. The experiments that we carry out show that our approach succeeds in balancing the amount of work between the processors, and that interesting speedups can be achieved with little effort.

This paper puts forward two useful methods for self-adaptation of the mutation distribution - the concepts of derandomization and cumulation. Principle shortcomings of the concept of mutative strategy parameter control and two levels of derandomization are reviewed. Basic demands on the self-adaptation of arbitrary (normal) mutation distributions are developed. Applying arbitrary, normal mutation distributions is equiv-alent to applying a general, linear problem encoding. The underlying objective of mutative strategy parameter control is roughly to favor previously selected mutation steps in the future. If this objective is pursued rigor-ously, a completely derandomized self-adaptation scheme results, which adapts arbitrary normal mutation distributions. This scheme, called covariance matrix adaptation (CMA), meets the previously stated demands. It can still be considerably improved by cumulation - utilizing an evolution path rather than single search steps. Simulations on various test functions reveal local and global search properties of the evolution strategy with and without covariance matrix adaptation. Their performances are comparable only on perfectly scaled functions. On badly scaled, non-separable functions usually a speed up factor of several orders of magnitude is ob-served. On moderately mis-scaled functions a speed up factor of three to ten can be expected.

Recent advances in multi-core and many-core processors requires programmers to exploit an increasing amount of parallelism from their applications. Data parallel languages such as CUDA and OpenCL make it possible to take advantage of such processors, but still require a large amount of effort from programmers.
 A number of parallelizing source-to-source compilers have recently been developed to ease programming of multi-core and many-core processors. This work presents and evaluates a number of such tools, focused in particular on C-to-CUDA transformations targeting GPUs. We compare these tools both qualitatively and quantitatively to each other and identify their strengths and weaknesses.
 In this paper, we address the weaknesses by presenting a new classification of algorithms. This classification is used in a new source-to-source compiler, which is based on the algorithmic skeletons technique. The compiler generates target code based on skeletons of parallel structures, which can be seen as parameterisable library implementations for a set of algorithm classes. We furthermore demonstrate that the presented compiler requires little modifications to the original sequential source code, generates readable code for further fine-tuning, and delivers superior performance compared to other tools for a set of 8 image processing kernels.

Two types of low cost-per-iteration gradient descent metho ds have been extensively studied in parallel. One is online or stochastic gradient descent ( OGD/SG ), and the other is randomzied coordinate descent (RBCD). In this paper, we combine the two types o f methods together and propose online randomized block coordinate descent (ORBCD). At each itera t on, ORBCD only computes the partial gradient of one block coordinate of one mini-batch samples. ORBCD is well suited for the composite minimization problem where one function is the average of th e losses of a large number of samples and the other is a simple regularizer defined on high dimensional variables. We show that the iteration complexity of ORBCD has the same order as OGD or SGD. For strongly convex functions, by reducing the variance of stochastic gradients, we show that ORBCD can con verge at a geometric rate in expectation, matching the convergence rate of SGD with variance reductio n and RBCD.

We introduce a new theoretical derivation, evaluation methods, and extensive empirical analysis for an automatic query expansion framework in which model estimation is cast as a robust constrained optimization problem. This framework provides a powerful method for modeling and solving complex expansion problems, by allowing multiple sources of domain knowledge or evidence to be encoded as simultaneous optimization constraints. Our robust optimization approach provides a clean theoretical way to model not only expansion benefit, but also expansion risk, by optimizing over uncertainty sets for the data. In addition, we introduce risk-reward curves to visualize expansion algorithm performance and analyze parameter sensitivity. We show that a robust approach significantly reduces the number and magnitude of expansion failures for a strong baseline algorithm, with no loss in average gain. Our approach is implemented as a highly efficient post-processing step that assumes little about the baseline expansion method used as input, making it easy to apply to existing expansion methods. We provide analysis showing that this approach is a natural and effective way to do selective expansion, automatically reducing or avoiding expansion in risky scenarios, and successfully attenuating noise in poor baseline methods.

OF THE THESIS Building Simple Annotation Tools