Question: Detecting parking spaces using a radar-generated target list.

This paper introduces a novel methodology for the instantaneous estimation of a vehicle's lateral velocity using high-resolution image radar data. Traditionally, the estimation of velocity and direction of movement for extended objects requires a tracking system to decipher the velocity vector from the radial velocity measurements provided by radar sensors. This process is time-consuming, often requiring multiple frames to achieve a stable estimate, which can be detrimental in scenarios involving cross-traffic. The proposed approach offers a robust, model-free solution that circumvents the need for time and space data association, characteristic of methods like the Kalman filter. Consequently, this method enables a nearly instantaneous (approximately 50 milliseconds) and unbiased estimation of the velocity vector for an extended target. The approach is designed to effectively manage noise and systematic variations within the radar signal, such as those caused by micro-Doppler effects from rotating wheels. Moreover, it is optimized to mitigate the impact of measurement errors not only in radial velocity but also in azimuth position, significantly enhancing the accuracy of the velocity vector estimation. The method's efficacy is further bolstered through the integration of data from multiple radar sensors, facilitating a more precise and reliable velocity estimation.

This paper addresses automatic interpretation of images of outdoor scenes. The method allows instances of objects from a number of generic classes to be identi ed: vegetation, buildings, vehicles, roads, etc., thereby enabling image databases to be queried on scene content. The feature set is based, in part, on psychophysical principles and includes measures of colour, texture and shape. Using a large database of ground-truth labelled images, a neural network is trained as a pattern classi er. The method is demonstrated on a large test set to provide highly accurate image interpretations, with over 90% of the image area labelled correctly. Scene Understanding Object Recognition Classi cation Image Segmentation Neural Networks

Despite the availability of lane departure and lane keeping systems for highway assistance, unmarked and winding rural roads still pose challenges to lane recognition systems. To detect an upcoming curve as soon as possible, the viewing range of image-based lane recognition systems has to be extended. This is done by evaluating 3D information obtained from stereo vision or imaging radar in this paper. Both sensors deliver evidence grids as the basis for road course estimation. Besides known Kalman Filter approaches, Particle Filters have recently gained interest since they offer the possibility to employ cues of a road, which can not be described as measurements needed for a Kalman Filter approach. We propose to combine both principles and their benefits in a Kalman Particle Filter. The comparison between the results gained from this recently published filter scheme and the classical approaches using real world data proves the advantages of the Kalman Particle Filter.

Autonomous video surveillance and monitoring has a rich history. Many deployed systems are able to reliably track human motion in indoor and controlled outdoor environments. However, object detection and tracking at night remain very important problems for visual surveillance. The objects are often distant, small and their signatures have low contrast against the background. Traditional methods based on the analysis of the difference between successive frames and a background frame will do not work. In this paper, a novel real time object detection algorithm is proposed for night-time visual surveillance. The algorithm is based on contrast analysis. In the first stage, the contrast in local change over time is used to detect potential moving objects. Then motion prediction and spatial nearest neighbor data association are used to suppress false alarms. Experiments on real scenes show that the algorithm is effective for night-time object detection and tracking. 2007 Published by Elsevier Ltd on behalf of Pattern Recognition Society.

Ship detection is an important application of global monitoring of environment and security. In order to overcome the limitations by other systems, surveillance with satellite synthetic aperture radar (SAR) is used because of its possibility to provide ship detection at high resolution over wide swaths and in all weather conditions. A new X-band radar onboard the TerraSAR-X (TS-X) satellite gives access to spatial resolution as fine as 1 m. In this paper, first results on the combined use of TS-X ship detection, automatic identification system (AIS), and satellite AIS (SatAIS) is presented. The AIS system is an effective terrestrial method for tracking vessels in real time typically up to 40 km off the coast. SatAIS, as a space-based system, allows almost global coverage for monitoring of ships since not all ships operate their AIS and smaller ships are not equipped with AIS. The system is considered to be of cooperative nature. In this paper, the quality of TS-X images with respect to ship detection is evaluated, and a first assessment of its performance for ship detection is given. The velocity of a moving ship is estimated using complex TS-X data. As test cases, images were acquired over the North Sea, Baltic Sea, Atlantic Ocean, and Pacific Ocean in Stripmap mode with a resolution of 3 m at a coverage of 30 km 100 km. Simultaneous information on ship positions was available from TS-X and terrestrial as well as SatAIS. First results on the simultaneous superposition of SatAIS and high-resolution radar images are presented.

Short-range continuous-wave Doppler radar sensors have been mainly used for noncontact detection of various motions. In this paper, we investigate the feasibility to implement the function of a remote mouse, an input device of a computer, by recognizing human gestures based on a dual-channel Doppler radar sensor. Direct conversion architecture, symmetric subcarrier modulation, and bandpass sampling techniques are used to obtain a cost-effective solution. An arcsine algorithm and a motion imaging algorithm are proposed to linearly reconstruct the hand and finger motions in a 2-D plane from the demodulated Doppler phase shifts. Experimental results verified the effectiveness of the proposed architecture and algorithms. Different from the frequency-domain “micro-Doppler” approach, the proposed remote gesture recognition based on linear motion reconstruction is able to recognize definitive signatures for the corresponding motions, exhibiting promising potential in practical applications of human-computer interaction.

This paper presents the design and first test on a simulator of a vehicle trajectory-planning algorithm that adapts to traffic on a lane-structured infrastructure such as highways. The proposed algorithm is designed to run on a fail-safe embedded environment with low computational power, such as an engine control unit, to be implementable in commercial vehicles of the near future. The target platform has a clock frequency of less than 150 MHz, 150 kB RAM of memory, and a 3-MB program memory. The trajectory planning is performed by a two-step algorithm. The first step defines the feasible maneuvers with respect to the environment, aiming at minimizing the risk of a collision. The output of this step is a target group of maneuvers in the longitudinal direction (accelerating or decelerating), in the lateral direction (changing lanes), and in the combination of both directions. The second step is a more detailed evaluation of several possible trajectories within these maneuvers. The trajectories are optimized to additional performance indicators such as travel time, traffic rules, consumption, and comfort. The output of this module is a trajectory in the vehicle frame that represents the recommended vehicle state (position, heading, speed, and acceleration) for the following seconds.

Developing on-board automotive driver assistance systems aiming to alert drivers about driving environments, and possible collision with other vehicles has attracted a lot of attention lately. In these systems, robust and reliable vehicle detection is a critical step. This paper presents a review of recent vision-based on-road vehicle detection systems. Our focus is on systems where the camera is mounted on the vehicle rather than being fixed such as in traffic/driveway monitoring systems. First, we discuss the problem of on-road vehicle detection using optical sensors followed by a brief review of intelligent vehicle research worldwide. Then, we discuss active and passive sensors to set the stage for vision-based vehicle detection. Methods aiming to quickly hypothesize the location of vehicles in an image as well as to verify the hypothesized locations are reviewed next. Integrating detection with tracking is also reviewed to illustrate the benefits of exploiting temporal continuity for vehicle detection. Finally, we present a critical overview of the methods discussed, we assess their potential for future deployment, and we present directions for future research.

Vision-based vehicle detection is an important issue for advanced driver assistance systems. In this paper, we presented an improved multi-vehicle detection and tracking method using cascade Adaboost and Adaptive Kalman filter(AKF) with target identity awareness. A cascade Adaboost classifier using Haar-like features was built for vehicle detection, followed by a more comprehensive verification process which could refine the vehicle hypothesis in terms of both location and dimension. In vehicle tracking, each vehicle was tracked with independent identity by an Adaptive Kalman filter in collaboration with a data association approach. The AKF adaptively adjusted the measurement and process noise covariance through on-line stochastic modelling to compensate the dynamics changes. The data association correctly assigned different detections with tracks using global nearest neighbour(GNN) algorithm while considering the local validation. During tracking, a temporal context based track management was proposed to decide whether to initiate, maintain or terminate the tracks of different objects, thus suppressing the sparse false alarms and compensating the temporary detection failures. Finally, the proposed method was tested on various challenging real roads, and the experimental results showed that the vehicle detection performance was greatly improved with higher accuracy and robustness.

Due to the development of economy and technology, the people's demands on cars are growing and so are the problems, such as finding stolen car, banning violation and parking lot management. It will be time-consuming and low-efficiency if we do those jobs only by human because of the limitation of human being's concentration. Therefore, it has been a popular topic to develop intelligent monitoring system under new video technology within a decade. There are still many rooms for future development and application in the car license detection and recognition fields. For finding stolen car, we can integrate car license detection system and road monitoring system to analyze the videos and trace the objects, so we can gain high-efficiency and low-cost results. For parking lot management system, we can have the result of access management and automatic charge to reduce the human resource through car license recognition system. Automated toll of highway can be done through car license recognition system as well. Because the car license recognition system is mostly applied to security monitoring field and business purposes, the demand of the accuracy is quite strict. There are causes which make the inaccuracy of the license recognition, such as the lack of video resolution, too small license due to the distance and the light and shadow. We will discuss the license recognition system and how to use apply the super-resolution to overcome those above problems.

Reliable traffic light detection is one crucial key component for autonomous driving in urban areas. This includes the extraction of direction arrows contained within the traffic lights as an autonomous car will need this information for selecting the traffic light corresponding to its current lane. Current state of the art traffic light detection systems are not able to provide such information. Within this work we present a hierarchical traffic light detection algorithm, which is able to detect traffic lights and determine their state and contained direction information within one CNN forward pass. This Hierarchical DeepTLR (HDTLR) outperforms current state of the art traffic light detection algorithms in state aware detection and can detect traffic lights with direction information down to a size of 4 pixel in width at a frequency of 12 frames per second.


Question: Addressing Constraint Satisfaction Problems using Belief Propagation-guided decimation.

This paper explores the use of factor graphs and the sum-product algorithm in addressing complex functions that are dependent on numerous variables. The key strategy involves breaking down these global functions into a product of more manageable, local functions, each of which is a function of a subset of the total variables. This factorization can be graphically represented through a bipartite graph known as a factor graph. The paper introduces a versatile message-passing algorithm, the sum-product algorithm, designed to operate within this factor graph framework. By adhering to a single, straightforward computational rule, the sum-product algorithm is capable of computing—either precisely or approximately—a range of marginal functions that can be derived from the overarching global function. Notably, a multitude of algorithms from various fields, such as artificial intelligence, signal processing, and digital communications, can be seen as specific instances of the sum-product algorithm. These include notable algorithms such as the forward/backward algorithm, the Viterbi algorithm, the iterative "turbo" decoding algorithm, Pearl's belief propagation algorithm for Bayesian networks, the Kalman filter, and certain fast Fourier transform (FFT) algorithms. This highlights the sum-product algorithm's versatility and its foundational role in a wide array of computational techniques.

Many approaches to Bayesian image segmentation have used maximum a posteriori (MAP) estimation in conjunction with Markov random fields (MRF). Although this approach performs well, it has a number of disadvantages. In particular, exact MAP estimates cannot be computed, approximate MAP estimates are computationally expensive to compute, and unsupervised parameter estimation of the MRF is difficult. The authors propose a new approach to Bayesian image segmentation that directly addresses these problems. The new method replaces the MRF model with a novel multiscale random field (MSRF) and replaces the MAP estimator with a sequential MAP (SMAP) estimator derived from a novel estimation criteria. Together, the proposed estimator and model result in a segmentation algorithm that is not iterative and can be computed in time proportional to MN where M is the number of classes and N is the number of pixels. The also develop a computationally efficient method for unsupervised estimation of model parameters. Simulations on synthetic images indicate that the new algorithm performs better and requires much less computation than MAP estimation using simulated annealing. The algorithm is also found to improve classification accuracy when applied to the segmentation of multispectral remotely sensed images with ground truth data.

We present an approach for the verification of feed-forward neural networks in which all nodes have a piece-wise linear activation function. Such networks are often used in deep learning and have been shown to be hard to verify for modern satisfiability modulo theory (SMT) and integer linear programming (ILP) solvers. The starting point of our approach is the addition of a global linear approximation of the overall network behavior to the verification problem that helps with SMT-like reasoning over the network behavior. We present a specialized verification algorithm that employs this approximation in a search process in which it infers additional node phases for the non-linear nodes in the network from partial node phase assignments, similar to unit propagation in classical SAT solving. We also show how to infer additional conflict clauses and safe node fixtures from the results of the analysis steps performed during the search. The resulting approach is evaluated on collision avoidance and handwritten digit recognition case studies.

Natural graphs, such as social networks, email graphs, or instant messaging patterns, have become pervasive through the internet. These graphs are massive, often containing hundreds of millions of nodes and billions of edges. While some theoretical models have been proposed to study such graphs, their analysis is still difficult due to the scale and nature of the data.
 We propose a framework for large-scale graph decomposition and inference. To resolve the scale, our framework is distributed so that the data are partitioned over a shared-nothing set of machines. We propose a novel factorization technique that relies on partitioning a graph so as to minimize the number of neighboring vertices rather than edges across partitions. Our decomposition is based on a streaming algorithm. It is network-aware as it adapts to the network topology of the underlying computational hardware. We use local copies of the variables and an efficient asynchronous communication protocol to synchronize the replicated values in order to perform most of the computation without having to incur the cost of network communication. On a graph of 200 million vertices and 10 billion edges, derived from an email communication network, our algorithm retains convergence properties while allowing for almost linear scalability in the number of computers.

By reducing optimization to a sequence of small subproblems, working set methods achieve fast convergence times for many challenging problems. Despite excellent performance, theoretical understanding of working sets is limited, and implementations often resort to heuristics to determine subproblem size, makeup, and stopping criteria. We propose BLITZ, a fast working set algorithm accompanied by useful guarantees. Making no assumptions on data, our theory relates subproblem size to progress toward convergence. This result motivates methods for optimizing algorithmic parameters and discarding irrelevant variables as iterations progress. Applied to `1-regularized learning, BLITZ convincingly outperforms existing solvers in sequential, limited-memory, and distributed settings. BLITZ is not specific to `1-regularized learning, making the algorithm relevant to many applications involving sparsity or constraints.

Surveying a suite of algorithms that offer a solution to managing large document archives.

Graph partitioning (GP) applications are ubiquitous throughout mathematics, computer science, chemistry, physics, bio-science, machine learning, and complex systems. Post Moore's era supercomputing has provided us an opportunity to explore new approaches for traditional graph algorithms on quantum computing architectures. In this work, we explore graph partitioning using quantum annealing on the D-Wave 2X machine. Motivated by a recently proposed graph-based electronic structure theory applied to quantum molecular dynamics (QMD) simulations, graph partitioning is used for reducing the calculation of the density matrix into smaller subsystems rendering the calculation more computationally efficient. Unconstrained graph partitioning as community clustering based on the modularity metric can be naturally mapped into the Hamiltonian of the quantum annealer. On the other hand, when constraints are imposed for partitioning into equal parts and minimizing the number of cut edges between parts, a quadratic unconstrained binary optimization (QUBO) reformulation is required. This reformulation may employ the graph complement to fit the problem in the Chimera graph of the quantum annealer. Partitioning into 2 parts and k parts concurrently for arbitrary k are demonstrated with benchmark graphs, random graphs, and small material system density matrix based graphs. Results for graph partitioning using quantum and hybrid classical-quantum approaches are shown to be comparable to current "state of the art" methods and sometimes better.

Simple parallel algorithms for the maximal independent set (MIS) problem are presented. The first algorithm is a Monte Carlo algorithm with a very local property. The local property of this algorithm may make it a useful protocol design tool in distributed computing environments and artificial intelligence. One of the main contributions of this paper is the development of powerful and general techniques for converting Monte Carlo algorithms into deterministic algorithms. These techniques are used to convert the Monte Carlo algorithm for the MIS problem into a simple deterministic algorithm with the same parallel running time.

In order to meet the users’ demand, bike sharing systems must be regularly rebalanced. The problem of balancing bike sharing systems (BBSS) is concerned with designing optimal tours and operating instructions for relocating bikes among stations to maximally comply with the expected future bike demands. In this paper, we tackle the BBSS by means of Constraint Programming: first, we introduce two novel constraint models for the BBSS including a smart branching strategy that focusses on the most promising routes. Second, in order to speed-up the search process, we incorporate both models in a Large Neighborhood Search (LNS) approach that is adapted to the respective CP model. Third, we perform an extensive computational evaluation on instances based on real-world data, where we see that the LNS approach outperforms the Branch & Bound approach and is competitive with other existing approaches.

We present a novel approach to the matching of subgraphs for object recognition in computer vision. Feature similarities between object model and scene graph are complemented with a regularization term that measures differences of the relational structure. For the resulting quadratic integer program, a mathematically tight relaxation is derived by exploiting the degrees of freedom of the embedding space of positive semidefinite matrices. We show that the global minimum of the relaxed convex problem can be interpreted as probability distribution over the original space of matching matrices, providing a basis for efficiently sampling all close-to-optimal combinatorial matchings within the original solution space. As a result, the approach can even handle completely ambiguous situations, despite uniqueness of the relaxed convex problem. Exhaustive numerical experiments demonstrate the promising performance of the approach which – up to a single inevitable regularization parameter that weights feature similarity against structural similarity – is free of any further tuning parameters.

A PCFGparsingcandramaticallyreducethetime requiredto find theexactViterbi parseby conservatively estimatingoutsideViterbi probabilities.Wediscussvariousestimatesandgive efficientalgorithmsfor computingthem.On Penntreebanksentences, ourmostdetailedestimate reducesthetotalnumberof edgesprocessedto lessthan3%of thatrequiredby exhausti ve parsing,andevenasimplerestimatewhichcanbepre-computedin underaminutestill reduces thework by a factorof 5. ThealgorithmextendstheclassicA graphsearchprocedureto a certainhypergraphassociatedwith parsing.Unlike best-firstandfinite-beammethodsfor achieving thiskind of speed-up, theA parseris guaranteedto returnthemostlikely parse,not justanapproximation.Thealgorithmis alsocorrectfor awide rangeof parsercontrolstrategies andmaintainsaworst-casecubictime bound. A Parsing: FastExact Viterbi ParseSelection


Question: Addressing Constraint Satisfaction Problems using Belief Propagation-guided decimation.

This document explores the application of the Survey Propagation algorithm in solving constraint satisfaction problems, particularly focusing on random formulas composed of M clauses with K literals over N Boolean variables. The complexity of the problem peaks when the ratio α = M/N is near the critical threshold αc, dividing the domain where formulas are predominantly satisfiable from the region where they are unsatisfiable. Statistical physics insights suggest that this complexity arises from the clustering of solutions near αc. To navigate this challenging landscape, the authors introduce an innovative message passing algorithm that efficiently identifies satisfying assignments in this transitional zone. The algorithm is iterative and comprises two principal components. The first involves an enhanced message-passing mechanism, extending traditional methods such as Sum-Product or Belief Propagation, by transmitting messages that represent surveys over clusters of conventional messages. The second component leverages the detailed probabilistic insights gained from these surveys to fix variables, thereby simplifying the problem. The simplified problem is then resolved using conventional heuristics. This approach, detailed in the publication, marks a significant advancement in solving constraint satisfaction problems through the synergy of message passing and probabilistic insights.

The FP-growth algorithm is currently one of the fastest approaches to frequent item set mining. In this paper I describe a C implementation of this algorithm, which contains two variants of the core operation of computing a projection of an FP-tree (the fundamental data structure of the FP-growth algorithm). In addition, projected FP-trees are (optionally) pruned by removing items that have become infrequent due to the projection (an approach that has been called FP-Bonsai). I report experimental results comparing this implementation of the FP-growth algorithm with three other frequent item set mining algorithms I implemented (Apriori, Eclat, and Relim).

Since the l a s t I JCAI , the PLANNER problem s o l v i n g fo rma l i sm has con t i nued to deve lop . Our ep ls temo lgy f o r the f ounda t i ons f o r problem s o l v i n g has been ex tended . An overv iew of the f o rma l i sm Is g iven from an I n f o r m a t i o n p rocess ing v i e w p o i n t . A s imp le example Is e x p l a i n e d us ing snapshots of the s t a t e of the problem s o l v i n g as the example I s worked, f i n a l l y , c u r r e n t a p p l i c a t i o n s f o r the fo rma l i sm are I i s ted . 1 . The S t r u c t u r a l Foundat ions o f Problem S o l v i n g We would l i k e to develop a f o u n d a t i o n f o r problem s o l v i n g analogous In some ways to the c u r r e n t l y e x i s t i n g f ounda t i ons f o r mathemat ics . Thus we need to analyze the s t r u c t u r e of f o u n d a t i o n s f o r mathemat ics . A f o u n d a t i o n f o r mathematics must p rov ide a d e f i n i t i o n a l f o rma l i sm In which mathemat ica l o b j e c t s can be d e f i n e d and t h e i r e x i s t e n c e p roved . For example set t heo ry as a f o u n d a t i o n p rov ides t h a t o b j e c t s must be b u i l t out o f s e t s . Then the re must be a deduc t i ve f o r m a l i s m In which fundamental t r u t h s can be s t a t e d and the means p rov ided to deduce a d d i t i o n a l t r u t h s f rom those a l r e a d y e s t a b l i s h e d . Cur rent mathemat ica l f ounda t i ons such as set theo ry seem q u i t e n a t u r a l and adequate f o r the vas t body o f c l a s s i c a l mathemat ics . The o b j e c t s and reason ing of most mathemat ica l domains such as a n a l y s i s and a lgebra can be e a s i l y founded on set t h e o r y . The e x i s t e n c e o f c e r t a i n a s t r o n o m i c a l l y l a r g e c a r d i n a l s poses some problems f o r set t h e o r e t i c f o u n d a t i o n s . However, the problems posed seem to be of p r a c t i c a l Importance on l y t o c e r t a i n ca tego ry t h e o r i s t s . Foundat ions o f mathemat ics have devoted a g rea t deal o f a t t e n t i o n to the problems o f c o n s i s t e n c y and comple teness. The problem of cons i s t ency Is Impor tant s i nce I f the f ounda t i ons are i n c o n s i s t e n t then any fo rmu la whatsoever may be deduced thus t r i v i a l i z i n g the f o u n d a t i o n s . Semantics f o r f ounda t i ons o f mathemat ics are d e f i n e d model 167 t h e o r e t i c a l l y In terms o f the n o t i o n o f s a t i s f i a b i l i t y . The problem o f completeness is t h a t , f o r a f o u n d a t i o n of mathematics to be I n t u i t i v e l y s a t i s f a c t o r y a l l the t r u e fo rmu las shou ld be proveab le s ince a f o u n d a t i o n mathemat ics alms to be a theory of mathemat ica l t r u t h . S i m i l a r fundamental ques t i ons must be faced by a f o u n d a t i o n f o r problem s o l v i n g . However t he re are some impor tant d i f f e r e n c e s s ince a f o u n d a t i o n of problem s o l v i n g alms more to be a theory of a c t i o n s and purposes than a theory of mathemat ica l t r u t h . A f o u n d a t i o n f o r problem s o l v i n g must s p e c i f y a g o a l o r l e n t e d f o rma l i sm In which problems can be s t a t e d . Fur thermore the re must be a f o rma l i sm f o r s p e c i f y i n g the a l l o w a b l e methods o f s o l u t i o n of p rob lems. As p a r t o f the d e f i n i t i o n o f the f o r m a l i s m s , the f o l l o w i n g elements must be d e f i n e d : the data s t r u c t u r e , the c o n t r o l s t r u c t u r e , and the p r i m i t i v e p rocedures . The problem of what are a l l o w a b l e data s t r u c t u r e s f o r f a c t s about the wo r l d Immediate ly a r i s e s . Being a theory of a c t i o n s , a f o u n d a t i o n f o r problem s o l v i n g must c o n f r o n t the problem of change: How can account be taken of the changing s i t u a t i o n In the wor ld? In o rder f o r the re to be problem s o l v i n g , the re must be a problem s o l v e r . A f o u n d a t i o n f o r problem s o l v i n g must cons ide r how much knowledge and what k ind of knowledge problem s o l v e r s can have about themse lves . In c o n t r a s t to the f o u n d a t i o n o f mathemat ics , the semant ics f o r a f o u n d a t i o n f o r problem s o l v i n g should be d e f i n e d In terms of p r o p e r t i e s of p rocedures . We would l i k e to see mathemat ica l I n v e s t i g a t i o n s on the adequacy of the f o u n d a t i o n s f o r problem s o l v i n g p rov ided by PLANNER. In chap te r C of the d i s s e r t a t i o n , we have made the beg inn ings of one k ind of such an I n v e s t i g a t i o n . To be more s p e c i f i c a f o u n d a t i o n f o r problem s o l v i n g must concern I t s e l f w i t h the f o l l o w i n g complex o f t o p i c s . PROCEDURAL EMBEDDING: How can " r e a l w o r l d " knowledge be e f f e c t i v e l y embedded In p rocedures . What are good ways to express problem s o l u t i o n methods and how can p lans f o r the s o l u t i o n of problems be fo rmu la ted? GENERALIZED COMPILATION: What are good methods f o r t r a n s f o r m i n g h igh l e v e l g o a l o r l e n t e d language in to s p e c i f i c e f f i c i e n t a l g o r i t h m s .

The problem of simultaneously disseminating k messages in a large network of n nodes, in a decentralized and distributed manner, where nodes only have knowledge about their own contents, is studied. In every discrete time-step, each node selects a communication partner randomly, uniformly among all nodes and only one message can be transmitted. The goal is to disseminate rapidly, with high probability, all messages to all nodes. It is shown that a random linear coding (RLC) based protocol disseminates all messages to all nodes in time ck + O (√k ln(k) ln(n)), where c < 3.46 using pull-based dissemination and c < 5.96 using push-based dissemination. Simulations suggest that c < 2 might be a tighter bound. Thus, if k ≫ (ln(n))3, the time for simultaneous dissemination RLC is asymptotically at most ck, versus the Ω(k log<inf>2</inf>(n))3 time of sequential dissemination. Furthermore, when k ≫ (ln(n))3, the dissemination time is order optimal. When k ≪ (ln(n))2, RLC reduces dissemination time by a factor of Ω(√k/ln k) over sequential dissemination. The overhead of the RLC protocol is negligible for messages of reasonable size. A store-and-forward mechanism without coding is also considered. It is shown that this approach performs no better than a sequential approach when k=∞ n. Owing to the distributed nature of the system, the proof requires analysis of an appropriate time-varying Bernoulli process.

We describe in this paper a new approach to parallelize branchand-bound on a certain number of processors. We propose to split the optimization of the original problem into the optimization of several subproblems that can be optimized separately with the goal that the amount of work that each processor carries out is balanced between the processors, while achieving interesting speedups. The main innovation of our approach consists in the use of machine learning to create a function able to estimate the difficulty (number of nodes) of a subproblem of the original problem. We also present a set of features that we developed in order to characterize the encountered subproblems. These features are used as input of the function learned with machine learning in order to estimate the difficulty of a subproblem. The estimates of the numbers of nodes are then used to decide how to partition the original optimization tree into a given number of subproblems, and to decide how to distribute them among the available processors. The experiments that we carry out show that our approach succeeds in balancing the amount of work between the processors, and that interesting speedups can be achieved with little effort.

We consider the following problem: The Santa Claus has n presents that he wants to distribute among m kids. Each kid has an arbitrary value for each present. Let pij be the value that kid i has for present j. The Santa's goal is to distribute presents in such a way that the least lucky kid is as happy as possible, i.e he tries to maximize mini=1,...,m sumj ∈ Si pij where Si is a set of presents received by the i-th kid.Our main result is an O(log log m/log log log m) approximation algorithm for the restricted assignment case of the problem when pij ∈ pj,0 (i.e. when present j has either value pj or 0 for each kid). Our algorithm is based on rounding a certain natural exponentially large linear programming relaxation usually referred to as the configuration LP. We also show that the configuration LP has an integrality gap of Ω(m1/2) in the general case, when pij can be arbitrary.

This paper investigates the methods for learning Bayesian belief network (BN) based predictive models for classification. Our primary interests are in the unrestricted Bayesian network and Bayesian multi-net based classifiers. We present our algorithms for learning these classifiers and also the methods for fighting the overfitting problem. A natural method for feature subset selection is also studied. Using a set of standard classification problems, we empirically evaluate the performance of various BN based classifiers. The results show that the proposed BN and Bayes multi-net classifiers are competitive with (or superior to) the best known classifiers, based on both BN and other formalisms; and that the computational time for learning and using these classifiers is relatively small. We also briefly introduce our BN classifier learning system – BN PowerPredictor. We argue that BN based classifiers deserve more attention in the data mining community.

We present a new algorithm for Bayesian network structure learning, called Max-Min Hill-Climbing (MMHC). The algorithm combines ideas from local learning, constraint-based, and search-and-score techniques in a principled and effective way. It first reconstructs the skeleton of a Bayesian network and then performs a Bayesian-scoring greedy hill-climbing search to orient the edges. In our extensive empirical evaluation MMHC outperforms on average and in terms of various metrics several prototypical and state-of-the-art algorithms, namely the PC, Sparse Candidate, Three Phase Dependency Analysis, Optimal Reinsertion, Greedy Equivalence Search, and Greedy Search. These are the first empirical results simultaneously comparing most of the major Bayesian network algorithms against each other. MMHC offers certain theoretical advantages, specifically over the Sparse Candidate algorithm, corroborated by our experiments. MMHC and detailed results of our study are publicly available at http://www.dsl-lab.org/supplements/mmhc_paper/mmhc_index.html.

Algorithms for learning Bayesian networks from data have two components: a scoring metric and a search procedure. The scoring metric computes a score reeecting the goodness-of-t of the structure to the data. The search procedure tries to identify network structures with high scores. Heckerman et al. (1995) introduce a Bayesian metric, called the BDe metric, that computes the relative posterior probability of a network structure given data. In this paper, we show that the search problem of identifying a Bayesian network|among those where each node has at most K parents|that has a relative posterior probability greater than a given constant is NP-complete, when the BDe metric is used. 12.1 Introduction Recently, many researchers have begun to investigate methods for learning Bayesian networks. Many of these approaches have the same basic components: a scoring metric and a search procedure. The scoring metric takes a database of observed cases D and a network structure B S , and returns a score reeecting the goodness-of-t of the data to the structure. A search procedure generates networks for evaluation by the scoring metric. These approaches use the two components to identify a network structure or set of structures that can be used to predict future events or infer causal relationships. Cooper and Herskovits (1992)|herein referred to as CH|derive a Bayesian metric, which we call the BD metric, from a set of reasonable assumptions about learning Bayesian networks containing only discrete variables. Heckerman et al. (1995)|herein referred to as HGC|expand upon the work of CH to derive a new metric, which we call the BDe metric, which has the desirable property of likelihood equivalence. Likelihood equivalence says that the data cannot help to discriminate equivalent structures. We now present the BD metric derived by CH. We use B h S to denote the hypothesis that B S is an I-map of the distribution that generated the database. 2 Given a belief-network structure B S , we use i to denote the parents of x i. We use r i to denote the number of states of variable x i , and q i = Q x l 2 i r l to denote the number of instances of i. We use the integer j to index these instances. That is, we write i = j to denote the observation of the jth instance of the parents of x i. 1996 Springer-Verlag. 2 There is an …

Due to a tremendous increase in internet traffic, backbone routers must have the capability to forward massive incoming packets at several gigabits per second. IP address lookup is one of the most challenging tasks for high-speed packet forwarding. Some high-end routers have been implemented with hardware parallelism using ternary content addressable memory (TCAM). However, TCAM is much more expensive in terms of circuit complexity as well as power consumption. Therefore, efficient algorithmic solutions are essentially required to be implemented using network processors as low cost solutions. Among the state-of-the-art algorithms for IP address lookup, a binary search based on a balanced tree is effective in providing a low-cost solution. In order to construct a balanced search tree, the prefixes with the nesting relationship should be converted into completely disjointed prefixes. A leaf-pushing technique is very useful to eliminate the nesting relationship among prefixes [V. Srinivasan, G. Varghese, Fast address lookups using controlled prefix expansion, ACM Transactions on Computer Systems 17 (1) (1999) 1–40]. However, it creates duplicate prefixes, thus expanding the search tree. This paper proposes an efficient IP address lookup algorithm based on a small balanced tree using entry reduction. The leaf-pushing technique is used for creating the completely disjointed entries. In the leaf-pushed prefixes, there are numerous pairs of adjacent prefixes with similarities in prefix strings and output ports. The number of entries can be significantly reduced by the use of a new entry reduction method which merges pairs with these similar prefixes. After sorting the reduced disjointed entries, a small balanced tree is constructed with a very small node size. Based on this small balanced tree, a native binary search can be effectively used in address lookup issue. In addition, we propose a new multi-way search algorithm to improve a binary search for IPv4 address lookup. As a result, the proposed algorithms offer excellent lookup performance along with reduced memory requirements. Besides, these provide good scalability for large amounts of routing data and for the address migration toward IPv6. Using both various IPv4 and IPv6 routing data, the performance evaluation results demonstrate that the proposed algorithms have better performance in terms of lookup speed, memory requirement and scalability for the growth of entries and IPv6, as compared with other algorithms based on a binary search. 2011 Elsevier B.V. All rights reserved. . All rights reserved. x: +82 2 313 8053.

We report the current state of the graph isomorphism problem from the practical point of view. After describing the general principles of the refinement-individualization paradigm and proving its validity, we explain how it is implemented in several of the key programs. In particular, we bring the description of the best known program nauty up to date and describe an innovative approach called Traces that outperforms the competitors for many difficult graph classes. Detailed comparisons against saucy, Bliss and conauto are presented. Email addresses: bdm@cs.anu.edu.au (Brendan D. McKay), piperno@di.uniroma1.it (Adolfo Piperno). 1 Supported by the Australian Research Council. Preprint submitted to Elsevier — nautytraces2b — 11 December 2013 — ar X iv :1 30 1. 14 93 v1 [ cs .D M ] 8 J an 2 01 3


Question: Addressing Constraint Satisfaction Problems using Belief Propagation-guided decimation.

This paper presents a comprehensive message passing algorithm known as the Generalized Distributive Law (GDL), which serves as a unifying framework for various methodologies developed across different scientific disciplines. The GDL encapsulates and extends the work of numerous researchers from fields such as information theory, digital communications, signal processing, statistics, and artificial intelligence. Among its special cases are the Baum-Welch algorithm, the Fast Fourier Transform (FFT) on finite Abelian groups, the Gallager-Tanner-Wiberg decoding algorithm, Viterbi’s algorithm, the BCJR algorithm, Pearl’s belief propagation algorithm, the Shafer-Shenoy probability propagation algorithm, and the turbo decoding algorithm. While the GDL is theoretically guaranteed to provide exact solutions under the junction tree condition, it is noteworthy that it often yields approximate solutions in scenarios where exactness is not theoretically assured, such as in Gallager-Tanner-Wiberg decoding with cycles or turbo decoding. This observation is supported by a wealth of experimental evidence and a few theoretical proofs, indicating the algorithm’s robustness and versatility in practical applications.

Logic partitioning is an important issue in VLSI CAD, and has been an area of active research for at least the last 25 years. Numerous approaches have been developed and many different techniques have been combined for a wide range of applications. In this paper, we examine many of the existing techniques for logic bipartitioning and present a methodology for determining the best mix of approaches. The result is a novel bipartitioning algorithm that includes both new and pre-existing techniques. Our algorithm produces results that are at least 16% better than the state-of-the-art while also being efficient in run-time.

Variational inference methods, including mean field methods and loopy belief propagation, have been widely used for approximate probabilistic inference in graphical models. While often less accurate than MCMC, variational methods provide a fast deterministic approximation to marginal and conditional probabilities. Such approximations can be particularly useful in high dimensional problems where sampling methods are too slow to be effective. A limitation of current methods, however, is that they are restricted to parametric probabilistic models. MCMC does not have such a limitation; indeed, MCMC samplers have been developed for the Dirichlet process (DP), a nonparametric distribution on distributions (Ferguson, 1973) that is the cornerstone of Bayesian nonparametric statistics (Escobar & West, 1995; Neal, 2000). In this paper, we develop a mean-field variational approach to approximate inference for the Dirichlet process, where the approximate posterior is based on the truncated stick-breaking construction (Ishwaran & James, 2001). We compare our approach to DP samplers for Gaussian DP mixture models.

We consider the following problem: The Santa Claus has n presents that he wants to distribute among m kids. Each kid has an arbitrary value for each present. Let pij be the value that kid i has for present j. The Santa's goal is to distribute presents in such a way that the least lucky kid is as happy as possible, i.e he tries to maximize mini=1,...,m sumj ∈ Si pij where Si is a set of presents received by the i-th kid.Our main result is an O(log log m/log log log m) approximation algorithm for the restricted assignment case of the problem when pij ∈ pj,0 (i.e. when present j has either value pj or 0 for each kid). Our algorithm is based on rounding a certain natural exponentially large linear programming relaxation usually referred to as the configuration LP. We also show that the configuration LP has an integrality gap of Ω(m1/2) in the general case, when pij can be arbitrary.

The purpose of this paper is twofold. First, a brief exposition of the general backtrack technique and its history is given. Second, it is shown how the use of macros can considerably shorten the computation time in many cases. In particular, this technique has allowed the solution of two previously open combinatorial problems, the computation of new terms in a well-known series, and the substantial reduction in computation time for the solution to another combinatorial problem.

The goal of this paper is to generate insights about the differences between graph-based and logic-based approaches to multi-relational data mining by performing a case study of graph-based system, Subdue and the inductive logic programming system, CProgol. We identify three key factors for comparing graph-based and logic-based multi-relational data mining; namely, the ability to discover structurally large concepts, the ability to discover semantically complicated concepts and the ability to effectively utilize background knowledge. We perform an experimental comparison of Subdue and CProgol on the Mutagenesis domain and various artificially generated Bongard problems. Experimental results indicate that Subdue can significantly outperform CProgol while discovering structurally large multi-relational concepts. It is also observed that CProgol is better at learning semantically complicated concepts and it tends to use background knowledge more effectively than Subdue.

This paper addresses the problem of segmenting an image into regions. We define a predicate for measuring the evidence for a boundary between two regions using a graph-based representation of the image. We then develop an efficient segmentation algorithm based on this predicate, and show that although this algorithm makes greedy decisions it produces segmentations that satisfy global properties. We apply the algorithm to image segmentation using two different kinds of local neighborhoods in constructing the graph, and illustrate the results with both real and synthetic images. The algorithm runs in time nearly linear in the number of graph edges and is also fast in practice. An important characteristic of the method is its ability to preserve detail in low-variability image regions while ignoring detail in high-variability regions.

This paper introduces a probabilistic algorithm for multi-robot decision-making under uncertainty, which can be posed as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). Dec-POMDPs are inherently synchronous decision-making frameworks which require significant computational resources to be solved, making them infeasible for many real-world robotics applications. The Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP) was recently introduced as an extension of the Dec-POMDP that uses high-level macro-actions to allow large-scale, asynchronous decision-making. However, existing Dec-POSMDP solution methods have limited scalability or perform poorly as the problem size grows. This paper proposes a cross-entropy based Dec-POSMDP algorithm motivated by the combinatorial optimization literature. The algorithm is applied to a constrained package delivery domain, where it significantly outperforms existing Dec-POSMDP solution methods.

We propose a new stochastic algorithm (generalized simulated annealing) for computationally finding the global minimum of a given (not necessarily convex) energy/cost function defined in a continuous D-dimensional space. This algorithm recovers, as particular cases, the so called classical (“Boltzmann machine”) and fast (“Cauchy machine”) simulated annealings, and can be quicker than both. Key-words: Simulated annealing; Nonconvex optimization; Gradient descent; Generalized Statistical Mechanics.

Using constraint programming, we effectively model and efficiently solve the problem of balancing and minimising the traffic complexities of an airspace of adjacent sectors. The traffic complexity of a sector is here defined in terms of the numbers of flights within it, near its border, and on non-level segments within it. The allowed forms of complexity resolution are the changing of the take-off times of not yet airborne flights, the changing of the remaining approach times into the chosen airspace of already airborne flights by slowing down and speeding up within the two layers of feeder sectors around that airspace, as well as the changing of the levels of passage over way-points in that airspace. Experiments with actual European flight profiles obtained from the Central Flow Management Unit (CFMU) show that these forms of complexity resolution can lead to significant complexity reductions and rebalancing.

Neuromorphic hardware tends to pose limits on the connectivity of deep networks that one can run on them. But also generic hardware and software implementations of deep learning run more efficiently on sparse networks. Several methods exist for pruning connections of a neural network after it was trained without connectivity constraints. We present an algorithm, DEEP R, that enables us to train directly a sparsely connected neural network. DEEP R automatically rewires the network during supervised training so that connections are there where they are most needed for the task, while its total number is all the time strictly bounded. We demonstrate that DEEP R can be used to train very sparse feedforward and recurrent neural networks on standard benchmark tasks with just a minor loss in performance. DEEP R is based on a rigorous theoretical foundation that views rewiring as stochastic sampling of network configurations from a posterior.


Question: Addressing constraint satisfaction problems using belief propagation-guided decimation.

This paper outlines a novel technique for crafting long error-correcting codes from shorter codes, referred to as subcodes, and a bipartite graph. The process involves selecting precise subsets of the code digits that must conform to one of the subcodes, as dictated by the graph. Theoretical lower bounds for the rate and minimum distance of the newly created code are determined based on the properties of the graph and the subcodes. The proposed encoding and decoding methods capitalize on the code's modular structure, breaking down complex operations into simpler tasks associated with each subcode. Performance bounds for two decoding algorithms are established, and the asymptotic complexity of decoding for specific code and decoder types is examined. The decoding algorithms can efficiently utilize probability data from the channel receiver, such as reliability indicators, without significantly escalating computational demands. Moreover, it is demonstrated that selecting a transmission sequence for the digits that aligns with the graph and subcodes can significantly enhance the code's burst-error correction capabilities. These principles are elucidated through various examples.

**Question**: A Method for Solving Economic Dispatch Problems with Valve-Point Effects Using a Hybrid of Genetic Algorithm and Particle Swarm Optimization

**Documents**: A hybrid evolutionary algorithm for feature selection
Feature selection is a critical problem in machine learning and data mining. It aims to identify the most relevant features that can be used to build a model with high accuracy and good generalization, while also reducing the computational complexity of the learning process. This paper proposes a hybrid evolutionary algorithm that combines the strengths of genetic algorithms (GA) and particle swarm optimization (PSO) for feature selection. The proposed algorithm, referred to as HGA-PSO, utilizes the mutation and crossover operations of GA to explore the search space and the swarm intelligence of PSO to refine the solutions. The algorithm is designed to iteratively update the feature subsets, evaluating their performance using a fitness function that measures the predictive power of the model. The proposed HGA-PSO is applied to several datasets from the UCI machine learning repository, and its performance is compared to that of GA and PSO. The results indicate that HGA-PSO can achieve higher classification accuracy with fewer features, demonstrating its effectiveness in feature selection tasks.

**Similar Documents**: This work introduces a novel hybrid evolutionary algorithm, HGA-PSO, for the purpose of feature selection in machine learning and data mining. The primary goal is to pinpoint the most pertinent features that can lead to the construction of models with high accuracy and robust generalization capabilities, while simultaneously minimizing the computational burden of the learning

Network scaling algorithms such as the Pathfinder algorithm are used to prunemany different kinds of networks, including citation networks, randomnetworks, and social networks. However, this algorithm suffers from run time problems for large networks and online processing due to its O(n4) time complexity. In this article, we introduce a new alternative, the MST-Pathfinder algorithm, which will allow us to prune the original network to get its PFNET(∞, n −1) in justO(n2 · logn) time.Theunderlying idea comes from the fact that the union (superposition) of all the Minimum Spanning Trees extracted from a given network is equivalent to the PFNET resulting from the Pathfinder algorithmparameterized by a specific set of values (r = ∞ and q = n −1), those usually considered in many different applications. Although this property is well-known in the literature, it seems that no algorithm based on it has been proposed, up to now, to decrease the high computational cost of the original Pathfinder algorithm.We also present a mathematical proof of the correctness of this new alternative and test its good efficiency in two different case studies: one dedicated to the post-processing of large random graphs, and the other one to a real world case in which medium networks obtained by a cocitation analysis of the scientific domains in different countries are pruned.

Given a parallel program represented by a task graph, the objective of a scheduling algorithm is to minimize the overall execution time of the program by properly assigning the nodes of the graph to the processors. This multiprocessor scheduling problem is NP-complete even with simplifying assumptions and becomes more complex under relaxed assumptions such as arbitrary precedence constraints, and arbitrary task execution and communication times. The present literature on this topic is a large repertoire of heuristics that produce good solutions in a reasonable amount of time. These heuristics, however, have restricted applicability in a practical environment because they have a number of fundamental problems including high time complexity, lack of scalability, and no performance guarantee with respect to optimal solutions. Recently, genetic algorithms (GAs) have been widely reckoned as a useful vehicle for obtaining high quality or even optimal solutions for a broad range of combinatorial optimization problems. While a few GAs for scheduling have already been suggested, in this paper we propose a novel GA-based algorithm with an objective to simultaneously meet the goals of high performance, scalability, and fast running time. The proposed parallel genetic scheduling (PGS) algorithm itself is a parallel algorithm which generates high quality solutions in a short time. By encoding the scheduling list as a chromosome, the PGS algorithm can potentially generate an optimal scheduling list which in turn leads to an optimal schedule. The major strength of the PGS algorithm lies in its two efficient genetic operators: the order crossover and mutation. These operators effectively combine the building-blocks of good scheduling lists to construct better lists. The proposed algorithm is evaluated through a robust comparison with two heuristics best known in terms of performance and time complexity. It outperforms both heuristics while taking considerably less running time. When evaluated with random task graphs for which optimal solutions are known, the PGS algorithm generates optimal solutions for more than half of the test cases and close-to-optimal for the other half. © 1997 Academic Press

A large class of computational problems involve the determination of properties of graphs, digraphs, integers, arrays of integers, finite families of finite sets, boolean formulas and elements of other countable domains. Through simple encodings from such domains into the set of words over a finite alphabet these problems can be converted into language recognition problems, and we can inquire into their computational complexity. It is reasonable to consider such a problem satisfactorily solved when an algorithm for its solution is found which terminates within a number of steps bounded by a polynomial in the length of the input. Many problems with wide applicability – e.g., set cover, knapsack, hitting set, max cut, and satisfiability – lack a polynomial algorithm for solving them, but also lack a proof that no such polynomial algorithm exists. Hence, they remain “open problems.” This paper references the recent work, “On the Reducibility of Combinatorial Problems” [1]. BODY A large class of open problems are mutually convertible via poly-time reductions. Hence, either all can be solved in poly-time, or none can. REFERENCES [1] R. Karp. Reducibility Among Combinatorial Problems. In Complexity of Computer Computations, 1972. ∗With apologies to Professor Richard Karp. Volume X of Tiny Transactions on Computer Science This content is released under the Creative Commons Attribution-NonCommercial ShareAlike License. Permission to make digital or hard copies of all or part of this work is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. CC BY-NC-SA 3.0: http://creativecommons.org/licenses/by-nc-sa/3.0/.

We identify two families of canonical representations called ROBDD[∧î]C and ROBDD[∧T̂ ,i]T by augmenting ROBDD with two types of conjunctive decompositions. These representations cover the three existing languages ROBDD, ROBDD with as many implied literals as possible (ROBDDL∞), and AND/OR BDD. We introduce a new time efficiency criterion called rapidity which reflects the idea that exponential operations may be preferable if the language can be exponentially more succinct. Then we demonstrate that the expressivity, succinctness and operation rapidity do not decrease from ROBDD[∧T̂ ,i]T to ROBDD[∧î]C , and then to ROBDD[∧ î+1 ]C . We also demonstrate that ROBDD[∧î]C (i > 1) and ROBDD[∧T̂ ,i]T are not less tractable than ROBDD-L∞ and ROBDD, respectively. Finally, we develop a compiler for ROBDD[∧∞̂]C which significantly advances the compiling efficiency of canonical representations.

Physical problems offer scope for macro level parallelization of solution by their essential structure. For parallelization of electrical network simulation, the most natural structure based method is that of multiport decomposition. In this paper this method is used for the simulation of electrical networks consisting of resistances, voltage and current sources using a distributed cluster of weakly coupled processors. At the two levels in which equations are solved in this method the authors have used sparse LU for both levels in the first scheme and sparse LU in the inner level and conjugate gradient in the outer level in the second scheme. Results are presented for planar networks, for the cases where the numbers of slave processors are 1 and 2, and for circuit sizes up to 8.2 million nodes and 16.4 million edges using 8 slave processors. The authors use a cluster of Pentium IV processors linked through a 10/100MBPS Ethernet switch

Common wisdom in the automated data structure generation community states that declarative techniques have better usability than imperative techniques, while imperative techniques have better performance. We show that this reasoning is fundamentally flawed: if we go to the declarative limit and employ constraint logic programming (CLP), the CLP data structure generation has orders of magnitude better performance than comparable imperative techniques. Conversely, we observe and argue that when it comes to realistically complex data structures and properties, the CLP specifications become more obscure, indirect, and difficult to implement and understand than their imperative counterparts. We empirically evaluate three competing generation techniques, CLP, Korat, and UDITA, to validate these observations on more complex and interesting data structures than any prior work in this area. We explain why these observations are true, and discuss possible techniques for attaining the best of both worlds.

Model-based methods and deep neural networks have both been tremendously successful paradigms in machine learning. In model-based methods, we can easily express our problem domain knowledge in the constraints of the model at the expense of difficulties during inference. Deterministic deep neural networks are constructed in such a way that inference is straightforward, but we sacrifice the ability to easily incorporate problem domain knowledge. The goal of this paper is to provide a general strategy to obtain the advantages of both approaches while avoiding many of their disadvantages. The general idea can be summarized as follows: given a model-based approach that requires an iterative inference method, we unfold the iterations into a layer-wise structure analogous to a neural network. We then de-couple the model parameters across layers to obtain novel neural-network-like architectures that can easily be trained discriminatively using gradient-based methods. The resulting formula combines the expressive power of a conventional deep network with the internal structure of the model-based approach, while allowing inference to be performed in a fixed number of layers that can be optimized for best performance. We show how this framework can be applied to the non-negative matrix factorization to obtain a novel non-negative deep neural network architecture, that can be trained with a multiplicative back-propagation-style update algorithm. We present experiments in the domain of speech enhancement, where we show that the resulting model is able to outperform conventional neural network while only requiring a fraction of the number of parameters. We believe this is due to the ability afforded by our framework to incorporate problem level assumptions into the architecture of the deep network. arXiv.org This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c ©Mitsubishi Electric Research Laboratories, Inc., 2014 201 Broadway, Cambridge, Massachusetts 02139

Novel low-density signature (LDS) structure is proposed for transmission and detection of symbol-synchronous communication over memoryless Gaussian channel. Given N as the processing gain, under this new arrangement, users' symbols are spread over N chips but virtually only dv < N chips that contain nonzero-values. The spread symbol is then so uniquely interleaved as the sampled, at chip rate, received signal contains the contribution from only dc < K number of users, where K denotes the total number of users in the system. Furthermore, a near-optimum chip-level iterative soft-in-soft-out (SISO) multiuser decoding (MUD), which is based on message passing algorithm (MPA) technique, is proposed to approximate optimum detection by efficiently exploiting the LDS structure. Given beta = K/N as the system loading, our simulation suggested that the proposed system alongside the proposed detection technique, in AWGN channel, can achieve an overall performance that is close to single-user performance, even when the system has 200% loading, i.e., when beta = 2. Its robustness against near-far effect and its performance behavior that is very similar to optimum detection are demonstrated in this paper. In addition, the complexity required for detection is now exponential to dc instead of K as in conventional code division multiple access (CDMA) structure employing optimum multiuser detector.

This work presents two new score functions based on the Bayesian Dirichlet equivalent uniform (BDeu) score for learning Bayesian network structures. They consider the sensitivity of BDeu to varying parameters of the Dirichlet prior. The scores take on the most adversary and the most beneficial priors among those within a contamination set around the symmetric one. We build these scores in such way that they are decomposable and can be computed efficiently. Because of that, they can be integrated into any state-of-the-art structure learning method that explores the space of directed acyclic graphs and allows decomposable scores. Empirical results suggest that our scores outperform the standard BDeu score in terms of the likelihood of unseen data and in terms of edge discovery with respect to the true network, at least when the training sample size is small. We discuss the relation between these new scores and the accuracy of inferred models. Moreover, our new criteria can be used to identify the amount of data after which learning is saturated, that is, additional data are of little help to improve the resulting model.

We present the design and implementation of a new inexact Newton type algorithm for solving large-scale bundle adjustment problems with tens of thousands of images. We explore the use of Conjugate Gradients for calculating the Newton step and its performance as a function of some simple and computationally efficient preconditioners. We show that the common Schur complement trick is not limited to factorization-based methods and that it can be interpreted as a form of preconditioning. Using photos from a street-side dataset and several community photo collections, we generate a variety of bundle adjustment problems and use them to evaluate the performance of six different bundle adjustment algorithms. Our experiments show that truncated Newton methods, when paired with relatively simple preconditioners, offer state of the art performance for large-scale bundle adjustment. The code, test problems and detailed performance data are available at http://grail.cs.washington.edu/projects/bal.


Question: Addressing constraint satisfaction problems using belief propagation-guided decimation.

This document provides an in-depth tutorial on the theory and application of hidden Markov models (HMMs) in the field of speech recognition. Unlike conventional statistical models, HMMs are uniquely suited for dealing with temporal sequences, which makes them particularly effective in speech processing tasks. The tutorial begins by introducing the fundamental concepts and mathematical foundations of HMMs, including their state transition probabilities and emission probabilities. It then delves into the three primary problems associated with HMMs: evaluation, decoding, and learning. The evaluation problem involves calculating the probability of a given sequence of observations, the decoding problem is concerned with identifying the most likely sequence of hidden states given the observations, and the learning problem aims to estimate the model parameters from a set of training data. The document also covers advanced topics such as continuous density HMMs, which are essential for handling continuous speech data. Furthermore, it discusses various techniques for improving the performance of HMMs in speech recognition, such as context-dependent modeling, decision trees, and neural network integration. Finally, the tutorial presents case studies and real-world applications of HMMs in speech recognition, highlighting their versatility and effectiveness in this domain.

**Similar Documents**: This comprehensive guide explores the theory and practical applications of hidden Markov models (HMMs), with a focus on their role in speech recognition technologies. Unlike standard statistical models, HMMs are particularly adept at handling temporal sequences, a critical feature in speech processing applications. The guide starts by laying out the foundational concepts and mathematical underpinnings of HMMs, encompassing state transition probabilities and emission probabilities. It then addresses the three main challenges in HMMs: the evaluation problem, which deals with calculating the probability of a specific sequence of observations; the decoding problem, which aims to determine the most probable sequence of hidden states given the observations; and the learning problem, which seeks to estimate the model parameters from a dataset of training examples. The document also explores advanced topics, such as continuous density HMMs, which are crucial for managing continuous speech data. Additionally, it examines various strategies for enhancing the performance of HMMs in speech recognition, including context-dependent modeling, decision tree methodologies, and the integration of neural networks. The guide concludes with case studies and real-world illustrations of HMMs in speech recognition, underscoring their adaptability and potency in this area.

Neuromorphic hardware tends to pose limits on the connectivity of deep networks that one can run on them. But also generic hardware and software implementations of deep learning run more efficiently on sparse networks. Several methods exist for pruning connections of a neural network after it was trained without connectivity constraints. We present an algorithm, DEEP R, that enables us to train directly a sparsely connected neural network. DEEP R automatically rewires the network during supervised training so that connections are there where they are most needed for the task, while its total number is all the time strictly bounded. We demonstrate that DEEP R can be used to train very sparse feedforward and recurrent neural networks on standard benchmark tasks with just a minor loss in performance. DEEP R is based on a rigorous theoretical foundation that views rewiring as stochastic sampling of network configurations from a posterior.

A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible (1) objective comparisons between solutions using alternative network architectures, (2) objective stopping rules for network pruning or growing procedures, (3) objective choice of magnitude and type of weight decay terms or additive regularizers (for penalizing large weights, etc.), (4) a measure of the effective number of well-determined parameters in a model, (5) quantified estimates of the error bars on network parameters and on network output, and (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian "evidence" automatically embodies "Occam's razor," penalizing overflexible and overcomplex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalization ability and the Bayesian evidence is obtained.

We consider the problem of estimating sparse graphs by a lasso penalty applied to the inverse covariance matrix. Using a coordinate descent procedure for the lasso, we develop a simple algorithm--the graphical lasso--that is remarkably fast: It solves a 1000-node problem ( approximately 500,000 parameters) in at most a minute and is 30-4000 times faster than competing methods. It also provides a conceptual link between the exact problem and the approximation suggested by Meinshausen and Bühlmann (2006). We illustrate the method on some cell-signaling data from proteomics.

Probabilistic soft logic (PSL) is a framework for collective, probabilistic reasoning in relational domains. PSL uses first order logic rules as a template language for graphical models over random variables with soft truth values from the interval [0, 1]. Inference in this setting is a continuous optimization task, which can be solved efficiently. This paper provides an overview of the PSL language and its techniques for inference and weight learning. An implementation of PSL is available at http://psl.umiacs.umd.edu/.

We address item relocation problems in graphs in this paper. We assume items placed in vertices of an undirected graph with at most one item per vertex. Items can be moved across edges while various constraints depending on the type of relocation problem must be satisfied. We introduce a general problem formulation that encompasses known types of item relocation problems such as multi-agent path finding (MAPF) and token swapping (TSWAP). In this formulation we express two new types of relocation problems derived from token swapping that we call token rotation (TROT) and token permutation (TPERM). Our solving approach for item relocation combines satisfiability modulo theory (SMT) with conflict-based search (CBS). We interpret CBS in the SMT framework where we start with the basic model and refine the model with a collision resolution constraint whenever a collision between items occurs in the current solution. The key difference between the standard CBS and our SMT-based modification of CBS (SMT-CBS) is that the standard CBS branches the search to resolve the collision while in SMT-CBS we iteratively add a single disjunctive collision resolution constraint. Experimental evaluation on several benchmarks shows that the SMT-CBS algorithm significantly outperforms the standard CBS. We also compared SMT-CBS with a modification of the SAT-based MDD-SAT solver that uses an eager modeling of item relocation in which all potential collisions are eliminated by constrains in advance. Experiments show that lazy approach in SMT-CBS produce fewer constraint than MDD-SAT and also achieves faster solving run-times.

This article provides next step towards solving speed bottleneck of any system that intensively uses convolutions operations (e.g. CNN). Method described in the article is applied on deformable part models (DPM) algorithm. Method described here is based on multidimensional tensors and provides efficient tradeoff between DPM performance and accuracy. Experiments on various databases, including Pascal VOC, show that the proposed method allows decreasing a number of convolutions up to 4.5 times compared with DPM v.5, while maintaining similar accuracy. If insignificant accuracy degradation is allowable, higher computational gain can be achieved. The method consists of filters tensor decomposition and convolutions shortening using the decomposed filter. Mathematical overview of the proposed method as well as simulation results are provided.

We describe a new algorithm GUESS-AND-CHECK for computing algebraic equation invariants. These invariants are of the form ∧ifi(x1, . . . , xn) = 0, where each fi is a polynomial over the variables x1, . . . , xn of the program. Two novel features of our algorithm are:(1) it is data driven, that is, invariants are derived from data generated from concrete executions of the program, and (2) it is sound and complete, that is, the algorithm terminates with the correct invariant if it exists. The GUESS-AND-CHECK algorithm proceeds iteratively in two phases. The “guess” phase uses linear algebra techniques to efficiently derive a candidate invariant from data. This candidate invariant is subsequently validated in a “check” phase. If the check phase fails to validate the candidate invariant, then the proof of invalidity is used to generate more data and this is used to find better candidate invariants in a subsequent guess phase. Therefore, GUESS-AND-CHECK iteratively invokes the guess and check phases until it finds the desired invariant. In terms of implementation, off-the-shelf linear algebra solvers are used to implement the guess phase, and off-the-shelf decision procedures are used to implement the check phase. We have evaluated our technique on a number of benchmark programs from recent papers on invariant generation. Our results are encouraging – we are able to efficiently compute algebraic invariants in all cases.

Sergei Arnautov, Bohdan Trach, Franz Gregor, Thomas Knauth, Andre Martin, Christian Priebe, Joshua Lind, Divya Muthukumaran, Dan O’Keeffe, Mark L Stillwell, David Goltzsche, David Eyers, Rüdiger Kapitza, Peter Pietzuch, and Christof Fetzer Fakultät Informatik, TU Dresden, christof.fetzer@tu-dresden.de Dept. of Computing, Imperial College London, prp@imperial.ac.uk Informatik, TU Braunschweig, rrkapitz@ibr.cs.tu-bs.de Dept. of Computer Science, University of Otago, dme@cs.otago.ac.nz

Large graphs involving millions of vertices are common in many practical applications and are challenging to process. Practical-time implementations using high-end computers are reported but are accessible only to a few. Graphics Processing Units (GPUs) of today have high computation power and low price. They have a restrictive programming model and are tricky to use. The G80 line of Nvidia GPUs can be treated as a SIMD processor array using the CUDA programming model. We present a few fundamental algorithms – including breadth first search, single source shortest path, and all-pairs shortest path – using CUDA on large graphs. We can compute the single source shortest path on a 10 million vertex graph in 1.5 seconds using the Nvidia 8800GTX GPU costing $600. In some cases optimal sequential algorithm is not the fastest on the GPU architecture. GPUs have great potential as high-performance co-processors.

The purpose of this paper is twofold. First, a brief exposition of the general backtrack technique and its history is given. Second, it is shown how the use of macros can considerably shorten the computation time in many cases. In particular, this technique has allowed the solution of two previously open combinatorial problems, the computation of new terms in a well-known series, and the substantial reduction in computation time for the solution to another combinatorial problem.


Question: WIR: Warp Instruction Reuse for Reducing Repetitive Computations in Graphics Processing Units

This paper introduces a novel solution, Criticality-Aware Warp Acceleration (CAWA), aimed at addressing the critical warp execution challenge in GPGPU workloads. Despite the efficiency of GPU architectures in handling parallel workloads through massive multi-threading and rapid context-switching, recent studies have revealed that GPGPU applications often face significant stall latencies, which are difficult to mitigate with the high concurrency of threads or warps. This leads to varying execution time disparities among parallel warps, negatively impacting overall GPU performance.
To combat this issue, CAWA offers a coordinated approach to efficiently allocate compute and memory resources for accelerating the execution of critical warps. This solution encompasses three key components: (1) A criticality predictor, which utilizes instruction and stall information to pinpoint the critical warp within a thread-block. (2) A criticality-aware warp scheduler, designed to prioritize the allocation of time resources to the critical warp, thereby optimizing its execution. (3) A criticality-aware cache reuse predictor, which aids in critical warp acceleration by maintaining latency-critical and useful cache blocks in the L1 data cache.
The primary objective of CAWA is to eliminate the significant execution time disparity among warps, thereby enhancing resource utilization in GPGPU workloads. Evaluation outcomes demonstrate that, under CAWA's coordinated scheduler and cache prioritization scheme, GPGPU workload performance can be enhanced by 23%, outperforming other state-of-the-art schedulers like GTO and 2-level schedulers, which offer improvements of 16% and -2%, respectively.

SIMD vectorization has received significant attention in the past decade as an important method to accelerate scientific applications, media and embedded applications on SIMD architectures such as Intel® SSE, AVX, and IBM* AltiVec. However, most of the focus has been directed at loops, effectively executing their iterations on multiple SIMD lanes concurrently relying upon program hints and compiler analysis. This paper presents a set of new C/C++ high-level vector extensions for SIMD programming, and the Intel® C++ product compiler that is extended to translate these vector extensions and produce optimized SIMD instruction sequences of vectorized functions and loops. For a function, our main idea is to vectorize the entire function for callers instead of just vectorizing loops (if any) inside the function. It poses the challenge of dealing with complicated control-flow in the function body, and matching caller and callee for SIMD vector calls while vectorizing caller functions (or loops) and callee functions. Our compilation methods for automatically compiling vector extensions are described. We present performance results of several non-trivial visual computing, computational, and simulation workloads, utilizing SIMD units through the vector extensions on Intel® Multicore 128-bit SIMD processors, and we show that significant SIMD speedups (3.07x to 4.69x) are achieved over the serial execution.

Search-based graph queries, such as finding short paths and isomorphic subgraphs, are dominated by memory latency. If input graphs can be partitioned appropriately, large cluster-based computing platforms can run these queries. However, the lack of compute-bound processing at each vertex of the input graph and the constant need to retrieve neighbors implies low processor utilization. Furthermore, graph classes such as scale-free social networks lack the locality to make partitioning clearly effective. Massive multithreading is an alternative architectural paradigm, in which a large shared memory is combined with processors that have extra hardware to support many thread contexts. The processor speed is typically slower than normal, and there is no data cache. Rather than mitigating memory latency, multithreaded machines tolerate it. This paradigm is well aligned with the problem of graph search, as the high ratio of memory requests to computation can be tolerated via multithreading. In this paper, we introduce the multithreaded graph library (MTGL), generic graph query software for processing semantic graphs on multithreaded computers. This library currently runs on serial machines and the Cray MTA-2, but Sandia is developing a run-time system that will make it possible to run MTGL-based code on symmetric multiprocessors. We also introduce a multithreaded algorithm for connected components and a new heuristic for inexact subgraph isomorphism We explore the performance of these and other basic graph algorithms on large scale-free graphs. We conclude with a performance comparison between the Cray MTA-2 and Blue Gene/Light for s-t connectivity.

Influence Maximization aims to find the top-$(K)$ influential individuals to maximize the influence spread within a social network, which remains an important yet challenging problem. Proven to be NP-hard, the influence maximization problem attracts tremendous studies. Though there exist basic greedy algorithms which may provide good approximation to optimal result, they mainly suffer from low computational efficiency and excessively long execution time, limiting the application to large-scale social networks. In this paper, we present IMGPU, a novel framework to accelerate the influence maximization by leveraging the parallel processing capability of graphics processing unit (GPU). We first improve the existing greedy algorithms and design a bottom-up traversal algorithm with GPU implementation, which contains inherent parallelism. To best fit the proposed influence maximization algorithm with the GPU architecture, we further develop an adaptive K-level combination method to maximize the parallelism and reorganize the influence graph to minimize the potential divergence. We carry out comprehensive experiments with both real-world and sythetic social network traces and demonstrate that with IMGPU framework, we are able to outperform the state-of-the-art influence maximization algorithm up to a factor of 60, and show potential to scale up to extraordinarily large-scale networks.

Accelerating program performance via SIMD vector units is very common in modern processors, as evidenced by the use of SSE, MMX, VSE, and VSX SIMD instructions in multimedia, scientific, and embedded applications. To take full advantage of the vector capabilities, a compiler needs to generate efficient vector code automatically. However, most commercial and open-source compilers fall short of using the full potential of vector units, and only generate vector code for simple innermost loops. In this paper, we present the design and implementation of anauto-vectorization framework in the back-end of a dynamic compiler that not only generates optimized vector code but is also well integrated with the instruction scheduler and register allocator. The framework includes a novel{\em compile-time efficient dynamic programming-based} vector instruction selection algorithm for straight-line code that expands opportunities for vectorization in the following ways: (1) {\em scalar packing} explores opportunities of packing multiple scalar variables into short vectors, (2)judicious use of {\em shuffle} and {\em horizontal} vector operations, when possible, and (3) {\em algebraic reassociation} expands opportunities for vectorization by algebraic simplification. We report performance results on the impact of auto-vectorization on a set of standard numerical benchmarks using the Jikes RVM dynamic compilation environment. Our results show performance improvement of up to 57.71\% on an Intel Xeon processor, compared tonon-vectorized execution, with a modest increase in compile-time in the range from 0.87\% to 9.992\%. An investigation of the SIMD parallelization performed by v11.1 of the Intel Fortran Compiler (IFC) on three benchmarks shows that our system achieves speedup with vectorization in all three cases and IFC does not. Finally, a comparison of our approach with an implementation of the Super word Level Parallelization (SLP) algorithm from~\cite{larsen00}, shows that our approach yields a performance improvement of up to 13.78\% relative to SLP.

The state-of-the-art ARM processors provide multiple cores and SIMD instructions. OpenCL is a promising programming model for utilizing such parallel processing capability because of its SPMD programming model and built-in vector support. Moreover, it provides portability between multicore ARM processors and accelerators in embedded systems. In this paper, we introduce the design and implementation of an efficient OpenCL framework for multicore ARM processors. Computational tasks in a program are implemented as OpenCL kernels and run on all CPU cores in parallel by our OpenCL framework. Vector operations and built-in functions in OpenCL kernels are optimized using the NEON SIMD instruction set. We evaluate our OpenCL framework using 37 benchmark applications. The result shows that our approach is effective and promising.

One of the key contributions of this paper is the idea of treating the two-dimensional FPGA area as a one-dimensional array of custom instructions, which the paper refers to as linear hardware space. Each custom instruction occupies the full width of the array but can occupy different heights to allow for instructions of varying complexity. Instructions also have a common interface that includes pass-thrus which connect to the previous and next instructions in the array. This communication interface greatly simplifies the problem of relocating a custom instruction to a free area, allowing other instructions to potentially remain in-place. The communication paths for an instruction are always at the same relative positions, and run-time CAD operations (and their corresponding overheads) are completely avoided. Thus, the linear hardware space reduces the overheads and simplifies the process of run-time reconfiguration of custom processor instructions.

Compilers for vector or multiprocessor computers must have certain optimization features to successfully generate parallel code.

The main contribution of this paper is to show optimal parallel algorithms to compute the sum, the prefix-sums, and the summed area table on two memory machine models, the Discrete Memory Machine (DMM) and the Unified Memory Machine (UMM). The DMM and the UMM are theoretical parallel computing models that capture the essence of the shared memory and the global memory of GPUs. These models have three parameters, the number p of threads, and the width w of the memory, and the memory access latency l. We first show that the sum of n numbers can be computed in O(n w + nl p + l log n) time units on the DMM and the UMM. We then go on to show that Ω(n w + nl p + l log n) time units are necessary to compute the sum. We also present a parallel algorithm that computes the prefix-sums of n numbers in O( n w + nl p + l log n) time units on the DMM and the UMM. Finally, we show that the summed area table of size √ n × √n can be computed in O( n w + nl p + l log n) time units on the DMM and the UMM. Since the computation of the prefix-sums and the summed area table is at least as hard as the sum computation, these parallel algorithms are also optimal. key words: Memory machine models, prefix-sums computation, parallel algorithm, GPU, CUDA

Computers with multiple processor cores using shared memory are now ubiquitous. In this paper, we present several parallel geometric algorithms that specifically target this environment, with the goal of exploiting the additional computing power. The d-dimensional algorithms we describe are (a) spatial sorting of points, as is typically used for preprocessing before using incremental algorithms, (b) kd-tree construction, (c) axis-aligned box intersection computation, and finally (d) bulk insertion of points in Delaunay triangulations for mesh generation algorithms or simply computing Delaunay triangulations. We show experimental results for these algorithms in 3D, using our implementations based on the Computational Geometry Algorithms Library (CGAL, http://www.cgal.org/). This work is a step towards what we hope will become a parallel mode for CGAL, where algorithms automatically use the available parallel resources without requiring significant user intervention.

The use of graphical processors for distributed computation revolutionized the field of high performance scientific computing. As the Moore's Law era of computing draws to a close, the development of non-Von Neumann systems: neuromorphic processing units, and quantum annealers; again are redefining new territory for computational methods. While these technologies are still in their nascent stages, we discuss their potential to advance computing in two domains: machine learning, and solving constraint satisfaction problems. Each of these processors utilize fundamentally different theoretical models of computation. This raises questions about how to best use them in the design and implementation of applications. While many processors are being developed with a specific domain target, the ubiquity of spin-glass models and neural networks provides an avenue for multi-functional applications. This provides hints at the future infrastructure needed to integrate many next-generation processing units into conventional high-performance computing systems.