Question: A Riesz fractional model-based method for improving license plate detection and recognition.

This paper presents an innovative license plate recognition algorithm tailored for intelligent transportation systems, incorporating an advanced adaptive image segmentation approach (employing sliding concentric windows) alongside connected component analysis, complemented by a neural network for character recognition. The algorithm's effectiveness was evaluated using a dataset of 1,334 gray-scale vehicle images captured in natural settings, featuring diverse backgrounds and varying illumination conditions. The camera was positioned to focus on the license plate area, with the angle of view and distance adjusted according to the experimental setup. The segmentation process accurately isolated 1,287 license plates from the total input images (96.5%). The character recognition component utilizes a probabilistic neural network (PNN) with a 108-180-36 topology, achieving a recognition accuracy of 89.1% for complete plate identification. The PNN is specifically trained to discern alphanumeric characters from license plates, utilizing data derived from advanced image processing techniques. When combining the segmentation and recognition rates, the overall success rate for the license plate recognition algorithm stands at 86.0%. A comparative analysis with existing literature indicates that superior performance (ranging from 90% to 95%) has been reported under more controlled conditions, such as limited distance, restricted viewing angles, uniform illumination, and simpler backgrounds.

Running races, such as marathons, are broadly covered by professional as well as amateur photographers. This leads to a constantly growing number of photos covering a race, making the process of identifying a particular runner in such datasets difficult. Today, such identification is often done manually. In running races, each competitor has an identification number, called the Racing Bib Number (RBN), used to identify that competitor during the race. RBNs are usually printed on a paper or cardboard tag and pined onto the competitor’s T-shirt during the race. We introduce an automatic system that receives a set of natural images taken in running sports events and outputs the participants’ RBN.

Most current approaches to recognition aim to be scaleinvariant. However, the cues available for recognizing a 300 pixel tall object are qualitatively different from those for recognizing a 3 pixel tall object. We argue that for sensors with finite resolution, one should instead use scale-variant, or multiresolution representations that adapt in complexity to the size of a putative detection window. We describe a multiresolution model that acts as a deformable part-based model when scoring large instances and a rigid template with scoring small instances. We also examine the interplay of resolution and context, and demonstrate that context is most helpful for detecting low-resolution instances when local models are limited in discriminative power. We demonstrate impressive results on the Caltech Pedestrian benchmark, which contains object instances at a wide range of scales. Whereas recent state-of-theart methods demonstrate missed detection rates of 86%-37% at 1 falsepositive-per-image, our multiresolution model reduces the rate to 29%.

Image resolution can be improved when the relative displacements in image sequences are known accurately, and some knowledge of the imaging process is available. The proposed approach is similar to back-projection used in tomography. Examples of improved image resolution are given for gray-level and color images, when the unknown image displacements are computed from the image sequence.

Recognizing arbitrary multi-character text in unconstrained natural photographs is a hard problem. In this paper, we address an equally hard sub-problem in this domain viz. recognizing arbitrary multi-digit numbers from Street View imagery. Traditional approaches to solve this problem typically separate out the localization, segmentation, and recognition steps. In this paper we propose a unified approach that integrates these three steps via the use of a deep convolutional neural network that operates directly on the image pixels. We employ the DistBelief (Dean et al., 2012) implementation of deep neural networks in order to train large, distributed neural networks on high quality images. We find that the performance of this approach increases with the depth of the convolutional network, with the best performance occurring in the deepest architecture we trained, with eleven hidden layers. We evaluate this approach on the publicly available SVHN dataset and achieve over 96% accuracy in recognizing complete street numbers. We show that on a per-digit recognition task, we improve upon the state-of-the-art and achieve 97.84% accuracy. We also evaluate this approach on an even more challenging dataset generated from Street View imagery containing several tens of millions of street number annotations and achieve over 90% accuracy. Our evaluations further indicate that at specific operating thresholds, the performance of the proposed system is comparable to that of human operators. To date, our system has helped us extract close to 100 million physical street numbers from Street View imagery worldwide.

We propose and experimentally evaluate a software solution for automatic detection and classification of plant leaf diseases. The proposed solution is an improvement to the solution proposed in [1] as it provides faster and more accurate solution. The developed processing scheme consists of four main phases as in [1]. The following two steps are added successively after the segmentation phase. In the first step we identify the mostlygreen colored pixels. Next, these pixels are masked based on specific threshold values that are computed using Otsu's method, then those mostly green pixels are masked. The other additional step is that the pixels with zeros red, green and blue values and the pixels on the boundaries of the infected cluster (object) were completely removed. The experimental results demonstrate that the proposed technique is a robust technique for the detection of plant leaves diseases. The developed algorithm‟s efficiency can successfully detect and classify the examined diseases with a precision between 83% and 94%, and can achieve 20% speedup over the approach proposed in [1].

Moving vehicles are detected and tracked automatically in monocular image sequences from road traffic scenes recorded by a stationary camera. In order to exploit the a priori knowledge about shape and motion of vehicles in traffic scenes, a parameterized vehicle model is used for an intraframe matching process and a recursive estimator based on a motion model is used for motion estimation. An interpretation cycle supports the intraframe matching process with a state MAP-update step. Initial model hypotheses are generated using an image segmentation component which clusters coherently moving image features into candidate representations of images of a moving vehicle. The inclusion of an illumination model allows taking shadow edges of the vehicle into account during the matching process. Only such an elaborate combination of various techniques has enabled us to track vehicles under complex illumination conditions and over long (over 400 frames) monocular image sequences. Results on various real-world road traffic scenes are presented and open problems as well as future work are outlined.

This paper aims at real-time in-car video analysis to detect and track vehicles ahead for safety, autodriving, and target tracing. This paper describes a comprehensive approach to localizing target vehicles in video under various environmental conditions. The extracted geometry features from the video are continuously projected onto a 1-D profile and are constantly tracked. We rely on temporal information of features and their motion behaviors for vehicle identification, which compensates for the complexity in recognizing vehicle shapes, colors, and types. We probabilistically model the motion in the field of view according to the scene characteristic and the vehicle motion model. The hidden Markov model (HMM) is used to separate target vehicles from the background and track them probabilistically. We have investigated videos of day and night on different types of roads, showing that our approach is robust and effective in dealing with changes in environment and illumination and that real-time processing becomes possible for vehicle-borne cameras.

In this article, the three most used object detection approaches, Linear Binary Pattern cascade, Haarlike cascade, and Histogram of Oriented Gradients with Support Vector Machine are applied to automatic runway detection in high resolution satellite imagery and their results are compared. They have been employed predominantly for human feature recognition and this paper tests theirs applicability to runways. The results show that they can be indeed employed for this purpose with LBP and Haar approaches performing better than HOG+SVM.

Transport-based techniques for signal and data analysis have recently received increased interest. Given their ability to provide accurate generative models for signal intensities and other data distributions, they have been used in a variety of applications, including content-based retrieval, cancer detection, image superresolution, and statistical machine learning, to name a few, and they have been shown to produce state-of-the-art results. Moreover, the geometric characteristics of transport-related metrics have inspired new kinds of algorithms for interpreting the meaning of data distributions. Here, we provide a practical overview of the mathematical underpinnings of mass transport-related methods, including numerical implementation, as well as a review, with demonstrations, of several applications. Software accompanying this article is available from [43].

End-to-end Optical Character Recognition (OCR) systems are heavily used to convert document images into machine-readable text. Commercial and open-source OCR systems (like Abbyy, OCRopus, Tesseract etc.) have traditionally been optimized for contemporary documents like books, letters, memos, and other end-user documents. However, these systems are difficult to use equally well for digitizing historical document images, which contain degradations like non-uniform shading, bleed-through, and irregular layout; such degradations usually do not exist in contemporary document images.
 The open-source anyOCR is an end-to-end OCR pipeline, which contains state-of-the-art techniques that are required for digitizing degraded historical archives with high accuracy. However, high accuracy comes at a cost of high computational complexity that results in 1) long runtime that limits digitization of big collection of historical archives and 2) high energy consumption that is the most critical limiting factor for portable devices with constrained energy budget. Therefore, we are targeting energy efficient and high throughput acceleration of the anyOCR pipeline. Generalpurpose computing platforms fail to meet these requirements that makes custom hardware design mandatory. In this paper, we are presenting a new concept named iDocChip. It is a portable hybrid hardware-software FPGA-based accelerator that is characterized by low footprint meaning small size, high power efficiency that will allow using it in portable devices, and high throughput that will make it possible to process big collection of historical archives in real time without effecting the accuracy.
 In this paper, we focus on binarization, which is the second most critical step in the anyOCR pipeline after text-line recognizer that we have already presented in our previous publication [21]. The anyOCR system makes use of a Percentile Based Binarization method that is suitable for overcoming degradations like non-uniform shading and bleed-through. To the best of our knowledge, we propose the first hardware architecture of the PBB technique. Based on the new architecture, we present a hybrid hardware-software FPGA-based accelerator that outperforms the existing anyOCR software implementation running on i7-4790T in terms of runtime by factor of 21, while achieving energy efficiency of 10 Images/J that is higher than that achieved by low power embedded processors with negligible loss of recognition accuracy.


Question: A Riesz fractional model-based enhancement for license plate detection and recognition.

This paper explores the application of fractional differential equations within the context of Banach spaces, focusing specifically on impulsive fractional differential equations. By employing the Leray-Schauder fixed point theorem and the impulsive nonlinear singular version of the Gronwall inequality, the authors establish conditions under which PC-mild solutions for such equations can exist. These conditions are designed to be straightforward to verify. To substantiate the theoretical findings, a practical example is provided to illustrate the existence of solutions for impulsive fractional differential equations in Banach spaces.

**Question**: A Deep Learning Framework for Multi-Objective Optimization in Cloud Computing

**Documents**: A Multi-Objective Genetic Algorithm for Task Scheduling in Cloud Computing
In this paper, a multi-objective genetic algorithm (MOGA) is proposed for the task scheduling problem in cloud computing. The proposed algorithm aims to minimize the makespan and the cost simultaneously. In MOGA, each chromosome represents a possible solution to the task scheduling problem, and the fitness function evaluates the solution based on the makespan and cost. The proposed algorithm is compared with other algorithms such as NSGA-II and SPEA2 on standard benchmark problems. The results show that MOGA outperforms the other algorithms in terms of the Pareto front approximation.

**Similar Documents**: This paper presents a novel multi-objective genetic algorithm (MOGA) tailored for addressing task scheduling issues in cloud computing environments. The primary objectives of this algorithm are to reduce the makespan and minimize the overall cost of operations. In MOGA, a chromosome represents a potential solution to the scheduling problem, and its fitness is assessed based on both the makespan and the cost. To evaluate the performance of MOGA, it is benchmarked against established algorithms like NSGA-II and SPEA2. Comparative analysis reveals that MOGA surpasses these algorithms in terms of approximating the Pareto front, demonstrating its effectiveness in multi-objective optimization for cloud computing task scheduling.

A novel method is proposed in this paper to improve the recognition accuracy of Local Binary Pattern (LBP) on low-resolution face recognition. More precise descriptors and e®ectively face features can be extracted by combining multi-scale blocking center symmetric local binary pattern (CS-LBP) based on Gaussian pyramids and weighted principal component analysis (PCA) on low-resolution condition. Firstly, the features statistical histograms of face images are calculated by multi-scale blocking CS-LBP operator. Secondly, the stronger classi ̄cation and lower dimension features can be got by applying weighted PCA algorithm. Finally, the di®erent classi ̄ers are used to select the optimal classi ̄cation categories of low-resolution face set and calculate the recognition rate. The results in the ORL human face databases show that recognition rate can get 89.38% when the resolution of face image drops to 12 10 pixel and basically satisfy the practical requirements of recognition. The further comparison of other descriptors and experiments from videos proved that the novel algorithm can improve recognition accuracy.

Wavelet frame systems are known to be effective in capturing singularities from noisy and degraded images. In this paper, we introduce a new edge driven wavelet frame model for image restoration by approximating images as piecewise smooth functions. With an implicit representation of image singularities sets, the proposed model inflicts different strength of regularization on smooth and singular image regions and edges. The proposed edge driven model is robust to both image approximation and singularity estimation. The implicit formulation also enables an asymptotic analysis of the proposed models and a rigorous connection between the discrete model and a general continuous variational model. Finally, numerical results on image inpainting and deblurring show that the proposed model is compared favorably against several popular image restoration models.

Contrast enhancement of radiographies based on a multiscale decomposition of the images recently has proven to be a far more versatile and efficient method than regular unsharp-masking techniques, while containing these as a subset. In this paper, we compare the performance of two multiscale-methods, namely the Laplacian Pyramid and the fast wavelet transform (FWT). We find that enhancement based on the FWT suffers from one serious drawback-the introduction of visible artifacts when large structures are enhanced strongly. By contrast, the Laplacian Pyramid allows a smooth enhancement of large structures, such that visible artifacts can be avoided. Only for the enhancement of very small details, for denoising applications or compression of images, the FWT may have some advantages over the Laplacian Pyramid.

In this paper, we propose a computer vision based system for real-time robust traffic sign detection and recognition, especially developed for intelligent vehicle. In detection phase, a color-based segmentation method is used to scan the scene in order to quickly establish regions of interest (ROI). Sign candidates within ROIs are detected by a set of Haar wavelet features obtained from AdaBoost training. Then, the Speeded Up Robust Features (SURF) is applied for the sign recognition. SURF finds local invariant features in a candidate sign and matches these features to the features of template images that exist in data set. The recognition is performed by finding out the template image that gives the maximum number of matches. We have evaluated the proposed system on our intelligent vehicle SmartVII. A recognition accuracy of over 90% in real-time has been achieved.

In this paper, we propose a novel explicit image filter called guided filter. Derived from a local linear model, the guided filter computes the filtering output by considering the content of a guidance image, which can be the input image itself or another different image. The guided filter can be used as an edge-preserving smoothing operator like the popular bilateral filter [1], but it has better behaviors near edges. The guided filter is also a more generic concept beyond smoothing: It can transfer the structures of the guidance image to the filtering output, enabling new filtering applications like dehazing and guided feathering. Moreover, the guided filter naturally has a fast and nonapproximate linear time algorithm, regardless of the kernel size and the intensity range. Currently, it is one of the fastest edge-preserving filters. Experiments show that the guided filter is both effective and efficient in a great variety of computer vision and computer graphics applications, including edge-aware smoothing, detail enhancement, HDR compression, image matting/feathering, dehazing, joint upsampling, etc.

Advanced Driver Assistance Systems (ADAS) refer to various high-tech in-vehicle systems that are designed to increase road traffic safety by helping drivers gain better awareness of the road and its potential hazards as well as other drivers around them. The design of traffic sign recognition, one important subsystem of ADAS, has been a challenge problem for many years and hence become an important and active research topic in the area of intelligent transport systems. The realization of a real-time traffic sign recognition system is usually divided into three stages: detection, tracking and classification. This paper introduces the main difficulties in road sign recognition and briefly surveys the state-of-the-art technologies in this field with further discussions on the potential trend of development of road sign recognition.

This paper introduces novel hiding techniques to counter the detection of image manipulations through forensic analyses. The presented techniques allow to resize and rotate (parts of) bitmap images without leaving a periodic pattern in the local linear predictor coefficients, which has been exploited by prior art to detect traces of manipulation. A quantitative evaluation on a batch of test images proves the proposed method’s efficacy, while controlling for key parameters and for the retained image quality compared to conventional linear interpolation.

Updating road markings is one of the routine tasks of transportation agencies. Compared with traditional road inventory mapping techniques, vehicle-borne mobile light detection and ranging (LiDAR) systems can undertake the job safely and efficiently. However, current hurdles include software and computing challenges when handling huge volumes of highly dense and irregularly distributed 3-D mobile LiDAR point clouds. This paper presents the development and implementation aspects of an automated object extraction strategy for rapid and accurate road marking inventory. The proposed road marking extraction method is based on 2-D georeferenced feature (GRF) images, which are interpolated from 3-D road surface points through a modified inverse distance weighted (IDW) interpolation. Weighted neighboring difference histogram (WNDH)-based dynamic thresholding and multiscale tensor voting (MSTV) are proposed to segment and extract road markings from the noisy corrupted GRF images. The results obtained using 3-D point clouds acquired by a RIEGL VMX-450 mobile LiDAR system in a subtropical urban environment are encouraging.

This paper describes a method for detecting and recognizing road signs in gray-level and color images acquired by a single camera mounted on a moving vehicle. The method works in three stages. First, the search for the road sign is reduced to a suitable region of the image by using some a priori knowledge on the scene or color clues (when available). Secondly, a geometrical analysis of the edges extracted from the image is carried out, which generates candidates to be circular and triangular signs. Thirdly, a recognition stage tests by cross-correlation techniques each candidate which, if validated, is classi ed according to the data-base of signs. An extensive experimentation has shown that the method is robust against low-level noise corrupting edge detection and contour following, and works for images of cluttered urban streets as well as country roads and highways. A further improvement on the detection and recognition scheme has been obtained by means of temporal integration based on Kalman ltering methods of the extracted information. The proposed approach can be very helpful for the development of a system for driving assistance.

We approach recognition in the framework of deformable shape matching, relying on a new algorithm for finding correspondences between feature points. This algorithm sets up correspondence as an integer quadratic programming problem, where the cost function has terms based on similarity of corresponding geometric blur point descriptors as well as the geometric distortion between pairs of corresponding feature points. The algorithm handles outliers, and thus enables matching of exemplars to query images in the presence of occlusion and clutter. Given the correspondences, we estimate an aligning transform, typically a regularized thin plate spline, resulting in a dense correspondence between the two shapes. Object recognition is then handled in a nearest neighbor framework where the distance between exemplar and query is the matching cost between corresponding points. We show results on two datasets. One is the Caltech 101 dataset (Fei-Fei, Fergus and Perona), an extremely challenging dataset with large intraclass variation. Our approach yields a 48% correct classification rate, compared to Fei-Fei et al 's 16%. We also show results for localizing frontal and profile faces that are comparable to special purpose approaches tuned to faces.


Question: A Riesz fractional model-based method for improving license plate detection and recognition.

A Novel Approach for Vehicle License Plate Recognition Utilizing Character-specific Extremal Regions and Hybrid Discriminative Restricted Boltzmann Machines

In this paper, we propose an innovative method for recognizing vehicle license plates, combining the strengths of character-specific extremal regions (ERs) and hybrid discriminative restricted Boltzmann machines (HDRBMs). The process begins with a preliminary license plate detection (LPD) phase, employing top-hat transformation, vertical edge detection, and morphological operations, followed by a series of validations to ensure accuracy. Subsequently, character-specific ERs are identified within the potential license plate areas. A rigorous selection process for ERs is then conducted, enabling simultaneous character segmentation and a refinement of the initial LPD. The final step involves the application of an HDRBM pattern classifier, pre-trained offline, to accurately identify the characters. The proposed methodology is notably resilient to variations in lighting and weather conditions throughout the day, as demonstrated through extensive testing on comprehensive data sets, confirming its efficacy in challenging traffic environments.

Wavelet frame systems are known to be effective in capturing singularities from noisy and degraded images. In this paper, we introduce a new edge driven wavelet frame model for image restoration by approximating images as piecewise smooth functions. With an implicit representation of image singularities sets, the proposed model inflicts different strength of regularization on smooth and singular image regions and edges. The proposed edge driven model is robust to both image approximation and singularity estimation. The implicit formulation also enables an asymptotic analysis of the proposed models and a rigorous connection between the discrete model and a general continuous variational model. Finally, numerical results on image inpainting and deblurring show that the proposed model is compared favorably against several popular image restoration models.

Video surveillance is active research topic in computer vision research area for humans & vehicles, so it is used over a great extent. Multiple images generated using a fixed camera contains various objects, which are taken under different variations, illumination changes after that the object’s identity and orientation are provided to the user. This scheme is used to represent individual images as well as various objects classes in a single, scale and rotation invariant model.The objective is to improve object recognition accuracy for surveillance purposes & to detect multiple objects with sufficient level of scale invariance.Multiple objects detection& recognition is important in the analysis of video data and higher level security system. This method can efficiently detect the objects from query images as well as videos by extracting frames one by one. When given a query image at runtime, by generating the set of query features and it will find best match it to other sets within the database. Using SURF algorithm find the database object with the best feature matching, then object is present in the query image. Keywords— Image recognition, Query image, Local feature, Surveillance system, SURF algorithm.

We propose and experimentally evaluate a software solution for automatic detection and classification of plant leaf diseases. The proposed solution is an improvement to the solution proposed in [1] as it provides faster and more accurate solution. The developed processing scheme consists of four main phases as in [1]. The following two steps are added successively after the segmentation phase. In the first step we identify the mostlygreen colored pixels. Next, these pixels are masked based on specific threshold values that are computed using Otsu's method, then those mostly green pixels are masked. The other additional step is that the pixels with zeros red, green and blue values and the pixels on the boundaries of the infected cluster (object) were completely removed. The experimental results demonstrate that the proposed technique is a robust technique for the detection of plant leaves diseases. The developed algorithm‟s efficiency can successfully detect and classify the examined diseases with a precision between 83% and 94%, and can achieve 20% speedup over the approach proposed in [1].

Deep learning has been widely recognized as a promising approach in different computer vision applications. Specifically, one-stage object detector and two-stage object detector are regarded as the most important two groups of Convolutional Neural Network based object detection methods. One-stage object detector could usually outperform two-stage object detector in speed; However, it normally trails in detection accuracy, compared with two-stage object detectors. In this study, focal loss based RetinaNet, which works as one-stage object detector, is utilized to be able to well match the speed of regular one-stage detectors and also defeat two-stage detectors in accuracy, for vehicle detection. State-of-the-art performance result has been showed on the DETRAC vehicle dataset.

Threshold based segmentation methods can effectively separate objects from the background. The conventional Otsu's method can achieve good result only when the histogram of an image has two distinct peaks. In this paper, an iterative thresholding based on 2D improved Otsu method using a novel threshold value recognition function is proposed which is used to find the optimum threshold value in different types of histograms and separate into two classes. For the two classes separated by the above threshold value, mean values are computed. Based on the threshold value and the two mean values, the histogram of an image is divided iteratively into three regions, namely, the Foreground (FG) region, Background (BG) region and To-Be-Processed (TBP) region. The pixels greater than the larger mean belongs to FG region, the pixel which are lesser than the smaller mean belongs to BG region. The pixel that lies between the two mean values belongs to the TBP region and only this region is processed at next iteration, the FG and BG region are not processed further. Again for that TBP region, the novel threshold value recognition function is used to find a new threshold value and then the two mean values are computed to separate into three classes, namely, the FG, BG and TBP region. Then the new TBP region is processed in a similar manner iteratively until the two mean values computed at successive iterations are not changed. Finally all the previously determined foreground regions are combined separately and the background regions are combined separately to give the segmented image. The experimental results show that the proposed method performs well in detecting the objects and has better anti-noise performance compared with the existing methods.

Pedestrian detection is a rapidly evolving area in computer vision with key applications in intelligent vehicles, surveillance, and advanced robotics. The objective of this paper is to provide an overview of the current state of the art from both methodological and experimental perspectives. The first part of the paper consists of a survey. We cover the main components of a pedestrian detection system and the underlying models. The second (and larger) part of the paper contains a corresponding experimental study. We consider a diverse set of state-of-the-art systems: wavelet-based AdaBoost cascade, HOG/linSVM, NN/LRF, and combined shape-texture detection. Experiments are performed on an extensive data set captured onboard a vehicle driving through urban environment. The data set includes many thousands of training samples as well as a 27-minute test sequence involving more than 20,000 images with annotated pedestrian locations. We consider a generic evaluation setting and one specific to pedestrian detection onboard a vehicle. Results indicate a clear advantage of HOG/linSVM at higher image resolutions and lower processing speeds, and a superiority of the wavelet-based AdaBoost cascade approach at lower image resolutions and (near) real-time processing speeds. The data set (8.5 GB) is made public for benchmarking purposes.

On-road vehicle detection and rear-end crash prevention are demanding subjects in both academia and automotive industry. The paper focuses on monocular vision-based vehicle detection under challenging lighting conditions, being still an open topic in the area of driver assistance systems. The paper proposes an effective vehicle detection method based on multiple features analysis and Dempster-Shafer-based fusion theory. We also utilize a new idea of Adaptive Global Haar-like (AGHaar) features as a promising method for feature classification and vehicle detection in both daylight and night conditions. Validation tests and experimental results show superior detection results for day, night, rainy, and challenging conditions compared to state-of-the-art solutions.

The “German Traffic Sign Recognition Benchmark” is a multi-category classification competition held at IJCNN 2011. Automatic recognition of traffic signs is required in advanced driver assistance systems and constitutes a challenging real-world computer vision and pattern recognition problem. A comprehensive, lifelike dataset of more than 50,000 traffic sign images has been collected. It reflects the strong variations in visual appearance of signs due to distance, illumination, weather conditions, partial occlusions, and rotations. The images are complemented by several precomputed feature sets to allow for applying machine learning algorithms without background knowledge in image processing. The dataset comprises 43 classes with unbalanced class frequencies. Participants have to classify two test sets of more than 12,500 images each. Here, the results on the first of these sets, which was used in the first evaluation stage of the two-fold challenge, are reported. The methods employed by the participants who achieved the best results are briefly described and compared to human traffic sign recognition performance and baseline results.

With the proliferation of cameras on mobile devices there is an increased desire to image document pages as an alternative to scanning. However, the quality of captured document images is often lower than its scanned equivalent due to hardware limitations and stability issues. In this context, automatic assessment of the quality of captured images is useful for many applications. Although there has been a lot of work on developing computational methods and creating standard datasets for natural scene image quality assessment, until recently quality estimation of camera captured document images has not been given much attention. One traditional quality indicator for document images is the Optical Character Recognition (OCR) accuracy. In this work, we present a dataset of camera captured document images containing varying levels of focal-blur introduced manually during capture. For each image we obtained the character level OCR accuracy. Our dataset can be used to evaluate methods for predicting OCR quality of captured documents as well as enhancements. In order to make the dataset publicly and freely available, originals from two existing datasets University of Washington dataset and Tobacco Database were selected. We present a case study with three recent methods for predicting the OCR quality of images on our dataset.

Many computer vision algorithms rely on the assumptions of the pinhole camera model, but lens distortion with off-the-shelf cameras is usually significant enough to violate this assumption. Many methods for radial distortion estimation have been proposed, but they all have limitations. Robust automatic radial distortion estimation from a single natural image would be extremely useful for many applications, particularly those in human-made environments containing abundant lines. For example, it could be used in place of an extensive calibration procedure to get a mobile robot or quadrotor experiment up and running quickly in an indoor environment. We propose a new method for automatic radial distortion estimation based on the plumb-line approach. The method works from a single image and does not require a special calibration pattern. It is based on Fitzgibbon’s division model, robust estimation of circular arcs, and robust estimation of distortion parameters. We perform an extensive empirical study of the method on synthetic images. We include a comparative statistical analysis of how different circle fitting methods contribute to accurate distortion parameter estimation. We finally provide qualitative results on a wide variety of challenging real images. The experiments demonstrate the method’s ability to accurately identify distortion parameters and remove distortion from images.


Question: A Riesz fractional model-based enhancement for license plate detection and recognition.

This paper delves into the evaluation of feature sets for robust visual object recognition, with a specific focus on human detection using linear SVM-based methods. Through a comprehensive analysis of existing edge and gradient-based descriptors, we demonstrate that a grid of Histograms of Oriented Gradient (HOG) descriptors outperforms conventional feature sets significantly in the context of human detection. Our experimental findings reveal that the key factors contributing to superior performance include the use of fine-scale gradients, precise orientation binning, relatively large spatial binning, and the implementation of high-quality local contrast normalization within overlapping descriptor blocks. The HOG descriptor approach yields near-perfect detection accuracy on the original MIT pedestrian database. To further validate the robustness and versatility of our method, we introduce a more demanding dataset consisting of over 1800 annotated human images, encompassing a broad spectrum of poses and diverse backgrounds. This expanded dataset allows for a more rigorous evaluation of the HOG descriptor's effectiveness under various real-world conditions.

In this paper, we propose a novel explicit image filter called guided filter. Derived from a local linear model, the guided filter computes the filtering output by considering the content of a guidance image, which can be the input image itself or another different image. The guided filter can be used as an edge-preserving smoothing operator like the popular bilateral filter [1], but it has better behaviors near edges. The guided filter is also a more generic concept beyond smoothing: It can transfer the structures of the guidance image to the filtering output, enabling new filtering applications like dehazing and guided feathering. Moreover, the guided filter naturally has a fast and nonapproximate linear time algorithm, regardless of the kernel size and the intensity range. Currently, it is one of the fastest edge-preserving filters. Experiments show that the guided filter is both effective and efficient in a great variety of computer vision and computer graphics applications, including edge-aware smoothing, detail enhancement, HDR compression, image matting/feathering, dehazing, joint upsampling, etc.

PRNU-based techniques guarantee a good forgery detection performance irrespective of the specific type of forgery. The presence or absence of the camera PRNU pattern is detected by a correlation test. Given the very low power of the PRNU signal, however, the correlation must be averaged over a pretty large window, reducing the algorithm's ability to reveal small forgeries. To improve resolution, we estimate correlation with a spatially adaptive filtering technique, with weights computed over a suitable pilot image. Implementation efficiency is achieved by resorting to the recently proposed guided filters. Experiments prove that the proposed filtering strategy allows for a much better detection performance in the case of small forgeries.

● Super resolution enhances the resolution of a low-res image. ● Many deep learning models today work fairly well with an upscaling factor of 4x but use only the downscaled image as input. ● We demonstrate two class-conditional GAN models that outperform state-of-the-art GANs for superresolution: ○ Explicitly passes in a known class parameter as input ○ Uses a separately trained classifier to inform a ResNet about character attributes ● Both improve the quality of the super-resolved output.

In recent years, QR codes have found a wide range of applications in business and industry. Moreover, their popularity and function are still growing (fast). Therefore, the processing of the images of QR codes is an important task. In this paper, we try to solve a problem encountered in QR decoding when the QR code is printed near the binding edge inside a book or on the surface of a cylindrical container. When a picture is taken on such a QR code, the picture is no longer in a rectangular shape, which is supposedly the “right” shape that any QR code should take, due to warping from the image capturing mechanism by the camera. We refer to such warping as cylinder-to-screen (C2S) warping. This warping translates into serious distortion of the original QR-code image (also referred to as QR image, for short) and thus causes failure in its decoding. In this paper, a mathematical model called the parametric cylindrical surface model (PCSM) is proposed. Then, based on it, a scheme is proposed to rectify C2S-warped QR images. Experiments show that the proposed scheme is effective, in the sense that many failed QR decodings become successful after the rectification.

We have shown that the left side null space of the autoregression (AR) matrix operator is the lexicographical presentation of the point spread function (PSF) on condition the AR parameters are common for original and blurred images. The method of inverse PSF evaluation with regularization functional as the function of surface area is offered. The inverse PSF was used for primary image estimation. Two methods of original image estimate optimization were designed basing on maximum entropy generalization of sought and blurred images conditional probability density and regularization. The first method uses balanced variations of convolution and deconvolution transforms to obtaining iterative schema of image optimization. The variations balance was defined by dynamic regularization basing on condition of iteration process convergence. The regularization has dynamic character because depends on current and previous image estimate variations. The second method implements the regularization of deconvolution optimization in curved space with metric defined on image estimate surface. It is basing on target functional invariance to fluctuations of optimal argument value. The given iterative schemas have faster convergence in comparison with known ones, so they can be used for reconstruction of high resolution images series in real time.

We present a fast and precise vision-based software intended for multiple robot localization. The core component of the proposed localization system is an efficient method for black and white circular pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision, and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost camera, its core algorithm is able to process hundreds of images per second while tracking hundreds of objects with millimeter precision. We propose a mathematical model of the method that allows to calculate its precision, area of coverage, and processing speed from the camera's intrinsic parameters and hardware's processing capacity. The correctness of the presented model and performance of the algorithm in real-world conditions are verified in several experiments. Apart from the method description, we also publish its source code; so, it can be used as an enabling technology for various mobile robotics problems.

In this paper, a new algorithm for vehicle logo recognition on the basis of an enhanced scale-invariant feature transform (SIFT)-based feature-matching scheme is proposed. This algorithm is assessed on a set of 1200 logo images that belong to ten distinctive vehicle manufacturers. A series of experiments are conducted, splitting the 1200 images to a training set and a testing set, respectively. It is shown that the enhanced matching approach proposed in this paper boosts the recognition accuracy compared with the standard SIFT-based feature-matching method. The reported results indicate a high recognition rate in vehicle logos and a fast processing time, making it suitable for real-time applications.

This paper introduces a new algorithm for enhancing image resolution from an image sequence. The approach we propose herein uses the integrating resampler for warping. The method is a direct computation, which is fundamentally different from the iterative back-projection approaches proposed in previous work. This paper shows that image-warping techniques may have a strong impact on the quality of image resolution enhancement. By coupling the degradation model of the imaging system directly into the integrating resampler, we can better approximate the warping characteristics of real sensors, which also significantly improve the quality of super-resolution images. Examples of super-resolutions are given for gray-scale images. Evaluations are made visuallyby comparing the resulting images and those using bi-linear resampling and back-projection and quantitativelyusing OCR as a fundamental measure. The paper shows that even when the images are qualitatively similar, quantitative differences appear in machine processing. q 2000 Elsevier Science B.V. All rights reserved. Keywords: Super-resolution; Imaging-consistent restoration/reconstruction; Integrating resampling; Integrating resampler; Quantitative evaluatio n; OCR; Bilinear resampling; Image reconstruction; Image restoration; Image warping 1. Background and introduction The basic premise of super-resolution is to combine images (and in some work scene/data models) to produce an image that has higher “resolution” than the original image. It is not sufficient to just resample the image—size is not the same as resolution. Increasing resolution could be viewed as either increasing the signal-to-noise ratio while keeping the size fixed and/or approximating the image at a larger size with reasonable approximations for frequencies higher than those representable at the original size. In either case, it requires additional information and assumptions. There are, of course, some fundamental limits on what can be gained; lens and photo-sites cause blur, and the discrete sampling causes frequencies above the Nyquist rate to alias. While in general the aliasing is theoretically impossible to separate, the problem occurs mostly along sharp “intensity edges”, so multiple images with slightly different imaging positions can be used to reduce its impact. There are two different groups of research that use the term super-resolution—one is based on multiple images, which will be considered within this paper; the other uses “scene/object” models. Early research on image-based super-resolution dates back to the work by Huang and Tsai [8]. They solved the problem in the frequency domain, but they disregarded sensor blurring. Furthermore, they assumed only inter-frame translations. Gross [7] solved the problem by first merging the low resolution images by interpolation and then obtaining the high-resolution image by deblurring the merged image. Again, only inter-frame translations were considered. Peleg and Keren [11,14] solved the problem by first estimating an initial guess of the high-resolution image, then simulating the imaging process and minimizing the difference between the observed and simulated low-resolution images. Irani and Peleg [9,10] used a back-projection method similar to that used in tomography to minimize the same difference. Bascle et al. [1] used the same back-projection method to minimize the difference between the predicted and actual images except that a simple motion blur model was included. All previous work ignores the impact of image warping techniques. In this paper, we show that warping techniques may have a strong impact on the quality of image resolution enhancement. Image warping has been around almost as long as image processing, allowing users to reshape the image geometry. Image warping requires the underlying image to be “resampled” at non-integer locations; hence, it requires image reconstruction. When the goal of warping is to produce output for human viewing, only moderately accurate image intensities are needed. In these cases, techniques Image and Vision Computing 18 (2000) 761–771 0262-8856/00/$ see front matter q 2000 Elsevier Science B.V. All rights reserved. PII: S0262-8856(99)00044-X www.elsevier.com/locate/imavis * Corresponding author. Tel.: 11-610-7584061. E-mail address:tboult@eecs.lehigh.edu (T.E. Boult). using bi-linear interpolation have been found sufficient. However, as a step for applications such as super-resolution, the precision of the warped intensity values is often important. For these problems, bi-linear image reconstruction may not be sufficient. This paper shows how the integrating resampler can be used to enhance image resolution. This builds upon the ideas of imaging-consistent restoration/reconstruction algorithms proposed in Ref. [2] and the integrating resampler proposed in Ref. [5]. A final contribution of this paper is a quantitative comparison and analysis of our techniques. In this paper, we use a character recognition rate based on a commercial OCR program. We chose this because it is easy to make it quantitative, and we consider this to be a good measure for other applications such as license-plate reading and other pattern recognition problems. In what follows in this paper, the image formation process and the relationship between restoration, reconstruction, and super-resolution is briefly reviewed in Section 2. The integrating resampler—an efficient method for warping using imaging-consistent reconstruction/restoration algorithms—is given in Section 3. This algorithm is well suited for today’s pipelined micro-processors. In addition, the integrating resampler can allow for modifications of the intensity values to better approximate the warping characteristics of real sensors. The super-resolution algorithm where we compare the integrating resampler with bi-linear resampling is presented in Section 4. Quantitative measurement of super-resolution imaging using OCR is shown in Section 5. 2. Image formation and image restoration In this section, we briefly review the image formation process as well as the sensor model proposed in Ref. [2]. Image formation is generally described as a sequence of filtering operations. See Fig. 1. Let f x; y be the intensity distribution of a scene at the front aperture of a lens. That distribution is acted upon by h1x; y; the blurring component of the lens, yielding f1x; y: A geometric distortion functionh2x; y is then applied to produce imagef2u; v: At this point, f2 strikes the image sensor where it undergoes additional blurring by a point spread functionh3u; v to generate image f3u; v: This blurring reflects the limitation of the sensor to accurately resolve each point without the influence of neighboring points. We choose to use a simple model wherein this blurring takes place within one pixel because for CCD and CID cameras, the physical boundaries between photo-sites generally allow only insignificant charge transfer between pixels. Imagef3 undergoes spatial sampling as it hits the discrete photo-sites in a CCD or CID camera. The combination of h3 with sampling is known as areas sampling. It reflects the finite size of the sampling area. If h3 is taken to be an impulse, then we have point sampling. This ideal, often assumed for theoretical considerations, is generally not true in practice. In either case, intensities in the sampled image Is are now defined only for integer values of u and v. The digital imageI u; v is obtained via an analog-todigital converter that quantizes the samples of Is. This completes the image formation process, leaving I u; v as the starting point for subsequent processing, including image reconstruction and restoration. Reconstruction and restoration start with I (and models for one or morehis), and seek to solve for one or more of the fjs. Recoveringf is known as image restoration and is of considerable interest in image processing. Note that recovering a functional form off2 is an intra-pixel restoration. Reconstruction limits itself to the problem of deriving the continuous functionf3. Restoration attempts to estimate the original input functionf2 (or f1). The goal of super-resolution is to recover a discrete approximation, with resolution higher than Is, to either f1 (plain super-resolution) orf (super-resolution with deblurring). Because super-resolution increases the signal-to-noise ratio, it significantly aids in the ill-condition problem of deblurring. 3. Imaging-consistency and the integrating resampler An algorithm is called imaging-consistent if it is the exact solution for some input function, which, according to the imaging model, would have generated the measured input. For image reconstruction, we achieve this by first restoring the image to yieldf2, then blurring it again by the pixel’s PSF. Details can be found in Ref. [2]. Only one-dimensional image models are presented herein; higher dimensions are treated separately. M.-C. Chiang, T.E. Boult / Image and Vision Computing 18 (2000) 761–771 762 Fig. 1. The image formation process and the relationship between restoration, reconstruction, and super-resolution. The first imaging-consistent method to consider is based on a piecewise quadratic model for the image. If we assume a Rect PSF filter (1 inside and 0 outside the pixel), the imaging consistent algorithm is easy to derive. To ensure that the function is continuous and that the method is local, we define the value of the reconstruction at the pixel boundarieski andki11 to be equal toEi andEi11: Any method of approximation could be used to compute Ei, though our examples will only include linear interpolation and cubic convolution. See [3, Section 2.3] for more details. Given the valuesEi at the pixel edges, an additional constraint, that the integral across the pixel must equal Vi, results in exactly three constraints: qi21=2  Ei ; qi1=2  Ei11; Z1=2 2 1=2 qix dx Vi : 1 From Eq. (1), one can det

This paper proposes and validates a real-time on-road vehicle detection system, which uses a single camera for the purpose of intelligent driver assistance. A three-step vehicle detection framework is presented to detect and track the target vehicle within an image. In the first step, probable vehicle locations are hypothesized using pattern recognition. The vehicle candidates are then verified in the hypothesis verification step. In this step, lane detection is used to filter vehicle candidates that are not within the lane region of interest. In the final step tracking and online learning are implemented to optimize the detection algorithm during misdetection and temporary occlusion. Good detection performance and accuracy was observed in highway driving environments with minimal shadows.

This paper addresses the problem of rain streak removal from a single image. Rain streaks impair visibility of an image and introduce undesirable interference that can severely affect the performance of computer vision algorithms. Rain streak removal can be formulated as a layer decomposition problem, with a rain streak layer superimposed on a background layer containing the true scene content. Existing decomposition methods that address this problem employ either dictionary learning methods or impose a low rank structure on the appearance of the rain streaks. While these methods can improve the overall visibility, they tend to leave too many rain streaks in the background image or over-smooth the background image. In this paper, we propose an effective method that uses simple patch-based priors for both the background and rain layers. These priors are based on Gaussian mixture models and can accommodate multiple orientations and scales of the rain streaks. This simple approach removes rain streaks better than the existing methods qualitatively and quantitatively. We overview our method and demonstrate its effectiveness over prior work on a number of examples.


Question: A Riesz fractional model-based enhancement for license plate detection and recognition.

This document outlines a Vehicle License Plate Recognition (VLPR) system designed for integration into Intelligent Transportation Systems (ITS). The core of the VLPR system, and arguably its most challenging component, is the license plate localization process. To tackle this, a novel license plate localization method is introduced, leveraging Maximally Stable Extremal Region (MSER) feature analysis. The process begins with the MSER detector extracting potential character regions from the image. Subsequently, the precise locations of license plates are deduced based on the standard arrangement of characters on license plates. A key benefit of this localization method is its robustness, requiring minimal assumptions about environmental factors such as lighting and weather conditions. Post-localization, the system proceeds to identify the characters and color of the license plate, thereby finalizing the VLPR system. The system's efficacy is validated through testing on a proprietary dataset, with outcomes demonstrating its capability to accurately locate and recognize license plates within images, underscoring the system's reliability and efficiency in real-world scenarios.

It plays an important role to accurately track multiple vehicles in intelligent transportation, especially in intelligent vehicles. Due to complicated traffic environments it is difficult to track multiple vehicles accurately and robustly, especially when there are occlusions among vehicles. To alleviate these problems, a new approach is proposed to track multiple vehicles with the combination of robust detection and two classifiers. An improved ViBe algorithm is proposed for robust and accurate detection of multiple vehicles. It uses the gray-scale spatial information to build dictionary of pixel life length to make ghost shadows and object’s residual shadows quickly blended into the samples of the background. The improved algorithm takes good post-processing method to restrain dynamic noise. In this paper, we also design a method using two classifiers to further attack the problem of failure to track vehicles with occlusions and interference. It classifies tracking rectangles with confidence values between two thresholds through combining local binary pattern with support vector machine (SVM) classifier and then using a convolutional neural network (CNN) classifier for the second time to remove the interference areas between vehicles and other moving objects. The two classifiers method has both time efficiency advantage of SVM and high accuracy advantage of CNN. Comparing with several existing methods, the qualitative and quantitative analysis of our experiment results showed that the proposed method not only effectively removed the ghost shadows, and improved the detection accuracy and real-time performance, but also was robust to deal with the occlusion of multiple vehicles in various traffic scenes.

We consider the problem of document binarization as a pre-processing step for optical character recognition (OCR) for the purpose of keyword search of historical printed documents. A number of promising techniques from the literature for binarization, pre-filtering, and post-binarization denoising were implemented along with newly developed methods for binarization: an error diffusion binarization, a multiresolutional version of Otsu’s binarization, and denoising by despeckling. The OCR in the ABBYY FineReader 7.1 SDK is used as a black box metric to compare methods. Results for 12 pages from six newspapers of differing quality show that performance varies widely by image, but that the classic Otsu method and Otsu-based methods perform best on average. 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

In this paper, we propose a new Reduced Reference Image Quality Metric for multiply degraded images based on a features extraction step and its combination. The selected features are extracted from the original image and its degraded version. Some of them aim to quantify the level of the considered degradation types, while the others quantify its sharpness. These features are then combined to obtain a single value, which corresponds to the predicted subjective score. Our method has been evaluated and compared in terms of correlation with subjective judgments to some recent methods by using the LIVE Multiply Distorted Image Quality Database.

A feature extraction method using the chaincode representation of fingerprint ridge contours is presented for use by Automatic Fingerprint Identification Systems. The representation allows efficient image quality enhancement and detection of fine minutiae feature points. For image enhancement a given fingerprint image is first binarized after a quick averaging to generate its chaincode representation. The direction field is estimated from a set of selected chaincodes. The original gray scale image is then enhanced using connected component analysis and a dynamic filtering scheme that takes advantage of the knowledge gained from the estimated direction flow of the contours. For feature extraction, the enhanced fingerprint image is carefully binarized using a local-global binarization algorithm for generating the chaincode representation. The minutiae are generated using a sophisticated ridge contour following procedure. Visual inspection of the experiment images shows that the method is very effective.

Image defects which could be caused by the digitization process or by faults in the imaging set-up (for example, bad lighting) can be corrected using Image Enhancement techniques. Once the image is in good condition, the Measurement Extraction operations can be used to obtain useful information from the image. Course Objectives “Introduction to Digital Image Processing” is two-day hands-on course that provide fundamental concepts and techniques for digital image processing and the software principles used in their practical implementation.

Contrast enhancement of radiographies based on a multiscale decomposition of the images recently has proven to be a far more versatile and efficient method than regular unsharp-masking techniques, while containing these as a subset. In this paper, we compare the performance of two multiscale-methods, namely the Laplacian Pyramid and the fast wavelet transform (FWT). We find that enhancement based on the FWT suffers from one serious drawback-the introduction of visible artifacts when large structures are enhanced strongly. By contrast, the Laplacian Pyramid allows a smooth enhancement of large structures, such that visible artifacts can be avoided. Only for the enhancement of very small details, for denoising applications or compression of images, the FWT may have some advantages over the Laplacian Pyramid.

In this paper, we propose a framework for both gradient descent image and object alignment in the Fourier domain. Our method centers upon the classical Lucas & Kanade (LK) algorithm where we represent the source and template/model in the complex 2D Fourier domain rather than in the spatial 2D domain. We refer to our approach as the Fourier LK (FLK) algorithm. The FLK formulation is advantageous when one preprocesses the source image and template/model with a bank of filters (e.g., oriented edges, Gabor, etc.) as 1) it can handle substantial illumination variations, 2) the inefficient preprocessing filter bank step can be subsumed within the FLK algorithm as a sparse diagonal weighting matrix, 3) unlike traditional LK, the computational cost is invariant to the number of filters and as a result is far more efficient, and 4) this approach can be extended to the Inverse Compositional (IC) form of the LK algorithm where nearly all steps (including Fourier transform and filter bank preprocessing) can be precomputed, leading to an extremely efficient and robust approach to gradient descent image matching. Further, these computational savings translate to nonrigid object alignment tasks that are considered extensions of the LK algorithm, such as those found in Active Appearance Models (AAMs).

Aerial images are subject to various types of noise, which restricts the recognition and analysis of images, target monitoring, and search services. At present, deep learning is successful in image recognition. However, traditional convolutional neural networks (CNNs) extract the main features of an image to predict directly and are limited by the requirements of the training sample size (i.e., a small data size is not successful enough). In this paper, using a small sample size, we propose an aerial-image denoising recognition model based on CNNs with a multi-scale residual learning approach. The proposed model has the following three advantages: (1) Instead of directly learning latent clean images, the proposed model learns the noise from noisy images and then subtracts the learned residual from the noisy images to obtain reconstructed (denoised) images; (2) The developed image denoising recognition model is beneficial to small training datasets; (3) We use multi-scale residual learning as a learning approach, and dropout is introduced into the model architecture to force the network to learn to generalize well enough. Our experimental results on aerial-image denoising recognition reveal that the proposed approach is highly superior to the other state-of-the-art methods.

This paper proposes an edge-directed interpolation algorithm for natural images. The basic idea is to first estimate local covariance coefficients from a low-resolution image and then use these covariance estimates to adapt the interpolation at a higher resolution based on the geometric duality between the low-resolution covariance and the high-resolution covariance. The edge-directed property of covariance-based adaptation attributes to its capability of tuning the interpolation coefficients to match an arbitrarily oriented step edge. A hybrid approach of switching between bilinear interpolation and covariance-based adaptive interpolation is proposed to reduce the overall computational complexity. Two important applications of the new interpolation algorithm are studied: resolution enhancement of grayscale images and reconstruction of color images from CCD samples. Simulation results demonstrate that our new interpolation algorithm substantially improves the subjective quality of the interpolated images over conventional linear interpolation.

This paper starts from the observation that multiple top performing pedestrian detectors can be modelled by using an intermediate layer filtering low-level features in combination with a boosted decision forest. Based on this observation we propose a unifying framework and experimentally explore different filter families. We report extensive results enabling a systematic analysis. Using filtered channel features we obtain top performance on the challenging Caltech and KITTI datasets, while using only HOG+LUV as low-level features. When adding optical flow features we further improve detection quality and report the best known results on the Caltech dataset, reaching 93% recall at 1 FPPI.


Question: Propagating uncertainty through the hyperbolic tangent function for application in reservoir computing systems.

This paper explores the application of Gaussian process priors in time series forecasting, particularly focusing on multi-step ahead predictions in non-linear dynamic systems. In traditional forecasting, multi-step predictions are typically achieved through iterative one-step ahead forecasts. For a given state-space model, where the system's output at time \(t\) is influenced by previous outputs, predictions often hinge on point estimates of these prior outputs. However, this approach neglects the uncertainty associated with intermediate predictions, which can accumulate and significantly impact the accuracy of long-term forecasts. To address this issue, we propose a method that integrates the uncertainty about intermediate regressor values into the forecasting process. By employing an analytical Gaussian approximation, we are able to formally account for this uncertainty, thereby updating and refining the uncertainty estimates on the current prediction. This method not only enhances the robustness of multi-step predictions but also provides a more accurate representation of the prediction intervals, crucial for decision-making in time series analysis.

A method for solving boundary value problems (BVPs) is introduced using artificial neural networks (ANNs) for irregular domain boundaries with mixed Dirichlet/Neumann boundary conditions (BCs). The approximate ANN solution automatically satisfies BCs at all stages of training, including before training commences. This method is simpler than other ANN methods for solving BVPs due to its unconstrained nature and because automatic satisfaction of Dirichlet BCs provides a good starting approximate solution for significant portions of the domain. Automatic satisfaction of BCs is accomplished by the introduction of an innovative length factor. Several examples of BVP solution are presented for both linear and nonlinear differential equations in two and three dimensions. Error norms in the approximate solution on the order of 10-4 to 10-5 are reported for all example problems.

In this paper, a reverse carry propagate adder (RCPA) is presented. In the RCPA structure, the carry signal propagates in a counter-flow manner from the most significant bit to the least significant bit; hence, the carry input signal has higher significance than the output carry. This method of carry propagation leads to higher stability in the presence of delay variations. Three implementations of the reverse carry propagate full-adder (RCPFA) cell with different delay, power, energy, and accuracy levels are introduced. The proposed structure may be combined with an exact (forward) carry adder to form hybrid adders with tunable levels of accuracy. The design parameters of the proposed RCPA implementations and some hybrid adders realized utilizing these structures are studied and compared with those of the state-of-the-art approximate adders using HSPICE simulations in a 45-nm CMOS technology. The results indicate that employing the proposed RCPAs in the hybrid adders may provide, on average, 27%, 6%, and 31% improvements in delay, energy, and energy-delay-product while providing higher levels of accuracy. In addition, the structure is more resilient to delay variation compared to the conventional approximate adder. Finally, the efficacy of the proposed RCPAs is investigated in the discrete cosine transform (DCT) block of the JPEG compression and finite-impulse response (FIR) filter applications. The investigation reveals 60% and 39% energy saving in the DCT of JPEG and FIR filter, respectively, for the proposed RCPAs.

This study applies generalized polynomial chaos theory to model complex nonlinear multibody dynamic systems operating in the presence of parametric and external uncertainty. Theoretical and computational aspects of this methodology are discussed in the companion paper “Modeling Multibody Dynamic Systems With Uncertainties. Part I: Theoretical and Computational Aspects”. In this paper we illustrate the methodology on selected test cases. The combined effects of parametric and forcing uncertainties are studied for a quarter car model. The uncertainty distributions in the system response in both time and frequency domains are validated against Monte-Carlo simulations. Results indicate that polynomial chaos is more efficient than Monte Carlo and more accurate than statistical linearization. The results of the direct collocation approach are similar to the ones obtained with the Galerkin approach. A stochastic terrain model is constructed using a truncated Karhunen-Loeve expansion. The application of polynomial chaos to differential-algebraic systems is illustrated using the constrained pendulum problem. Limitations of the polynomial chaos approach are studied on two different test problems, one with multiple attractor points, and the second with a chaotic evolution and a nonlinear attractor set. The overall conclusion is that, despite its limitations, generalized polynomial chaos is a powerful approach for the simulation of multibody dynamic systems with uncertainties.

Gaussian processes (GPs) are a good choice for function approximation as they are flexible, robust to overfitting, and provide well-calibrated predictive uncertainty. Deep Gaussian processes (DGPs) are multi-layer generalizations of GPs, but inference in these models has proved challenging. Existing approaches to inference in DGP models assume approximate posteriors that force independence between the layers, and do not work well in practice. We present a doubly stochastic variational inference algorithm that does not force independence between layers. With our method of inference we demonstrate that a DGP model can be used effectively on data ranging in size from hundreds to a billion points. We provide strong empirical evidence that our inference scheme for DGPs works well in practice in both classification and regression.

In this paper, we present a survey of recent works in developing neuromorphic or neuro-inspired hardware systems. In particular, we focus on those systems which can either learn from data in an unsupervised or online supervised manner. We present algorithms and architectures developed specially to support on-chip learning. Emphasis is placed on hardware friendly modifications of standard algorithms, such as backpropagation, as well as novel algorithms, such as structural plasticity, developed specially for low-resolution synapses. We cover works related to both spike-based and more traditional non-spike-based algorithms. This is followed by developments in novel devices, such as floating-gate MOS, memristors, and spintronic devices. CMOS circuit innovations for on-chip learning and CMOS interface circuits for post-CMOS devices, such as memristors, are presented. Common architectures, such as crossbar or island style arrays, are discussed, along with their relative merits and demerits. Finally, we present some possible applications of neuromorphic hardware, such as brain–machine interfaces, robotics, etc., and identify future research trends in the field.

We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure1.

Low precision networks in the reinforcement learning (RL) setting are relatively unexplored because of the limitations of binary activations for function approximation. Here, in the discrete action ATARI domain, we demonstrate, for the first time, that low precision policy distillation from a high precision network provides a principled, practical way to train an RL agent. As an application, on 10 different ATARI games, we demonstrate real-time end-to-end game playing on lowpower neuromorphic hardware by converting a sequence of game frames into discrete actions.

The problem of learning a general input-output relation using a layered neural network is discussed in a statistical framework. By imposing the consistency condition that the error minimization be equivalent to a likelihood maximization for training the network, the authors arrive at a Gibbs distribution on a canonical ensemble of networks with the same architecture. This statistical description enables them to evaluate the probability of a correct prediction of an independent example, after training the network on a given training set. The prediction probability is highly correlated with the generalization ability of the network, as measured outside the training set. This suggests a general and practical criterion for training layered networks by minimizing prediction errors. The authors demonstrate the utility of this criterion for selecting the optimal architecture in the continuity problem. As a theoretical application of the statistical formalism, they discuss the question of learning curves and estimate the sufficient training size needed for correct generalization, in a simple example.<<ETX>>

The accurate and quick modelling of inverse lens distortion to rectify images or predict the image coordinates of real world objects has long been a challenge. This work investigates the use of artificial neural networks to perform this modelling. Several architectures are investigated and multiple methods of training the networks are used to achieve the best results. The error is expressed as a physical displacement on the imaging chip so that a fair comparison can be made between other published results which are based on cameras with different resolutions. It is shown that the novel application of locally optimised neural networks to residual lens calibration data yields an inverse distortion modelling that is highly competitive with prior published results.

The Differentiable Neural Computer (DNC) can learn algorithmic and question answering tasks. An analysis of its internal activation patterns reveals three problems: Most importantly, the lack of key-value separation makes the address distribution resulting from content-based look-up noisy and flat, since the value influences the score calculation, although only the key should. Second, DNC’s deallocation of memory results in aliasing, which is a problem for content-based look-up. Thirdly, chaining memory reads with the temporal linkage matrix exponentially degrades the quality of the address distribution. Our proposed fixes of these problems yield improved performance on arithmetic tasks, and also improve the mean error rate on the bAbI question answering dataset by 43%.


Question: Propagating uncertainty through the tanh function for use in reservoir computing applications.

This paper delves into the concept of reservoir computing (RC), a novel paradigm for state-space models that utilizes a fixed, complex state transition structure known as the reservoir, coupled with an adjustable readout mechanism from the state space. The reservoir's intricate design is intended to encapsulate a vast array of features from the input data, which are then harnessed by the reservoir-to-output mapping. Despite the burgeoning field of RC and its numerous practical successes, it has faced criticism for lacking a principled approach. The creation of reservoirs often involves a series of random model-building steps, necessitating iterative trials and adjustments by both researchers and practitioners.

To address this gap and initiate a more systematic exploration of RC, we focus on a prominent subclass of RC techniques, the echo state network (ESN), and pose fundamental questions: What is the least complex configuration required for the reservoir to yield competitive models, and what is the memory capacity (MC) of such simplified reservoirs? Through extensive experimentation on a diverse set of benchmark time series datasets, originating from various domains and possessing distinct characteristics, as well as through rigorous theoretical analysis, we reveal that a straightforward, deterministically crafted cyclic reservoir can match the performance of standard ESN methodologies. Remarkably, the short-term memory of these linear cyclic reservoirs can be tuned to approach the theoretically optimal value, showcasing the potential for a simpler, yet equally effective, reservoir design in RC applications.

Deep neural networks (DNNs) are powerful types of artificial neural networks (ANNs) that use several hidden layers. They have recently gained considerable attention in the speech transcription and image recognition community (Krizhevsky et al., 2012) for their superior predictive properties including robustness to overfitting. However their application to financial market prediction has not been previously researched, partly because of their computational complexity. This paper describes the application of DNNs to predicting financial market movement directions. A critical step in the viability of the approach in practice is the ability to effectively deploy the algorithm on general purpose high performance computing infrastructure. Using an Intel Xeon Phi co-processor with 61 cores, we describe the process for efficient implementation of the batched stochastic gradient descent algorithm and demonstrate a 11.4x speedup on the Intel Xeon Phi over a serial implementation on the Intel Xeon.

In this paper, we explore different ways to extend a recurrent neural network (RNN) to a deep RNN. We start by arguing that the concept of depth in an RNN is not as clear as it is in feedforward neural networks. By carefully analyzing and understanding the architecture of an RNN, however, we find three points of an RNN which may be made deeper; (1) input-to-hidden function, (2) hidden-tohidden transition and (3) hidden-to-output function. Based on this observation, we propose two novel architectures of a deep RNN which are orthogonal to an earlier attempt of stacking multiple recurrent layers to build a deep RNN (Schmidhuber, 1992; El Hihi and Bengio, 1996). We provide an alternative interpretation of these deep RNNs using a novel framework based on neural operators. The proposed deep RNNs are empirically evaluated on the tasks of polyphonic music prediction and language modeling. The experimental result supports our claim that the proposed deep RNNs benefit from the depth and outperform the conventional, shallow RNNs.

Neuromorphic hardware tends to pose limits on the connectivity of deep networks that one can run on them. But also generic hardware and software implementations of deep learning run more efficiently on sparse networks. Several methods exist for pruning connections of a neural network after it was trained without connectivity constraints. We present an algorithm, DEEP R, that enables us to train directly a sparsely connected neural network. DEEP R automatically rewires the network during supervised training so that connections are there where they are most needed for the task, while its total number is all the time strictly bounded. We demonstrate that DEEP R can be used to train very sparse feedforward and recurrent neural networks on standard benchmark tasks with just a minor loss in performance. DEEP R is based on a rigorous theoretical foundation that views rewiring as stochastic sampling of network configurations from a posterior.

Received () Revised () Recurrent nets are in principle capable to store past inputs to produce the currently desired output. Because of this property recurrent nets are used in time series prediction and process control. Practical applications involve temporal dependencies spanning many time steps, e.g. between relevant inputs and desired outputs. In this case, however, gradient based learning methods take too much time. The extremely increased learning time arises because the error vanishes as it gets propagated back. In this article the decaying error ow is theoretically analyzed. Then methods trying to overcome vanishing gradients are brieey discussed. Finally, experiments comparing conventional algorithms and alternative methods are presented. With advanced methods long time lag problems can be solved in reasonable time.

We present a simple regularization technique for Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) units. Dropout, the most successful technique for regularizing neural networks, does not work well with RNNs and LSTMs. In this paper, we show how to correctly apply dropout to LSTMs, and show that it substantially reduces overfitting on a variety of tasks. These tasks include language modeling, speech recognition, and machine translation.

The modeling and identification of a mechanical system is the most important issue for many control systems in order to realize the desired control specifications. In particular, the friction characteristics often deteriorate the control performance, such as in the fast and precise positioning performance in industrial robots, the force estimation accuracy based on a disturbance observer, and the posture control performance of an inverted pendulum robot. Rolling friction tends to cause overshoot, undershoot, or limit cycles of the target value in positioning systems. In previous research, some model structures for rolling friction have been proposed to express the hysteresis characteristics in order to overcome these control issues. However, it is difficult to identify the correct parameters for precise modeling. In this paper, the modeling of rolling friction based on a Recurrent Neural Network (RNN) using Long Short-Term Memory (LSTM) is proposed to precisely express the rolling friction characteristics. The initial value design of the RNN during supervised learning is also presented to achieve a better model. The effectiveness of the proposed approach is verified by comparison with conventional friction models using an actual experimental setup.

Speech recognition is largely taking advantage of deep learning, showing that substantial benefits can be obtained by modern Recurrent Neural Networks (RNNs). The most popular RNNs are Long Short-Term Memory (LSTMs), which typically reach state-of-the-art performance in many tasks thanks to their ability to learn long-term dependencies and robustness to vanishing gradients. Nevertheless, LSTMs have a rather complex design with three multiplicative gates, that might impair their efficient implementation. An attempt to simplify LSTMs has recently led to Gated Recurrent Units (GRUs), which are based on just two multiplicative gates. This paper builds on these efforts by further revising GRUs and proposing a simplified architecture potentially more suitable for speech recognition. The contribution of this work is two-fold. First, we suggest to remove the reset gate in the GRU design, resulting in a more efficient single-gate architecture. Second, we propose to replace tanh with ReLU activations in the state update equations. Results show that, in our implementation, the revised architecture reduces the per-epoch training time with more than 30% and consistently improves recognition performance across different tasks, input features, and noisy conditions when compared to a standard GRU.

We introduce BinaryNet, a method which trains DNNs with binary weights and activations when computing parameters’ gradient. We show that it is possible to train a Multi Layer Perceptron (MLP) on MNIST and ConvNets on CIFAR-10 and SVHN with BinaryNet and achieve nearly state-of-the-art results. At run-time, BinaryNet drastically reduces memory usage and replaces most multiplications by 1-bit exclusive-not-or (XNOR) operations, which might have a big impact on both general-purpose and dedicated Deep Learning hardware. We wrote a binary matrix multiplication GPU kernel with which it is possible to run our MNIST MLP 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The code for BinaryNet is available.

The increasing use of deep neural networks for safety-critical applications, such as autonomous driving and flight control, raises concerns about their safety and reliability. Formal verification can address these concerns by guaranteeing that a deep learning system operates as intended, but the state-of-the-art is limited to small systems. In this work-in-progress report we give an overview of our work on mitigating this difficulty, by pursuing two complementary directions: devising scalable verification techniques, and identifying design choices that result in deep learning systems that are more amenable to verification. ACM Reference Format: Lindsey Kuper, Guy Katz, Justin Gottschlich, Kyle Julian, Clark Barrett, and Mykel J. Kochenderfer. 2018. Toward Scalable Verification for SafetyCritical Deep Networks. In Proceedings of SysML Conference (SysML). ACM, New York, NY, USA, 3 pages.

In recent years significant progress has been made in successfully training recurrent neural networks (RNNs) on sequence learning problems involving long range temporal dependencies. The progress has been made on three fronts: (a) Algorithmic improvements involving sophisticated optimization techniques, (b) network design involving complex hidden layer nodes and specialized recurrent layer connections and (c) weight initialization methods. In this paper, we focus on recently proposed weight initialization with identity matrix for the recurrent weights in a RNN. This initialization is specifically proposed for hidden nodes with Rectified Linear Unit (ReLU) non linearity. We offer a simple dynamical systems perspective on weight initialization process, which allows us to propose a modified weight initialization strategy. We show that this initialization technique leads to successfully training RNNs composed of ReLUs. We demonstrate that our proposal produces comparable or better solution for three toy problems involving long range temporal structure: the addition problem, the multiplication problem and the MNIST classification problem using sequence of pixels. In addition, we present results for a benchmark action recognition problem.