Question: Identifying perceptual artifacts in compressed video sequences.

The complexity of image quality assessment is a critical issue in numerous image processing domains, despite significant endeavors to establish objective metrics that align with human perception. This paper delves into the underlying reasons for the persistent challenges in image quality assessment, primarily attributing them to the limitations of the error sensitivity based framework, a method that has been widely adopted in previous approaches. By highlighting the framework's weaknesses, we aim to shift the paradigm in the design of image quality metrics. We propose a novel perspective that emphasizes the human visual system's primary function—extracting structural information from visual scenes—and its remarkable adaptability for this task. Consequently, we argue that an evaluation of structural distortion could serve as a reliable proxy for perceived image distortion. Guided by this new perspective, we have developed a straightforward yet potent algorithm for image quality indexing, which our preliminary results indicate holds considerable promise.

**Question**: The role of technology in enhancing the efficiency of public transportation systems

**Documents**: A survey on the use of smart cards in public transportation
Smart cards have been widely used in public transportation systems to improve the efficiency of ticketing and fare collection. This paper surveys the use of smart cards in public transportation systems around the world. The paper begins with an introduction to smart cards and their working principles. It then describes the various types of smart cards and their applications in public transportation. The paper also discusses the benefits and challenges of using smart cards in public transportation systems. The benefits include reduced fare evasion, improved passenger convenience, and faster boarding times. The challenges include the cost of implementing the technology, the need for interoperability between different systems, and the potential for technical failures. The paper concludes with a discussion of future trends in the use of smart cards in public transportation systems.

**Similar Documents**: This paper offers a comprehensive review of the integration of smart card technology within public transportation systems, exploring its impact on efficiency and passenger experience. Starting with a foundational explanation of smart cards and their operational mechanisms, the paper delves into the diverse array of smart card types and their specific applications within the public transportation sector. The discussion then turns to the advantages and obstacles associated with smart card implementation. Among the benefits are a decrease in fare evasion, enhanced convenience for passengers, and expedited boarding processes. Conversely, challenges encompass the substantial costs of technology deployment, the imperative for interoperability across disparate systems, and the risk of technical malfunctions. The paper culminates in an analysis of emerging trends in smart card usage within public transportation, shedding light on potential future developments in this transformative technology.

The embedded zerotree wavelet algorithm (EZW) is a simple, yet remarkably effective, image compression algorithm, having the property that the bits in the bit stream are generated in order of importance, yielding a fully embedded code. The embedded code represents a sequence of binary decisions that distinguish a n image from the “null” image. Using a n embedded coding algorithm, a n encoder can terminate the encoding a t any point thereby allowing a target rate or target distortion metric to be met exactly. Also, given a bit stream, the decoder can cease decoding a t any point in the bit stream and still produce exactly the same image that would have been encoded a t the bit rate corresponding to the truncated bit stream. In addition to producing a fully embedded bit stream, EZW consistently produces compression results that a re competitive with virtually all known compression algorithms on standard test images. Yet this performance is achieved with a technique that requires absolutely no training, no pre-stored tables or codebooks, and requires no prior knowledge of the image source. The EZW algorithm is based on four key concepts: 1) a discrete wavelet transform or hierarchical subband decomposition, 2) prediction of the absence of significant information across scales by exploiting the self-similarity inherent in images, 3) entropy-coded successive-approximation quantization, and 4) universal lossless data compression which is achieved via adaptive arithmetic coding.

Automatically learned quality assessment for images has recently become a hot topic due to its usefulness in a wide variety of applications, such as evaluating image capture pipelines, storage techniques, and sharing media. Despite the subjective nature of this problem, most existing methods only predict the mean opinion score provided by data sets, such as AVA and TID2013. Our approach differs from others in that we predict the distribution of human opinion scores using a convolutional neural network. Our architecture also has the advantage of being significantly simpler than other methods with comparable performance. Our proposed approach relies on the success (and retraining) of proven, state-of-the-art deep object recognition networks. Our resulting network can be used to not only score images reliably and with high correlation to human perception, but also to assist with adaptation and optimization of photo editing/enhancement algorithms in a photographic pipeline. All this is done without need for a “golden” reference image, consequently allowing for single-image, semantic- and perceptually-aware, no-reference quality assessment.

Video and images acquired by a visual system are seriously degraded under hazy and foggy weather, which will affect the detection, tracking, and recognition of targets. Thus, restoring the true scene from such a foggy video or image is of significance. The main goal of this paper was to summarize current video and image defogging algorithms. We first presented a review of the detection and classification method of a foggy image. Then, we summarized existing image defogging algorithms, including image restoration algorithms, image contrast enhancement algorithms, and fusion-based defogging algorithms. We also presented current video defogging algorithms. We summarized objective image quality assessment methods that have been widely used for the comparison of different defogging algorithms, followed by an experimental comparison of various classical image defogging algorithms. Finally, we presented the problems of video and image defogging which need to be further studied. The code of all algorithms will be available at <;uri xlink:href="http://www.yongxu.org/lunwen.html" xlink:type="simple">http://www.yongxu.org/lunwen.html<;/uri>.

Convolutional autoencoders have emerged as popular methods for unsupervised defect segmentation on image data. Most commonly, this task is performed by thresholding a per-pixel reconstruction error based on an `-distance. This procedure, however, leads to large residuals whenever the reconstruction includes slight localization inaccuracies around edges. It also fails to reveal defective regions that have been visually altered when intensity values stay roughly consistent. We show that these problems prevent these approaches from being applied to complex real-world scenarios and that they cannot be easily avoided by employing more elaborate architectures such as variational or feature matching autoencoders. We propose to use a perceptual loss function based on structural similarity that examines inter-dependencies between local image regions, taking into account luminance, contrast, and structural information, instead of simply comparing single pixel values. It achieves significant performance gains on a challenging real-world dataset of nanofibrous materials and a novel dataset of two woven fabrics over state-of-the-art approaches for unsupervised defect segmentation that use per-pixel reconstruction

There has been a growing interest in using different approaches to improve the coding efficiency of modern video codec in recent years as demand for web-based video consumption increases. In this paper, we propose a model-based approach that uses texture analysis/synthesis to reconstruct blocks in texture regions of a video to achieve potential coding gains using the AV1 codec developed by the Alliance for Open Media (AOM). The proposed method uses convolutional neural networks to extract texture regions in a frame, which are then reconstructed using a global motion model. Our preliminary results show an increase in coding efficiency while maintaining satisfactory visual quality. Introduction With the increasing amount of videos being created and consumed, better video compression tools are needed to provide fast transmission and high visual quality. Modern video coding standards utilize spatial and temporal redundancy in the videos to achieve high coding efficiency and high visual quality with motion compensation techniques and 2-D orthogonal transforms. However, efficient exploitation of statistical dependencies measured by a mean squared error (MSE) does not always produce the best psychovisual result, and may require higher data rate to preserve detail information in the video. Recent advancement in GPU computing has enabled the analysis of large scale data using deep learning method. Deep learning techniques have shown promising performance in many applications such as object detection, natural language process, and synthetic images generation [1, 2, 3, 4]. Several methods have been developed for video applications to improve coding efficiency using deep learning. In [5], sample adaptive offset (SAO) is replaced by a CNN-based in-loop filter (IFCNN) to improve the coding efficiency in HEVC. By learning the predicted residue between the quantized reconstructed frames obtained after deblocking filter (DF) and the original frames, IFCNN is able to reconstruct video frames with higher quality without requiring any bit transmission during coding process. Similar to [5], [6] proposes a Variable-filter-size Residue-learning CNN (VRCNN) to improving coding efficiency by replacing DF and SAO in HEVC. VRCNN is based on the concept of ARCNN [7] which is originally designed for JPEG applications. Instead of only using spatial information to train a CNN to reduce the coding artifacts in HEVC, [8] proposed a spatial temporal residue network (STResNet) as an additional in-loop filter after SAO. A rate-distortion optimization strategy is used to control the on/off switch of the proposed in-loop filter. There are also some works that have been done in the decoder of HEVC to improve the coding efficiency. In [9], a deep CNN-based auto decoder (DCAD) is implemented in the decoder of HEVC to improve the video quality of decoded video. DCAD is trained to learn the predict residue between decoded video frames and original video frames. By adding the predicted residual generated from DCAD to the compressed video frames, this method enhances the compressed video frames to higher quality. In summary, the above methods improve the coding efficiency by enhancing the quality of reconstructed video frames. However, they require different trained models for video reconstruction at different quantization levels. We are interested in developing deep learning approaches to only encode visually relevant information and use a different coding method for “perceptually insignificant” regions in a frame, which can lead to substantial data rate reductions while maintaining visual quality. In particular, we have developed a model based approach that can be used to improve the coding efficiency by identifying texture areas in a video frame that contain detail irrelevant information, which the viewer does not perceive specific details and can be skipped or encoded at a much lower data rate. The task is then to divide a frame into “perceptually insignificant” texture region and then use a texture model for the pixels in that region. In 1959, Schreiber and colleagues proposed a coding method that divides an image into textures and edges and used it in image coding [10]. This work was later extended by using the human visual system and statistical model to determine the texture region [11, 12, 13]. More recently, several groups have focused on adapting perceptual based approaches to the video coding framework [14]. In our previous work [15], we introduced a texture analyzer before encoding the input sequences to identify detail irrelevant regions in the frame which are classified into different texture classes. At the encoder, no inter-frame prediction is performed for these regions. Instead, displacement of the entire texture region is modeled by just one set of motion parameters. Therefore, only the model parameters are transmitted to the decoder for reconstructing the texture regions using a texture synthesizer. Non-texture regions in the frame are coded conventionally. Since this method uses feature extraction based on texture segmentation technique, a proper set of parameters are required to achieve accurate texture segmentation for different videos. Deep learning methods usually do not require such parameter tuning for inference. As a result, deep learning techniques can be developed to perform texture segmentation and classification for the proposed model-based video coding. A Fisher vector convolution neural networks (FVCNN) that can produce segmentation labels for different texture classes was proposed in [16]. One of the advantage of FV-CNN is that the image input size is flexible and is not limited by the network architecture. Instead of doing pixel-wise classification on texture regions, a texture classification CNN network was described in [17]. To reduce computational expenses, [17] uses a small classification network to classify image patches with size of 227 × 227. A smaller network is needed to classify smaller image patches in our case. In this paper, we propose a block-based texture segmentation method to extract texture region in a video frame using convolutional neural networks. The block-based segmentation network classifies each 16 × 16 block in a frame as texture or non-texture. The identified texture region is then synthesized using the temporal correlations among the frames. Our method was implemented using the AOM/AV1 codec. Preliminary results show significant bitrate savings while maintaining a satisfactory visual quality.

Recently dense trajectories were shown to be an efficient video representation for action recognition and achieved state-of-the-art results on a variety of datasets. This paper improves their performance by taking into account camera motion to correct them. To estimate camera motion, we match feature points between frames using SURF descriptors and dense optical flow, which are shown to be complementary. These matches are, then, used to robustly estimate a homography with RANSAC. Human motion is in general different from camera motion and generates inconsistent matches. To improve the estimation, a human detector is employed to remove these matches. Given the estimated camera motion, we remove trajectories consistent with it. We also use this estimation to cancel out camera motion from the optical flow. This significantly improves motion-based descriptors, such as HOF and MBH. Experimental results on four challenging action datasets (i.e., Hollywood2, HMDB51, Olympic Sports and UCF50) significantly outperform the current state of the art.

V isual information retrieval systems have entered a new era. First-generation systems allowed access to images and videos through textual data. 1,2 Typical searches for these systems include, for example, " all images of paintings of the Florentine school of the 15th century " or " all images by Cezanne with landscapes. " Such systems expressed information through alphanumeric keywords or scripts. They employed representation schemes like relational models, frame models, and object-oriented models. On the other hand, current-generation retrieval systems support full retrieval by visual content. 3,4 Access to visual information is not only performed at a conceptual level, using keywords as in the textu-al domain, but also at a perceptual level, using objective measurements of visual content. In these systems, image processing, pattern recognition , and computer vision constitute an integral part of the system's architecture and operation. They objectively analyze pixel distribution and extract the content descriptors automatically from raw sensory data. Image content descriptors are commonly represented as feature vectors, whose elements correspond to significant parameters that model image attributes. Therefore, visual attributes are regarded as points in a multidimen-sional feature space, where point closeness reflects feature similarity. These advances (for comprehensive reviews of the field, see the " Further Reading " sidebar) have paved the way for third-generation systems, featuring full multimedia data management and networking support. Forthcoming standards such as MPEG-4 and MPEG-7 (see the Nack and Lindsay article in this issue) provide the framework for efficient representation, processing, and retrieval of visual information. Yet many problems must still be addressed and solved before these technologies can emerge. An important issue is the design of indexing structures for efficient retrieval from large, possibly distributed , multimedia data repositories. To achieve this goal, image and video content descriptors can be internally organized and accessed through multidimensional index structures. 5 A second key problem is to bridge the semantic gap between the system and users. That is, devise representations capturing visual content at high semantic levels especially relevant for retrieval tasks. Specifically, automatically obtaining a representation of high-level visual content remains an open issue. Virtually all the systems based on automatic storage and retrieval of visual information proposed so far use low-level perceptual representations of pictorial data, which have limited semantics. Building up a representation proves tantamount to defining a model of the world, possibly through a formal description language, whose semantics capture only a few …

This paper proposes a two-stage adaptive method for restoration of images corrupted with impulse noise. In the first stage, the pixels which are most likely contaminated by noise are detected based on their intensity values. In the second stage, an efficient average filtering algorithm is used to remove those noisy pixels from the image. Only pixels which are determined to be noisy in the first stage are processed in the second stage. The remaining pixels of the first stage are not processed further and are just copied to their corresponding locations in the restored image. The experimental results for the proposed method demonstrate that it is faster and simpler than even median filtering, and it is very efficient for images corrupted with a wide range of impulse noise densities varying from 10% to 90%. Because of its simplicity, high speed, and low computational complexity, the proposed method can be used in realtime digital image applications, e.g., in consumer electronic products such as digital televisions and cameras.

This paper presents a video stabilization algorithm based on the extraction and tracking of scale invariant feature transform features through video frames. Implementation of SIFT operator is analyzed and adapted to be used in a feature-based motion estimation algorithm. SIFT features are extracted from video frames and then their trajectory is evaluated to estimate interframe motion. A modified version of iterative least squares method is adopted to avoid estimation errors and features are tracked as they appear in nearby frames to improve video stability. Intentional camera motion is eventually filtered with adaptive motion vector integration. Results confirm the effectiveness of the method.

Various methods of automatic shot boundary detection have been proposed and claimed to perform reliably. A the detection of edits is fundamental to any kind of video analysis since it segments a video into its basic com the shots, only few comparative investigations on early shot boundary detection algorithms have been pu These investigations mainly concentrate on measuring the edit detection performance, however, do not con algorithms’ ability to classify the types and to locate the boundaries of the edits correctly. This paper extend comparative investigations. More recent algorithms designed explicitly to detect specific complex editing ope such as fades and dissolves are taken into account, and their ability to classify the types and locate the boun such edits are examined. The algorithms’ performance is measured in terms of hit rate, number of false hits, a rate for hard cuts, fades, and dissolves over a large and diverse set of video sequences. The experiments while hard cuts and fades can be detected reliably, dissolves are still an open research issue. The false hit rat solves is usually unacceptably high, ranging from 50% up to over 400%. Moreover, all algorithms seem to fai roughly the same conditions.


Question: Identifying perceptual artifacts in compressed video data streams.

This study evaluates various instrumental models designed to assess image dissimilarity and quality, as perceived by human observers. The models' effectiveness in predicting both image dissimilarity and quality is scrutinized using two distinct datasets: one comprising images affected by noise and blur, and another consisting of JPEG-compressed images. The investigation employs a comparative analysis to gauge the models' performance in accurately reflecting human perception. (1998 Elsevier Science B.V. All rights reserved.)

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design
An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.

**Similar Documents**: This article presents an innovative methodology, the Hybrid Genetic Algorithm-Particle Swarm Optimization (HGAPSO), tailored for the automated design of recurrent neural/fuzzy networks. Diverging from conventional methods, HGAPSO integrates the strengths of genetic algorithms (GA) and particle swarm optimization (PSO) to introduce a novel evolutionary learning algorithm. The process of generating a new generation in HGAP

Video summaries provide condensed and succinct representations of the content of a video stream through a combination of still images, video segments, graphical representations and textual descriptors. This paper presents a conceptual framework for video summarisation derived from the research literature and used as a means for surveying the research literature. The framework distinguishes between video summarisation techniques (the methods used to process content from a source video stream to achieve a summarisation of that stream) and video summaries (outputs of video summarisation techniques). Video summarisation techniques are considered within three broad categories: internal (analyse information sourced directly from the video stream), external (analyse information not sourced directly from the video stream) and hybrid (analyse a combination of internal and external information). Video summaries are considered as a function of the type of content they are derived from (object, event, perception or feature based) and the functionality offered to the user for their consumption (interactive or static, personalised or generic). It is argued that video summarisation would benefit from greater incorporation of external information, particularly user based information that is unobtrusively sourced, in order to overcome longstanding challenges such as the semantic gap and providing video summaries that have greater relevance to individual users. 2007 Elsevier Inc. All rights reserved.

The problem of visualizing high dynamic range images on the devices with restricted dynamic range has recently gained a lot of interest in the computer graphics community. Various so-called tone mapping operators have been proposed to face this issue. The field of tone mapping assumes thorough knowledge of both the objective and subjective attributes of an image. However, there no published analysis of such attributes exists so far. In this paper, we present an overview of image attributes which are used extensively in different tone mapping methods. Furthermore, we propose a scheme of relationships between these attributes, leading to the definition of an overall quality measure which we call naturalness. We present results of the subjective psychophysical testing that we have performed to prove the proposed relationship scheme. Our effort sets the stage for well-founded quality comparisons between tone mapping operators. By providing good definitions of the different attributes, comparisons, be they user-driven or fully automatic, are made possible at all.

Image processing is one of the most important areas of signal processing. Image processing is applicable for the improvement of pictorial improvement for human perception and autonomous machine application also the efficient storage and transmission. In this paper guided filter is proposed which can be applicable to both images as well as to video. This guided filter has many advantages over the previous version of filter like mean filter, median filter, sobel filter, bilateral filter, etc. Among of those bilateral filter is more popular. But this bilateral filter has certain limitations like gradient reversal effect this introduces false edges in the image. Proposed guided filter can smooth image/video and also preserved the edge that is information at the edges of image/video but this filter has better behaviors near edges. Also the fast implementation of guided filter is possible. This filter also has algorithm implementation regardless of kernel size. Guided filter gives better output than that of bilateral filter after filtering this bilateral filter gives output like cartoon this also a problem with this. The guided filter is both effective and efficient in many applications of computer vision and computer graphics, etc. Filtering techniques are used in image and video processing applications for various technologies.

This paper introduces a new compositional framework for classifying color correction methods according to their two main computational units. The framework was used to dissect fifteen among the best color correction algorithms and the computational units so derived, with the addition of four new units specifically designed for this work, were then reassembled in a combinatorial way to originate about one hundred distinct color correction methods, most of which never considered before. The above color correction methods were tested on three different existing datasets, including both real and artificial color transformations, plus a novel dataset of real image pairs categorized according to the kind of color alterations induced by specific acquisition setups. Differently from previous evaluations, special emphasis was given to effectiveness in real world applications, such as image mosaicing and stitching, where robustness with respect to strong image misalignments and light scattering effects is required. Experimental evidence is provided for the first time in terms of the most recent perceptual image quality metrics, which are known to be the closest to human judgment. Comparative results show that combinations of the new computational units are the most effective for real stitching scenarios, regardless of the specific source of color alteration. On the other hand, in the case of accurate image alignment and artificial color alterations, the best performing methods either use one of the new computational units, or are made up of fresh combinations of existing units.

Since the MPEG-1 Layer III encoding technology is nowadays widely used it might be interesting to gain knowledge of how this powerful compression/decompression scheme actually functions. How come the MPEG-1 Layer III is capable of reduc ing the bit rate with a factor of 12 without almost any audible degradation? Would it be fairly easy to implement this encoding algorithm? This paper will answer these questions and give further additional detailed information.

Anumber of performance issues must be addressed in order to transmit continuous media stream over the Internet with acceptable quality [9, 6] . These include reducing jitter and recovering from packet losses mainly due to congestion in the routers . Several methods can be used to deal with the loss of a packet such as : retransmission schemes, error concealment, error resilience, interleaving, and FEC (forward error correction) . Each has its advantages/disadvantages depending on the network scenario and conditions (RTT, propagation delay, topology, channel error rate, congestion in the sender/receiver path, etc.) . In this work we focus on FEC techniques. FEC works by adding redundant information to the stream being transmitted in order to reconstruct the original stream at the receiver when packet losses occur. Among its advantages are the small delays introduced to recover information compared to retransmission, for instance, and simplicity of implementation . A clear disadvantage is the increase in the transmission rate due to the redundancy added. One of the simplest approaches among FEC techniques that have been proposed in the literature [11] is to add a packet to each group of N 1 packets with a payload equal to the result of an XOR operation performed in the group. Clearly, if one packet out of the N is lost, the stream is completely recovered . This is accomplished at the expense ofextra bandwidth, in this example an increase of 1/(N 1) . In [4] a clever FEC scheme was proposed in which each packet carries a sample of the previous transmitted packet but compression is used to reduce the overhead . In [7] another technique was developed aimed at maintaining a good compromise between recovery efficiency and bandwidth usage. Since packet losses are mostly due to congestion, increasing the amount of protection using FEC may be unfair to flow controlled streams and also may even have an adverse effect on the results [1] . Therefore, the FEC algorithm should be carefully chosen according to the error characteristics ofthe path between the sender and receiver. Another issue is to develop accurate models ofthe loss process . Many studies exist to characterize the loss process [2, 14, 8] . Simple models such as the Bernoulli process, the Gilbert model and sophisticated Markov models have been proposed. Using either the Bernoulli or the Gilbert process, the work of[3] propose an adaptive algorithm based on the FECscheme of[4] . Recently, the work of Salamatian and Vaton [13] has shown that Hidden

Supervoxel segmentation has strong potential to be incorporated into early video analysis as superpixel segmentation has in image analysis. However, there are many plausible supervoxel methods and little understanding as to when and where each is most appropriate. Indeed, we are not aware of a single comparative study on supervoxel segmentation. To that end, we study seven supervoxel algorithms, including both off-line and streaming methods, in the context of what we consider to be a good supervoxel: namely, spatiotemporal uniformity, object/region boundary detection, region compression and parsimony. For the evaluation we propose a comprehensive suite of seven quality metrics to measure these desirable supervoxel characteristics. In addition, we evaluate the methods in a supervoxel classification task as a proxy for subsequent high-level uses of the supervoxels in video analysis. We use six existing benchmark video datasets with a variety of content-types and dense human annotations. Our findings have led us to conclusive evidence that the hierarchical graph-based (GBH), segmentation by weighted aggregation (SWA) and temporal superpixels (TSP) methods are the top-performers among the seven methods. They all perform well in terms of segmentation accuracy, but vary in regard to the other desiderata: GBH captures object boundaries best; SWA has the best potential for region compression; and TSP achieves the best undersegmentation error.

Motion detection is very important in image processing. One way of detecting motion is using optical flow. Optical flow cannot be computed locally, since only one independent measurement is available from the image sequence at a point, while the flow velocity has two components. A second constraint is needed. The method used for finding the optical flow in this project is assuming that the apparent velocity of the brightness pattern varies smoothly almost everywhere in the image. This technique is later used in developing software for motion detection which has the capability to carry out four types of motion detection. The motion detection software presented in this project also can highlight motion region, count motion level as well as counting object numbers. Many objects such as vehicles and human from video streams can be recognized by applying optical flow technique. Keywords—Background modeling, Motion detection, Optical flow, Velocity smoothness constant, motion trajectories.

We are in an age often referred to as the information age. In this information age, because we believe that information leads to power and success, from the technologies such as computers, satellites, etc., we have been collecting tremendous amounts of information. Our ability to analyze and understand massive datasets lags far behind our ability to gather and store data. Image and video data contains abundant, rich information for data miners to explore. On one hand, the rich literature on image and video data analysis will naturally provide many advanced methods that may help mining other kinds of data. On the other hand, recent research on data mining will also provide some new, interesting methods that may benefit image and video data retrieval and analysis. Today, a lot of data is available everywhere but the ability to understand and make use of that data is very less. Whether the context is business, medicine, science or government, the datasets themselves are of little value. What is of value is the knowledge that can be inferred from the data and put to use. We need systems which would analyze the data for us. This paper basically aims to find out important pixels of an image using one of the classification technique named as decision tree (ID-3). Our aim is to separate the important and unimportant pixels of an image using simple rules. Further one of the compression techniques named as Huffman algorithm is applied for image compression. Finally, resultant image is stored with lesser space complexity .

In this paper, a neural network approach for color classification in HSV color space based on color JND (just noticeable difference) is presented. The HSV color samples are generated using an interactive tool based on JND concept and these samples are used for supervised training. A four layer feed forward neural network is trained for classifying a given color pair as similar or dissimilar pair. An interactive tool for color pixel comparison is developed for testing the color classifier on real life images. Research shows that neural network classifier in HSV color space works better than RGB classifier in terms of efficiency for segmentation purposes. Thus the experimentation results can be used for segmenting real life images.


Question: A unified kernel method for twin learning in similarity and clustering tasks.

This paper presents a novel technique for nonlinear dimensionality reduction known as locally linear embedding (LLE), which is particularly useful in the fields of science that rely on data analysis and visualization. The challenge of handling large, multivariate datasets often leads to the need for dimensionality reduction, aiming to find concise representations of high-dimensional data. LLE, as an unsupervised learning algorithm, achieves this by computing low-dimensional embeddings that preserve the neighborhood relationships of the original high-dimensional inputs. Unlike clustering methods that focus on local dimensionality reduction, LLE creates a unified, global coordinate system of lower dimensionality for all data points, avoiding the pitfalls of local minima in its optimization process. By capitalizing on the local symmetries in linear reconstructions, LLE is capable of uncovering the global structure of complex, nonlinear manifolds, including those found in image datasets of faces and textual data from documents. This makes LLE a versatile tool for uncovering patterns in a wide range of datasets, from visual imagery to textual information.

Metric learning has been shown to be highly effective to improve the performance of nearest neighbor classification. In this paper, we address the problem of metric learning for symmetric positive definite (SPD) matrices such as covariance matrices, which arise in many real-world applications. Naively using standard Mahalanobis metric learning methods under the Euclidean geometry for SPD matrices is not appropriate, because the difference of SPD matrices can be a non-SPD matrix and thus the obtained solution can be uninterpretable. To cope with this problem, we propose to use a properly parameterized LogEuclidean distance and optimize the metric with respect to kernel-target alignment, which is a supervised criterion for kernel learning. Then the resulting non-trivial optimization problem is solved by utilizing the Riemannian geometry. Finally, we experimentally demonstrate the usefulness of our LogEuclidean metric learning algorithm on real-world classification tasks for EEG signals and texture patches.

This paper addresses the nearest neighbor search problem under inner product similarity and introduces a compact code-based approach. The idea is to approximate a vector using the composition of several elements selected from a source dictionary and to represent this vector by a short code composed of the indices of the selected elements. The inner product between a query vector and a database vector is efficiently estimated from the query vector and the short code of the database vector. We show the superior performance of the proposed group M -selection algorithm that selects M elements from M source dictionaries for vector approximation in terms of search accuracy and efficiency for compact codes of the same length via theoretical and empirical analysis. Experimental results on large-scale datasets (1M and 1B SIFT features, 1M linear models and Netflix) demonstrate the superiority of the proposed approach.

This survey is an introduction to positive definite kernels and the set of methods they have inspired in the machine learning literature, namely kernel methods. We first discuss some properties of positive definite kernels as well as reproducing kernel Hibert spaces, the natural extension of the set of functions {k(x, ·), x ∈ X} associated with a kernel k defined on a space X . We discuss at length the construction of kernel functions that take advantage of well-known statistical models. We provide an overview of numerous data-analysis methods which take advantage of reproducing kernel Hilbert spaces and discuss the idea of combining several kernels to improve the performance on certain tasks. We also provide a short cookbook of different kernels which are particularly useful for certain datatypes such as images, graphs or speech segments. Remark: This report is a draft. I apologize in advance for the numerous mistakes, typos, and not always well written parts it contains. Comments and suggestions will be highly appreciated.

The proliferation and ubiquity of temporal data across many disciplines has generated substantial interest in the analysis and mining of time series. Clustering is one of the most popular data mining methods, not only due to its exploratory power, but also as a preprocessing step or subroutine for other techniques. In this paper, we describe k-Shape, a novel algorithm for time-series clustering. k-Shape relies on a scalable iterative refinement procedure, which creates homogeneous and well-separated clusters. As its distance measure, k-Shape uses a normalized version of the cross-correlation measure in order to consider the shapes of time series while comparing them. Based on the properties of that distance measure, we develop a method to compute cluster centroids, which are used in every iteration to update the assignment of time series to clusters. An extensive experimental evaluation against partitional, hierarchical, and spectral clustering methods, with the most competitive distance measures, showed the robustness of k-Shape. Overall, k-Shape emerges as a domain-independent, highly accurate, and efficient clustering approach for time series with broad applications.

This paper introduces a shape-based similarity measure, called the angular metric for shape similarity (AMSS), for time series data. Unlike most similarity or dissimilarity measures, AMSS is based not on individual data points of a time series but on vectors equivalently representing it. AMSS treats a time series as a vector sequence to focus on the shape of the data and compares data shapes by employing a variant of cosine similarity. AMSS is, by design, expected to be robust to time and amplitude shifting and scaling, but sensitive to short-term oscillations. To deal with the potential drawback, ensemble learning is adopted, which integrates data smoothing when AMSS is used for classification. Evaluative experiments reveal distinct properties of AMSS and its effectiveness when applied in the ensemble framework as compared to existing measures.

All clustering algorithms process unlabeled data and, consequently, suffer from two problems: (P1) choosing and validating the correct number of clusters and (P2) insuring that algorithmic labels correspond to meaningful physical labels. Clustering algorithms such as hard and fuzzy c-means, based on optimizing sums of squared errors objective functions, suffer from a third problem: (P3) a tendency to recommend solutions that equalize cluster populations. The semi-supervised c-means algorithms introduced in this paper attempt to overcome these three problems for problem domains where a few data from each class can be labeled. Segmentation of magnetic resonance images is a problem of this type and we use it to illustrate the new algorithm. Our examples show that the semi-supervised approach provides MRI segmentations that are superior to ordinary fuzzy c-means and to the crisp k-nearest neighbor rule and further, that the new method ameliorates (P1)-(P3). Cluster analysis Fuzzy c-means Partial supervision Image segmentation Magnetic resonance images

In semantic web, ontology plays an important role to provide formal definitions of concepts and relationships. Therefore, communicating similar ontologies becomes essential to provide ontologies interpretability and extendibility. Thus, it is inevitable to have similar but not the same ontologies in a particular domain, since there might be several definitions for a given concept. This paper presents a method to combine similarity measures of different categories without having ontology instances or any user feedbacks towards aligning two given ontologies. To align different ontologies efficiently, K Nearest Neighbor (KNN), Support Vector Machine (SVM), Decision Tree (DT) and AdaBoost classifiers are investigated. Each classifier is optimized based on the lower cost and better classification rate. Experimental results demonstrate that the F-measure criterion improves to 99% using feature selection and combination of AdaBoost and DT classifiers, which is highly comparable, and outperforms the previous reported F-measures. Computer Engineering Department, Shahid Chamran University, Ahvaz, Iran.

This paper introduces a new similarity measure, the covering similarity, which we formally define for evaluating the similarity between a symbolic sequence and a set of symbolic sequences. A pairwise similarity can also be directly derived from the covering similarity to compare two symbolic sequences. An efficient implementation to compute the covering similarity is proposed which uses a suffix-tree data structure, but other implementations, based on suffix array for instance, are possible and are possibly necessary for handling very large-scale problems. We have used this similarity to isolate attack sequences from normal sequences in the scope of host-based intrusion detection. We have assessed the covering similarity on two well-known benchmarks in the field. In view of the results reported on these two datasets for the state-of-the-art methods, according to the comparative study, we have carried out based on three challenging similarity measures commonly used for string processing, or in bioinformatics, we show that the covering similarity is particularly relevant to address the detection of anomalies in sequences of system calls.

The amounts of currently produced data emphasize the importance of techniques for efficient data processing. Searching big data collections according to similarity of data well corresponds to human perception. This paper is focused on similarity search using the concept of sketches – a compact bit string representations of data objects compared by Hamming distance, which can be used for filtering big datasets. The object-to-sketch transformation is a form of the dimensionality reduction and thus there are two basic contradictory requirements: (1) The length of the sketches should be small for efficient manipulation, but (2) longer sketches retain more information about the data objects. First, we study various sketching methods for data modeled by metric space and we analyse their quality. Specifically, we study importance of several sketch properties for similarity search and we propose a high quality sketching technique. Further, we focus on the length of sketches by studying mutual influence of sketch properties such as correlation of their bits and the intrinsic dimensionality of a set of sketches. The outcome is an equation that allows us to estimate a suitable length of sketches for an arbitrary given dataset. Finally, we empirically verify proposed approach on two real-life datasets.

We address the problem of visual category recognition by learning an image-to-image distance function that attempts to satisfy the following property: the distance between images from the same category should be less than the distance between images from different categories. We use patch-based feature vectors common in object recognition work as a basis for our image-to-image distance functions. Our large-margin formulation for learning the distance functions is similar to formulations used in the machine learning literature on distance metric learning, however we differ in that we learn local distance functions¿a different parameterized function for every image of our training set¿whereas typically a single global distance function is learned. This was a novel approach first introduced in Frome, Singer, & Malik, NIPS 2006. In that work we learned the local distance functions independently, and the outputs of these functions could not be compared at test time without the use of additional heuristics or training. Here we introduce a different approach that has the advantage that it learns distance functions that are globally consistent in that they can be directly compared for purposes of retrieval and classification. The output of the learning algorithm are weights assigned to the image features, which is intuitively appealing in the computer vision setting: some features are more salient than others, and which are more salient depends on the category, or image, being considered. We train and test using the Caltech 101 object recognition benchmark.


Question: Twin learning for similarity and clustering: a unified method using kernel functions.

This paper introduces a novel approach for robust subspace clustering that utilizes a smoothed rank approximation, offering a more accurate solution than traditional nuclear norm methods. In various fields from signal processing to machine learning, the problem of minimizing matrix rank under affine constraints is common. The nuclear norm, while a convex relaxation that can precisely recover rank under certain theoretical conditions, often fails to achieve optimal results in practical scenarios. To address this limitation, we propose an enhanced rank approximation based on the Logarithm-Determinant (LogDet) function. Our approach is specifically tailored for subspace clustering, capable of handling diverse error types and noise. A robust optimization strategy is devised, complete with theoretical proof of convergence to a stationary point. When applied to tasks such as face clustering and motion segmentation, the proposed method outperforms leading subspace clustering algorithms, showcasing its effectiveness and potential for real-world applications.

We consider the problem of learning a local metric in order to enhance the performance of nearest neighbor classification. Conventional metric learning methods attempt to separate data distributions in a purely discriminative manner; here we show how to take advantage of information from parametric generative models. We focus on the bias in the information-theoretic error arising from finite sampling effects, and find an appropriate local metric that maximally reduces the bias based upon knowledge from generative models. As a byproduct, the asymptotic theoretical analysis in this work relates metric learning to dimensionality reduction from a novel perspective, which was not understood from previous discriminative approaches. Empirical experiments show that this learned local metric enhances the discriminative nearest neighbor performance on various datasets using simple class conditional generative models such as a Gaussian.

We describe the application of kernel methods to Natural Language Processing (NLP) problems. In many NLP tasks the objects being modeled are strings, trees, graphs or other discrete structures which require some mechanism to convert them into feature vectors. We describe kernels for various natural language structures, allowing rich, high dimensional representations of these structures. We show how a kernel over trees can be applied to parsing using the voted perceptron algorithm, and we give experimental results on the ATIS corpus of parse trees.

A string similarity measure quantifies the similarity between two text strings for approximate string matching or comparison. For example, the strings "Sam" and "Samuel" can be considered similar. Most existing work that computes the similarity of two strings only considers syntactic similarities, e.g., number of common words or q-grams. While these are indeed indicators of similarity, there are many important cases where syntactically different strings can represent the same real-world object. For example, "Bill" is a short form of "William". Given a collection of predefined synonyms, the purpose of the paper is to explore such existing knowledge to evaluate string similarity measures more effectively and efficiently, thereby boosting the quality of string matching.
 In particular, we first present an expansion-based framework to measure string similarities efficiently while considering synonyms. Because using synonyms in similarity measures is, while expressive, computationally expensive (NP-hard), we propose an efficient algorithm, called selective-expansion, which guarantees the optimality in many real scenarios. We then study a novel indexing structure called SI-tree, which combines both signature and length filtering strategies, for efficient string similarity joins with synonyms. We develop an estimator to approximate the size of candidates to enable an online selection of signature filters to further improve the efficiency. This estimator provides strong low-error, high-confidence guarantees while requiring only logarithmic space and time costs, thus making our method attractive both in theory and in practice. Finally, the results from an empirical study of the algorithms verify the effectiveness and efficiency of our approach.

Users are constantly involved in a multitude of activities in ever-changing context. Analyzing activities in contextrich environments has become a great challenge in contextawareness research. Traditional methods for activity recognition, such as classification, cannot cope with the variety and dynamicity of context and activities. In this paper, we propose an activity recognition approach that incorporates unsupervised learning. We analyze the feasibility of applying subspace clustering—a specific type of unsupervised learning— to high-dimensional, heterogeneous sensory input. Then we present the correspondence between clustering output and classification input. This approach has the potential to discover implicit, evolving activities, and can provide valuable assistance to traditional classification based methods. As sensors become prevalent means in context detection and information channels proliferate to make context sharing easier, it is increasingly challenging to interpret context and analyze its effects on the activities (Lim and Dey 2010). We argue that applying traditional approaches to activity recognition may become more and more difficult to apply in context and activity-rich environments. In the literature, context attributes used for learning activities are chosen by either empirical assumption or dimension reduction to render a small set of features (Krause, Smailagic, and Siewiorek 2006). These approaches are infeasible in face of a broad spectrum of context information. The most significant drawback is that they fail to acknowledge the large variety of features needed to describe different activities. For activity recognition, most previous works applied supervised learning approaches that aimed at predicting activities among a set of known classes(Ferscha et al. 2004). These approaches, however, are also challenged when coping with new and fast evolving activities. Unsupervised learning, particularly clustering, has been highly successful for revealing implicit relationships and regularities in large data sets. Intuitively, we can envisage an activity recognition approach that applies clustering to context history. The clusters, representing frequent context patterns, can suggest activities and their contextual conditions. The results can be used independently for analyzing and interpreting activities. Furthermore, clusters can reveal the scopes and conditions Copyright c © 2011, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. of activities and interactions. This information is valuable when determining the scopes of information sharing in pervasive environments. Although clustering is a promising approach in discovering associations within context, the feasibility of traditional clustering is questionable in dealing with high dimensionality and heterogeneity of context data. In this paper, we will first conduct a detailed analysis about the challenges to apply clustering for activity recognition. Afterwards we introduce two recent subspace clustering methods that can address these challenges. Lastly, based on the analysis of unsupervised activity recognition, we will propose an activity recognition framework that incorporates clustering in conventional classification. We will show that the two directions are complementary to each other, and developing a hybrid approach will greatly benefit activity context awareness. Unsupervised activity learning

In this paper we introduce the deep kernelized autoencoder, a neural network model that allows an explicit approximation of (i) the mapping from an input space to an arbitrary, user-specified kernel space and (ii) the back-projection from such a kernel space to input space. The proposed method is based on traditional autoencoders and is trained through a new unsupervised loss function. During training, we optimize both the reconstruction accuracy of input samples and the alignment between a kernel matrix given as prior and the inner products of the hidden representations computed by the autoencoder. Kernel alignment provides control over the hidden representation learned by the autoencoder. Experiments have been performed to evaluate both reconstruction and kernel alignment performance. Additionally, we applied our method to emulate kPCA on a denoising task obtaining promising results.

This paper presents a new approach to hierarchical clustering of very large data sets, named GridClustering. The method organizes unlike the conventional methods the space surrounding the patterns and not the patterns. It uses a multidimensional grid data structure. The resulting block partitioning of the value space is clustered via a topological neighbor search. The Grid-Clustering method is able to deliver structural pattern distribution information for very large data sets. It superceeds all conventional hierarchical algorithms in runtime behavior and memory space requirements. The algorithm was analyzed within a testbed and suitable values for the tunable parameters of the algorithm are proposed. A comparison of the executions times to other commonly used clustering algorithms and a heuristic runtime analysis is given.

Word similarity computation has important applications in many areas, such as natural language processing, intelligence information retrieval, document clustering, automatic answering, word sense disambiguation, machine translation etc. This article makes an intensive study of word similarity computation based on HowNet and the word similarity is computed in three steps: (1) computes sememes similarity, (2) computes concepts similarity using the weight sum method of sememes similarity, (3) takes the maximum similarity of concepts as words similarity. This article mainly introduces numbering the sememes and presents a more accurate method of decomposing the concept description type. The experiment shows that the algorithm of word similarity presented in this article is simple and feasible, and the computed results are relatively consistent with awareness.

Name disambiguation can occur when one is seeking a list of publications of an author who has used different name variations and when there are multiple other authors with the same name. We present an efficient integrative framework for solving the name disambiguation problem: a blocking method retrieves candidate classes of authors with similar names and a clustering method, DBSCAN, clusters papers by author. The distance metric between papers used in DBSCAN is calculated by an online active selection support vector machine algorithm (LASVM), yielding a simpler model, lower test errors and faster prediction time than a standard SVM. We prove that by recasting transitivity as density reachability in DBSCAN, transitivity is guaranteed for core points. For evaluation, we manually annotated 3,355 papers yielding 490 authors and achieved 90.6% pairwise-F1. For scalability, authors in the entire CiteSeer dataset, over 700,000 papers, were readily disambiguated.

Unlabeled document collections are becoming increasingly common and available; mining such data sets represents a major contemporary challenge. Using words as features, text documents are often represented as high-dimensional and sparse vectors–a few thousand dimensions and a sparsity of 95 to 99% is typical. In this paper, we study a certain spherical k-means algorithm for clustering such document vectors. The algorithm outputs k disjoint clusters each with a concept vector that is the centroid of the cluster normalized to have unit Euclidean norm. As our first contribution, we empirically demonstrate that, owing to the high-dimensionality and sparsity of the text data, the clusters produced by the algorithm have a certain “fractal-like” and “self-similar” behavior. As our second contribution, we introduce concept decompositions to approximate the matrix of document vectors; these decompositions are obtained by taking the least-squares approximation onto the linear subspace spanned by all the concept vectors. We empirically establish that the approximation errors of the concept decompositions are close to the best possible, namely, to truncated singular value decompositions. As our third contribution, we show that the concept vectors are localized in the word space, are sparse, and tend towards orthonormality. In contrast, the singular vectors are global in the word space and are dense. Nonetheless, we observe the surprising fact that the linear subspaces spanned by the concept vectors and the leading singular vectors are quite close in the sense of small principal angles between them. In conclusion, the concept vectors produced by the spherical k-means algorithm constitute a powerful sparse and localized “basis” for text data sets.

One of the key obstacles in making learning protocols realistic in applications is the need to supervise them, a costly process that often requires hiring domain experts. We consider the framework to use the world knowledge as indirect supervision. World knowledge is general-purpose knowledge, which is not designed for any specific domain. Then, the key challenges are how to adapt the world knowledge to domains and how to represent it for learning. In this article, we provide an example of using world knowledge for domain-dependent document clustering. We provide three ways to specify the world knowledge to domains by resolving the ambiguity of the entities and their types, and represent the data with world knowledge as a heterogeneous information network. Then, we propose a clustering algorithm that can cluster multiple types and incorporate the sub-type information as constraints. In the experiments, we use two existing knowledge bases as our sources of world knowledge. One is Freebase, which is collaboratively collected knowledge about entities and their organizations. The other is YAGO2, a knowledge base automatically extracted from Wikipedia and maps knowledge to the linguistic knowledge base, WordNet. Experimental results on two text benchmark datasets (20newsgroups and RCV1) show that incorporating world knowledge as indirect supervision can significantly outperform the state-of-the-art clustering algorithms as well as clustering algorithms enhanced with world knowledge features.
 A preliminary version of this work appeared in the proceedings of KDD 2015 [Wang et al. 2015a]. This journal version has made several major improvements. First, we have proposed a new and general learning framework for machine learning with world knowledge as indirect supervision, where document clustering is a special case in the original paper. Second, in order to make our unsupervised semantic parsing method more understandable, we add several real cases from the original sentences to the resulting logic forms with all the necessary information. Third, we add details of the three semantic filtering methods and conduct deep analysis of the three semantic filters, by using case studies to show why the conceptualization-based semantic filter can produce more accurate indirect supervision. Finally, in addition to the experiment on 20 newsgroup data and Freebase, we have extended the experiments on clustering results by using all the combinations of text (20 newsgroup, MCAT, CCAT, ECAT) and world knowledge sources (Freebase, YAGO2).


Question: A unified kernel method for twin learning in similarity and clustering tasks.

This paper introduces a novel clustering method, leveraging sparse representation (SR), designed to group data points originating from various low-dimensional linear or affine subspaces embedded within a high-dimensional space. The methodology is anchored in the premise that every point within the union of these subspaces possesses a SR relative to a dictionary constructed from the remaining data points. Notably, while identifying such a SR is generally an NP-hard problem, our primary innovation lies in demonstrating that, under certain conditions, this SR can be accurately determined through l1 optimization. Following the determination of SRs, the data is segmented by applying spectral clustering to a similarity matrix derived from these representations. Our method is robust, capable of managing noise, outliers, and incomplete data. To illustrate its efficacy, we apply our subspace clustering technique to the challenge of segmenting multiple motions in video sequences. Rigorous testing on a diverse set of 167 video sequences reveals that our approach markedly surpasses current state-of-the-art methods in terms of performance.

We propose a new method, called SimClus, for clustering with lower bound on similarity. Instead of accepting k the number of clusters to find, the alternative similarity-based approach imposes a lower bound on the similarity between an object and its corresponding cluster representative (with one representative per cluster). SimClus achieves a O(log n) approximation bound on the number of clusters, whereas for the best previous algorithm the bound can be as poor as O(n). Experiments on real and synthetic datasets show that our algorithm produces more than 40% fewer representative objects, yet offers the same or better clustering quality. We also propose a dynamic variant of the algorithm, which can be effectively used in an on-line setting.

Compatibility between items, such as clothes and shoes, is a major factor among customer’s purchasing decisions. However, learning “compatibility” is challenging due to (1) broader notions of compatibility than those of similarity, (2) the asymmetric nature of compatibility, and (3) only a small set of compatible and incompatible items are observed. We propose an end-to-end trainable system to embed each item into a latent vector and project a query item into K compatible prototypes in the same space. These prototypes reflect the broad notions of compatibility. We refer to both the embedding and prototypes as “Compatibility Family”. In our learned space, we introduce a novel Projected Compatibility Distance (PCD) function which is differentiable and ensures diversity by aiming for at least one prototype to be close to a compatible item, whereas none of the prototypes are close to an incompatible item. We evaluate our system on a toy dataset, two Amazon product datasets, and Polyvore outfit dataset. Our method consistently achieves state-of-the-art performance. Finally, we show that we can visualize the candidate compatible prototypes using a Metric-regularized Conditional Generative Adversarial Network (MrCGAN), where the input is a projected prototype and the output is a generated image of a compatible item. We ask human evaluators to judge the relative compatibility between our generated images and images generated by CGANs conditioned directly on query items. Our generated images are significantly preferred, with roughly twice the number of votes as others.

The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples (n→∞). The next phase is likely to focus on algorithms capable of learning from very few labeled examples (n → 1), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a “good” representation for supervised learning, characterized by small sample complexity (n). We consider the case of visual object recognition though the theory applies to other domains. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translations, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and unique (discriminative) signature can be computed for each image patch, I, in terms of empirical distributions of the dot-products between I and a set of templates stored during unsupervised learning. A module performing filtering and pooling, like the simple and complex cells described by Hubel and Wiesel, can compute such estimates. Hierarchical architectures consisting of this basic Hubel-Wiesel moduli inherit its properties of invariance, stability, and discriminability while capturing the compositional organization of the visual world in terms of wholes and parts. The theory extends existing deep learning convolutional architectures for image and speech recognition. It also suggests that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and discriminative for recognition—and that this representation may be continuously learned in an unsupervised way during development and visual experience. This work was supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF 1231216. http://cbmm.mit.edu CBMM Paper March, 2014 1–24 ar X iv :1 31 1. 41 58 v5 [ cs .C V ] 1 1 M ar 2 01 4 Unsupervised learning of invariant representations with low sample complexity: the magic of sensory cortex or a new framework for machine learning? Fabio Anselmi ∗ †, Joel Z. Leibo ∗ , Lorenzo Rosasco ∗ † , Jim Mutch ∗ , Andrea Tacchetti ∗ † , and Tomaso Poggio ∗ † ∗Center for Brains, Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA 02139, and †Istituto Italiano di Tecnologia, Genova, 16163 The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples (n→∞). The next phase is likely to focus on algorithms capable of learning from very few labeled examples (n → 1), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a “good” representation for supervised learning, characterized by small sample complexity (n). We consider the case of visual object recognition though the theory applies to other domains. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translations, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and unique (discriminative) signature can be computed for each image patch, I, in terms of empirical distributions of the dot-products between I and a set of templates stored during unsupervised learning. A module performing filtering and pooling, like the simple and complex cells described by Hubel and Wiesel, can compute such estimates. Hierarchical architectures consisting of this basic Hubel-Wiesel moduli inherit its properties of invariance, stability, and discriminability while capturing the compositional organization of the visual world in terms of wholes and parts. The theory extends existing deep learning convolutional architectures for image and speech recognition. It also suggests that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and discriminative for recognition—and that this representation may be continuously learned in an unsupervised way during development and visual experience.1 Invariance | Hierarchy | Convolutional networks | Visual cortex It is known that Hubel and Wiesel’s original proposal [1] for visual area V1—of a module consisting of complex cells (C-units) combining the outputs of sets of simple cells (S-units) with identical orientation preferences but differing retinal positions—can be used to construct translationinvariant detectors. This is the insight underlying many networks for visual recognition, including HMAX [2] and convolutional neural nets [3, 4]. We show here how the original idea can be expanded into a comprehensive theory of visual recognition relevant for computer vision and possibly for visual cortex. The first step in the theory is the conjecture that a representation of images and image patches, with a feature vector that is invariant to a broad range of transformations—such as translation, scale, expression of a face, pose of a body, and viewpoint—makes it possible to recognize objects from only a few labeled examples, as humans do. The second step is proving that hierarchical architectures of Hubel-Wiesel (‘HW’) modules (indicated by ∧ in Fig. 1) can provide such invariant representations while maintaining discriminative information about the original image. Each ∧ -module provides a feature vector, which we call a signature, for the part of the visual field that is inside its “receptive field”; the signature is invariant to (R) affine transformations within the receptive field. The hierarchical architecture, since it computes a set of signatures for l=4

The objective of cluster analysis is to partition a set of data points into several groups based on a suitable distance measure. We first propose a model called local gravitation among data points. In this model, each data point is viewed as an object with mass, and associated with a local resultant force (LRF) generated by its neighbors. The motivation of this paper is that there exist distinct differences between the LRFs (including magnitudes and directions) of the data points close to the cluster centers and at the boundary of the clusters. To capture these differences efficiently, two new local measures named centrality and coordination are further investigated. Based on empirical observations, two new clustering methods called local gravitation clustering and communication with local agents are designed, and several test cases are conducted to verify their effectiveness. The experiments on synthetic data sets and real-world data sets indicate that both clustering approaches achieve good performance on most of the data sets.

Semantic similarity is an essential component of many Natural Language Processing applications. However, prior methods for computing semantic similarity often operate at different levels, e.g., single words or entire documents, which requires adapting the method for each data type. We present a unified approach to semantic similarity that operates at multiple levels, all the way from comparing word senses to comparing text documents. Our method leverages a common probabilistic representation over word senses in order to compare different types of linguistic data. This unified representation shows state-ofthe-art performance on three tasks: semantic textual similarity, word similarity, and word sense coarsening.

Recent developments in computer vision have shown that local features can provide efficient representations suitable for robust object recognition. Support Vector Machines have been established as powerful learning algorithms with good generalization capabilities. In this paper, we combine these two approaches and propose a general kernel method for recognition with local features. We show that the proposed kernel satisfies the Mercer condition and that it is suitable for many established local feature frameworks. Large-scale recognition results are presented on three different databases, which demonstrate that SVMs with the proposed kernel perform better than standard matching techniques on local features. In addition, experiments on noisy and occluded images show that local feature representations significantly outperform global approaches.

We argue that similarity judgments are inferences about generative processes, and that two objects appear similar when they are likely to have been generated by the same process. We describe a formal model based on this idea and show how featural and spatial models emerge as special cases. We compare our approach to the transformational approach, and present an experiment where our model performs better than a transformational model. Every object is the outcome of a generative process. An animal grows from a fertilized egg into an adult, a city develops from a settlement into a metropolis, and an artifact is assembled from a pile of raw materials according to the plan of its designer. Observations like these motivate the generative approach, which proposes that an object may be understood by thinking about the process that generated it. The promise of the approach is that apparently complex objects may be produced by simple processes, an insight that has proved productive across disciplines including biology [18], physics [21], and architecture [1]. To give two celebrated examples from biology, the shape of a pinecone and the markings on a cheetah’s tail can be generated by remarkably simple processes of growth. These patterns can be characterized much more compactly by describing their causal history than by attempting to describe them directly. Leyton has argued that the generative approach provides a general framework for understanding cognition. Applications of the approach can be found in generative theories of perception [12], memory [12], language [3], categorization [2], and music [11]. This paper offers a generative theory of similarity, a notion often invoked by models of high-level cognition. We argue that two objects are similar to the extent that they seem to have been generated by the same underlying process. The literature on similarity covers settings that extend from the comparison of simple stimuli like tones and colored patches to the comparison of highly-structured objects like narratives. The generative approach is relevant to the entire spectrum of applications, but we are particularly interested in high-level similarity. In particular, we are interested in how similarity judgments draw on intuitive theories, or systems of rich conceptual knowledge [15]. Generative processes and theories are intimately linked. Murphy [14], for example, defines a theory as ‘a set of causal relations that collectively generate or explain the phenomena in a domain.’ We hope that our generative theory provides a framework in which to model how similarity judgments emerge from intuitive theories. We develop a formal theory of similarity and compare it to three existing theories. The featural account [20] suggests that the similarity of two objects is a function of their common and distinctive features, the spatial account suggests that similarity is inversely proportional to distance in a spatial representation, [19] and the transformation account suggests that similarity depends on the number of operations required to transform one object into the other [6]. We show that versions of each of these approaches emerge as special cases of our generative approach, and present an experiment that directly compares our approach with the transformation account. A fourth theory suggests that similarity relies on a process of analogical mapping [5]. We will not discuss this approach in detail, but finish by suggesting how a generative approach to analogy differs from the standard view. Generative processes and similarity Before describing our formal model, we give an informal motivation for a generative approach to similarity. Suppose we are shown a prototype object and asked to describe similar objects we might find in the world. There are two kinds of answers: small perturbations of the prototype, or objects produced by small perturbations of the process that generated the prototype. The second strategy is likely to be more successful than the first, since many perturbations of the prototype will not arise from any plausible generative process, and thus could never appear in practice. By construction, however, an object produced by a perturbation of an existing generative process will have a plausible causal history. To give a concrete example, suppose the prototype is a bug generated by a biological process of growth (Figure 1ii). The bug in i is a small perturbation of the prototype, but seems unlikely to arise since legs are generated in pairs. A perturbation of the generative process might produce a bug with more segments, such as the bug in iii. If we hope to find a bug that is similar but not identical to the prototype, iii is a better bet than i. A sceptic might argue that this one-shot learning problem can be solved by taking the intersection of the set of objects similar to the prototype and the set of obi) ii) Prototype iii) Figure 1: Three bugs. Which is more similar to the prototype — i or iii? jects that are likely to exist. The second set depends critically on generative processes, but the first set (and therefore the notion of similarity) need not. We think it more likely that the notion of similarity is ultimately grounded in the world, and that it evolved for the purpose of comparing real-world objects. If so, then knowledge about what kinds of objects are likely to exist may be deeply bound up with the notion of similarity. The one-shot learning problem is of practical importance, but is not the standard context in which similarity is discussed. More commonly, subjects are shown a pair of objects and asked to rate the similarity of the pair. Note that both objects are observed to exist and the previous argument does not apply. Yet generative processes are still important, since they help pick out the features critical for the similarity comparison. Suppose, for instance, that a forest-dweller discovers a nutritious mushroom. Which is more similar to the mushroom: a mushroom identical except for its size, or a mushroom identical except for its color? Knowing how mushrooms are formed suggests that size is not a key feature. Mushrooms grow from small to large, and the final size of a plant depends on factors like the amount of sunlight it received and the fertility of the soil that it grew in. Reflections like these suggest that the differently-sized mushroom should be judged more similar. A final reason why generative processes matter is that they are deeply related to essentialism. Medin and Ortony [13] note that ‘surface features are frequently constrained by, and sometimes generated by, the deeper, more central parts of objects.’ Even if we observe only the surface features of two objects, it may make sense to judge their similarity by comparing the deeper properties inferred to generate the surface features. Yet we can say more: just as surface features are generated by the essence of the object, the essence itself has a generative history. Surface features are often reliable guides to the essence of an object, but the object’s causal history is a still more reliable indicator, if not a defining criterion of its essence. Keil [9] discusses the case of an animal that is born a skunk, then undergoes surgery that leaves it looking exactly like a raccoon. Since the animal is generated in the same way as a skunk (born of skunk parents), we conclude that it remains a skunk, no matter how it appears on the surface. These examples suggest that the generative approach may help to explain a broad class of theory-dependent inferences. We now present a formal model that attempts to capture the intuitions behind all of these cases. A computational theory of similarity Given a domain D, we develop a theory that specifies the similarity between any two samples from D. A sample from D will usually contain a single object, but working with similarities between sets of objects is useful for some applications. We formalize a generative process as a probability distribution over D that depends on parameter vector θ. Suppose that s1 and s2 are samples from D. We consider two hypotheses: H1 holds that s1 and s2 are independent samples from a single generative process, and H2 holds that the samples are generated from two independently chosen processes. Similarity is defined as the probability that the objects are generated by the same process: that is, the relative posterior probability of H1 compared to H2: sim(s1, s2) = P (H1|s1, s2)

We describe the application of kernel methods to Natural Language Processing (NLP) problems. In many NLP tasks the objects being modeled are strings, trees, graphs or other discrete structures which require some mechanism to convert them into feature vectors. We describe kernels for various natural language structures, allowing rich, high dimensional representations of these structures. We show how a kernel over trees can be applied to parsing using the voted perceptron algorithm, and we give experimental results on the ATIS corpus of parse trees.

Many machine learning applications require the ability to learn from and reason about noisy multi-relational data. To address this, several effective representations have been developed that provide both a language for expressing the structural regularities of a domain, and principled support for probabilistic inference. In addition to these two aspects, however, many applications also involve a third aspect–the need to reason about similarities–which has not been directly supported in existing frameworks. This paper introduces probabilistic similarity logic (PSL), a general-purpose framework for joint reasoning about similarity in relational domains that incorporates probabilistic reasoning about similarities and relational structure in a principled way. PSL can integrate any existing domainspecific similarity measures and also supports reasoning about similarities between sets of entities. We provide efficient inference and learning techniques for PSL and demonstrate its effectiveness both in common relational tasks and in settings that require reasoning about similarity.

Kernel method has been developed as one of the standard approaches for nonlinear learning, which however, does not scale to large data set due to its quadratic complexity in the number of samples. A number of kernel approximation methods have thus been proposed in the recent years, among which the random features method gains much popularity due to its simplicity and direct reduction of nonlinear problem to a linear one. Different random feature functions have since been proposed to approximate a variety of kernel functions. Among them the Random Binning (RB) feature, proposed in the first random-feature paper [21], has drawn much less attention than the Random Fourier (RF) feature proposed also in [21]. In this work, we observe that the RB features, with right choice of optimization solver, could be orders-of-magnitude more efficient than other random features and kernel approximation methods under the same requirement of accuracy. We thus propose the first analysis of RB from the perspective of optimization, which by interpreting RB as a Randomized Block Coordinate Descent in the infinite-dimensional space, gives a faster convergence rate compared to that of other random features. In particular, we show that by drawing R random grids with at least κ number of non-empty bins per grid in expectation, RB method achieves a convergence rate of O(1/κ R)), which not only sharpens its O(1/√R) rate from Monte Carlo analysis, but also shows a κ times speedup over other random features under the same analysis framework. In addition, we demonstrate another advantage of RB in the L1-regularized setting, where unlike other random features, a RB-based Coordinate Descent solver can be parallelized with guaranteed speedup proportional to κ. Our extensive experiments demonstrate the superior performance of the RB features over other random features and kernel approximation methods.