Question: Exploration of feature selection methods' performance.

This paper delves into the issue of evaluating attribute quality in machine learning, considering both independent and interdependent attributes. Kira and Rendell introduced the RELIEF algorithm, which proved highly effective in attribute estimation for two-class problems, accommodating both discrete and continuous data. Here, we present an in-depth analysis of RELIEF and propose enhancements to tackle noisy, incomplete, and multi-class datasets. These extensions are rigorously validated using a range of artificial datasets and a widely recognized real-world case study, demonstrating their efficacy in complex data environments.

The development of data-mining applications such as classification and clustering has shown the need for machine learning algorithms to be applied to large scale data. In this paper we present the comparison of different classification techniques using Waikato Environment for Knowledge Analysis or in short, WEKA. WEKA is an open source software which consists of a collection of machine learning algorithms for data mining tasks. The aim of this paper is to investigate the performance of different classification or clustering methods for a set of large data. The algorithm or methods tested are Bayes Network, Radial Basis Function, Pruned Tree, Single Conjunctive Rule Learner and Nearest Neighbors Algorithm. A fundamental review on the selected technique is presented for introduction purposes. The data breast cancer data with a total data of 6291 and a dimension of 699 rows and 9 columns will be used to test and justify the differences between the classification methods or algorithms. Subsequently, the classification technique that has the potential to significantly improve the common or conventional methods will be suggested for use in large scale data, bioinformatics or other general applications. Keywords— Machine Learning, Data Mining, WEKA, Classification, Bioinformatics.

Supervised classification is one of the tasks most frequently carried out by so-called Intelligent Systems. Thus, a large number of techniques have been developed based on Artificial Intelligence (Logic-based techniques, Perceptron-based techniques) and Statistics (Bayesian Networks, Instance-based techniques). The goal of supervised learning is to build a concise model of the distribution of class labels in terms of predictor features. The resulting classifier is then used to assign class labels to the testing instances where the values of the predictor features are known, but the value of the class label is unknown. This paper describes various classification algorithms and the recent attempt for improving classification accuracy—ensembles of classifiers.

With the goal of reducing computational costs without sacrificing accuracy, we describe two algorithms to find sets of prototypes for nearest neighbor classification. Here, the term “prototypes” refers to the reference instances used in a nearest neighbor computation — the instances with respect to which similarity is assessed in order to assign a class to a new data item. Both algorithms rely on stochastic techniques to search the space of sets of prototypes and are simple to implement. The first is a Monte Carlo sampling algorithm; the second applies random mutation hill climbing. On four datasets we show that only three or four prototypes sufficed to give predictive accuracy equal or superior to a basic nearest neighbor algorithm whose run-time storage costs were approximately 10 to 200 times greater. We briefly investigate how random mutation hill climbing may be applied to select features and prototypes simultaneously. Finally, we explain the performance of the sampling algorithm on these datasets in terms of a statistical measure of the extent of clustering displayed by the target classes.

Classification is part of various applications and it is an important problem that represents active research topic. Support vector machine is one of the widely used and very powerful classifier. The accuracy of support vector machine highly depends on learning parameters. Optimal parameters can be efficiently determined by using swarm intelligence algorithms. In this paper, we proposed recent elephant herding optimization algorithm for support vector machine parameter tuning. The proposed approach is tested on standard datasets and it was compared to other approaches from literature. The results of computational experiments show that our proposed algorithm outperformed genetic algorithms and grid search considering accuracy of classification.

An intrusion Detection System (IDS) provides alarts against intrusion attacks where a traditional firewall fails. Machine learning algorithms aim to detect anomalies using supervised and unsupervised approaches. Features selection techniques identify important features and remove irrelevant and redundant attributes to reduce the dimensionality of feature space. This paper presents a features selection framework for efficient network anomaly detection using different machine learning classifiers. The framework applies different strategies by using filter and wrapper features selection methodologies. The aim of this framework is to select the minimum number of features that achieve the highest accuracy. UNSW-NB15 dataset is used in the experimental results to evaluate the proposed framework. The results show that by using 18 features from one of the filter ranking methods and applying J48 as a classifier, an accuracy of 88% is achieved.

Decision Tree Classifiers (DTC's) are used successfully in many diverse areas such as radar signal classification, character recognition, remote sensing, medical diagnosis, expert systems, and speech recognition, to name only a few. Perhaps, the most important feature of DTC's is their capability to break down a complex decision-making process into a collection of simpler decisions, thus providing a solution which is often easier to interpret. This paper presents a survey of current methods for DTC designs and the various existing issues. After considering potential advantages of DTC's over single stage classifiers, subjects of tree structure design, feature selection at each internal node, and decision and search strategies are discussed_ Finally, several remarks are made concerning possible future research directions.

In pattern recognition problems it has been noted that beyond a certain point the inclusion of additional parameters (that have been estimated) leads to higher probabilities of error. A simple problem has been formulated where the probability of error approaches zero as the dimensionality increases and all the parameters are known; on the other hand, the probability of error approaches one-half as the dimensionality increases and parameters are estimated.

We describe the two most commonly used systems for induction of decision trees for classi cation: C4.5 and CART. We highlight the methods and di erent decisions made in each system with respect to splitting criteria, pruning, noise handling, and other di erentiating features. We describe how rules can be derived from decision trees and point to some di erence in the induction of regression trees. We conclude with some pointers to advanced techniques, including ensemble methods, oblique splits, grafting, and coping with large data.

As the number of features in pattern recognition applications continuously grows, new algorithms are necessary to reduce the dimensionality of the feature space while producing comparable results. For example, a dynamic area of research, activity recognition, produces large quantities of high-velocity, high-dimensionality data that require real time classification. While dimensionality reduction approaches such as principle component analysis (PCA) and feature selection work well for datasets of reasonable size and dimensionality, they fail on big data. A possible approach to classification of high-dimensionality datasets is to combine a typical classifier, fuzzy k-nearest neighbor in our case (FKNN), with feature reduction by random projection (RP). As opposed to PCA where one projection matrix is computed based on least square optimality, in RP, a projection matrix is chosen at random multiple times. As the random projection procedure is repeated many times, the question is how to aggregate the values of the classifier obtained in each projection. In this paper we present a fusion strategy for RP FKNN, denoted as RPFKNN. The fusion strategy is based on the class membership values produced by FKNN and classification accuracy in each projection. We test RPFKNN on several synthetic and activity recognition datasets.

Success of optical character recognition depends on a number of factors, two of which are feature extraction and classi cation algorithms. In this paper we look at the results of the application of a set of classi ers to datasets obtained through various basic feature extraction methods.


Question: Exploration of feature selection methods' performance.

This document introduces an unsupervised learning algorithm designed to categorize reviews as either positive (thumbs up) or negative (thumbs down). The classification hinges on the average semantic orientation of phrases within the review that include adjectives or adverbs. A phrase's semantic orientation is considered positive when it conveys favorable connotations (e.g., "subtle nuances"), and negative when it suggests unfavorable ones (e.g., "very cavalier"). The semantic orientation of a phrase is quantified by calculating the mutual information between the phrase and the term "excellent," then subtracting the mutual information between the phrase and the term "poor." A review is deemed positive if the average semantic orientation of its phrases is above zero. When tested on a dataset of 410 reviews from Epinions across four domains—automobiles, banks, movies, and travel destinations—the algorithm achieves a mean accuracy of 74%. This accuracy varies, with automobile reviews showing the highest accuracy at 84%, while movie reviews exhibit the lowest at 66%.

**Question**: A Comparative Study of Different Feature Selection Methods for Text Classification

**Documents**: A Comparative Study of Feature Selection Methods for Text Classification
This paper presents a comparative study of various feature selection methods for text classification. The methods evaluated include Chi-Square, Information Gain, and Document Frequency. The study employs a corpus of 10,000 documents, categorized into five classes, for the evaluation of these methods. The performance of each method is assessed using a Support Vector Machine (SVM) classifier. The results indicate that the Information Gain method performs the best, followed by Chi-Square and Document Frequency. The study also highlights the importance of selecting the appropriate number of features for optimal classification accuracy.

**Similar Documents**: This document offers a comparative analysis of different feature selection techniques used in text classification tasks. Among the methods reviewed are Chi-Square, Information Gain, and Document Frequency. The evaluation is carried out on a dataset consisting of 10,000 documents, which are divided into five distinct categories. To assess the effectiveness of each method, a Support Vector Machine (SVM) classifier is utilized. The findings reveal that Information Gain emerges as the most effective method, closely followed by Chi-Square and Document Frequency. Moreover, the study underscores the significance of determining the ideal number of features to enhance classification accuracy.

Sentiment Analysis (SA) research has increased tremendously in recent times .Sentiment analysis means to extract opinion of users from review documents. Sentiment classification using Machine learning (ML ) methods faces the problem of high dimensionality of feature vector. Therefore, a feature selection method is required to eliminate the irrelevant and noisy features from the feature vector for efficient working of ML algorithms Rough set theory provides an important concept for feature reduction called reduct. The cost of reduct set computation is highly influenced by the attribute size of the dataset where the problem of finding reducts has been proven as NPhard problems.Different feature selection are applied on different data set, Experimental results show that mRMR is better compared to IG for sentiment classification, Hybrid feature selection method based on the RST and Information Gain (IG) is better compared to the previous methods. Proposed methods are evaluated on four standard datasets viz. Movie review, Product (book, DVD, and electronics) reviewed datasets, and Experimental results show that hybrid feature selection method outperforms than feature selection methods for sentimental classification.

Ensemble learning is an intensively studies technique in machine learning and pattern recognition. Recent work in computational biology has seen an increasing use of ensemble learning methods due to their unique advantages in dealing with small sample size, high-dimensionality, and complexity data structures. The aim of this article is two-fold. First, it is to provide a review of the most widely used ensemble learning methods and their application in various bioinformatics problems, including the main topics of gene expression, mass spectrometry-based proteomics, gene-gene interaction identification from genome-wide association studies, and prediction of regulatory elements from DNA and protein sequences. Second, we try to identify and summarize future trends of ensemble methods in bioinformatics. Promising directions such as ensemble of support vector machine, meta-ensemble, and ensemble based feature selection are discussed.

In recent years, text classification have been widely used. Dimension of text data has increased more and more. Working of almost all classification algorithms is directly related to dimension. In high dimension data set, working of classification algorithms both takes time and occurs over fitting problem. So feature selection is crucial for machine learning techniques. In this study, frequently used feature selection metrics Chi Square (CHI), Information Gain (IG) and Odds Ratio (OR) have been applied. At the same time the method Relevancy Frequency (RF) proposed as term weighting method has been used as feature selection method in this study. It is used for tf.idf term as weighting method, Sequential Minimal Optimization (SMO) and Naive Bayes (NB) in the classification algorithm. Experimental results show that RF gives successful results.

Very recently tracking was approached using classification techniques such as support vector machines. The object to be tracked is discriminated by a classifier from the background. In a similar spirit we propose a novel on-line AdaBoost feature selection algorithm for tracking. The distinct advantage of our method is its capability of on-line training. This allows to adapt the classifier while tracking the object. Therefore appearance changes of the object (e.g. out of plane rotations, illumination changes) are handled quite naturally. Moreover, depending on the background the algorithm selects the most discriminating features for tracking resulting in stable tracking results. By using fast computable features (e.g. Haar-like wavelets, orientation histograms, local binary patterns) the algorithm runs in real-time. We demonstrate the performance of the algorithm on several (publically available) video sequences.

While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but ignored. This article reviews the current practice and then theoretically and empirically examines several suitable tests. Based on that, we recommend a set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparison of more classifiers over multiple data sets. Results of the latter can also be neatly presented with the newly introduced CD (critical difference) diagrams.

Detection of malicious software (malware) using ma chine learning methods has been explored extensively to enable fas t detection of new released malware. The performance of these classifiers depen ds on the induction algorithms being used. In order to benefit from mul tiple different classifiers, and exploit their strengths we suggest using an ens emble method that will combine the results of the individual classifiers i nto one final result to achieve overall higher detection accuracy. In this paper we evaluate several combining methods using five different base inducers (C4.5 Dec ision Tree, Naïve Bayes, KNN, VFI and OneR) on five malware datasets. The mai n goal is to find the best combining method for the task of detecting mal icious files in terms of accuracy, AUC and execution time.

The ability to predict performance of students is very crucial in our present education system. We can use data mining concepts for this purpose. ID3 algorithm is one of the famous algorithms present today to generate decision trees. But this algorithm has a shortcoming that it is inclined to attributes with many values. So , this research aims to overcome this shortcoming of the algorithm by using gain ratio(instead of information gain) as well as by giving weights to each attribute at every decision making point. Several other algorithms like J48 and Naive Bayes classification algorithm are also applied on the dataset. The WEKA tool was used for the analysis of J48 and Naive Bayes algorithms. The results are compared and presented. The dataset used in our study is taken from the School of Computing Sciences and Engineering (SCSE), VIT University. Keyworddata mining, educational data mining (EDM), decision tree, gain ratio, weighted ID3

Robust statistics is a branch of statistics which includes statistical methods capable of dealing adequately with the presence of outliers. In this paper, we propose an anomaly detection method that combines a feature selection algorithm and an outlier detection method, which makes extensive use of robust statistics. Feature selection is based on a mutual information metric for which we have developed a robust estimator; it also includes a novel and automatic procedure for determining the number of relevant features. Outlier detection is based on robust Principal Component Analysis (PCA) which, opposite to classical PCA, is not sensitive to outliers and precludes the necessity of training using a reliably labeled dataset, a strong advantage from the operational point of view. To evaluate our method we designed a network scenario capable of producing a perfect ground-truth under real (but controlled) traffic conditions. Results show the significant improvements of our method over the corresponding classical ones. Moreover, despite being a largely overlooked issue in the context of anomaly detection, feature selection is found to be an important preprocessing step, allowing adaption to different network conditions and inducing significant performance gains.

1 Department of Decision Sciences and Information Management, K.U.Leuven Naamsestraat 69, B-3000 Leuven, Belgium {David.Martens;Johan.Huysmans; Bart.Baesens;Jan.Vanthienen}@econ.kuleuven.be 2 School of Computing, National University of Singapore, 3 Science Drive 2, Singapore 117543, Singapore rudys@comp.nus.edu.sg 3 University of Southampton, School of Management, Highfield Southampton, SO17 1BJ, UK Bart@soton.ac.uk

Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, ***, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.


Question: Evaluation of feature selection methods in performance analysis.

This paper explores the automatic creation and clustering of semantically similar words as a key component in advancing natural language processing capabilities. We introduce a novel word similarity metric that hinges on the distributional characteristics of words within their textual context. This metric facilitates the development of a thesaurus derived from a corpus that has undergone syntactic analysis. Subsequently, we unveil an innovative assessment framework to evaluate the efficacy of our automatically generated thesaurus. Our empirical findings reveal that the thesaurus is markedly more aligned with WordNet, a widely recognized lexical database, than Roget's Thesaurus, a traditional lexical resource. This comparison underscores the enhanced accuracy and relevance of our thesaurus in the realm of natural language understanding and processing.

This article presents an algorithm for inducing multiclass decision trees with multivariate tests at internal decision nodes. Each test is constructed by training a linear machine and eliminating variables in a controlled manner. Empirical results demonstrate that the algorithm builds small accurate trees across a variety of tasks.

-This paper evaluates the performance both of some texture measures which have been successfully used in various applications and of some new promising approaches proposed recently. For classification a method based on Kullback discrimination of sample and prototype distributions is used. The classification results for single features with one-dimensional feature value distributions and for pairs of complementary features with two-dimensional distributions are presented Texture analysis Classification Feature distribution Kullback discriminant Performance evaluation Brodatz textures l. I N T R O D U C T I O N Texture is an important characteristic for the analysis of many types of images. A wide variety of measures for discriminating textures have been proposed, c1'2~ Comparative studies to evaluate the performance of some texture measures have been carried out by Weszka et al., ~3~ Du Bufet al. ~4~ and Ohanian and Dubes, ~s) for

We outline a design for a FACS-based facial expression recognition system and describe in more detail the implementation of two of its main components. Firstly we look at how features that are useful from a pattern analysis point of view can be extracted from a raw input image. We show that good results can be obtained by using the method of local binary patterns (LPB) to generate a large number of candidate features and then selecting from them using fast correlation-based filtering (FCBF). Secondly we show how Platt scaling can be used to improve the performance of an error-correcting output code (ECOC) classifier.

As the number of features in pattern recognition applications continuously grows, new algorithms are necessary to reduce the dimensionality of the feature space while producing comparable results. For example, a dynamic area of research, activity recognition, produces large quantities of high-velocity, high-dimensionality data that require real time classification. While dimensionality reduction approaches such as principle component analysis (PCA) and feature selection work well for datasets of reasonable size and dimensionality, they fail on big data. A possible approach to classification of high-dimensionality datasets is to combine a typical classifier, fuzzy k-nearest neighbor in our case (FKNN), with feature reduction by random projection (RP). As opposed to PCA where one projection matrix is computed based on least square optimality, in RP, a projection matrix is chosen at random multiple times. As the random projection procedure is repeated many times, the question is how to aggregate the values of the classifier obtained in each projection. In this paper we present a fusion strategy for RP FKNN, denoted as RPFKNN. The fusion strategy is based on the class membership values produced by FKNN and classification accuracy in each projection. We test RPFKNN on several synthetic and activity recognition datasets.

Knowledge discovery in databases is a rapidly growing field, whose development is driven by strong research interests as well as urgent practical, social, and economical needs. While the last few years knowledge discovery tools have been used mainly in research environments, sophisticated software products are now rapidly emerging. In this paper, we provide an overview of common knowledge discovery tasks and approaches to solve these tasks. We propose a feature classification scheme that can be used to study knowledge and data mining software. This scheme is based on the software's general characteristics, database connectivity, and data mining characteristics. We then apply our feature classification scheme to investigate 43 software products, which are either research prototypes or commercially available. Finally, we specify features that we consider important for knowledge discovery software to possess in order to accommodate its users effectively, as well as issues that are either not addressed or insufficiently solved yet.

Activity recognition fits within the bigger framework of context awareness. In this paper, we report on our efforts to recognize user activity from accelerometer data. Activity recognition is formulated as a classification problem. Performance of base-level classifiers and meta-level classifiers is compared. Plurality Voting is found to perform consistently well across different settings.

Model selection strategies for machine learning algorithm s typically involve the numerical optimisation of an appropriate model selection criterion, ofte n based on an estimator of generalisation performance, such as k-fold cross-validation. The error of such an estimator can b e broken down into bias and variance components. While unbiasedness is oft en cited as a beneficial quality of a model selection criterion, we demonstrate that a low varian ce is at least as important, as a nonnegligible variance introduces the potential for over-fitt ing in model selection as well as in training the model. While this observation is in hindsight perhaps rat her obvious, the degradation in performance due to over-fitting the model selection criterion can b e surprisingly large, an observation that appears to have received little attention in the machine lea rning literature to date. In this paper, we show that the effects of this form of over-fitting are often of c mparable magnitude to differences in performance between learning algorithms, and thus canno t be ignored in empirical evaluation. Furthermore, we show that some common performance evaluati on practices are susceptible to a form of selection bias as a result of this form of over-fitting and hence are unreliable. We discuss methods to avoid over-fitting in model selection and sub sequent selection bias in performance evaluation, which we hope will be incorporated into best pra ctice. While this study concentrates on cross-validation based model selection, the findings are quit general and apply to any model selection practice involving the optimisation of a model se lection criterion evaluated over a finite sample of data, including maximisation of the Bayesian evid nce and optimisation of performance bounds.

We study the use of support vector machines (SVM's) in classifying e-mail as spam or nonspam by comparing it to three other classification algorithms: Ripper, Rocchio, and boosting decision trees. These four algorithms were tested on two different data sets: one data set where the number of features were constrained to the 1000 best features and another data set where the dimensionality was over 7000. SVM's performed best when using binary features. For both data sets, boosting trees and SVM's had acceptable test performance in terms of accuracy and speed. However, SVM's had significantly less training time.

In this age of computer science each and every thing becomes intelligent and perform task as human. For that purpose there are various tools, techniques and methods are proposed. Support vector machine is a model for statistics and computer science, to perform supervised learning, methods that are used to make analysis of data and recognize patterns. SVM is mostly used for classification and regression analysis. And in the same way k-nearest neighbor algorithm is a classification algorithm used to classify data using training examples. In this paper we use SVM and KNN algorithm to classify data and get prediction (find hidden patterns) for target. Here we use medical patients nominal data to classify and discover the data pattern to predict future disease, Uses data mining which is use to classify text analysis in future.

We present attribute bagging (AB), a technique for improving the accuracy and stability of classi#er ensembles induced using random subsets of features. AB is a wrapper method that can be used with any learning algorithm. It establishes an appropriate attribute subset size and then randomly selects subsets of features, creating projections of the training set on which the ensemble classi#ers are built. The induced classi#ers are then used for voting. This article compares the performance of our AB method with bagging and other algorithms on a hand-pose recognition dataset. It is shown that AB gives consistently better results than bagging, both in accuracy and stability. The performance of ensemble voting in bagging and the AB method as a function of the attribute subset size and the number of voters for both weighted and unweighted voting is tested and discussed. We also demonstrate that ranking the attribute subsets by their classi#cation accuracy and voting using only the best subsets further improves the resulting performance of the ensemble. ? 2002 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.


Question: A Bayesian approach to multi-object tracking utilizing motion context from multiple objects.

This paper delves into the exploration of feature sets for robust visual object recognition, with a focus on linear SVM-based human detection as a case study. Through an extensive review and experimental analysis of existing edge and gradient-based descriptors, it is revealed that grids of histograms of oriented gradient (HOG) descriptors surpass other feature sets in performance for human detection tasks. The study meticulously examines the impact of each computational stage on overall performance, highlighting the significance of fine-scale gradients, detailed orientation binning, coarser spatial binning, and high-quality local contrast normalization within overlapping descriptor blocks. These findings are instrumental in achieving nearly perfect classification on the original MIT pedestrian database. To further challenge the robustness of the HOG descriptor approach, a more demanding dataset is introduced, consisting of over 1800 annotated human images that encompass a broad spectrum of poses and background variations, thereby pushing the boundaries of object recognition accuracy.

We describe a system to detect objects in three-dimensional space using video and inertial sensors (accelerometer and gyrometer), ubiquitous in modern mobile platforms from phones to drones. Inertials afford the ability to impose class-specific scale priors for objects, and provide a global orientation reference. A minimal sufficient representation, the posterior of semantic (identity) and syntactic (pose) attributes of objects in space, can be decomposed into a geometric term, which can be maintained by a localization-and-mapping filter, and a likelihood function, which can be approximated by a discriminatively-trained convolutional neural network The resulting system can process the video stream causally in real time, and provides a representation of objects in the scene that is persistent: Confidence in the presence of objects grows with evidence, and objects previously seen are kept in memory even when temporarily occluded, with their return into view automatically predicted to prime re-detection.

Tracking by detection is a common approach to solving the Multiple Object Tracking problem. In this paper we show how deep metric learning can be used to improve three aspects of tracking by detection. We train a convolutional neural network to learn an embedding function in a Siamese configuration on a large person re-identification dataset offline. It is then used to improve the online performance of tracking while retaining a high frame rate. We use this learned appearance metric to robustly build estimates of pedestrian’s trajectories in the MOT16 dataset. In breaking with the tracking by detection model, we use our appearance metric to propose detections using the predicted state of a tracklet as a prior in the case where the detector fails. This method achieves competitive results in evaluation, especially among online, realtime approaches. We present an ablative study showing the impact of each of the three uses of our deep appearance metric.

In this paper we have proposed and designed FindingHuMo (Finding Human Motion), a real-time user tracking system for Smart Environments. FindingHuMo can perform device-free tracking of multiple (unknown and variable number of) users in the Hallway Environments, just from non-invasive and anonymous (not user specific) binary motion sensor data stream. The significance of our designed system are as follows: (a) fast tracking of individual targets from binary motion data stream from a static wireless sensor network in the infrastructure. This needs to resolve unreliable node sequences, system noise and path ambiguity, (b) Scaling for multi-user tracking where user motion trajectories may crossover with each other in all possible ways. This needs to resolve path ambiguity to isolate overlapping trajectories, FindingHumo applies the following techniques on the collected motion data stream: (i) a proposed motion data driven adaptive order Hidden Markov Model with Viterbi decoding (called Adaptive-HMM), and then (ii) an innovative path disambiguation algorithm (called CPDA). Using this methodology the system accurately detects and isolates motion trajectories of individual users. The system performance is illustrated with results from real-time system deployment experience in a Smart Environment.

Recent approaches on visual scene understanding attempt to build a scene graph – a computational representation of objects and their pairwise relationships. Such rich semantic representation is very appealing, yet difficult to obtain from a single image, especially when considering complex spatial arrangements in the scene. Differently, an image sequence conveys useful information using the multi-view geometric relations arising from camera motions. Indeed, object relationships are naturally related to the 3D scene structure. To this end, this paper proposes a system that first computes the geometrical location of objects in a generic scene and then efficiently constructs scene graphs from video by embedding such geometrical reasoning. Such compelling representation is obtained using a new model where geometric and visual features are merged using an RNN framework. We report results on a dataset we created for the task of 3D scene graph generation in multiple views.

To help accelerate progress in multi-target, multi-camera tracking systems, we present (i) a new pair of precision-recall measures of performance that treats errors of all types uniformly and emphasizes correct identification over sources of error; (ii) the largest fully-annotated and calibrated data set to date with more than 2 million frames of 1080 p, 60 fps video taken by 8 cameras observing more than 2,700 identities over 85 min; and (iii) a reference software system as a comparison baseline. We show that (i) our measures properly account for bottom-line identity match performance in the multi-camera setting; (ii) our data set poses realistic challenges to current trackers; and (iii) the performance of our system is comparable to the state of the art.

We introduce a generative model for learning person and costume specific detectors from labeled examples. We demonstrate the model on the task of localizing and naming actors in long video sequences. More specifically, the actor's head and shoulders are each represented as a constellation of optional color regions. Detection can proceed despite changes in view-point and partial occlusions. We explain how to learn the models from a small number of labeled key frames or video tracks, and how to detect novel appearances of the actors in a maximum likelihood framework. We present results on a challenging movie example, with 81% recall in actor detection (coverage) and 89% precision in actor identification (naming).

In this paper we introduce a new salient object segmentation method, which is based on combining a saliency measure with a conditional random field (CRF) model. The proposed saliency measure is formulated using a statistical framework and local feature contrast in illumination, color, and motion information. The resulting saliency map is then used in a CRF model to define an energy minimization based segmentation approach, which aims to recover well-defined salient objects. The method is efficiently implemented by using the integral histogram approach and graph cut solvers. Compared to previous approaches the introduced method is among the few which are applicable to both still images and videos including motion cues. The experiments show that our approach outperforms the current state-of-the-art methods in both qualitative and quantitative terms.

Abnormal behavior detection in crowd scenes is continuously a challenge in the field of computer vision. For tackling this problem, this paper starts from a novel structure modeling of crowd behavior. We first propose an informative structural context descriptor (SCD) for describing the crowd individual, which originally introduces the potential energy function of particle's interforce in solid-state physics to intuitively conduct vision contextual cueing. For computing the crowd SCD variation effectively, we then design a robust multi-object tracker to associate the targets in different frames, which employs the incremental analytical ability of the 3-D discrete cosine transform (DCT). By online spatial-temporal analyzing the SCD variation of the crowd, the abnormality is finally localized. Our contribution mainly lies on three aspects: 1) the new exploration of abnormal detection from structure modeling where the motion difference between individuals is computed by a novel selective histogram of optical flow that makes the proposed method can deal with more kinds of anomalies; 2) the SCD description that can effectively represent the relationship among the individuals; and 3) the 3-D DCT multi-object tracker that can robustly associate the limited number of (instead of all) targets which makes the tracking analysis in high density crowd situation feasible. Experimental results on several publicly available crowd video datasets verify the effectiveness of the proposed method.

Visual tracking is a challenging problem in computer vision. Most state-of-the-art visual trackers either rely on luminance information or use simple color representations for image description. Contrary to visual tracking, for object recognition and detection, sophisticated color features when combined with luminance have shown to provide excellent performance. Due to the complexity of the tracking problem, the desired color feature should be computationally efficient, and possess a certain amount of photometric invariance while maintaining high discriminative power. This paper investigates the contribution of color in a tracking-by-detection framework. Our results suggest that color attributes provides superior performance for visual tracking. We further propose an adaptive low-dimensional variant of color attributes. Both quantitative and attribute-based evaluations are performed on 41 challenging benchmark color sequences. The proposed approach improves the baseline intensity-based tracker by 24 % in median distance precision. Furthermore, we show that our approach outperforms state-of-the-art tracking methods while running at more than 100 frames per second.

Most current approaches to recognition aim to be scaleinvariant. However, the cues available for recognizing a 300 pixel tall object are qualitatively different from those for recognizing a 3 pixel tall object. We argue that for sensors with finite resolution, one should instead use scale-variant, or multiresolution representations that adapt in complexity to the size of a putative detection window. We describe a multiresolution model that acts as a deformable part-based model when scoring large instances and a rigid template with scoring small instances. We also examine the interplay of resolution and context, and demonstrate that context is most helpful for detecting low-resolution instances when local models are limited in discriminative power. We demonstrate impressive results on the Caltech Pedestrian benchmark, which contains object instances at a wide range of scales. Whereas recent state-of-theart methods demonstrate missed detection rates of 86%-37% at 1 falsepositive-per-image, our multiresolution model reduces the rate to 29%.


Question: A Bayesian approach to multi-object tracking utilizing motion context from multiple objects.

In the realm of real-time pedestrian detection, the balance between speed and accuracy is paramount. Traditional methods often employ a densely sampled image pyramid, feature computation at each scale, and sliding window classification using a fixed-scale model (Figure 1a). While effective, this approach can be computationally intensive, making it less suitable for applications requiring high-speed detection. Viola and Jones [6] introduced a groundbreaking method that leverages shift and scale invariant features, enabling the placement of a trained detector at any location and scale without the need for an image pyramid (Figure 1b). This classifier pyramid approach significantly accelerates multiscale detection. However, this technique has limitations, as not all features exhibit scale invariance, particularly gradient histograms, which restricts its broader applicability.

We present a novel strategy for achieving fast multiscale detection while maintaining high accuracy (Figure 1c). Our method approximates features across multiple scales using a sparsely sampled image pyramid with a step size of an entire octave. Within each octave, we deploy a classifier pyramid. This approach mirrors the accuracy of densely sampled image pyramids while matching the speed of a classifier pyramid applied to a single-scale image. This innovation paves the way for efficient pedestrian detection in real-time systems without compromising on accuracy.

In this paper we introduce Co-Fusion, a dense SLAM system that takes a live stream of RGB-D images as input and segments the scene into different objects (using either motion or semantic cues) while simultaneously tracking and reconstructing their 3D shape in real time. We use a multiple model fitting approach where each object can move independently from the background and still be effectively tracked and its shape fused over time using only the information from pixels associated with that object label. Previous attempts to deal with dynamic scenes have typically considered moving regions as outliers, and consequently do not model their shape or track their motion over time. In contrast, we enable the robot to maintain 3D models for each of the segmented objects and to improve them over time through fusion. As a result, our system can enable a robot to maintain a scene description at the object level which has the potential to allow interactions with its working environment; even in the case of dynamic scenes.

Modeling and tracking the driving environment is a complex problem due to the heterogeneous nature of the real world. In many situations, modeling the obstacles and the driving surfaces can be achieved by the use of geometrical objects, and tracking becomes the problem of estimating the parameters of these objects. In the more complex cases, the scene can be modeled and tracked as an occupancy grid. This paper presents a novel occupancy grid tracking solution based on particles for tracking the dynamic driving environment. The particles will have a dual nature-they will denote hypotheses, as in the particle filtering algorithms, but they will also be the building blocks of our modeled world. The particles have position and speed, and they can migrate in the grid from cell to cell, depending on their motion model and motion parameters, but they will be also created and destroyed using a weighting-resampling mechanism that is specific to particle filtering algorithms. The tracking algorithm will be centered on particles, instead of cells. An obstacle grid derived from processing a stereovision-generated elevation map is used as measurement information, and the measurement model takes into account the uncertainties of the stereo reconstruction. The resulting system is a flexible real-time tracking solution for dynamic unstructured driving environments.

We analyze the computer vision task of pixel-level background subtraction. We present recursive equations that are used to constantly update the parameters of a Gaussian mixture model and to simultaneously select the appropriate number of components for each pixel. We also present a simple non-parametric adaptive density estimation method. The two methods are compared with each other and with some previously proposed algorithms. 2005 Elsevier B.V. All rights reserved.

Tracking-by-learning strategies have been effective in solving many challenging problems in visual tracking, in which the learning sample generation and labeling play important roles for final performance. Since the concern of deep learning based approaches has shown an impressive performance in different vision tasks, how to properly apply the learning model, such as CNN, to an online tracking framework is still challenging. In this paper, to overcome the overfitting problem caused by straight-forward incorporation, we propose an online tracking framework by constructing a CNN based adaptive appearance model to generate more reliable training data over time. With a reformative Metropolis-Hastings re-sampling scheme to reshape particles for a better state posterior representation during online learning, the proposed tracking outperforms most of the state-of-art trackers on challenging benchmark video sequences.

Motion detection is very important in image processing. One way of detecting motion is using optical flow. Optical flow cannot be computed locally, since only one independent measurement is available from the image sequence at a point, while the flow velocity has two components. A second constraint is needed. The method used for finding the optical flow in this project is assuming that the apparent velocity of the brightness pattern varies smoothly almost everywhere in the image. This technique is later used in developing software for motion detection which has the capability to carry out four types of motion detection. The motion detection software presented in this project also can highlight motion region, count motion level as well as counting object numbers. Many objects such as vehicles and human from video streams can be recognized by applying optical flow technique. Keywords—Background modeling, Motion detection, Optical flow, Velocity smoothness constant, motion trajectories.

Very recently tracking was approached using classification techniques such as support vector machines. The object to be tracked is discriminated by a classifier from the background. In a similar spirit we propose a novel on-line AdaBoost feature selection algorithm for tracking. The distinct advantage of our method is its capability of on-line training. This allows to adapt the classifier while tracking the object. Therefore appearance changes of the object (e.g. out of plane rotations, illumination changes) are handled quite naturally. Moreover, depending on the background the algorithm selects the most discriminating features for tracking resulting in stable tracking results. By using fast computable features (e.g. Haar-like wavelets, orientation histograms, local binary patterns) the algorithm runs in real-time. We demonstrate the performance of the algorithm on several (publically available) video sequences.

The limited nature of robot sensors make many important robo tics problems partially observable. These problems may require the s yst m to perform complex information-gathering operations. One approach to so lving these problems is to create plans inbelief-space , the space of probability distributions over the underlying state of the system. The belief-space plan encodes a st rategy for performing a task while gaining information as necessary. Most approach es to belief-space planning rely upon representing belief state in a particular way (typically as a Gaussian). Unfortunately, this can lead to large errors between the assumed density representation of belief state and the true belief state. Th is paper proposes a new sample-based approach to belief-space planning that has fix ed computational complexity while allowing arbitrary implementations of Bayes filtering to be used to track belief state. The approach is illustrated in the conte xt of a simple example and compared to a prior approach. Then, we propose an applicatio n of the technique to an instance of the grasp synthesis problem where a robot mu st simultaneously localize and grasp an object given initially uncertain obje ct parameters by planning information-gathering behavior. Experimental results ar e presented that demonstrate the approach to be capable of actively localizing and graspi ng boxes that are presented to the robot in uncertain and hard-to-localize config urations.

Visual object tracking is a significant computer vision task which can be applied to many domains, such as visual surveillance, human computer interaction, and video compression. Despite extensive research on this topic, it still suffers from difficulties in handling complex object appearance changes caused by factors such as illumination variation, partial occlusion, shape deformation, and camera motion. Therefore, effective modeling of the 2D appearance of tracked objects is a key issue for the success of a visual tracker. In the literature, researchers have proposed a variety of 2D appearance models.
 To help readers swiftly learn the recent advances in 2D appearance models for visual object tracking, we contribute this survey, which provides a detailed review of the existing 2D appearance models. In particular, this survey takes a module-based architecture that enables readers to easily grasp the key points of visual object tracking. In this survey, we first decompose the problem of appearance modeling into two different processing stages: visual representation and statistical modeling. Then, different 2D appearance models are categorized and discussed with respect to their composition modules. Finally, we address several issues of interest as well as the remaining challenges for future research on this topic.
 The contributions of this survey are fourfold. First, we review the literature of visual representations according to their feature-construction mechanisms (i.e., local and global). Second, the existing statistical modeling schemes for tracking-by-detection are reviewed according to their model-construction mechanisms: generative, discriminative, and hybrid generative-discriminative. Third, each type of visual representations or statistical modeling techniques is analyzed and discussed from a theoretical or practical viewpoint. Fourth, the existing benchmark resources (e.g., source codes and video datasets) are examined in this survey.

Model-based recognition and motion tracking depends upon the ability to solve for projection and model parameters that will best fit a 3-D model to matching 2-D image features. This paper extends current methods of parameter solving to handle objects with arbitrary curved surfaces and with any number of internal parameters representing articulations, variable dimensions, or surface deformations. Numerical stabilization methods are developed that take account of inherent inaccuracies in the image measurements and allow useful solutions to be determined even when there are fewer matches than unknown parameters. The LevenbergMarquardt method is used to always ensure convergence of the solution. These techniques allow model-based vision to be used for a much wider class of problems than was possible with previous methods. Their application is demonstrated for tracking the motion of curved, parameterized objects. This paper has been published in IEEE Transactions on Pattern Analysis and Machine Intelligence, 13, 5 (May 1991), pp. 441–450.

We advocate the use of Gaussian Process Dynamical Models (GPDMs) for learning human pose and motion priors for 3D people tracking. A GPDM provides a lowdimensional embedding of human motion data, with a density function that gives higher probability to poses and motions close to the training data. With Bayesian model averaging a GPDM can be learned from relatively small amounts of data, and it generalizes gracefully to motions outside the training set. Here we modify the GPDM to permit learning from motions with significant stylistic variation. The resulting priors are effective for tracking a range of human walking styles, despite weak and noisy image measurements and significant occlusions.


Question: A Bayesian approach to multi-object tracking utilizing motion context from various objects.

This paper introduces a robust mobile vision system designed for multi-person tracking in complex, crowded urban environments. The system innovatively merges continuous visual odometry computation with tracking-by-detection techniques to effectively monitor pedestrians, even when faced with intermittent occlusions and the inherent motion of the camera rig. Recognizing the importance of leveraging comprehensive visual information for reliable performance, the proposed system consolidates various visual processing components—visual odometry, pedestrian detection, depth estimation, and tracking—into a unified framework. This integration fosters multiple cognitive feedback loops among these components, enhancing their collective performance. A key innovation is a novel feedback mechanism from the object detector to the visual odometry module. This connection leverages the semantic insights from detection to refine and stabilize the localization process, a critical factor in maintaining tracking accuracy. To mitigate the potential risks associated with feedback loops, such as the amplification of errors, the system incorporates automated failure detection and recovery protocols. These protocols ensure that the system can continue to operate effectively, even when a component becomes unreliable or fails. The robustness and efficacy of the proposed system are demonstrated through extensive experimental evaluations on challenging, long-duration video sequences captured in busy inner-city settings. The results highlight the system's ability to achieve stable tracking performance in scenarios that were previously too complex for reliable tracking.

Motion detection is very important in image processing. One way of detecting motion is using optical flow. Optical flow cannot be computed locally, since only one independent measurement is available from the image sequence at a point, while the flow velocity has two components. A second constraint is needed. The method used for finding the optical flow in this project is assuming that the apparent velocity of the brightness pattern varies smoothly almost everywhere in the image. This technique is later used in developing software for motion detection which has the capability to carry out four types of motion detection. The motion detection software presented in this project also can highlight motion region, count motion level as well as counting object numbers. Many objects such as vehicles and human from video streams can be recognized by applying optical flow technique. Keywords—Background modeling, Motion detection, Optical flow, Velocity smoothness constant, motion trajectories.

Capabilities of inference and prediction are the significant components of visual systems. Visual path prediction is an important and challenging task among them, with the goal to infer the future path of a visual object in a static scene. This task is complicated as it needs high-level semantic understandings of both the scenes and underlying motion patterns in video sequences. In practice, cluttered situations have also raised higher demands on the effectiveness and robustness of models. Motivated by these observations, we propose a deep learning framework, which simultaneously performs deep feature learning for visual representation in conjunction with spatiotemporal context modeling. After that, a unified path-planning scheme is proposed to make accurate path prediction based on the analytic results returned by the deep context models. The highly effective visual representation and deep context models ensure that our framework makes a deep semantic understanding of the scenes and motion patterns, consequently improving the performance on visual path prediction task. In experiments, we extensively evaluate the model's performance by constructing two large benchmark datasets from the adaptation of video tracking datasets. The qualitative and quantitative experimental results show that our approach outperforms the state-of-the-art approaches and owns a better generalization capability.

In this paper, we present a simple yet effective Boolean map based representation that exploits connectivity cues for visual tracking. We describe a target object with histogram of oriented gradients and raw color features, of which each one is characterized by a set of Boolean maps generated by uniformly thresholding their values. The Boolean maps effectively encode multi-scale connectivity cues of the target with different granularities. The fine-grained Boolean maps capture spatially structural details that are effective for precise target localization while the coarse-grained ones encode global shape information that are robust to large target appearance variations. Finally, all the Boolean maps form together a robust representation that can be approximated by an explicit feature map of the intersection kernel, which is fed into a logistic regression classifier with online update, and the target location is estimated within a particle filter framework. The proposed representation scheme is computationally efficient and facilitates achieving favorable performance in terms of accuracy and robustness against the state-of-the-art tracking methods on a large benchmark dataset of 50 image sequences.

One of the major obstacles to using Bayesian methods for pattern recognition has been its computational expense. This thesis presents an approximation technique that can perform Bayesian inference faster and more accurately than previously possible. This method, "Expectation Propagation," unifies and generalizes two previous techniques: assumeddensity filtering, an extension of the Kalman filter, and loopy belief propagation, an extension of belief propagation in Bayesian networks. The unification shows how both of these algorithms can be viewed as approximating the true posterior distribution with a simpler distribution, which is close in the sense of KL-divergence. Expectation Propagation exploits the best of both algorithms: the generality of assumed-density filtering and the accuracy of loopy belief propagation. Loopy belief propagation, because it propagates exact belief states, is useful for limited types of belief networks, such as purely discrete networks. Expectation Propagation approximates the belief states with expectations, such as means and variances, giving it much wider scope. Expectation Propagation also extends belief propagation in the opposite direction-propagating richer belief states which incorporate correlations between variables. This framework is demonstrated in a variety of statistical models using synthetic and real-world data. On Gaussian mixture problems, Expectation Propagation is found, for the same amount of computation, to be convincingly better than rival approximation techniques: Monte Carlo, Laplace's method, and variational Bayes. For pattern recognition, Expectation Propagation provides an algorithm for training Bayes Point Machine classifiers that is faster and more accurate than any previously known. The resulting classifiers outperform Support Vector Machines on several standard datasets, in addition to having a comparable training time. Expectation Propagation can also be used to choose an appropriate feature set for classification, via Bayesian model selection. Thesis Supervisor: Rosalind Picard Title: Associate Professor of Media Arts and Sciences

While intelligent environments are often cited as a reason for doing work on visual person-tracking, really making an intelligent environment exposes many realworld problems in visual tracking that must be solved to make the technology practical. In the context of our EasyLiving project in intelligent environments, we created a practical person-tracking system that solves most of the real-world problems. It uses two sets of color stereo cameras for tracking multiple people during live demonstrations in a living room. The stereo images are used for locating people, and the color images are used for maintaining their identities. The system runs quickly enough to make the room feel responsive, and it tracks multiple people standing, walking, sitting, occluding, and entering and leaving the

We propose a novel similarity measure of two image sequences based on shapeme histograms. The idea of shapeme histogram has been used for single image/texture recognition, but is used here to solve the sequence-tosequence matching problem. We develop techniques to represent each sequence as a set of shapeme histograms, which captures different variations of the object appearances within the sequence. These shapeme histograms are computed from the set of 2D invariant features that are stable across multiple images in the sequence, and therefore minimizes the effect of both background clutter, and 2D pose variations. We define sequence similarity measure as the similarity of the most similar pair of images from both sequences. This definition maximizes the chance of matching between two sequences of the same object, because it requires only part of the sequences being similar. We also introduce a weighting scheme to conduct an implicit feature selection process during the matching of two shapeme histograms. Experiments on clustering image sequences of tracked objects demonstrate the efficacy of the proposed method.

In this work, we evaluate the performance of the popular dense trajectories approach on first-person action recognition datasets. A person moving around with a wearable camera will actively interact with humans and objects and also passively observe others interacting. Hence, in order to represent real-world scenarios, the dataset must contain actions from first-person perspective as well as third-person perspective. For this purpose, we introduce a new dataset which contains actions from both the perspectives captured using a head-mounted camera. We employ a motion pyramidal structure for grouping the dense trajectory features. The relative strengths of motion along the trajectories are used to compute different bag-of-words descriptors and concatenated to form a single descriptor for the action. The motion pyramidal approach performs better than the baseline improved trajectory descriptors. The method achieves 96.7% on the JPL interaction dataset and 61.8% on our NUS interaction dataset. The same is used to detect actions in long video sequences and achieves average precision of 0.79 on JPL interaction dataset.

Continuous quantities are ubiquitous in models of real-world phenomena, but are surprisingly difficult to reason about automatically. Probabilistic graphical models such as Bayesian networks and Markov random fields, and algorithms for approximate inference such as belief propagation (BP), have proven to be powerful tools in a wide range of applications in statistics and artificial intelligence. However, applying these methods to models with continuous variables remains a challenging task. In this work we describe an extension of BP to continuous variable models, generalizing particle filtering, and Gaussian mixture filtering techniques for time series to more complex models. We illustrate the power of the resulting nonparametric BP algorithm via two applications: kinematic tracking of visual motion and distributed localization in sensor networks.

The increasing availability of video data, through existing traffic cameras or dedicated field data collection, paves the way for the collection of massive datasets about the microscopic behaviour of road users using computer vision techniques. Analysis of such datasets helps to understand the normal road user behaviour, and it can be used for realistic prediction of future motion and computing surrogate safety indicators. A multi-level motion pattern learning framework is developed that enables automated scene interpretation, anomalous behaviour detection and surrogate safety analysis. Firstly, Points of interest (POI) are learnt based on Gaussian Mixture Model and the Expectation Maximization algorithm and then used to form activity paths (APs). Secondly, motion patterns, represented by trajectory prototypes, are learnt from road users’ trajectories in each AP using a two-stage trajectory clustering method based on spatial then temporal (speed) information. Finally, motion prediction relies on matching at each instant partial trajectories to the learnt prototypes to evaluate the potential for collision by computing indicators. An intersection case study demonstrates the framework ability in many ways: it helps to reduce the computation cost up to 90 %, clean trajectories dataset from tracking outliers, use actual trajectories as prototypes without any preand post-processing and predict future motion realistically to compute surrogate safety indicators.

The ability to identify objects or region correspondences between consecutive frames of a given hand-drawn animation sequence is an indispensable tool for automating animation modification tasks such as sequence-wide recoloring or shape-editing of a specific animated character. Existing correspondence identification methods heavily rely on appearance features, but these features alone are insufficient to reliably identify region correspondences when there exist occlusions or when two or more objects share similar appearances. To resolve the above problems, manual assistance is often required. In this paper, we propose a new correspondence identification method which considers both appearance features and motions of regions in a global manner. We formulate correspondence likelihoods between temporal region pairs as a network flow graph problem which can be solved by a well-established optimization algorithm. We have evaluated our method with various animation sequences and results show that our method consistently outperforms the state-of-the-art methods without any user guidance.


Question: A Bayesian approach to multi-object tracking utilizing motion context from multiple objects.

This paper presents an innovative tracking system designed to handle the complexities of monitoring a fluctuating number of non-rigid objects, particularly in scenarios where the observation models and target distributions exhibit high levels of non-linearity and non-Gaussian characteristics. The primary challenges in this domain arise from the dynamic nature of the objects and the intricate interplay between them, often involving overlaps and ambiguous identities. To address these issues, we propose a method that integrates the capabilities of two established algorithms: the mixture particle filter and Adaboost. This system is showcased in the context of tracking hockey players within video sequences. The essence of our approach lies in the synergy between these two algorithms. The mixture particle filter, known for its efficacy in multi-target tracking due to its assignment of a unique mixture component to each player, is complemented by the robust detection capabilities of Adaboost. A critical aspect of the mixture particle filter is the selection of the proposal distribution and the management of objects entering or exiting the scene. We address this by utilizing a mixture model that factors in the dynamic models of individual players alongside the detection hypotheses generated by Adaboost. The Adaboost-driven proposal distribution enables rapid identification of new players entering the scene, while the filtering process ensures continuous tracking of each player. The amalgamation of Adaboost with the mixture particle filter yields a streamlined, fully automated system for multiple object tracking, capable of handling the intricate dynamics of non-rigid object movements.

Occupancy grid mapping is a well-known environment perception approach. A grid map divides the environment into cells and estimates the occupancy probability of each cell based on sensor measurements. An important extension is the Bayesian occupancy filter (BOF), which additionally estimates the dynamic state of grid cells and allows modeling changing environments. In recent years, the BOF attracted more and more attention, especially sequential Monte Carlo implementations (SMC-BOF), requiring less computational costs. An advantage compared to classical object tracking approaches is the object-free representation of arbitrarily shaped obstacles and free-space areas. Unfortunately, publications about BOF based on laser measurements report that grid cells representing big, contiguous, stationary obstacles are often mistaken as moving with the velocity of the ego vehicle (ghost movements). This paper presents a method to fuse laser and radar measurement data with the SMC-BOF. It shows that the doppler information of radar measurements significantly improves the dynamic estimation of the grid map, reduces ghost movements, and in general leads to a faster convergence of the dynamic estimation.

Intelligent visual surveillance for road vehicles is the key to developing autonomous intelligent traffic systems. Recently, traffic incident detection employing computer vision and image processing has attracted much attention. In this paper, a probabilistic model for predicting traffic accidents using three-dimensional (3-D) model-based vehicle tracking is proposed. Sample data including motion trajectories are first obtained by 3-D model-based vehicle tracking. A fuzzy self-organizing neural network algorithm is then applied to learn activity patterns from the sample trajectories. Finally, vehicle activity is predicted by locating and matching each partial trajectory with the learned activity patterns, and the occurrence probability of a traffic accident is determined. Experiments show the effectiveness of the proposed algorithms.

A common method for real-time segmentation of moving regions in image sequences involves “background subtraction,” or thresholding the error between an estimate of the image without moving objects and the current image. The numerous approaches to this problem differ in the type of background model used and the procedure used to update the model. This paper discusses modeling each pixel as a mixture of Gaussians and using an on-line approximation to update the model. The Gaussian distributions of the adaptive mixture model are then evaluated to determine which are most likely to result from a background process. Each pixel is classified based on whether the Gaussian distribution which represents it most effectively is considered part of the background model. This results in a stable, real-time outdoor tracker which reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. This system has been run almost continuously for 16 months, 24 hours a day, through rain and snow.

The aim of the radar systems is to collect information about their surroundings. In many scenarios besides static targets there are numerous moving objects with very different characteristics, such as extent, movement behavior or micro-Doppler spread. It would be most desirable to have algorithms that extract all information on static and moving object automatically, without a system operator. In this paper, we present measurements conducted with a commercially available high-resolution multi-channel linear frequency-modulated continuous-wave radar and algorithms that do not only produce radar images but a description of the scenario on a higher level. After conventional spectrum estimation and thresholding, we present a clustering stage that combines individual detections and generates representations of each target individually. This stage is followed by a Kalman filter based multi-target tracking block. The tracker allows us to follow each target and collect its properties over time. With this method of jointly estimating tracks and characteristics of each individual target in a scenario, inputs for classifiers can be generated. Which, in turn, will be able to generate information that could be used for driver assistance or alarm trigger systems.

Object occlusion is widespread in video surveillance due to the influence of angle and environment, which brings a great impact on the target tracking and makes the development of video object tracking encountered meeting many constraints. The challenge in video object tracking is how to track accurately when the target is obscured. We use the frame difference method to detect the target and smooth the target trajectory by using Kalman filter. When the target is occluded, we can select the appropriate fitting section after smoothing trajectories and use the least square method to fit the target motion path adaptively, so we can predict the object location. By comparing the experimental results with the actual motion of the target, it shows that the algorithm can be used to track most of the occlusion targets precisely.

The concept of miss distance-Euclidean, Mahalanobis, etc.-is a fundamental, far-reaching, and taken-for-granted element of the engineering theory and practice of single-target systems. In this paper we introduce a comprehensive L/sub p/-type theory of distance metrics for multitarget (and, more generally, multiobject) systems. We show that this theory extends, and provides a rigorous theoretical basis for, an intuitively appealing optimal-assignment approach proposed by Drummond for evaluating the performance of multitarget tracking algorithms. We describe tractable computational approaches for computing such metrics based on standard optimal assignment or convex optimization techniques. We describe the potentially far-reaching implications of these metrics for applications such as performance evaluation and sensor management. In the former case, we demonstrate the application of multitarget miss-distance metrics as measures of effectiveness (MoEs) for multitarget tracking algorithms.

Autonomous driving poses unique challenges for vehicle environment perception in complex driving environments. Due to the uncertain nature of the vehicle environment and imperfection of any perception framework, multiple stages of estimation might be necessary to achieve the desired performance. However, it is highly possible that the estimation of one stage might result in output estimates with significant auto/cross-correlation, which would pass to another stage. In such situations, a decorrelation procedure is required. We present an object tracking approach taking into consideration the auto-correlation (of the velocity components) introduced by an upfront dynamic occupancy gridmap system. More specifically, we use a linear state-space system to approximately “reconstruct” the nonlinear estimation/mapping procedure for the purpose of auto-correlation quantification. We focus on demonstrating an estimation improvement of the proposed decorrelation tracker over a direct Kalman filter (i.e., KF that ignores the auto-correlation). For this we recorded several scenarios of different target motion behaviors. It is shown that the decorrelation tracker indeed introduces noticable estimation improvement in particularly velocity space if the target moves in a relatively smooth manner.

We present a system that estimates the motion of a stereo head or a single moving camera based on video input. The system operates in real-time with low delay and the motion estimates are used for navigational purposes. The front end of the system is a feature tracker. Point features are matched between pairs of frames and linked into image trajectories at video rate. Robust estimates of the camera motion are then produced from the feature tracks using a geometric hypothesize-and-test architecture. This generates motion estimates from visual input alone. No prior knowledge of the scene nor the motion is necessary. The visual estimates can also be used in conjunction with information from other sources such as GPS, inertia sensors, wheel encoders, etc. The pose estimation method has been applied successfully to video from aerial, automotive and handheld platforms. We focus on results obtained with a stereo-head mounted on an autonomous ground vehicle. We give examples of camera trajectories estimated in real-time purely from images over previously unseen distances (600 meters) and periods of time .

In image sequence processing kalman filtering is used for an adaptive background estimation, in order to separate the foreground from the background. The presented work is an approach which takes into account that changing illumination should be considered in the background estimation, and should not be detected as foreground. The new approach assumes a stationary CCD cameras with fixed focal length and considers non-rigid objects moving non-continuously like human bodies. Furthermore, statistic based methods are used to overcome the problems caused by shadow borders and the adaptation, when the background is covered by the foreground.

State-of-the-art motion estimation algorithms suffer from three major problems: Poorly textured regions, occlusions and small scale image structures. Based on the Gestalt principles of grouping we propose to incorporate a low level image segmentation process in order to tackle these problems. Our new motion estimation algorithm is based on non-local total variation regularization which allows us to integrate the low level image segmentation process in a unified variational framework. Numerical results on the Mid-dlebury optical flow benchmark data set demonstrate that we can cope with the aforementioned problems.


Question: A Bayesian approach to multi-object tracking utilizing motion context from multiple objects.

This paper presents a novel method for multi-target tracking in crowded scenes using a single camera, focusing on the progressive association of detection responses into increasingly longer track fragments, culminating in the identification of individual target trajectories. Unlike conventional approaches that typically employ handcrafted, parametric models to determine tracklet affinity, our method automatically selects from a variety of features and non-parametric models. This selection and combination process is optimized to enhance discriminative power on training data through the use of the HybridBoost algorithm. The algorithm is designed to address the dual challenge of ranking and classification inherent in tracklet association: it ranks correct associations higher than alternatives and simultaneously rejects incorrect associations when no further linking is appropriate. The effectiveness of this approach is demonstrated through experiments involving pedestrian tracking in complex datasets, where it outperforms leading algorithms in terms of tracking accuracy.

In this letter, we address the autonomous flight of a small quadrotor, enabling tracking of a moving object. The 15-cm diameter, 250-g robot relies only on onboard sensors (a single camera and an inertial measurement unit) and computers, and can detect, localize, and track moving objects. Our key contributions include the relative pose estimate of a spherical target as well as the planning algorithm, which considers the dynamics of the underactuated robot, the actuator limitations, and the field of view constraints. We show simulation and experimental results to demonstrate feasibility and performance, as well as robustness to abrupt variations in target motion.

In this paper, we propose a new approach for anomaly detection in video surveillance. This approach is based on a nonparametric Bayesian regression model built upon Gaussian process priors. It establishes a set of basic vectors describing motion patterns from low-level features via online clustering, and then constructs a Gaussian process regression model to approximate the distribution of motion patterns in kernel space. We analyze di®erent anomaly measure criterions derived from Gaussian process regression model and compare their performances. To reduce

Bicycles that share the road with intelligent vehicles present particular challenges for automated perception systems. Bicycle detection is important because bicycles share the road with vehicles and can move at comparable speeds in urban environments. From a computer vision standpoint, bicycle detection is challenging as bicycle's appearance can change dramatically between viewpoints and a person riding on the bicycle is a non-rigid object. In this paper, we present a vision-based framework to detect and track bicycles that takes into account these issues. A mixture model of multiple viewpoints is defined and trained via a Support Vector Machine (SVM) to detect bicycles under a variety of circumstances. Each component of the model uses a part-based representation and known geometric context is used to improve overall detection efficiency. An extended Kalman filter (EKF) is used to estimate the position and velocity of the bicycle in vehicle coordinates. We demonstrate the effectiveness of this approach through a series of experiments run on video data of moving bicycles captured from a vehicle-mounted camera

Recently dense trajectories were shown to be an efficient video representation for action recognition and achieved state-of-the-art results on a variety of datasets. This paper improves their performance by taking into account camera motion to correct them. To estimate camera motion, we match feature points between frames using SURF descriptors and dense optical flow, which are shown to be complementary. These matches are, then, used to robustly estimate a homography with RANSAC. Human motion is in general different from camera motion and generates inconsistent matches. To improve the estimation, a human detector is employed to remove these matches. Given the estimated camera motion, we remove trajectories consistent with it. We also use this estimation to cancel out camera motion from the optical flow. This significantly improves motion-based descriptors, such as HOF and MBH. Experimental results on four challenging action datasets (i.e., Hollywood2, HMDB51, Olympic Sports and UCF50) significantly outperform the current state of the art.

A lane-detection system is an important component of many intelligent transportation systems. We present a robust lane-detection-and-tracking algorithm to deal with challenging scenarios such as a lane curvature, worn lane markings, lane changes, and emerging, ending, merging, and splitting lanes. We first present a comparative study to find a good real-time lane-marking classifier. Once detection is done, the lane markings are grouped into lane-boundary hypotheses. We group left and right lane boundaries separately to effectively handle merging and splitting lanes. A fast and robust algorithm, based on random-sample consensus and particle filtering, is proposed to generate a large number of hypotheses in real time. The generated hypotheses are evaluated and grouped based on a probabilistic framework. The suggested framework effectively combines a likelihood-based object-recognition algorithm with a Markov-style process (tracking) and can also be applied to general-part-based object-tracking problems. An experimental result on local streets and highways shows that the suggested algorithm is very reliable.

As an alternative to the tracking-based approaches that heavily depend on accurate detection of moving objects, which often fail for crowded scenarios, we present a pixelwise method that employs dual foregrounds to extract temporally static image regions. Depending on the application, these regions indicate objects that do not constitute the original background but were brought into the scene at a subsequent time, such as abandoned and removed items, illegally parked vehicles. We construct separate longand short-term backgrounds that are implemented as pixelwise multivariate Gaussian models. Background parameters are adapted online using a Bayesian update mechanism imposed at different learning rates. By comparing each frame with these models, we estimate two foregrounds. We infer an evidence score at each pixel by applying a set of hypotheses on the foreground responses, and then aggregate the evidence in time to provide temporal consistency. Unlike optical flow-based approaches that smear boundaries, our method can accurately segment out objects even if they are fully occluded. It does not require on-site training to compensate for particular imaging conditions. While having a low-computational load, it readily lends itself to parallelization if further speed improvement is necessary.

The long-standing goal of localizing every object in an image remains elusive. Manually annotating objects is quite expensive despite crowd engineering innovations. Current state-of-the-art automatic object detectors can accurately detect at most a few objects per image. This paper brings together the latest advancements in object detection and in crowd engineering into a principled framework for accurately and efficiently localizing objects in images. The input to the system is an image to annotate and a set of annotation constraints: desired precision, utility and/or human cost of the labeling. The output is a set of object annotations, informed by human feedback and computer vision. Our model seamlessly integrates multiple computer vision models with multiple sources of human input in a Markov Decision Process. We empirically validate the effectiveness of our human-in-the-loop labeling approach on the ILSVRC2014 object detection dataset.

This paper proposes an integrated Bayesian frame work for feature-based simultaneous localization and map building (SLAM) in the general case of uncertain feature number and data association. By modeling the measurements and feature map as random finite sets (RFSs), a formulation of the feature-based SLAM problem is presented that jointly estimates the number and location of the features, as well as the vehicle trajectory. More concisely, the joint posterior distribution of the set-valued map and vehicle trajectory is propagated forward in time as measurements arrive, thereby incorporating both data association and feature management into a single recursion. Furthermore, the Bayes optimality of the proposed approach is established. A first-order solution, which is coined as the probability hypothesis density (PHD) SLAM filter, is derived, which jointly propagates the posterior PHD of the map and the posterior distribution of the vehicle trajectory. A Rao-Blackwellized (RB) implementation of the PHD-SLAM filter is proposed based on the Gaussian-mixture PHD filter (for the map) and a particle filter (for the vehicle trajectory). Simulated and experimental results demonstrate the merits of the proposed approach, particularly in situations of high clutter and data association ambiguity.

Humans acquire their most basic physical concepts early in development, and continue to enrich and expand their intuitive physics throughout life as they are exposed to more and varied dynamical environments. We introduce a hierarchical Bayesian framework to explain how people can learn physical parameters at multiple levels. In contrast to previous Bayesian models of theory acquisition (Tenenbaum, Kemp, Griffiths, & Goodman, 2011), we work with more expressive probabilistic program representations suitable for learning the forces and properties that govern how objects interact in dynamic scenes unfolding over time. We compare our model to human learners on a challenging task of estimating multiple physical parameters in novel microworlds given short movies. This task requires people to reason simultaneously about multiple interacting physical laws and properties. People are generally able to learn in this setting and are consistent in their judgments. Yet they also make systematic errors indicative of the approximations people might make in solving this computationally demanding problem with limited computational resources. We propose two approximations that complement the top-down Bayesian approach. One approximation model relies on a more bottom-up feature-based inference scheme. The second approximation combines the strengths of the bottom-up and top-down approaches, by taking the feature-based inference as its point of departure for a search in physical-parameter space.

In this work, we introduce a large high-diversity database for generic object tracking, called GOT-10k. GOT-10k is backboned by the semantic hierarchy of WordNet [1]. It populates a majority of 563 object classes and 87 motion patterns in real-world, resulting in a scale of over 10 thousand video segments and 1.5 million bounding boxes. To our knowledge, GOT-10k is by far the richest motion trajectory dataset, and its coverage of object classes is more than a magnitude wider than similar scale counterparts [20], [23]. By publishing GOT-10k, we hope to encourage the development of generic purposed trackers that work for a wide range of moving objects and under diverse real-world scenarios. To promote generalization and avoid the evaluation results biased to seen classes, we follow the one-shot principle [35] in dataset splitting where training and testing classes are zero-overlapped. We also carry out a series of analytical experiments to select a compact while highly representative testing subset – it embodies 84 object classes and 32 motion patterns with only 180 video segments, allowing for efficient evaluation. Finally, we train and evaluate a number of representative trackers on GOT-10k and analyze their performance. The evaluation results suggest that tracking in real-world unconstrained videos is far from being solved, and only 40% of frames are successfully tracked using top ranking trackers. The database and toolkits are publicly available at https://got-10k.github.io.