Question: Active metric learning for categorization of remotely sensed hyperspectral imagery.

We present an innovative method termed Query by Committee (QBC), designed to optimize the learning process in classification tasks. This approach involves training a group of models, referred to as a committee, on a shared dataset. The selection of subsequent queries is guided by the principle of maximal disagreement among the committee members. To explore the efficacy of QBC, we apply it to two simplified models: the high-low game and the learning of one perceptron by another. As the quantity of queries approaches infinity, the QBC algorithm demonstrates an asymptotically finite information gain, leading to a generalization error that decreases at an exponential rate with an increasing number of examples. This is a stark contrast to the learning outcomes from randomly selected inputs, where information gain tends towards zero and generalization error diminishes at a much slower inverse power law rate. Our findings suggest that the characteristic of asymptotically finite information gain might be a key factor in identifying effective query algorithms. This approach could be particularly relevant in scenarios where data labeling is costly, such as in the classification of remotely sensed hyperspectral images, where the QBC method could potentially enhance the efficiency and accuracy of the classification process.

Ship classification is the key step in maritime surveillance using synthetic aperture radar (SAR) imagery. In this letter, we develop a new ship classification method in TerraSAR-X images based on sparse representation in feature space, in which the sparse representation classification (SRC) method is exploited. In particular, to describe the ship more accurately and to reduce the dimension of the dictionary in SRC, we propose to employ a representative feature vector to construct the dictionary instead of utilizing the image pixels directly. By testing on a ship data set collected from TerraSAR-X images, we show that the proposed method is superior to traditional methods such as the template matching (TM), K-nearest neighbor (K-NN), Bayes and Support Vector Machines (SVM).

Neural network such as an autoencoder (AE) and a convolutional autoencoder (CAE) have been successfully applied in image feature extraction. For the statistical distribution of polarimetric synthetic aperture radar (POLSAR) data, we combine the Wishart distance measurement into the training process of the AE and the CAE. In this paper, a new type of AE and CAE is specially defined, which we name them Wishart-AE (WAE) and Wishart-CAE (WCAE). Furthermore, we connect the WAE or the WCAE with a softmax classifier to compose a classification model for the purpose of POLSAR image classification. Compared with AE and CAE models, WAE and WCAE models can achieve higher classification accuracy because they could obtain the classification features, which are more suitable for POLSAR data. What is more, the WCAE model utilizes the local spatial information of a POLSAR image when compared with the WAE model. A convolutional natural network (CNN), which also makes use of the spatial information, has been widely applied in image classification, but our WCAE model is time-saving than the CNN model. Given the above, our methods not only improve the classification performance but also save the experimental time. Experimental results on four POLSAR datasets also demonstrate that our proposed methods are significantly effective.

In this paper we propose a convolutional neural network (CNN), which allows to identify corresponding patches of very high resolution (VHR) optical and SAR imagery of complex urban scenes. Instead of a siamese architecture as conventionally used in CNNs designed for image matching, we resort to a pseudo-siamese configuration with no interconnection between the two streams for SAR and optical imagery. The network is trained with automatically generated training data and does not resort to any hand-crafted features. First evaluations show that the network is able to predict corresponding patches with high accuracy, thus indicating great potential for further development to a generalized multi-sensor matching procedure.

Inspired by recent development of artificial satellite, remote sensing images have attracted extensive attention. Recently, notable progress has been made in scene classification and target detection. However, it is still not clear how to describe the remote sensing image content with accurate and concise sentences. In this paper, we investigate to describe the remote sensing images with accurate and flexible sentences. First, some annotated instructions are presented to better describe the remote sensing images considering the special characteristics of remote sensing images. Second, in order to exhaustively exploit the contents of remote sensing images, a large-scale aerial image data set is constructed for remote sensing image caption. Finally, a comprehensive review is presented on the proposed data set to fully advance the task of remote sensing caption. Extensive experiments on the proposed data set demonstrate that the content of the remote sensing image can be completely described by generating language descriptions. The data set is available at https://github.com/201528014227051/RSICD_optimal.

Grouping images into semantically meaningful categories using low-level visual features is a challenging and important problem in content-based image retrieval. Based on these groupings, eeective indices can be built for an image database. In this paper, we show how a speciic high-level classiication problem (city images vs. landscapes) can be solved from relatively simple low-level features geared for the particular classes. We have developed a procedure to qualitatively measure the saliency of a feature towards a classiication problem based on the plot of the intra-class and inter-class distance distributions. We use this approach to determine the discriminative power of the following features: color histogram, color coherence vector, DCT coeecient, edge direction histogram, and edge direction coherence vector. We determine that the edge direction-based features have the most discriminative power for the classiication problem of interest here. A weighted k-NN classiier is used for the classiication which results in an accuracy of 93:9% when evaluated on an image database of 2; 716 images using the leave-one-out method. This approach has been extended to further classify 528 landscape images into forests, mountains, and sunset/sunrise classes. First, the input images are classiied as sunset/sunrise images vs. forest & mountain images (94:5% accuracy) and then the forest & mountain images are classiied as forest images or mountain images (91:7% accuracy). We are currently identifying further semantic classes to assign to images as well as extracting low level features which are salient for these classes. Our nal goal is to combine multiple 2-class classiiers into a single hierarchical classiier.

All clustering algorithms process unlabeled data and, consequently, suffer from two problems: (P1) choosing and validating the correct number of clusters and (P2) insuring that algorithmic labels correspond to meaningful physical labels. Clustering algorithms such as hard and fuzzy c-means, based on optimizing sums of squared errors objective functions, suffer from a third problem: (P3) a tendency to recommend solutions that equalize cluster populations. The semi-supervised c-means algorithms introduced in this paper attempt to overcome these three problems for problem domains where a few data from each class can be labeled. Segmentation of magnetic resonance images is a problem of this type and we use it to illustrate the new algorithm. Our examples show that the semi-supervised approach provides MRI segmentations that are superior to ordinary fuzzy c-means and to the crisp k-nearest neighbor rule and further, that the new method ameliorates (P1)-(P3). Cluster analysis Fuzzy c-means Partial supervision Image segmentation Magnetic resonance images

Segmentation of Synthetic Aperture Radar (SAR) images has several uses, but it is a difficult task due to a number of properties related to SAR images. In this article we show how Convolutional Neural Networks (CNNs) can easily be trained for SAR image segmentation with good results. Besides this contribution we also suggest a new way to do pixel wise annotation of SAR images that replaces a human expert manual segmentation process, which is both slow and troublesome. Our method for annotation relies on 3D CAD models of objects and scene, and converts these to labels for all pixels in a SAR image. Our algorithms are evaluated on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset which was released by the Defence Advanced Research Projects Agency during the 1990s. The method is not restricted to the type of targets imaged in MSTAR but can easily be extended to any SAR data where prior information about scene geometries can be estimated.

Existing fine-grained visual categorization methods often suffer from three challenges: lack of training data, large number of fine-grained categories, and high intraclass vs. low inter-class variance. In this work we propose a generic iterative framework for fine-grained categorization and dataset bootstrapping that handles these three challenges. Using deep metric learning with humans in the loop, we learn a low dimensional feature embedding with anchor points on manifolds for each category. These anchor points capture intra-class variances and remain discriminative between classes. In each round, images with high confidence scores from our model are sent to humans for labeling. By comparing with exemplar images, labelers mark each candidate image as either a "true positive" or a "false positive." True positives are added into our current dataset and false positives are regarded as "hard negatives" for our metric learning model. Then the model is retrained with an expanded dataset and hard negatives for the next round. To demonstrate the effectiveness of the proposed framework, we bootstrap a fine-grained flower dataset with 620 categories from Instagram images. The proposed deep metric learning scheme is evaluated on both our dataset and the CUB-200-2001 Birds dataset. Experimental evaluations show significant performance gain using dataset bootstrapping and demonstrate state-of-the-art results achieved by the proposed deep metric learning methods.

The objective of the current study was to analyze the seasonal effect on differentiating tree species in an urban environment using multi-temporal hyperspectral data, Light Detection And Ranging (LiDAR) data, and a tree species database collected from the field. Two Airborne Imaging Spectrometer for Applications (AISA) hyperspectral images were collected, covering the Summer and Fall seasons. In order to make both datasets spatially and spectrally compatible, several preprocessing steps, including band reduction and a spatial degradation, were performed. An object-oriented classification was performed on both images using training data collected randomly from the tree species database. The seven dominant tree species (Gleditsia triacanthos, Acer saccharum, Tilia Americana, Quercus palustris, Pinus strobus and Picea glauca) were used in the classification. The results from this analysis did not show any major difference in overall accuracy between the two seasons. Overall accuracy was approximately 57% for the Summer dataset and 56% for the Fall dataset. However, the Fall dataset provided more consistent results for all tree species while the Summer dataset had a few higher individual class accuracies. Further, adding LiDAR into the classification improved the results by 19% for both fall and summer. This is mainly due to the removal of shadow effect and the addition of elevation data to separate low and high vegetation.

Metric learning has been shown to be highly effective to improve the performance of nearest neighbor classification. In this paper, we address the problem of metric learning for symmetric positive definite (SPD) matrices such as covariance matrices, which arise in many real-world applications. Naively using standard Mahalanobis metric learning methods under the Euclidean geometry for SPD matrices is not appropriate, because the difference of SPD matrices can be a non-SPD matrix and thus the obtained solution can be uninterpretable. To cope with this problem, we propose to use a properly parameterized LogEuclidean distance and optimize the metric with respect to kernel-target alignment, which is a supervised criterion for kernel learning. Then the resulting non-trivial optimization problem is solved by utilizing the Riemannian geometry. Finally, we experimentally demonstrate the usefulness of our LogEuclidean metric learning algorithm on real-world classification tasks for EEG signals and texture patches.


Question: Active metric learning technique for categorization of remotely sensed hyperspectral images.

This paper introduces an innovative framework, Adaptive Manifold Learning, aimed at optimizing the process of finding low-dimensional representations of high-dimensional data. Traditional manifold learning techniques often struggle with accurately defining locality, approximating the manifold locally, and seamlessly integrating local structures into a comprehensive global parameterization. Our proposed methods tackle two critical challenges in this domain: 1) dynamically adjusting the sizes of local neighborhoods when establishing a connectivity structure among high-dimensional data points, and 2) mitigating adaptive bias in the local low-dimensional embedding, considering both the manifold's curvature variations and the interaction with the data set's sampling density. We showcase the superior performance of our techniques in enhancing manifold learning algorithms through comprehensive evaluations using both synthetic and real-world data sets, thereby underscoring their effectiveness in refining manifold learning outcomes.

The number of digital images is growing extremely rapidly, and so is the need for their classification. But, as more images of pre-defined categories become available, they also become more diverse and cover finer semantic differences. Ultimately, the categories themselves need to be divided into subcategories to account for that semantic refinement. Image classification in general has improved significantly over the last few years, but it still requires a massive amount of manually annotated data. Subdividing categories into subcategories multiples the number of labels, aggravating the annotation problem. Hence, we can expect the annotations to be refined only for a subset of the already labeled data, and exploit coarser labeled data to improve classification. In this work, we investigate how coarse category labels can be used to improve the classification of subcategories. To this end, we adopt the framework of Random Forests and propose a regularized objective function that takes into account relations between categories and subcategories. Compared to approaches that disregard the extra coarse labeled data, we achieve a relative improvement in subcategory classification accuracy of up to 22% in our large-scale image classification experiments.

Many classification problems involve instances that are unlabeled, multi-view and multi-class. However, few technique has been benchmarked for this complex scenario, with a notable exception that combines co-trained naive bayes (CoT-NB) with BCH coding. In this paper, we benchmark the performance of co-regularized least square regression (CoR-LS) for semi-supervised multi-view multi-class classification. We find it performed consistently and significantly better than CoT-NB over eight data sets at different scales. We also find for CoR-LS identity coding is optimal on large data sets and BCH coding is optimal on small data sets. Optimal scoring, a data-dependent coding scheme, often provides near-optimal performance.

We extend the traditional active learning framework to include feedback on features in addition to labeling instances, and we execute a careful study of the effects of feature selection and human feedback on features in the setting of text categorization. Our experiments on a variety of categorization tasks indicate that there is significant potential in improving classifier performance by feature re-weighting, beyond that achieved via membership queries alone (traditional active learning) if we have access to an oracle that can point to the important (most predictive) features. Our experiments on human subjects indicate that human feedback on feature relevance can identify a sufficient proportion of the most relevant features (over 50% in our experiments). We find that on average, labeling a feature takes much less time than labeling a document. We devise an algorithm that interleaves labeling features and documents which significantly accelerates standard active learning in our simulation experiments. Feature feedback can complement traditional active learning in applications such as news filtering, e-mail classification, and personalization, where the human teacher can have significant knowledge on the relevance of features.

We propose a new method for clustering based on finding maximum margin hyperplanes through data. By reformulating the problem in terms of the implied equivalence relation matrix, we can pose the problem as a convex integer program. Although this still yields a difficult computational problem, the hard-clustering constraints can be relaxed to a soft-clustering formulation which can be feasibly solved with a semidefinite program. Since our clustering technique only depends on the data through the kernel matrix, we can easily achieve nonlinear clusterings in the same manner as spectral clustering. Experimental results show that our maximum margin clustering technique often obtains more accurate results than conventional clustering methods. The real benefit of our approach, however, is that it leads naturally to a semi-supervised training method for support vector machines. By maximizing the margin simultaneously on labeled and unlabeled training data, we achieve state of the art performance by using a single, integrated learning principle.

We present a Bag of Words-based active object categorization technique implemented and tested on a humanoid robot. The robot is trained to categorize objects that are handed to it by a human operator. The robot uses hand and head motions to actively acquire a number of different views. A view planning scheme using entropy minimization reduces the number of views needed to achieve a valid decision. Categorization results are significantly improved by active elimination of background features using robot arm motion. Our experiments cover both, categorization when the object is handed to the robot in a fixed pose at training and testing, and object pose independent categorization. Results on a 4-class object database demonstrate the classification efficiency, a significant gain from multi-view compared to single-view classification, and the advantage of view planning. We conclude that humanoid robotic systems can be successfully applied to actively categorize objects a task with many potential applications ranging from edutainment to active surveillance.

Estimation of in-field biomass and crop composition is important for both farmers and researchers. Using close-up high resolution images of the crops, crop species can be distinguished using image processing. In the current study, deep convolutional neural networks for semantic segmentation (or pixel-wise classification) of cluttered classes in RGB images was explored in case of catch crops and volunteer barley cereal. The dataset consisted of RGB images from a plot trial using oil radish as catch crops in barley. The images were captured using a high-end consumer camera mounted on a tractor. The images were manually annotated in 7 classes: oil radish, barley, weed, stump, soil, equipment and unknown. Data argumentation was used to artificially increase the dataset by transposing and flipping the images. A modified version of VGG-16 deep neural network was used. First, the last fully-connected layers were converted to convolutional layer and the depth was modified to cope with our number of classes. Secondly, a deconvolutional layer with a 32 stride was added between the last fully-connected layer and the softmax classification layer to ensure that the output layer has the same size as the input. Preliminary results using this network show a pixel accuracy of 79% and a frequency weighted intersection over union of 66%. These preliminary results indicate great potential in deep convolutional networks for segmentation of plant species in cluttered RGB images.

Keywords: Image segmentation Plant leaf Complicated background Watershed segmentation Hu geometric moments Zernike moment Moving center hypersphere (MCH) classifier a b s t r a c t Classifying plant leaves has so far been an important and difficult task, especially for leaves with complicated background where some interferents and overlapping phenomena may exist. In this paper, an efficient classification framework for leaf images with complicated background is proposed. First, a so-called automatic marker-controlled watershed segmen-tation method combined with pre-segmentation and morphological operation is introduced to segment leaf images with complicated background based on the prior shape information. Then, seven Hu geometric moments and sixteen Zernike moments are extracted as shape features from segmented binary images after leafstalk removal. In addition , a moving center hypersphere (MCH) classifier which can efficiently compress feature data is designed to address obtained mass high-dimensional shape features. Finally, experimental results on some practical plant leaves show that proposed classification framework works well while classifying leaf images with complicated background. There are twenty classes of practical plant leaves successfully classified and the average correct classification rate is up to 92.6%. Plants play the most important part in the cycle of nature. They are the primary producers that sustain all other life forms including people. This is because plants are the only organisms that can convert light energy from the sun into food. Animals, incapable of making their own food, depend directly or indirectly on plants for their supply of food. Moreover, all of the oxygen available for living organisms comes from plants. Plants are also the primary habitat for thousands of other organisms. In addition, many of the fuel people use today, such as coal, natural gas and gasoline, were made from plants that lived millions of years ago. However, in recent years, people have been seriously destroying the natural environments, so that many plants constantly die and even die out every year. Conversely, the resulting ecological crisis has brought many serious consequences including land desertion, climate anomaly, land flood, and so on, which have menaced the survival of human being and the development of society. Now, people have realized the importance and urgency of protecting plant resource. Besides taking effective measures to protect plants, it is important for ordinary people to know or classify plants which can further enhance the public's consciousness of plant protection. Therefore, besides professional botanists, many non-professional researchers have paid more …

The primary problem dealt with in this paper is the following. Given some description of a visual object, find that object in an actual photograph. Part of the solution to this problem is the specification of a descriptive scheme, and a metric on which to base the decision of "goodness" of matching or detection.

Learning to recognize of object classes is one of the most important functionalities of vision. It is estimated that humans are able to learn tens of thousands of visual categori es in their life. Given the photometric and geometric variabilities displayed by objects as well as the high degree of intra-clas s variabilities, we hypothesize that humans achieve such a fe at by using knowledge and information cumulated throughout the learning process. In recent years, a handful of pioneering p apers have applied various forms of knowledge transfer algorithms to the problem of learning object classes. We first review some o f these papers by loosely grouping them into three categories : transfer through prior parameters, transfer through shared features or parts, and transfer through contextual information. In the second half of the paper, we detail a recent algorithm proposed by the author. This incremental learning scheme us es information from object classes previously learned in the f orm of prior models to train a new object class model. Training images can be presented in an incremental way. We present experimental results tested with this model on a large numbe r of object categories.

We present a method for training multi-label, massively multi-class image classification models, that is faster and more accurate than supervision via a sigmoid cross-entropy loss (logistic regression). Our method consists in embedding high-dimensional sparse labels onto a lower-dimensional dense sphere of unit-normed vectors, and treating the classification problem as a cosine proximity regression problem on this sphere. We test our method on a dataset of 300 million high-resolution images with 17,000 labels, where it yields considerably faster convergence, as well as a 7% higher mean average precision compared to logistic regression.


Question: Active-metric learning technique for categorization of remotely sensed hyperspectral images.

This paper offers an in-depth review of manifold-learning-based feature extraction methods for the classification of hyperspectral data, highlighting their effectiveness in capturing the complex geometric and topological structures inherent in hyperspectral images (HSI). With advancements in hyperspectral sensing technology, the ability to characterize spectral signatures across diverse physical and biological systems has expanded, prompting the development of novel methodologies to extract valuable information from these datasets. Manifold learning techniques have proven particularly advantageous in addressing the nonlinearity and sparsity often observed in HSI data. The review underscores the potential of these techniques in remote sensing classification tasks, while also acknowledging the challenges and areas for future research. Notably, the impact of physical phenomena on electromagnetic energy propagation and reflection can be mitigated by manifold learning, yet the effectiveness of these methods can vary with data characteristics, especially at lower spatial resolutions. Users are advised to assess the inherent nonlinearity of their datasets before applying manifold learning. Due to its data-driven nature, the performance of manifold learning is highly dependent on the specific dataset, and no single method can universally outperform others. Parameter tuning is essential for nonlinear manifold-learning algorithms, though they typically offer stable results across a range of values. Compared to linear methods, these algorithms have higher computational demands, which can be a limitation for large-scale remote sensing applications. There is significant potential for advancing manifold learning in the analysis of hyperspectral and multisource remote sensing data. However, the assumption of smooth manifolds may not hold for all datasets, and the presence of distinctly different spectral classes can lead to multiple manifolds or submanifolds that are challenging to represent within a unified framework. Developing methods to effectively characterize and integrate these submanifolds remains an open research challenge, with hierarchical manifold structures showing promise. Most research to date has concentrated on feature extraction from single images, assuming uniformity across the scene. Future studies should explore the integration of global and local embedding techniques in dynamic, multitemporal environments and consider their application in semisupervised and active learning frameworks.

All clustering algorithms process unlabeled data and, consequently, suffer from two problems: (P1) choosing and validating the correct number of clusters and (P2) insuring that algorithmic labels correspond to meaningful physical labels. Clustering algorithms such as hard and fuzzy c-means, based on optimizing sums of squared errors objective functions, suffer from a third problem: (P3) a tendency to recommend solutions that equalize cluster populations. The semi-supervised c-means algorithms introduced in this paper attempt to overcome these three problems for problem domains where a few data from each class can be labeled. Segmentation of magnetic resonance images is a problem of this type and we use it to illustrate the new algorithm. Our examples show that the semi-supervised approach provides MRI segmentations that are superior to ordinary fuzzy c-means and to the crisp k-nearest neighbor rule and further, that the new method ameliorates (P1)-(P3). Cluster analysis Fuzzy c-means Partial supervision Image segmentation Magnetic resonance images

Machine learning algorithms, despite their increasing success in handling object recognition tasks, still seldom perform without error. Often the process of understanding why the algorithm has failed is the task of the human who, using domain knowledge and contextual information, can discover systematic shortcomings in either the data or the algorithm. This paper presents an approach where the process of reasoning about errors emerging from a machine learning framework is automated using symbolic techniques. By utilizing spatial and geometrical reasoning between objects in a scene, the system is able to describe misclassified regions in relation to its context. The system is demonstrated in the remote sensing domain where objects and entities are detected in satellite images.

Hyperspectral imaging is an important visual modality with growing interest and range of applications. The latter, however, is hindered by the fact that existing devices are limited in either spatial, spectral, and/or temporal resolution, while yet being both complicated and expensive. We present a low cost and fast method to recover high quality hyperspectral images directly from RGB. Our approach first leverages hyperspectral prior in order to create a sparse dictionary of hyperspectral signatures and their corresponding RGB projections. Describing novel RGB images via the latter then facilitates reconstruction of the hyperspectral image via the former. A novel, larger-than-ever database of hyperspectral images serves as a hyperspectral prior. This database further allows for evaluation of our methodology at an unprecedented scale, and is provided for the benefit of the research community. Our approach is fast, accurate, and provides high resolution hyperspectral cubes despite using RGB-only input.

Recent studies have demonstrated the usefulness of optical indices from hyperspectral remote sensing in the assessment of vegetation biophysical variables both in forestry and agriculture. Those indices are, however, the combined response to variations of several vegetation and environmental properties, such as Leaf Area Index (LAI), leaf chlorophyll content, canopy shadows, and background soil reflectance. Of particular significance to precision agriculture is chlorophyll content, an indicator of photosynthesis activity, which is related to the nitrogen concentration in green vegetation and serves as a measure of the crop response to nitrogen application. This paper presents a combined modeling and indices-based approach to predicting the crop chlorophyll content from remote sensing data while minimizing LAI (vegetation parameter) influence and underlying soil (background) effects. This combined method has been developed first using simulated data and followed by evaluation in terms of quantitative predictive capability using real hyperspectral airborne data. Simulations consisted of leaf and canopy reflectance modeling with PROSPECT and SAILH radiative transfer models. In this modeling study, we developed an index that integrates advantages of indices minimizing soil background effects and indices that are sensitive to chlorophyll concentration. Simulated data have shown that the proposed index Transformed Chlorophyll Absorption in Reflectance Index/Optimized Soil-Adjusted Vegetation Index (TCARI/OSAVI) is both very sensitive to chlorophyll content variations and very resistant to the variations of LAI and solar zenith angle. It was therefore possible to generate a predictive equation to estimate leaf chlorophyll content from the combined optical index derived from above-canopy reflectance. This relationship was evaluated by application to hyperspectral CASI imagery collected over corn crops in three experimental farms from Ontario and Quebec, Canada. The results presented here are from the L’Acadie, Quebec, Agriculture and AgriFood Canada research site. Images of predicted leaf chlorophyll content were generated. Evaluation showed chlorophyll variability over crop plots with various levels of nitrogen, and revealed an excellent agreement with ground truth, with a correlation of r = .81 between estimated and field measured chlorophyll content data. D 2002 Elsevier Science Inc. All rights reserved.

Scene content understanding facilitates a large number of applications, ranging from content-based image retrieval to other multimedia applications. Material detection refers to the problem of identifying key semantic material types (such as sky, grass, foliage, water, and snow in images. In this paper, we present a holistic approach to determining scene content, based on a set of individual material detection algorithms, as well as probabilistic spatial context models. A major limitation of individual material detectors is the significant number of misclassifications that occur because of the similarities in color and texture characteristics of various material types. We have developed a spatial context-aware material detection system that reduces misclassification by constraining the beliefs to conform to the probabilistic spatial context models. Experimental results show that the accuracy of materials detection is improved by 13% using the spatial context models over the individual material detectors themselves.

At the present time, one of the best methods for semiautomatic image segmentation seems to be the approach based on the fuzzy connectedness principle. First, we identify some deficiencies of this approach and propose a way to improve it, through the introduction of competitive learning. Second, we propose a different approach, based on watersheds. We show that the competitive fuzzy connectedness-based method outperforms the noncompetitive variant and generally (but not always) outperforms the watershed-based approach. The competitive variant of the fuzzy connectedness-based method can be a good alternative to the watersheds.

In the present research we have used GLCM and Gabor filters to extract texture features in order to classify PAN images. The main drawback of GLCM algorithm is its time-consuming nature. In this work, we proposed a fast GLCM algorithm to overcome the mentioned weakness of traditional GLCM. The fast GLCM is capable of extracting approximately the same features as the traditional GLCM does, but in a really much less time (in the best case, 180 times faster, and in the worst case, 30 times faster). The other weakness of the traditional GLCM is its lower accuracies in the region near the class borders. As Gabor filters are more powerful in border regions, we have tried to combine Gabor features with GLCM features. In this way we would compensate the latter mentioned weakness of GLCM. Experimental results show good capabilities of the proposed fast GLCM and the feature fusion method in classification of PAN images.

Focused on the issue that conventional remote sensing image classification methods have run into the bottlenecks in accuracy, a new remote sensing image classification method inspired by deep learning is proposed, which is based on Stacked Denoising Autoencoder. First, the deep network model is built through the stacked layers of Denoising Autoencoder. Then, with noised input, the unsupervised Greedy layer-wise training algorithm is used to train each layer in turn for more robust expressing, characteristics are obtained in supervised learning by Back Propagation (BP) neural network, and the whole network is optimized by error back propagation. Finally, Gaofen-1 satellite (GF-1) remote sensing data are used for evaluation, and the total accuracy and kappa accuracy reach 95.7% and 0.955, respectively, which are higher than that of the Support Vector Machine and Back Propagation neural network. The experiment results show that the proposed method can effectively improve the accuracy of remote sensing image classification.

Grouping images into semantically meaningful categories using low-level visual features is a challenging and important problem in content-based image retrieval. Based on these groupings, eeective indices can be built for an image database. In this paper, we show how a speciic high-level classiication problem (city images vs. landscapes) can be solved from relatively simple low-level features geared for the particular classes. We have developed a procedure to qualitatively measure the saliency of a feature towards a classiication problem based on the plot of the intra-class and inter-class distance distributions. We use this approach to determine the discriminative power of the following features: color histogram, color coherence vector, DCT coeecient, edge direction histogram, and edge direction coherence vector. We determine that the edge direction-based features have the most discriminative power for the classiication problem of interest here. A weighted k-NN classiier is used for the classiication which results in an accuracy of 93:9% when evaluated on an image database of 2; 716 images using the leave-one-out method. This approach has been extended to further classify 528 landscape images into forests, mountains, and sunset/sunrise classes. First, the input images are classiied as sunset/sunrise images vs. forest & mountain images (94:5% accuracy) and then the forest & mountain images are classiied as forest images or mountain images (91:7% accuracy). We are currently identifying further semantic classes to assign to images as well as extracting low level features which are salient for these classes. Our nal goal is to combine multiple 2-class classiiers into a single hierarchical classiier.

The objective of the current study was to analyze the seasonal effect on differentiating tree species in an urban environment using multi-temporal hyperspectral data, Light Detection And Ranging (LiDAR) data, and a tree species database collected from the field. Two Airborne Imaging Spectrometer for Applications (AISA) hyperspectral images were collected, covering the Summer and Fall seasons. In order to make both datasets spatially and spectrally compatible, several preprocessing steps, including band reduction and a spatial degradation, were performed. An object-oriented classification was performed on both images using training data collected randomly from the tree species database. The seven dominant tree species (Gleditsia triacanthos, Acer saccharum, Tilia Americana, Quercus palustris, Pinus strobus and Picea glauca) were used in the classification. The results from this analysis did not show any major difference in overall accuracy between the two seasons. Overall accuracy was approximately 57% for the Summer dataset and 56% for the Fall dataset. However, the Fall dataset provided more consistent results for all tree species while the Summer dataset had a few higher individual class accuracies. Further, adding LiDAR into the classification improved the results by 19% for both fall and summer. This is mainly due to the removal of shadow effect and the addition of elevation data to separate low and high vegetation.


Question: Active metric learning technique for categorization of remotely sensed hyperspectral images.

This paper introduces Stochastic Neighbor Embedding (SNE), a probabilistic technique for placing high-dimensional objects or those defined by pairwise dissimilarities into a low-dimensional space while preserving their neighbor relationships. SNE centers a Gaussian on each high-dimensional object and uses the densities under this Gaussian (or the specified dissimilarities) to define a probability distribution over the object's potential neighbors. The goal is to approximate this distribution as accurately as possible when the same operation is performed on the low-dimensional representations of the objects. The chosen cost function is the sum of Kullback-Leibler divergences, one for each object, which facilitates a straightforward gradient for adjusting the positions of the low-dimensional representations. Distinct from other dimensionality reduction methods, SNE's probabilistic nature allows for the representation of each object as a mixture of widely separated low-dimensional representations. This capability is crucial for handling ambiguous objects, such as the document count vector for the word "bank," which can have versions close to both "river" and "finance" without necessitating that the representations of outdoor concepts be proximate to those of corporate concepts. This makes SNE particularly suitable for tasks like the classification of remotely sensed hyperspectral images, where the data might contain complex and ambiguous relationships.

We present an unsupervised technique for visual learning, which is based on density estimation in high-dimensional spaces using an eigenspace decomposition. Two types of density estimates are derived for modeling the training data: a multivariate Gaussian (for unimodal distributions) and a Mixture-of-Gaussians model (for multimodal distributions). These probability densities are then used to formulate a maximum-likelihood estimation framework for visual search and target detection for automatic object recognition and coding. Our learning technique is applied to the probabilistic visual modeling, detection, recognition, and coding of human faces and nonrigid objects, such as hands.

This paper presents an automatic image segmentation method for polarimetric synthetic aperture radar data. It utilizes the full polarimetric information and incorporates texture by modeling with a non-Gaussian distribution for the complex scattering coefficients. The modeling is based upon the well-known product model, with a Gamma-distributed texture parameter leading to the K-Wishart model for the covariance matrix. The automatic clustering is achieved through a finite mixture model estimated with a modified expectation maximization algorithm. We include an additional goodness-of-fit test stage that allows for splitting and merging of clusters. This not only improves the model fit of the clusters, but also dynamically selects the appropriate number of clusters. The resulting image segmentation depicts the statistically significant clusters within the image. A key feature is that the degree of sub-sampling of the input image will affect the detail level of the clustering, revealing only the major classes or a variable level of detail. Real-world examples are shown to demonstrate the technique.

Purely, data-driven large scale image classification has been achieved using various feature descriptors like SIFT, HOG etc. Major milestone in this regards is Convolutional Neural Networks (CNN) based methods which learn optimal feature descriptors as filters. Little attention has been given to the use of domain knowledge. Ontology plays an important role in learning to categorize images into abstract classes where there may not be a clear visual connect between category and image, for example identifying image mood - happy, sad and neutral. Our algorithm combines CNN and ontology priors to infer abstract patterns in Indian Monument Images. We use a transfer learning based approach in which, knowledge of domain is transferred to CNN while training (top down transfer) and inference is made using CNN prediction and ontology tree/priors (bottom up transfer ). We classify images to categories like Tomb, Fort and Mosque. We demonstrate that our method improves remarkably over logistic classifier and other transfer learning approach. We conclude with a remark on possible applications of the model and note about scaling this to bigger ontology.

In this paper, an improved version of the moth-flame optimization (MFO) algorithm for image segmentation is proposed to effectively enhance the optimal multilevel thresholding of satellite images. Multilevel thresholding is one of the most widely used methods for image segmentation, as it has efficient processing ability and easy implementation. However, as the number of threshold values increase, it consequently becomes computationally expensive. To overcome this problem, the nature-inspired meta-heuristic named multilevel thresholding moth-flame optimization algorithm (MTMFO) for multilevel thresholding was developed. The improved method proposed herein was tested on various satellite images tested against five different existing methods: the genetic algorithm (GA), the differential evolution (DE) algorithm, the artificial bee colony (ABC) algorithm, the particle swarm optimization (PSO) algorithm, and the moth-flame optimization (MFO) algorithm for solving multilevel satellite image thresholding problems. Experimental results indicate that the MTMFO more effectively and accurately identifies the optimal threshold values with respect to the other state-of-the-art optimization algorithms.

Machine learning and pattern recognition techniques are being increasingly employed in functional magnetic resonance imaging (fMRI) data analysis. By taking into account the full spatial pattern of brain activity measured simultaneously at many locations, these methods allow detecting subtle, non-strictly localized effects that may remain invisible to the conventional analysis with univariate statistical methods. In typical fMRI applications, pattern recognition algorithms "learn" a functional relationship between brain response patterns and a perceptual, cognitive or behavioral state of a subject expressed in terms of a label, which may assume discrete (classification) or continuous (regression) values. This learned functional relationship is then used to predict the unseen labels from a new data set ("brain reading"). In this article, we describe the mathematical foundations of machine learning applications in fMRI. We focus on two methods, support vector machines and relevance vector machines, which are respectively suited for the classification and regression of fMRI patterns. Furthermore, by means of several examples and applications, we illustrate and discuss the methodological challenges of using machine learning algorithms in the context of fMRI data analysis.

The research work deals with an approach to perform texture and morphological based retrieval on a corpus of Basmati rice grain images. The work has been carried out using Image Warping and Image analysis approach. The method has been employed to normalize food grain images and hence eliminating the effects of orientation using image warping technique with proper scaling. The approach has been tested on sufficient number of basmati rice grain images of rice based on intensity, position and orientation. A digital image analysis algorithm based on color, morphological and textural features was developed to identify the six varieties of basmati rice seeds which are widely planted in India. Nine color and nine morphological and textural features were used for discriminant analysis. A back propagation neural network-based classifier was developed to identify the unknown grain types. The color and textural features were presented to the neural network for training purposes. The trained network was then used to identify the unknown grain types. It is also to find the percentage purity of hulled basmati rice grain sample by image processing technique. Commercially the purity test of basmati rice sample is done according to the size of the grain kernel (full, half or broken). By image processing we can also identify any broken basmati rice grains. Here we discuss the various procedures used to obtain the percentage quality of basmati rice grains. Keywords— warping, Image rectification, Image segmentation, Edge Detection, blurring image, Thresholding, Percentage Purity, Pixel area.

This paper describes a classification based tree detection method for autonomous navigation of forest vehicles in forest environment. Fusion of color, and texture cues has been used to segment the image into tree trunk and background objects. The segmentation of images into tree trunk and background objects is a challenging task due to high variations of illumination, effect of different color shades, non-homogeneous bark texture, shadows and foreshortening. To accomplish this, the approach has been to find the best combinations of color, and texture descriptors, and classification techniques. An additional task has been to estimate the distance between forest vehicle and the base of segmented trees using monocular vision. A simple heuristic distance measurement method is proposed that is based on pixel height and a reference width. The performance of various color and texture operators, and accuracy of classifiers has been evaluated using cross validation techniques.

The number of digital images is growing extremely rapidly, and so is the need for their classification. But, as more images of pre-defined categories become available, they also become more diverse and cover finer semantic differences. Ultimately, the categories themselves need to be divided into subcategories to account for that semantic refinement. Image classification in general has improved significantly over the last few years, but it still requires a massive amount of manually annotated data. Subdividing categories into subcategories multiples the number of labels, aggravating the annotation problem. Hence, we can expect the annotations to be refined only for a subset of the already labeled data, and exploit coarser labeled data to improve classification. In this work, we investigate how coarse category labels can be used to improve the classification of subcategories. To this end, we adopt the framework of Random Forests and propose a regularized objective function that takes into account relations between categories and subcategories. Compared to approaches that disregard the extra coarse labeled data, we achieve a relative improvement in subcategory classification accuracy of up to 22% in our large-scale image classification experiments.

We extend the traditional active learning framework to include feedback on features in addition to labeling instances, and we execute a careful study of the effects of feature selection and human feedback on features in the setting of text categorization. Our experiments on a variety of categorization tasks indicate that there is significant potential in improving classifier performance by feature re-weighting, beyond that achieved via membership queries alone (traditional active learning) if we have access to an oracle that can point to the important (most predictive) features. Our experiments on human subjects indicate that human feedback on feature relevance can identify a sufficient proportion of the most relevant features (over 50% in our experiments). We find that on average, labeling a feature takes much less time than labeling a document. We devise an algorithm that interleaves labeling features and documents which significantly accelerates standard active learning in our simulation experiments. Feature feedback can complement traditional active learning in applications such as news filtering, e-mail classification, and personalization, where the human teacher can have significant knowledge on the relevance of features.

We propose an iterative crowd-enabled active learning algorithm for building high-precision visual classifiers from unlabeled images. Our method employs domain experts to identify a small number of examples of a specific visual event. These expert-labeled examples seed a classifier, which is then iteratively trained by active querying of a non-expert crowd. These non-experts actively refine the classifiers at every iteration by answering simple binary questions about the classifiers’ detections. The advantage of this approach is that experts efficiently shepherd an unsophisticated crowd into training a classifier capable of fine-grained distinctions. This obviates the need to label an entire dataset to obtain high-precision classifiers. We find these classifiers are advantageous for creating a large vocabulary of visual attributes for specialized taxonomies. We demonstrate our crowd active learning pipeline by creating classifiers for attributes related to North American birds and fashion.


Question: Active-metric learning technique for the classification of remotely sensed hyperspectral images.

This paper introduces a novel technique for conducting nonlinear principal component analysis, which significantly enhances the efficiency of computing principal components in high-dimensional spaces. The method leverages integral operator kernel functions to facilitate the computation of principal components in feature spaces that are nonlinearly related to the input space, such as the space of all possible five-pixel products in 16x16 images. The derivation of this innovative approach is detailed, and its effectiveness is demonstrated through experimental results on polynomial feature extraction for pattern recognition tasks. Specifically, the technique is applied to the classification of remotely sensed hyperspectral images, showing its capability to improve classification accuracy by capturing complex nonlinear relationships within the data. This approach represents a significant advancement in the field of remote sensing image analysis, offering a powerful tool for extracting meaningful information from hyperspectral data.

Existing fine-grained visual categorization methods often suffer from three challenges: lack of training data, large number of fine-grained categories, and high intraclass vs. low inter-class variance. In this work we propose a generic iterative framework for fine-grained categorization and dataset bootstrapping that handles these three challenges. Using deep metric learning with humans in the loop, we learn a low dimensional feature embedding with anchor points on manifolds for each category. These anchor points capture intra-class variances and remain discriminative between classes. In each round, images with high confidence scores from our model are sent to humans for labeling. By comparing with exemplar images, labelers mark each candidate image as either a "true positive" or a "false positive." True positives are added into our current dataset and false positives are regarded as "hard negatives" for our metric learning model. Then the model is retrained with an expanded dataset and hard negatives for the next round. To demonstrate the effectiveness of the proposed framework, we bootstrap a fine-grained flower dataset with 620 categories from Instagram images. The proposed deep metric learning scheme is evaluated on both our dataset and the CUB-200-2001 Birds dataset. Experimental evaluations show significant performance gain using dataset bootstrapping and demonstrate state-of-the-art results achieved by the proposed deep metric learning methods.

We are interested in the problem of adaptive learning in the context of automatic speech recognition (ASR). In this paper, we propose an active learning algorithm for ASR. Automatic speech recognition systems are trained using human supervision to provide transcriptions of speech utterances. The goal of Active Learning is to minimize the human supervision for training acoustic and language models and to maximize the performance given the transcribed and untranscribed data. Active learning aims at reducing the number of training examples to be labeled by automatically processing the unlabeled examples, and then selecting the most informative ones with respect to a given cost function for a human to label. In this paper we describe how to estimate the confidence score for each utterance through an on-line algorithm using the lattice output of a speech recognizer. The utterance scores are filtered through the informativeness function and an optimal subset of training samples is selected. The active learning algorithm has been applied to both batch and on-line learning scheme and we have experimented with different selective sampling algorithms. Our experiments show that by using active learning the amount of labeled data needed for a given word accuracy can be reduced by more than 60% with respect to random sampling.

The fully automated analysis of 3D point clouds is of great importance in photogrammetry, remote sensing and computer vision. For reliably extracting objects such as buildings, road inventory or vegetation, many approaches rely on the results of a point cloud classification, where each 3D point is assigned a respective semantic class label. Such an assignment, in turn, typically involves statistical methods for feature extraction and machine learning. Whereas the different components in the processing workflow have extensively, but separately been investigated in recent years, the respective connection by sharing the results of crucial tasks across all components has not yet been addressed. This connection not only encapsulates the interrelated issues of neighborhood selection and feature extraction, but also the issue of how to involve spatial context in the classification step. In this paper, we present a novel and generic approach for 3D scene analysis which relies on (i) individually optimized 3D neighborhoods for (ii) the extraction of distinctive geometric features and (iii) the contextual classification of point cloud data. For a labeled benchmark dataset, we demonstrate the beneficial impact of involving contextual information in the classification process and that using individual 3D neighborhoods of optimal size significantly increases the quality of the results for both pointwise and contextual classification.

This paper proposes an efficient sparse metric learning algorithm in high dimensional space via an l1-penalized log-determinant regularization. Compare to the most existing distance metric learning algorithms, the proposed algorithm exploits the sparsity nature underlying the intrinsic high dimensional feature space. This sparsity prior of learning distance metric serves to regularize the complexity of the distance model especially in the "less example number p and high dimension d" setting. Theoretically, by analogy to the covariance estimation problem, we find the proposed distance learning algorithm has a consistent result at rate O (√m2 log d)/n) to the target distance matrix with at most m nonzeros per row. Moreover, from the implementation perspective, this l1-penalized log-determinant formulation can be efficiently optimized in a block coordinate descent fashion which is much faster than the standard semi-definite programming which has been widely adopted in many other advanced distance learning algorithms. We compare this algorithm with other state-of-the-art ones on various datasets and competitive results are obtained.

The objective of the current study was to analyze the seasonal effect on differentiating tree species in an urban environment using multi-temporal hyperspectral data, Light Detection And Ranging (LiDAR) data, and a tree species database collected from the field. Two Airborne Imaging Spectrometer for Applications (AISA) hyperspectral images were collected, covering the Summer and Fall seasons. In order to make both datasets spatially and spectrally compatible, several preprocessing steps, including band reduction and a spatial degradation, were performed. An object-oriented classification was performed on both images using training data collected randomly from the tree species database. The seven dominant tree species (Gleditsia triacanthos, Acer saccharum, Tilia Americana, Quercus palustris, Pinus strobus and Picea glauca) were used in the classification. The results from this analysis did not show any major difference in overall accuracy between the two seasons. Overall accuracy was approximately 57% for the Summer dataset and 56% for the Fall dataset. However, the Fall dataset provided more consistent results for all tree species while the Summer dataset had a few higher individual class accuracies. Further, adding LiDAR into the classification improved the results by 19% for both fall and summer. This is mainly due to the removal of shadow effect and the addition of elevation data to separate low and high vegetation.

Tensor representation is helpful to reduce the small sample size problem in discriminative subspace selection. As pointed by this paper, this is mainly because the structure information of objects in computer vision research is a reasonable constraint to reduce the number of unknown parameters used to represent a learning model. Therefore, we apply this information to the vector-based learning and generalize the vector-based learning to the tensor-based learning as the supervised tensor learning (STL) framework, which accepts tensors as input. To obtain the solution of STL, the alternating projection optimization procedure is developed. The STL framework is a combination of the convex optimization and the operations in multilinear algebra. The tensor representation helps reduce the overfitting problem in vector-based learning. Based on STL and its alternating projection optimization procedure, we generalize support vector machines, minimax probability machine, Fisher discriminant analysis, and distance metric learning, to support tensor machines, tensor minimax probability machine, tensor Fisher discriminant analysis, and the multiple distance metrics learning, respectively. We also study the iterative procedure for feature extraction within STL. To examine the effectiveness of STL, we implement the tensor minimax probability machine for image classification. By comparing with minimax probability machine, the tensor version reduces the overfitting problem.

Hyperspectral imaging is an important visual modality with growing interest and range of applications. The latter, however, is hindered by the fact that existing devices are limited in either spatial, spectral, and/or temporal resolution, while yet being both complicated and expensive. We present a low cost and fast method to recover high quality hyperspectral images directly from RGB. Our approach first leverages hyperspectral prior in order to create a sparse dictionary of hyperspectral signatures and their corresponding RGB projections. Describing novel RGB images via the latter then facilitates reconstruction of the hyperspectral image via the former. A novel, larger-than-ever database of hyperspectral images serves as a hyperspectral prior. This database further allows for evaluation of our methodology at an unprecedented scale, and is provided for the benefit of the research community. Our approach is fast, accurate, and provides high resolution hyperspectral cubes despite using RGB-only input.

In entity matching, a fundamental issue while training a classifier to label pairs of entities as either duplicates or non-duplicates is the one of selecting informative training examples. Although active learning presents an attractive solution to this problem, previous approaches minimize the misclassification rate (0-1 loss) of the classifier, which is an unsuitable metric for entity matching due to class imbalance (i.e., many more non-duplicate pairs than duplicate pairs). To address this, a recent paper [1] proposes to maximize recall of the classifier under the constraint that its precision should be greater than a specified threshold. However, the proposed technique requires the labels of all n input pairs in the worst-case.
 Our main result is an active learning algorithm that approximately maximizes recall of the classifier while respecting a precision constraint with provably sub-linear label complexity (under certain distributional assumptions). Our algorithm uses as a black-box any active learning module that minimizes 0-1 loss. We show that label complexity of our algorithm is at most log n times the label complexity of the black-box, and also bound the difference in the recall of classifier learnt by our algorithm and the recall of the optimal classifier satisfying the precision constraint. We provide an empirical evaluation of our algorithm on several real-world matching data sets that demonstrates the effectiveness of our approach.

In this paper, a unique approach for road extraction utilizing pixel spectral information for classification and image segmentation-derived object features was developed. In this approach, road extraction was performed in two steps. In the first step, support vector machine (SVM) was employed merely to classify the image into two groups of categories: a road group and a non-road group. For this classification, support vector machine (SVM) achieved higher accuracy than Gaussian maximum likelihood (GML). In the second step, the road group image was segmented into geometrically homogeneous objects using a region growing technique based on a similarity criterion, with higher weighting on shape factors over spectral criteria. A simple thresholding on the shape index and density features derived from these objects was performed to extract road features, which were further processed by thinning and vectorization to obtain road centerlines. The experiment showed the proposed approach worked well with images comprised by both rural and urban area features. Introduction Road information not only plays a central role in the transportation application, but also is an important data layer in Geographical Information Systems (GIS). Automated road extraction can save time and labor to a great degree in updating a road spatial database. Various road extraction approaches have been developed. Xiong (2001) grouped these methods into five categories: ridge finding, heuristic reasoning, dynamic programming, statistical inference, and map matching. In ridge finding, edge operators are performed on images to derive edge magnitude and direction, followed by a thresholding and thinning process to obtain ridge pixels (Nevatia and Babu, 1980; Treash and Amaratunga, 2000). Alternatively, gradient direction profile analysis can be performed to generate edge pixels (Gong and Wang, 1997). Ridge points are linked to produce the road segments. Heuristic reasoning is a knowledge-based method in which a series of pre-set rules on road characteristics such as shape index, the distance between image primitives, fragments trend, and contextual information are employed to detect and connect image primitives or antiparallel linear edges to road segments (McKeown, et al., 1985; Zhu and Yeh, 1986). In the dynamic programming method, roads are modeled with a set of mathematical equations on the derivatives of gray values and select characteristics of roads, such as smooth curves, homogeneous surface, narrow linear features, and relatively constant width. Dynamic programming is employed to solve the optimization problem Road Extraction Using SVM and Image Segmentation Mingjun Song and Daniel Civco (Gruen and Li, 1995). In the statistical inference method, linear features are modeled as a Markov point process or a geometric-stochastic model on the road width, direction, intensity and background intensity, and maximum a posteriori probability is used to estimate the road network (Barzohar and Cooper, 1996; Stoica, et al., 2000). In a map matching method, existing road maps are used as starting point to update the road network. In general, two steps are involved: first, a mapimage matching algorithm is employed to match the roads on the map to the image; second, new roads are searched based on the assumption that they are connected to existing roads (Stilla, 1995). Xiong’s classification on road extraction methods is only a generalization, and some other methods may combine different techniques. Active contour models, known as snakes, are also used in road extraction (Gruen and Li, 1997; Agouris, et al., 2001). A snake is a spline with minimized energy driven by internal spline and external image forces (Park, et al., 2001). In general, external image forces are represented by the gradient magnitude of an image, which attracts snakes to contours with strong edges. Internal forces are given by a continuity term and a curvature term expressed by the differences of adjacent snaxels, which are vertex nodes of the snake, with weights coming from training data, which control the shape and smoothness of the snakes. Through the optimization, the snake evolves from its initial position to desired position with minimized energy. Park and Kim (2001) used template matching to extract road segments in which a road template was formed around the road seed, and an adaptive least squares matching algorithm was used to detect a target window with similar transformation. This method assumes a small difference in brightness values between template and target windows. Most of these road extraction methods require some road seeds as starting points, which are in general provided by users, and road segments evolve under a certain model. Sometimes control points are needed to correct the evolution of roads (Zhao, et al., 2002). Further, these methods use black-and-white aerial photographs or the panchromatic band of high-resolution satellite images and therefore the geometric characteristics of roads alone play a critical role. Boggess (1993) used a classification method incorporating texture and neural networks to extract roads by classifying roads and other features from Landsat TM imagery, but obtained numerous false-inclusions. Roberts, et al. (2001) developed a spectral mixture library using hyperspectral images to extract roads, but the use of spectral information alone does not capture the spatial properties of these curvilinear features. P H OTO G R A M M E T R I C E N G I N E E R I N G & R E M OT E S E N S I N G December 2004 1 3 6 5 Center for Land use Education and Research, Department of Natural Resources Management and Engineering, The University of Connecticut U-4087, 1376 Storrs Road, Storrs, CT 06269-4087 (mingjun.song@uconn.edu, daniel.civco@ uconn.edu). Photogrammetric Engineering & Remote Sensing Vol. 70, No. 12, December 2004, pp. 1365–1371. 0099-1112/04/7012–1365/$3.00/0 © 2004 American Society for Photogrammetry and Remote Sensing LFX-536.qxd 11/9/04 16:13 Page 1365

Texture is one of the important characteristics used in identifying objects or regions of interest in an image, whether the image be a photomicrograph, an aerial photograph, or a satellite image. This paper describes some easily computable textural features based on graytone spatial dependancies, and illustrates their application in categoryidentification tasks of three different kinds of image data: photomicrographs of five kinds of sandstones, 1:20 000 panchromatic aerial photographs of eight land-use categories, and Earth Resources Technology Satellite (ERTS) multispecial imagery containing seven land-use categories. We use two kinds of decision rules: one for which the decision regions are convex polyhedra (a piecewise linear decision rule), and one for which the decision regions are rectangular parallelpipeds (a min-max decision rule). In each experiment the data set was divided into two parts, a training set and a test set. Test set identification accuracy is 89 percent for the photomicrographs, 82 percent for the aerial photographic imagery, and 83 percent for the satellite imagery. These results indicate that the easily computable textural features probably have a general applicability for a wide variety of image-classification applications.


Question: Ad hoc retrieval experiments utilizing WordNet and thesauri constructed automatically.

An Innovative Approach to Query Expansion: Leveraging Concept Similarity and Automatic Thesauri Construction

Query expansion techniques have been a subject of extensive research, often yielding mixed outcomes in terms of enhancing retrieval effectiveness. This paper introduces a novel probabilistic query expansion model that harnesses the power of an automatically generated similarity thesaurus. The uniqueness of this thesaurus lies in its reflection of domain-specific knowledge, derived directly from the collection it represents. Two pivotal challenges in query expansion are tackled in this work: the identification and the prioritization of supplementary search terms. Unlike conventional approaches, our method selects terms that closely align with the overarching concept of the query, rather than merely those that are similar to individual query terms. Experimental results affirm that this approach significantly boosts retrieval performance, as evidenced by improvements in both recall-precision metrics and overall usefulness. This innovation underscores the potential of concept-based query expansion in refining search outcomes.

In this paper, we describe a system called TagAssist that provides tag suggestions for new blog posts by utilizing existing tagged posts. The system is able to increase the quality of suggested tags by performing lossless compression over existing tag data. In addition, the system employs a set of metrics to evaluate the quality of a potential tag suggestion. Coupled with the ability for users to manually add tags, TagAssist can ease the burden of tagging and increase the utility of retrieval and browsing systems built on top of tagging data.

The semantic representation of individual word senses and concepts is of fundamental importance to several applications in Natural Language Processing. To date, concept modeling techniques have in the main based their representation either on lexicographic resources, such as WordNet, or on encyclopedic resources, such as Wikipedia. We propose a vector representation technique that combines the complementary knowledge of both these types of resource. Thanks to its use of explicit semantics combined with a novel cluster-based dimensionality reduction and an effective weighting scheme, our representation attains state-of-the-art performance on multiple datasets in two standard benchmarks: word similarity and sense clustering. We are releasing our vector representations at http://lcl.uniroma1.it/nasari/.

Word sense disambiguation assumes word senses. Within the lexicography and linguistics literature, they are known to be very slippery entities. The paper looks at problems with existing accounts of ‘word sense’ and describes the various kinds of ways in which a word’s meaning can deviate from its core meaning. An analysis is presented in which word senses are abstractions from clusters of corpus citations, in accordance with current lexicographic practice. The corpus citations, not the word senses, are the basic objects in the ontology. The corpus citations will be clustered into senses according to the purposes of whoever or whatever does the clustering. In the absence of such purposes, word senses do not exist. Word sense disambiguation also needs a set of word senses to disambiguate between. In most recent work, the set has been taken from a general-purpose lexical resource, with the assumption that the lexical resource describes the word senses of English/French/: : : , between which NLP applications will need to disambiguate. The implication of the paper is, by contrast, that word senses exist only relative to a task.

Word Sense Disambiguation (WSD) is a key enabling-technology. Supervised WSD techniques are the best performing in public evaluations, but need large amounts of hand-tagging data. Existing hand-annotated corpora like SemCor (Miller et al., 1993), which is annotated with WordNet senses (Fellbaum, 1998) allow for a small improvement over the simple most frequent sense heuristic, as attested in the all-words track of the last Senseval competition (Snyder and Palmer, 2004). In theory, larger amounts of training data (SemCor has approx. 500M words) would improve the performance of supervised WSD, but no current project exists to provide such an expensive resource. Another problem of the supervised approach is that the inventory and distribution of senses changes dramatically from one domain to the other, requiring additional hand-tagging of corpora (Martı́nez and Agirre, 2000; Koeling et al., 2005). Supervised WSD is based on the “fixed-list of senses” paradigm, where the senses for a target word are a closed list coming from a dictionary or lexicon. Lexicographers and semanticists have long warned about the problems of such an approach, where senses are listed separately as discrete entities, and have argued in favor of more complex representations, where, for instance, senses are dense regions in a continuum (Cruse, 2000). Unsupervised Word Sense Induction and Discrimination (WSID, also known as corpus-based unsupervised systems) has followed this line of thinking, and tries to induce word senses directly from the corpus. Typical WSID systems involve clustering techniques, which group together similar examples. Given a set of induced clusters (which represent word uses or senses1), each new occurrence of the target word will be compared to the clusters and the most similar cluster will be selected as its sense. One of the problems of unsupervised systems is that of managing to do a fair evaluation. Most of current unsupervised systems are evaluated in-house, with a brief comparison to a re-implementation of a former system, leading to a proliferation of unsupervised systems with little ground to compare among them. The goal of this task is to allow for comparison across sense-induction and discrimination systems, and also to compare these systems to other supervised and knowledge-based systems. The paper is organizes as follows. Section 2 presents the evaluation framework used in this task. Section 3 presents the systems that participated in the task, and the official results. Finally, Section 4 draws the conclusions.

We propose a framework of word-vector adaptation, which makes vectors of visually similar concepts close to each other. Here, word vectors are real-valued vector representation of words, e.g., word2vec representation. Our basic idea is to assume that each concept has some hypernyms that are important to determine its visual features. For example, for a concept Swallow with hypernyms Bird, Animal and Entity, we believe Bird is the most important since birds have common visual features with their feathers etc. Adapted word vectors are obtained for each word by taking a weighted sum of a given original word vector and its hypernym word vectors. Our weight optimization makes vectors of visually similar concepts close to each other, by giving a large weight for such important hypernyms. We apply the adapted word vectors to zero-shot learning on the TRECVID 2014 semantic indexing dataset. We achieved 0.083 of Mean Average Precision, which is the best performance without using TRECVID training data to the best of our knowledge.

As the number of Internet users and the number of accessible Web pages grows, it is becoming increasingly difficult for users to find documents that are relevant to their particular needs. Users must either browse through a large hierarchy of concepts to find the information for which they are looking or submit a query to a publicly available search engine and wade through hundreds of results, most of them irrelevant. The core of the problem is that whether the user is browsing or searching, whether they are an eighth grade student or a Nobel prize winner, the identical information is selected and it is presented the same way. In this paper, we report on research that adapts information navigation based on a user profile structured as a weighted concept hierarchy. A user may create his or her own concept hierarchy and use them for browsing Web sites. Or, the user profile may be created from a reference ontology by ‘watching over the user’s shoulder’ while they browse. We show that these automatically created profiles reflect the user’s interests quite well and they are able to produce moderate improvements when applied to search results.

We propose a novel graph-based approach for constructing concept hierarchy from a large text corpus. Our algorithm, GraBTax, incorporates both statistical co-occurrences and lexical similarity in optimizing the structure of the taxonomy. To automatically generate topic-dependent taxonomies from a large text corpus, GraBTax first extracts topical terms and their relationships from the corpus. The algorithm then constructs a weighted graph representing topics and their associations. A graph partitioning algorithm is then used to re-cursively partition the topic graph into a tax-onomy. For evaluation, we apply GraBTax to articles, primarily computer science, in the CiteSeerX digital library and search engine. The quality of the resulting concept hierarchy is assessed by both human judges and comparison with Wikipedia categories.

a r t i c l e i n f o a b s t r a c t We present an automatic approach to the construction of BabelNet, a very large, wide-coverage multilingual semantic network. Key to our approach is the integration of lexicographic and encyclopedic knowledge from WordNet and Wikipedia. In addition, Machine Translation is applied to enrich the resource with lexical information for all languages. We first conduct in vitro experiments on new and existing gold-standard datasets to show the high quality and coverage of BabelNet. We then show that our lexical resource can be used successfully to perform both monolingual and cross-lingual Word Sense Disambiguation: thanks to its wide lexical coverage and novel semantic relations, we are able to achieve state-of the-art results on three different SemEval evaluation tasks.

Previous studies have shown that semantically meaningful representations of words and text can be acquired through neural embedding models. In particular, paragraph vector (PV) models have shown impressive performance in some natural language processing tasks by estimating a document (topic) level language model. Integrating the PV models with traditional language model approaches to retrieval, however, produces unstable performance and limited improvements. In this paper, we formally discuss three intrinsic problems of the original PV model that restrict its performance in retrieval tasks. We also describe modifications to the model that make it more suitable for the IR task, and show their impact through experiments and case studies. The three issues we address are (1) the unregulated training process of PV is vulnerable to short document over-fitting that produces length bias in the final retrieval model; (2) the corpus-based negative sampling of PV leads to a weighting scheme for words that overly suppresses the importance of frequent words; and (3) the lack of word-context information makes PV unable to capture word substitution relationships.

This research proposes an expanded semantic graph definition that serves as a basis for an expanded semantic graph representation and graph matching approach. This representation separates the content and context and adds a number of semantic structures that encapsulate inferred information. The expanded semantic graph approach facilitates finding additional matches, identifying and eliminating poor matches, and prioritizing matches based on how much new knowledge is provided. By focusing on information of interest, doing pre-processing, and reducing processing requirements, the approach is applicable to problems where related information of interest is sought across a massive body of free text documents. Key aspects of the approach include (1) expanding the nodes and edges through inference using DL-Safe rules, abductive hypotheses, and syntactic patterns, (2) separating semantic content into nodes and semantic context into edges, and (3) applying relatedness measures on a node, edge, and sub graph basis. Results from tests using a ground-truthed subset of a large dataset of law enforcement investigator emails illustrate the benefits of these approaches.


Question: Experiments with ad hoc retrieval utilizing WordNet and thesauri constructed automatically.

This document presents a method, coined PhraseFinder, for the automatic construction of collection-dependent association thesauri utilizing extensive full-text document databases. Despite their widespread use in both commercial and experimental information retrieval systems, thesauri have not consistently proven to enhance retrieval performance, and their automatic creation for large text collections poses significant challenges. The proposed approach, PhraseFinder, aims to address these issues by generating association thesauri that can be seamlessly integrated into natural language queries within INQUERY, an information retrieval system grounded in probabilistic inference networks. To assess the effectiveness of different association thesauri types and thesauri developed for diverse collections, a series of experiments are carried out within the INQUERY environment. These experiments aim to evaluate the impact of automatically constructed thesauri on retrieval performance, highlighting the potential benefits and limitations of using association thesauri in information retrieval tasks.

As the number of Internet users and the number of accessible Web pages grows, it is becoming increasingly difficult for users to find documents that are relevant to their particular needs. Users must either browse through a large hierarchy of concepts to find the information for which they are looking or submit a query to a publicly available search engine and wade through hundreds of results, most of them irrelevant. The core of the problem is that whether the user is browsing or searching, whether they are an eighth grade student or a Nobel prize winner, the identical information is selected and it is presented the same way. In this paper, we report on research that adapts information navigation based on a user profile structured as a weighted concept hierarchy. A user may create his or her own concept hierarchy and use them for browsing Web sites. Or, the user profile may be created from a reference ontology by ‘watching over the user’s shoulder’ while they browse. We show that these automatically created profiles reflect the user’s interests quite well and they are able to produce moderate improvements when applied to search results.

Manually and automatically constructed concept graphs (or semantic networks), in which the nodes correspond to words or phrases and the typed edges designate semantic relationships between words and phrases, have been previously shown to be rich sources of effective latent concepts for query expansion. However, finding good expansion concepts for a given query in large and dense concept graphs is a challenging problem, since the number of candidate concepts that are related to query terms and phrases and need to be examined increases exponentially with the distance from the original query concepts. In this paper, we propose a two-stage feature-based method for sequential selection of the most effective concepts for query expansion from a concept graph. In the first stage, the proposed method weighs the concepts according to different types of computationally inexpensive features, including collection and concept graph statistics. In the second stage, a sequential concept selection algorithm utilizing more expensive features is applied to find the most effective expansion concepts at different distances from the original query concepts. Experiments on TREC datasets of different type indicate that the proposed method achieves significant improvement in retrieval accuracy over state-of-the-art methods for query expansion using concept graphs.

STRAND (Resnik, 1998) is a languageindependent system for automatic discovery of text in parallel translation on the World Wide Web. This paper extends the preliminary STRAND results by adding automatic language identification, scaling up by orders of magnitude, and formally evaluating performance. The most recent end-product is an automatically acquired parallel corpus comprising 2491 English-French document pairs, approximately 1.5 million words per language. 1 I n t r o d u c t i o n Text in parallel translation is a valuable resource in natural language processing. Statistical methods in machine translation (e.g. (Brown et al., 1990)) typically rely on large quantities of bilingual text aligned at the document or sentence level, and a number of approaches in the burgeoning field of crosslanguage information retrieval exploit parallel corpora either in place of or in addition to mappings between languages based on information from bilingual dictionaries (Davis and Dunning, 1995; Landauer and Littman, 1990; Hull and Oard, 1997; Oard, 1997). Despite the utility of such data, however, sources of bilingual text are subject to such limitations as licensing restrictions, usage fees, restricted domains or genres, and dated text (such as 1980's Canadian politics); or such sources simply may not exist for * This work was supported by Department of Defense contract MDA90496C1250, DARPA/ITO Contract N66001-97-C-8540, and a research grant from Sun Microsystems Laboratories. The author gratefully acknowledges the comments of the anonymous reviewers, helpful discussions with Dan Melamed and Doug Oard, and the assistance of Jeff Allen in the French-English experimental evaluation. language pairs of interest. Although the majority of Web content is in English, it also shows great promise as a source of multilingual content. Using figures from the Babel survey of multilinguality on the Web (h tZp : / /www. i s o c . o r g / ) , it is possible to estimate that as of June, 1997, there were on the order of 63000 primarily non-English Web servers, ranging over 14 languages. Moreover, a followup investigation of the non-English servers suggests that nearly a third contain some useful cross-language data, such as parallel English on the page or links to parallel English pages the follow-up also found pages in five languages not identified by the Babel study (Catalan, Chinese, Hungarian, Icelandic, and Arabic; Michael Littman, personal communication). Given the continued explosive increase in the size of the Web, the trend toward business organizations that cross national boundaries, and high levels of competition for consumers in a global marketplace, it seems impossible not to view multilingual content on the Web as an expanding resource. Moreover, it is a dynamic resource, changing in content as the world changes. For example, Diekema et al., in a presentation at the 1998 TREC-7 conference (Voorhees and Harman, 1998), observed that the performance of their cross-language information retrieval was hurt by lexical gaps such as Bosnia/Bosniethis illustrates a highly topical missing pair in their static lexical resource (which was based on WordNet 1.5). And Gey et al., also at TREC-7, observed that in doing cross-language retrieval using commercial machine translation systems, gaps in the lexicon (their example was acupuncture/Akupunktur) could make the difference between precision of 0.08 and precision of 0.83 on individual queries. ttesnik (1998) presented an algorithm called

We propose that entity queries are generated via a two-step process: users first select entity facts that can distinguish target entities from the others; and then choose words to describe each selected fact. Based on this query generation paradigm, we propose a new entity representation model named as entity factoid hierarchy. An entity factoid hierarchy is a tree structure composed of factoid nodes. A factoid node describes one or more facts about the entity in different information granularities. The entity factoid hierarchy is constructed via a factor graph model, and the inference on the factor graph is achieved by a modified variant of Multiple-try Metropolis algorithm. Entity retrieval is performed by decomposing entity queries and computing the query likelihood on the entity factoid hierarchy. Using an array of benchmark datasets, we demonstrate that our proposed framework significantly improves the retrieval performance over existing models.

The openly available biomedical literature contains over 5 billion words in publication abstracts and full texts. Recent advances in unsupervised language processing methods have made it possible to make use of such large unannotated corpora for building statistical language models and inducing high quality vector space representations, which are, in turn, of utility in many tasks such as text classification, named entity recognition and query expansion. In this study, we introduce the first set of such language resources created from analysis of the entire available biomedical literature, including a dataset of all 1to 5-grams and their probabilities in these texts and new models of word semantics. We discuss the opportunities created by these resources and demonstrate their application. All resources introduced in this study are available under open licenses at http://bio.nlplab.org.

We combine search in triple stores with full-text search into what we call \emph{semantic full-text search}. We provide a fully functional web application that allows the incremental construction of complex queries on the English Wikipedia combined with the facts from Freebase. The user is guided by context-sensitive suggestions of matching words, instances, classes, and relations after each keystroke. We also provide a powerful API, which may be used for research tasks or as a back end, e.g., for a question answering system. Our web application and public API are available under \url{http://broccoli.cs.uni-freiburg.de}.

Categorization of documents is challenging, as the number o f discriminating words can be very large. We present a nearest neighbor classification scheme for text categoriz ation in which the importance of discriminating words is learned using mutual information and weight adjustment tec hniques. The nearest neighbors for a particular document are then computed based on the matching words and their weigh ts. We evaluate our scheme on both synthetic and real world documents. Our experiments with synthetic data sets s how that this scheme is robust under different emulated conditions. Empirical results on real world documents demo nstrate that this scheme outperforms state of the art classification algorithms such as C4.5, RIPPER, Rainbow, an d PEBLS.

Vector space word representations are useful for many natural language processing applications. The diversity of techniques for computing vector representations and the large number of evaluation benchmarks makes reliable comparison a tedious task both for researchers developing new vector space models and for those wishing to use them. We present a website and suite of offline tools that that facilitate evaluation of word vectors on standard lexical semantics benchmarks and permit exchange and archival by users who wish to find good vectors for their applications. The system is accessible at: www.wordvectors.org.

We examine the problems with automated recommendation systems when information about user preferences is limited. We equate the problem to one of content similarity measurement and apply techniques from Natural Language Processing to the domain of movie recommendation. We describe two algorithms, a naïve word-space approach and a more sophisticated approach using topic signatures, and evaluate their performance compared to baseline, gold standard, and commercial systems.

We propose a self-supervised word segmentation technique for text segmentation in Chinese information retrieval. This method combines the advantages of traditional dictionary based, character based and mutual information based approaches, while overcoming many of their shortcomings. Experiments on TREC data show this method is promising. Our method is completely language independent and unsupervised, which provides a promising avenue for constructing accurate multi-lingual or cross-lingual information retrieval systems that are flexible and adaptive. We find that although the segmentation accuracy of self-supervised segmentation is not as high as some other segmentation methods, it is enough to give good retrieval performance. It is commonly believed that word segmentation accuracy is monotonically related to retrieval performance in Chinese information retrieval. However, for Chinese, we find that the relationship between segmentation and retrieval performance is in fact nonmonotonic; that is, at around 70% word segmentation accuracy an over-segmentation phenomenon begins to occur which leads to a reduction in information retrieval performance. We demonstrate this effect by presenting an empirical investigation of information retrieval on Chinese TREC data, using a wide variety of word segmentation algorithms with word segmentation accuracies ranging from 44% to 95%, including 70% word segmentation accuracy from our self-supervised word-segmentation approach. It appears that the main reason for the drop in retrieval performance is that correct compounds and collocations are preserved by accurate segmenters, while they are broken up by less accurate (but reasonable) segmenters, to a surprising advantage. This suggests that words themselves might be too broad a notion to conveniently capture the general semantic meaning of Chinese text. Our research suggests machine learning techniques can play an important role in building adaptable information retrieval systems and different evaluation standards for word segmentation should be given to different applications.