Question: An improved search technique incorporating wildcards and morphological variations in the Google Books Ngram Viewer.

This study delves into the quantifiable dynamics of language evolution, specifically focusing on the regularization of English verbs over a historical span of 1,200 years. Language, grounded in grammatical rules, undergoes cultural evolution, allowing for the transformation and replacement of these rules over time. In English, this process has led to the predominance of the dental suffix ‘-ed’ for the indication of past tense, replacing a once-complex system of conjugations inherited from its proto-Germanic roots. The research examines the rise of this linguistic rule and the concurrent decline of its exceptions, known as irregular verbs. A comprehensive dataset of verbs that have undergone conjugational evolution for over a millennium has been compiled, with particular attention to the inflectional changes of 177 Old English irregular verbs. Among these, 145 maintained their irregularity through Middle English, and 98 continue to be irregular in contemporary usage. The study further explores how the pace of regularization is influenced by the frequency of word usage. It is observed that the half-life of an irregular verb is inversely proportional to the square root of its usage frequency, indicating that verbs with lower frequency regularize at a faster rate compared to more commonly used verbs. This investigation offers a detailed, quantitative analysis of the regularization process, highlighting how ancestral linguistic forms gradually give way to emerging rules.

Retrieving the lyrics of a sung recording from a database of text documents is a research topic that has not received much attention so far. Such a retrieval system has many practical applications, e.g. for karaoke applications or for indexing large song databases by their lyric content. We present a new method for lyrics retrieval. An acoustic model trained on singing is used to obtain phoneme probabilities from sung queries, which are then mapped to phoneme sequences. These are compared against lines of textual lyrics in a large corpus in order to retrieve the best-matching song. The approach is tested on three sung datasets. Lyrics are retrieved from a set of 300 possible songs (12,000 lines of lyrics). The results are highly encouraging and could be used further to perform automatic lyrics alignment and keyword spotting for large databases of songs, or for retrieving lyrics from the internet.

In this paper we present BabelNet 1.1, a brand-new release of the largest “encyclopedic dictionary”, obtained from the automatic integration of the most popular computational lexicon of English, i.e. WordNet, and the largest multilingual Web encyclopedia, i.e. Wikipedia. BabelNet 1.1 covers 6 languages and comes with a renewed Web interface, graph explorer and programmatic API. BabelNet is available online at http://www.babelnet.org.

Automatic lexical acquisition has been an active area of research in computational linguistics for over two decades, but the automatic identification of new word-senses has received attention only very recently. Previous work on this topic has been limited by the availability of appropriate evaluation resources. In this paper we present the largest corpus-based dataset of diachronic sense differences to date, which we believe will encourage further work in this area. We then describe several extensions to a state-of-the-art topic modelling approach for identifying new word-senses. This adapted method shows superior performance on our dataset of two different corpus pairs to that of the original method for both: (a) types having taken on a novel sense over time; and (b) the token instances of such novel senses.

We present an efficient algorithm to estimate large modified Kneser-Ney models including interpolation. Streaming and sorting enables the algorithm to scale to much larger models by using a fixed amount of RAM and variable amount of disk. Using one machine with 140 GB RAM for 2.8 days, we built an unpruned model on 126 billion tokens. Machine translation experiments with this model show improvement of 0.8 BLEU point over constrained systems for the 2013 Workshop on Machine Translation task in three language pairs. Our algorithm is also faster for small models: we estimated a model on 302 million tokens using 7.7% of the RAM and 14.0% of the wall time taken by SRILM. The code is open source as part of KenLM.

Query expansion methods have been studied for a long time - with debatable success in many instances. In this paper we present a probabilistic query expansion model based on a similarity thesaurus which was constructed automatically. A similarity thesaurus reflects domain knowledge about the particular collection from which it is constructed. We address the two important issues with query expansion: the selection and the weighting of additional search terms. In contrast to earlier methods, our queries are expanded by adding those terms that are most similar to the concept of the query, rather than selecting terms that are similar to the query terms. Our experiments show that this kind of query expansion results in a notable improvement in the retrieval effectiveness when measured using both recall-precision and usefulness.

The Mongue-Elkan method is a general text string comparison method based on an internal character-based similarity measure (e.g. edit distance) combined with a token level (i.e. word level) similarity measure. We propose a generalization of this method based on the notion of the generalized arithmetic mean instead of the simple average used in the expression to calculate the Monge-Elkan method. The experiments carried out with 12 well-known name-matching data sets show that the proposed approach outperforms the original Monge-Elkan method when character-based measures are used to compare tokens.

The naive Bayes classiier, currently experiencing a renaissance in machine learning, has long been a core technique in information retrieval. We review some of the variations of naive Bayes models used for text retrieval and classiication, focusing on the distributional assumptions made about word occurrences in documents.

0957-4174/$ see front matter 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.eswa.2013.05.050 ⇑ Corresponding author at: LT3 Language and Translation Technology Team, University College Ghent, Groot-Brittanniëlaan 45, 9000 Ghent, Belgium. Tel.: +32 9 224 97 53. E-mail addresses: bart.desmet@hogent.be (B. Desmet), veronique.hoste@ hogent.be (V. Hoste). Bart Desmet a,b,⇑, Véronique Hoste a,c

Citation field extraction entails segmenting a citation string into its constituent parts, such as title, authors, publisher and year. Despite the importance of this task, there is a lack of well-annotated citation data. This paper presents a new labeled dataset for citation extraction that, in comparison to the previous standard dataset, exceeds four-times more data, supplies detailed nested labels rather than coarse-grained flat labels, and is derived from four different academic fields rather than one. We describe our new dataset in detail, and provide baseline experimental results from a state-of-the-art extraction method.

String matching is the problem of finding all the occurrences of a pattern in a text. We propose a very fast new family of string matching algorithms based on hashing q-grams. The new algorithms are the fastest on many cases, in particular, on small size alphabets. © 2007 Elsevier B.V. All rights reserved.


Question: An evolutionary cost-sensitive extreme learning machine with subspace extension approach.

This document presents the Gravitational Search Algorithm (GSA), a novel optimization technique inspired by the principles of gravity and mass interactions observed in the natural world. Unlike other heuristic methods that draw inspiration from swarm behaviors, GSA introduces a unique approach where the searcher agents are modeled as masses. These masses interact with one another following the laws of Newtonian physics, specifically gravity and motion. The efficacy of GSA has been rigorously tested against established heuristic search algorithms, and the results affirm its superior performance in tackling a variety of nonlinear function optimization problems. This work was published in 2009 by Elsevier Inc., with all rights reserved.

However, this document does not directly address the question regarding the Evolutionary Cost-sensitive Extreme Learning Machine and Subspace Extension. It focuses on the introduction and validation of the Gravitational Search Algorithm as a powerful heuristic optimization method, highlighting its ability to outperform other algorithms in solving nonlinear function problems. The document's semantic style is preserved, maintaining a scientific and descriptive tone while retaining the complexity and length of the original text.

Ensemble learning is an intensively studies technique in machine learning and pattern recognition. Recent work in computational biology has seen an increasing use of ensemble learning methods due to their unique advantages in dealing with small sample size, high-dimensionality, and complexity data structures. The aim of this article is two-fold. First, it is to provide a review of the most widely used ensemble learning methods and their application in various bioinformatics problems, including the main topics of gene expression, mass spectrometry-based proteomics, gene-gene interaction identification from genome-wide association studies, and prediction of regulatory elements from DNA and protein sequences. Second, we try to identify and summarize future trends of ensemble methods in bioinformatics. Promising directions such as ensemble of support vector machine, meta-ensemble, and ensemble based feature selection are discussed.

To date, almost all experimental evaluations of machine learning-based recognition algorithms in computer vision have taken the form of “closed set” recognition, whereby all testing classes are known at training time. A more realistic scenario for vision applications is “open set” recognition, where incomplete knowledge of the world is present at training time, and unknown classes can be submitted to an algorithm during testing. This paper explores the nature of open set recognition and formalizes its definition as a constrained minimization problem. The open set recognition problem is not well addressed by existing algorithms because it requires strong generalization. As a step toward a solution, we introduce a novel “1-vs-set machine,” which sculpts a decision space from the marginal distances of a 1-class or binary SVM with a linear kernel. This methodology applies to several different applications in computer vision where open set recognition is a challenging problem, including object recognition and face verification. We consider both in this work, with large scale cross-dataset experiments performed over the Caltech 256 and ImageNet sets, as well as face matching experiments performed over the Labeled Faces in the Wild set. The experiments highlight the effectiveness of machines adapted for open set evaluation compared to existing 1-class and binary SVMs for the same tasks.

Support vector machines (SVM) is originally designed for binary classification. To extend it to multi-class scenario, a typical conventional way is to decompose an M-class problem into a series of two-class problems, for which one-against-all is the earliest and one of the most widely used implementations. However, certain theoretical analysis reveals a drawback, i.e., the competence of each classifier is totally neglected when the results of classification from the multiple classifiers are combined for the final decision. To overcome this limitation, this paper introduces reliability measures into the multi-class framework. Two measures are designed: static reliability measure (SRM) and dynamic reliability measure (DRM). SRM works on a collective basis and yields a constant value regardless of the location of the test sample. DRM, on the other hand, accounts for the spatial variation of the classifier's performance. Based on these two reliability measures, a new decision strategy for the one-against-all method is proposed, which is tested on benchmark data sets and demonstrates its effectiveness.

We introduce an estimation of distribution algorithm (EDA) based on co-evolution of fitness maximizers and fitness predictors for improving the performance of evolutionary search when evaluations are prohibitively expensive. Fitness predictors are lightweight objects which, given an evolving individual, heuristically approximate the true fitness. The predictors are trained by their ability to correctly differentiate between good and bad solutions using reduced computation. We apply co-evolving fitness prediction to symbolic regression and measure its impact. Our results show that the additional computational investment in training the co-evolving fitness predictors can greatly enhance both speed and convergence of individual solutions while overall reducing the number of evaluations. In application to symbolic regression, the advantage of using fitness predictors grows as the complexity of models increases.

Multilabel classification is prevalent in many real-world applications where data instances may be associated with multiple labels simultaneously. In multilabel classification, exploiting label correlations is an essential but nontrivial task. Most of the existing multilabel learning algorithms are either ineffective or computationally demanding and less scalable in exploiting label correlations. In this paper, we propose a co-evolutionary multilabel hypernetwork (Co-MLHN) as an attempt to exploit label correlations in an effective and efficient way. To this end, we firstly convert the traditional hypernetwork into a multilabel hypernetwork (MLHN) where label correlations are explicitly represented. We then propose a co-evolutionary learning algorithm to learn an integrated classification model for all labels. The proposed Co-MLHN exploits arbitrary order label correlations and has linear computational complexity with respect to the number of labels. Empirical studies on a broad range of multilabel data sets demonstrate that Co-MLHN achieves competitive results against state-of-the-art multilabel learning algorithms, in terms of both classification performance and scalability with respect to the number of labels.

Subspace learning methods are very sensitive to the illumination, translation, and rotation variances in image recognition. Thus, they have not obtained promising performance for palmprint recognition so far. In this paper, we propose a new descriptor of palmprint named histogram of oriented lines (HOL), which is a variant of histogram of oriented gradients (HOG). HOL is not very sensitive to changes of illumination, and has the robustness against small transformations because slight translations and rotations make small histogram value changes. Based on HOL, even some simple subspace learning methods can achieve high recognition rates.

This paper provides new insight into kernel methods by using data selection. The kernel trick is used to select from the data a relevant subset forming a basis in a feature space F . Thus the selected vectors de,ne a subspace in F . Then, the data is projected onto this subspace where classical algorithms are applied. We show that kernel methods like generalized discriminant analysis (Neural Comput. 12 (2000) 2385) or kernel principal component analysis (Neural Comput. 10 (1998) 1299) can be expressed more easily. Moreover, it will turn out that the size of the basis is related to the complexity of the model. Therefore, the data selection leads to a complexity control and thus to a better generalization. The approach covers a wide range of algorithms. We investigate the function approximation on real classi,cation problems and on a regression problem. c © 2003 Elsevier B.V. All rights reserved.

In this paper we examine the application of the random forest classifier for the all relevant feature selection problem. To this end we first examine two recently proposed all relevant feature selection algorithms, both being a random forest wrappers, on a series of synthetic data sets with varying size. We show that reasonable accuracy of predictions can be achieved and that heuristic algorithms that were designed to handle the all relevant problem, have performance that is close to that of the reference ideal algorithm. Then, we apply one of the algorithms to four families of semi-synthetic data sets to assess how the properties of particular data set influence results of feature selection. Finally we test the procedure using a well-known gene expression data set. The relevance of nearly all previously established important genes was confirmed, moreover the relevance of several new ones is discovered.

The development of data-mining applications such as classification and clustering has shown the need for machine learning algorithms to be applied to large scale data. In this paper we present the comparison of different classification techniques using Waikato Environment for Knowledge Analysis or in short, WEKA. WEKA is an open source software which consists of a collection of machine learning algorithms for data mining tasks. The aim of this paper is to investigate the performance of different classification or clustering methods for a set of large data. The algorithm or methods tested are Bayes Network, Radial Basis Function, Pruned Tree, Single Conjunctive Rule Learner and Nearest Neighbors Algorithm. A fundamental review on the selected technique is presented for introduction purposes. The data breast cancer data with a total data of 6291 and a dimension of 699 rows and 9 columns will be used to test and justify the differences between the classification methods or algorithms. Subsequently, the classification technique that has the potential to significantly improve the common or conventional methods will be suggested for use in large scale data, bioinformatics or other general applications. Keywords— Machine Learning, Data Mining, WEKA, Classification, Bioinformatics.

Support vector machines (SVM) is an effective tool for building good credit scoring models. However, the performance of the model depends on its parameters’ setting. In this study, we use direct search method to optimize the SVM-based credit scoring model and compare it with other three parameters optimization methods, such as grid search, method based on design of experiment (DOE) and genetic algorithm (GA). Two real-world credit datasets are selected to demonstrate the effectiveness and feasibility of the method. The results show that the direct search method can find the effective model with high classification accuracy and good robustness and keep less dependency on the initial search space or point setting.


Question: An evolutionary cost-sensitive approach for extreme learning machines with subspace extension.

This paper addresses critical aspects of scalability and the level of supervision needed in contemporary Mahalanobis metric learning techniques, highlighting the computational challenges and the impracticality of fully supervised labels for massive datasets. The paper presents an efficient and straightforward method for learning a distance metric from equivalence constraints, adopting a statistical inference approach. Unlike previous methods that often involve intricate optimization problems and time-consuming iterative processes, the proposed strategy is significantly faster, achieving several orders of magnitude improvement in computational speed. The efficacy of this approach is substantiated through a series of experiments on diverse and demanding datasets, encompassing facial recognition in uncontrolled settings, matching previously unseen object instances, and person re-identification across non-overlapping camera views. In the latter two scenarios, the proposed method outperforms existing state-of-the-art techniques, showcasing its robustness and versatility across various benchmark tests.

Semi-supervised learning approaches are trained using the full training (labeled) data and available testing (unlabeled) data. Demonstrations of the value of training with unlabeled data typically depend on a smoothness assumption relating the conditional expectation to high density regions of the marginal distribution and an inherent missing completely at random assumption for the labeling. So-called covariate shift poses a challenge for many existing semi-supervised or supervised learning techniques. Covariate shift models allow the marginal distributions of the labeled and unlabeled feature data to differ, but the conditional distribution of the response given the feature data is the same. An example of this occurs when a complete labeled data sample and then an unlabeled sample are obtained sequentially, as it would likely follow that the distributions of the feature data are quite different between samples. The value of using unlabeled data during training for the elastic net is justified geometrically in such practical covariate shift problems. The approach works by obtaining adjusted coefficients for unlabeled prediction which recalibrate the supervised elastic net to compromise: (i) maintaining elastic net predictions on the labeled data with (ii) shrinking unlabeled predictions to zero. Our approach is shown to dominate linear supervised alternatives on unlabeled response predictions when the unlabeled feature data are concentrated on a low dimensional manifold away from the labeled data and the true coefficient vector emphasizes directions away from this manifold. Large variance of the supervised predictions on the unlabeled set is reduced more than the increase in squared bias when the unlabeled responses are expected to be small, so an improved compromise within the bias-variance tradeoff is the rationale for this performance improvement. Performance is validated on simulated and real data.

Analysis of clinical records contributes to the Traditional Chinese Medicine (TCM) experience expansion and techniques promotion. More than two diagnostic classes (diagnostic syndromes) in the clinical records raise a popular data mining problem: multi-value classification. In this paper, we propose a novel multi-class classifier, named Multiple Asymmetric Partial Least Squares Classifier (MAPLSC). MAPLSC attempts to be robust facing imbalanced data distribution in the multi-value classification. Elaborated comparisons with other seven state-of-the-art methods on two TCM clinical datasets and four public microarray datasets demonstrate MAPLSC's remarkable improvements.

Clustering is a process that aims to group the similar records in one cluster and dissimilar records in different clusters. K-means is one of the most popular and well-known clustering technique for its simplicity and light weight. However, the main drawback of K-means clustering technique is that it requires a user (data miner) to estimate the number of clusters in advance. Another limitation of K-means is that it has a tendency to get stuck at local optima. In order to overcome these limitations many evolutionary algorithm based clustering techniques have been proposed since the 1990s and applied to various fields. In this paper, we present an up-to-date review of some major evolutionary algorithm based clustering techniques for the last twenty (20) years (1995-2015). A total of 63 ranked (i.e. based on citation reports and JCR/CORE rank) evolutionary algorithm based clustering approaches are reviewed. Maximum of the techniques do not require any user to define the number of clusters in advance. We present the limitations and advantages of some evolutionary algorithm based clustering techniques. We also present a thorough discussion and future research directions of evolutionary algorithm based clustering techniques.

The distribution information of data points in two classes as the structural information is inserted into the classifiers to improve their generalization performance. Recently many algorithms such as S-TWSVM has used this information to construct two nonparallel hyperplanes which each one lies as close as possible to one class and being far away from the other. It is well known that different classes have different data distribution in real world problems, thus the covariance matrices of these classes are not the same. In these situations, the Mahalanobis is often more popular than Euclidean as a measure of distance. In this paper, in addition to apply the idea of S-TWSVM, the classical Euclidean distance is replaced by Mahalanobis distance which leads to simultaneously consider the covariance matrices of the two classes. By this modification, the orientation information in two classes can be better exploited than S-TWSVM. The experiments indicate our proposed algorithm is often superior to other learning algorithms in terms of generalization performance.

When intelligent interfaces, such as intelligent desktop assistants, email classif iers, and recommender systems, customize themselves to a particular end user , such customizations can decrease productivity and increase frustration due to inacc urate predictions—especially in early stages when training data is limited. The end user can improve the learning algorithm by tediously labeling a substantial amoun t of additional training data, but this takes time and is too ad hoc to target a particular area of inaccuracy. To solve this problem, we propose new supervised and semi-supervised learning algorithms based on locally weighted logistic regression for feature labeling by end users , enabling them to point out which features are important for a class, rath er than provide new training instances. We first evaluate our algorithms against other feature labeling algorit hms under idealized conditions using feature labels generated by an oracle . In addition, another of our contributions is an evaluation of feature labeling algorithms under real world conditions using feature labels harvested from actual end users in our user study. Our user study is the first statistical user study for feature labeling invo lvi g a large number of end users (43 participants), all of whom have no background in machine learning. Our supervised and semi-supervised algorithms were among the best perfo rmers when compared to other feature labeling algorithms in the idealized setting and they are also robust to poor quality feature labels provided by ordinary end u sers in our study. We also perform an analysis to investigate the relative gains of incorporatin g the different sources of knowledge available in the labeled training set, the feature labels and the unlabeled data. Together, our results strongly suggest that feature labeling by e d users is both viable and effective for allowing end users to improve the learnin g algorithm behind their customized applicati ons. * Corresponding author E-mail address: dassh@eecs.oregonstate.edu (Shubhomoy Das, 1148 Kelley Engineering Center, Corvallis, OR 97331-5501, USA, Ph: 1-541-908-6949). 1 Early versions of portions of this work appeared in [36, 37] S. Das, T. Moore, W-K. Wong et. al./Artificial Intelligence © 2012 Elsevier B.V. All rights reserved.

Dual decomposition methods are the current state-of-the-art for training multiclass formulations of Support Vector Machines (SVMs). At every iteration, dual decomposition methods update a small subset of dual variables by solving a restricted optimization problem. In this paper, we propose an exact and efficient method for solving the restricted problem. In our method, the restricted problem is reduced to the well-known problem of Euclidean projection onto the positive simplex, which we can solve exactly in expected O(k) time, where k is the number of classes. We demonstrate that our method empirically achieves state-of-the-art convergence on several large-scale high-dimensional datasets.

Empirical evidence suggests that hashing is an effective strategy for dimensionality reduction and practical nonparametric estimation. In this paper we provide exponential tail bounds for feature hashing and show that the interaction between random subspaces is negligible with high probability. We demonstrate the feasibility of this approach with experimental results for a new use case --- multitask learning with hundreds of thousands of tasks.

Support vector machines have met with significant success in numerous real-world learning tasks. However, like most machine learning algorithms, they are generally applied using a randomly selected training set classified in advance. In many settings, we also have the option of using pool-based active learning. Instead of using a randomly selected training set, the learner has access to a pool of unlabeled instances and can request the labels for some number of them. We introduce a new algorithm for performing active learning with support vector machines, i.e., an algorithm for choosing which instances to request next. We provide a theoretical motivation for the algorithm using the notion of a version space. We present experimental results showing that employing our active learning method can significantly reduce the need for labeled training instances in both the standard inductive and transductive settings.

Redundant and irrelevant features in data have caused a long-term problem in network traffic classification. These features not only slow down the process of classification but also prevent a classifier from making accurate decisions, especially when coping with big data. In this paper, we propose a mutual information based algorithm that analytically selects the optimal feature for classification. This mutual information based feature selection algorithm can handle linearly and nonlinearly dependent data features. Its effectiveness is evaluated in the cases of network intrusion detection. An Intrusion Detection System (IDS), named Least Square Support Vector Machine based IDS (LSSVM-IDS), is built using the features selected by our proposed feature selection algorithm. The performance of LSSVM-IDS is evaluated using three intrusion detection evaluation datasets, namely KDD Cup 99, NSL-KDD and Kyoto 2006+ dataset. The evaluation results show that our feature selection algorithm contributes more critical features for LSSVM-IDS to achieve better accuracy and lower computational cost compared with the state-of-the-art methods.

Uncertainty sampling methods iteratively request class labels for training instances whose classes are uncertain despite the previous labeled instances. These methods can greatly reduce the number of instances that an expert need label. One problem with this approach is that the classifier best suited for an application may be too expensive to train or use during the selection of instances. We test the use of one classifier (a highly efficient probabilistic one) to select examples for training another (the C4.5 rule induction program). Despite being chosen by this heterogeneous approach, the uncertainty samples yielded classifiers with lower error rates than random samples ten times larger.


Question: An evolutionary cost-sensitive extreme learning machine with subspace extension approach.

The AR Face Database is a widely utilized resource in the field of facial recognition and image processing research. This comprehensive database consists of over 4,000 color images of 126 individuals, each captured under various conditions including different facial expressions, illumination angles, and occlusions. The images are collected in two separate sessions for each subject, allowing for the study of temporal variations in facial appearance. The AR Face Database is particularly valuable for the development and testing of algorithms designed to handle challenges such as pose variations, illumination changes, and partial occlusions, which are common in real-world facial recognition applications. Researchers can leverage this dataset to enhance the robustness and accuracy of their facial recognition systems, making it a crucial tool in advancing the field of computer vision and biometrics.

**Question**: Evolutionary Cost-sensitive Extreme Learning Machine and Subspace Extension

**Documents**: An evolutionary algorithm for subspace clustering

**Similar Documents**: This paper presents an innovative evolutionary algorithm tailored for subspace clustering, a technique aimed at identifying clusters in high-dimensional data, where each cluster may exist in a different subset of dimensions. Unlike traditional clustering methods that assume data points cluster in the entire feature space, subspace clustering recognizes that data may exhibit complex structures where clusters are formed in lower-dimensional subspaces. The proposed evolutionary algorithm efficiently explores the vast subspace possibilities by utilizing genetic operators such as crossover and mutation, combined with a fitness function that evaluates the quality of the subspaces based on cluster coherence and separability. This approach not only enhances the robustness of subspace clustering but also accelerates the discovery of meaningful patterns in large datasets. The effectiveness of the evolutionary algorithm is demonstrated through extensive experiments on various datasets, showcasing its ability to accurately identify subspace clusters and outperform existing methods in terms of clustering accuracy and computational efficiency.

Data stream classification poses many challenges to the data mining community. In this paper, we address four such major challenges, namely, infinite length, concept-drift, concept-evolution, and feature-evolution. Since a data stream is theoretically infinite in length, it is impractical to store and use all the historical data for training. Concept-drift is a common phenomenon in data streams, which occurs as a result of changes in the underlying concepts. Concept-evolution occurs as a result of new classes evolving in the stream. Feature-evolution is a frequently occurring process in many streams, such as text streams, in which new features (i.e., words or phrases) appear as the stream progresses. Most existing data stream classification techniques address only the first two challenges, and ignore the latter two. In this paper, we propose an ensemble classification framework, where each classifier is equipped with a novel class detector, to address concept-drift and concept-evolution. To address feature-evolution, we propose a feature set homogenization technique. We also enhance the novel class detection module by making it more adaptive to the evolving stream, and enabling it to detect more than one novel class at a time. Comparison with state-of-the-art data stream classification techniques establishes the effectiveness of the proposed approach.

Analysis of clinical records contributes to the Traditional Chinese Medicine (TCM) experience expansion and techniques promotion. More than two diagnostic classes (diagnostic syndromes) in the clinical records raise a popular data mining problem: multi-value classification. In this paper, we propose a novel multi-class classifier, named Multiple Asymmetric Partial Least Squares Classifier (MAPLSC). MAPLSC attempts to be robust facing imbalanced data distribution in the multi-value classification. Elaborated comparisons with other seven state-of-the-art methods on two TCM clinical datasets and four public microarray datasets demonstrate MAPLSC's remarkable improvements.

In this tutorial we give an overview of the basic ideas underlying Support Vector (SV) machines for function estimation. Furthermore, we include a summary of currently used algorithms for training SV machines, covering both the quadratic (or convex) programming part and advanced methods for dealing with large datasets. Finally, we mention somemodifications and extensions that have been applied to the standard SV algorithm, and discuss the aspect of regularization from a SV perspective.

PurposeThe immense quantity of available unstructured text documents serve as one of the largest source of information. Text classification can be an essential task for many purposes in information retrieval, such as document organization, text filtering and sentiment analysis. Ensemble learning has been extensively studied to construct efficient text classification schemes with higher predictive performance and generalization ability. Providing diversity among the classification algorithms is a key issue in the ensemble design. Design/methodology/approachAn ensemble scheme based on hybrid supervised clustering is presented for text classification. In the presented scheme, supervised hybrid clustering, which is based on cuckoo search algorithm and k-means, is introduced to partition the data samples of each class into clusters so that training subsets with higher diversities can be provided. Each classifier is trained on the diversified training subsets and the predictions of individual classifiers are combined by the majority voting rule. The predictive performance of the proposed classifier ensemble is compared to conventional classification algorithms (such as Naïve Bayes, logistic regression, support vector machines and C4.5 algorithm) and ensemble learning methods (such as AdaBoost, Bagging and Random Subspace) using eleven text benchmarks. FindingThe experimental results indicate that the presented classifier ensemble outperforms the conventional classification algorithms and ensemble learning methods for text classification. Originality/valueThe presented ensemble scheme is the first to use supervised clustering to obtain diverse ensemble for text classification

Classification of large, static collections of XML data has been intensively studied in the last several years. Recently however, the data processing paradigm is shifting from static to streaming data, where documents have to be processed online using limited memory and class definitions can change with time in an event called concept drift. As most existing XML classifiers are capable of processing only static data, there is a need to develop new approaches dedicated for streaming environments. In this paper, we propose a new classification algorithm for XML data streams called XSC. The algorithm uses incrementally mined frequent subtrees and a tree-subtree similarity measure to classify new documents in an associative manner. The proposed approach is experimentally evaluated against eight state-of-the-art stream classifiers on real and synthetic data. The results show that XSC performs significantly better than competitive algorithms in terms of accuracy and memory usage.

Clustering is a process that aims to group the similar records in one cluster and dissimilar records in different clusters. K-means is one of the most popular and well-known clustering technique for its simplicity and light weight. However, the main drawback of K-means clustering technique is that it requires a user (data miner) to estimate the number of clusters in advance. Another limitation of K-means is that it has a tendency to get stuck at local optima. In order to overcome these limitations many evolutionary algorithm based clustering techniques have been proposed since the 1990s and applied to various fields. In this paper, we present an up-to-date review of some major evolutionary algorithm based clustering techniques for the last twenty (20) years (1995-2015). A total of 63 ranked (i.e. based on citation reports and JCR/CORE rank) evolutionary algorithm based clustering approaches are reviewed. Maximum of the techniques do not require any user to define the number of clusters in advance. We present the limitations and advantages of some evolutionary algorithm based clustering techniques. We also present a thorough discussion and future research directions of evolutionary algorithm based clustering techniques.

In this paper, we propose a support vector machine with automatic confidence (SVMAC) for pattern classification. The main contributions of this work to learning machines are twofold. One is that we develop an algorithm for calculating the label confidence value of each training sample. Thus, the label confidence values of all of the training samples can be considered in training support vector machines. The other one is that we propose a method for incorporating the label confidence value of each training sample into learning and derive the corresponding quadratic programming problems. To demonstrate the e ff ctiveness of the proposed SVMACs, a series of experiments are performed on three benchmarking pattern classification problems and a challenging gender classification problem. Experimental results show that the generalization performance of our SVMACs is superior to that of traditional SVMs.

In this paper, we present a deep extension of Sparse Subspace Clustering, termed Deep Sparse Subspace Clustering (DSSC). Regularized by the unit sphere distribution assumption for the learned deep features, DSSC can infer a new data affinity matrix by simultaneously satisfying the sparsity principle of SSC and the nonlinearity given by neural networks. One of the appealing advantages brought by DSSC is: when original real-world data do not meet the class-specific linear subspace distribution assumption, DSSC can employ neural networks to make the assumption valid with its hierarchical nonlinear transformations. To the best of our knowledge, this is among the first deep learning based subspace clustering methods. Extensive experiments are conducted on four real-world datasets to show the proposed DSSC is significantly superior to 12 existing methods for subspace clustering.

Despite the great developments in information technology, particularly the Internet, computer networks, global information exchange, and its positive impact in all areas of daily life, it has also contributed to the development of penetration and intrusion which forms a high risk to the security of information organizations, government agencies, and causes large economic losses. There are many techniques designed for protection such as firewall and intrusion detection systems (IDS). IDS is a set of software and/or hardware techniques used to detect hacker's activities in computer systems. Two types of anomalies are used in IDS to detect intrusive activities different from normal user behavior. Misuse relies on the knowledge base that contains all known attack techniques and intrusion is discovered through research in this knowledge base. Artificial intelligence techniques have been introduced to improve the performance of these systems. The importance of IDS is to identify unauthorized access attempting to compromise confidentiality, integrity or availability of the computer network. This paper investigates the Intrusion Detection (ID) problem using three machine learning algorithms namely, BayesNet algorithm, Multi-Layer Perceptron (MLP), and Support Vector Machine (SVM). The algorithms are applied on a real, Management Information Based (MIB) dataset that is collected from real life environment. To enhance the detection process accuracy, a set of feature selection approaches is used; Infogain (IG), ReleifF (RF), and Genetic Search (GS). Our experiments show that the three feature selection methods have enhanced the classification performance. GS with bayesNet, MLP and SVM give high accuracy rates, more specifically the BayesNet with the GS accuracy rate is 99.9%.

We implemented versions of the SVM appropriate for one-class classification in the context of information retrieval. The experiments were conducted on the standard Reuters data set. For the SVM implementation we used both a version of Schölkopf et al. and a somewhat different version of one-class SVM based on identifying “outlier” data as representative of the second-class. We report on experiments with different kernels for both of these implementations and with different representations of the data, including binary vectors, tf-idf representation and a modification called “Hadamard” representation. Then we compared it with one-class versions of the algorithms prototype (Rocchio), nearest neighbor, naive Bayes, and finally a natural one-class neural network classification method based on “bottleneck” compression generated filters. The SVM approach as represented by Schölkopf was superior to all the methods except the neural network one, where it was, although occasionally worse, essentially comparable. However, the SVM methods turned out to be quite sensitive to the choice of representation and kernel in ways which are not well understood; therefore, for the time being leaving the neural network approach as the most robust.


Question: An evolutionary cost-sensitive approach for the extension of subspace in extreme learning machines.

This paper delves into the creation of advanced systems for understanding and predicting female facial beauty, marking a significant departure from previous research. Unlike prior efforts, which often required manual annotation of facial landmarks, our approach is fully automated, using raw pixel data as input. This methodology not only simplifies the data preprocessing phase but also broadens the scope of our study. Our dataset, significantly larger than those used in previous studies, encompasses a wide variety of facial attributes including pose, lighting, background, expression, age, and ethnicity, without imposing any restrictions. This diversity and scale present a greater challenge for the learning algorithms, but also lead to more robust and generalizable models.

Our findings highlight the superiority of a biologically-inspired model with multiple layers of trainable feature extractors over the traditional eigenface approach. This model is capable of producing predictions that closely align with human perceptions of beauty. Moreover, we have developed a novel visualization technique that provides insights into the learned model. Through this technique, we have uncovered a set of features that define beauty, transcending the conventional notions of averageness and symmetry. These discoveries not only advance the field of computer vision but also contribute to our understanding of human aesthetics.

In this paper, a novel algorithm is proposed for facial expression recognition by integrating curvelet transform and online sequential extreme learning machine (OSELM) with radial basis function (RBF) hidden node having optimal network architecture. In the proposed algorithm, the curvelet transform is firstly applied to each region of the face image divided into local regions instead of whole face image to reduce the curvelet coefficients too huge to classify. Feature set is then generated by calculating the entropy, the standard deviation and the mean of curvelet coefficients of each region. Finally, spherical clustering (SC) method is employed to the feature set to automatically determine the optimal hidden node number and RBF hidden node parameters of OSELM by aim of increasing classification accuracy and reducing the required time to select the hidden node number. So, the learning machine is called as OSELM-SC. It is constructed two groups of experiments: The aim of the first one is to evaluate the classification performance of OSELM-SC on the benchmark datasets, i.e., image segment, satellite image and DNA. The second one is to test the performance of the proposed facial expression recognition algorithm on the Japanese Female Facial Expression database and the Cohn-Kanade database. The obtained experimental results are compared against the state-of-the-art methods. The results demonstrate that the proposed algorithm can produce effective facial expression features and exhibit good recognition accuracy and robustness.

Many learning algorithms generate complex models that are difficult for a human to interpret, debug, and extend. In this paper, we address this challenge by proposing a new learning paradigm called correctable learning, where the learning algorithm receives external feedback about which data examples are incorrectly learned. We define a set of metrics which measure the correctability of a learning algorithm. We then propose a simple and efficient correctable learning algorithm which learns local models for different regions of the data space. Given an incorrect example, our method samples data in the neighborhood of that example and learns a new, more correct local model over that region. Experiments over multiple classification and ranking datasets show that our correctable learning algorithm offers significant improvements over the state-of-the-art techniques.

An approach to the construction of classifiers from imbalanced datasets is described. A dataset is imbalanced if the classification categories are not approximately equally represented. Often real-world data sets are predominately composed of “normal” examples with only a small percentage of “abnormal” or “interesting” examples. It is also the case that the cost of misclassifying an abnormal (interesting) example as a normal example is often much higher than the cost of the reverse error. Under-sampling of the majority (normal) class has been proposed as a good means of increasing the sensitivity of a classifier to the minority class. This paper shows that a combination of our method of over-sampling the minority (abnormal) class and under-sampling the majority (normal) class can achieve better classifier performance (in ROC space) than only under-sampling the majority class. This paper also shows that a combination of our method of over-sampling the minority class and under-sampling the majority class can achieve better classifier performance (in ROC space) than varying the loss ratios in Ripper or class priors in Naive Bayes. Our method of over-sampling the minority class involves creating synthetic minority class examples. Experiments are performed using C4.5, Ripper and a Naive Bayes classifier. The method is evaluated using the area under the Receiver Operating Characteristic curve (AUC) and the ROC convex hull strategy.

We propose a hypergraph-based active learning scheme which we term HS; HS generalizes the previously reported algorithm S originally proposed for graph-based active learning with pointwise queries [1]. Our HS method can accommodate hypergraph structures and allows one to ask both pointwise queries and pairwise queries. Based on a novel parametric system particularly designed for hypergraphs, we derive theoretical results on the query complexity of HS for the above described generalized settings. Both the theoretical and empirical results show that HS requires a significantly fewer number of queries than S when one uses S over a graph obtained from the corresponding hypergraph via clique expansion.

We present a simple, agnostic active learning algorithm that works for any hypothesis class of bounded VC dimension, and any data distribution. Our algorithm extends a scheme of Cohn, Atlas, and Ladner [6] to the agnostic setting, by (1) reformulating it using a reduction to supervised learning and (2) showing how to apply generalization bounds even for the non-i.i.d. samples that result from selective sampling. We provide a general characterization of the label complexity of our algorithm. This quantity is never more than the usual PAC sample complexity of supervised learning, and is exponentially smaller for some hypothesis classes and distributions. We also demonstrate improvements experimentally.

Large-scale learning problems require a plethora of labels that can be efficiently collected from crowdsourcing services at low cost. However, labels annotated by crowdsourced workers are often noisy, which inevitably degrades the performance of large-scale optimizations including the prevalent stochastic gradient descent (SGD). Specifically, these noisy labels adversely affect updates of the primal variable in conventional SGD. To solve this challenge, we propose a robust SGD mechanism called progressive stochastic learning (POSTAL), which naturally integrates the learning regime of curriculum learning (CL) with the update process of vanilla SGD. Our inspiration comes from the progressive learning process of CL, namely learning from “easy” tasks to “complex” tasks. Through the robust learning process of CL, POSTAL aims to yield robust updates of the primal variable on an ordered label sequence, namely, from “reliable” labels to “noisy” labels. To realize POSTAL mechanism, we design a cluster of “screening losses,” which sorts all labels from the reliable region to the noisy region. To sum up, POSTAL using screening losses ensures robust updates of the primal variable on reliable labels first, then on noisy labels incrementally until convergence. In theory, we derive the convergence rate of POSTAL realized by screening losses. Meanwhile, we provide the robustness analysis of representative screening losses. Experimental results on UCI1 simulated and Amazon Mechanical Turk crowdsourcing data sets show that the POSTAL using screening losses is more effective and robust than several existing baselines.1UCI is the abbreviation of University of California Irvine.

Multilabel classification is prevalent in many real-world applications where data instances may be associated with multiple labels simultaneously. In multilabel classification, exploiting label correlations is an essential but nontrivial task. Most of the existing multilabel learning algorithms are either ineffective or computationally demanding and less scalable in exploiting label correlations. In this paper, we propose a co-evolutionary multilabel hypernetwork (Co-MLHN) as an attempt to exploit label correlations in an effective and efficient way. To this end, we firstly convert the traditional hypernetwork into a multilabel hypernetwork (MLHN) where label correlations are explicitly represented. We then propose a co-evolutionary learning algorithm to learn an integrated classification model for all labels. The proposed Co-MLHN exploits arbitrary order label correlations and has linear computational complexity with respect to the number of labels. Empirical studies on a broad range of multilabel data sets demonstrate that Co-MLHN achieves competitive results against state-of-the-art multilabel learning algorithms, in terms of both classification performance and scalability with respect to the number of labels.

Feature subset selection and identification of appropriate classification method plays an important role to optimize the predictive performance of supervised machine learning system. Current literature makes isolated attempts to optimize the feature selection and classifier identification. However, feature set has an intrinsic relationship with classification technique and together they form a ‘model’ for classification task. In this paper, we propose AutoModeling that finds optimal learning model and jointly optimize the feature and hypothesis space to maximize performance measure objective function. It is an automated framework of selecting the ensemble model {selected feature subset, selected classifier} from a given superset of features and classifiers learned from given training dataset in a computational efficient manner. We introduce novel relax-greedy search with our proposed patience function as a wrapper feature selection that maximizes the predictive performance and eliminates the classical nesting effect. We perform extensive experimentations on different types of publicly available datasets and AutoModeling demonstrates superior performance over relevant state-of-the-art methods, expert-driven manual methods and deep neural networks.

In many applications, data collected are highly skewed where data of one class clearly dominates data from the other classes. Most existing classification systems that perform well on balanced data give very poor performance on imbalanced data, especially for the minority class data. Existing work on improving the quality of classification on imbalanced data include over-sampling, under-sampling, and methods that make modifications to the existing classification systems. This paper discusses the BEV system for classifying imbalanced data. The system is developed based on the ideas from the "Bagging" classification ensemble. The motivation behind the scheme is to maximally use the minority class data without creating synthetic data or making changes to the existing classification systems. Experimental results using real world imbalanced data show the effectiveness of the system.

Dual decomposition methods are the current state-of-the-art for training multiclass formulations of Support Vector Machines (SVMs). At every iteration, dual decomposition methods update a small subset of dual variables by solving a restricted optimization problem. In this paper, we propose an exact and efficient method for solving the restricted problem. In our method, the restricted problem is reduced to the well-known problem of Euclidean projection onto the positive simplex, which we can solve exactly in expected O(k) time, where k is the number of classes. We demonstrate that our method empirically achieves state-of-the-art convergence on several large-scale high-dimensional datasets.


Question: An evolutionary cost-sensitive extreme learning machine with subspace extension approach.

This paper challenges conventional neural network theories by proposing an innovative incremental constructive approach for single-hidden-layer feedforward networks (SLFNs) with random hidden nodes. Traditionally, SLFNs using additive or radial basis function (RBF) hidden nodes are recognized as universal approximators when all network parameters are adjustable. However, adjusting all parameters often leads to complex and inefficient learning processes, especially when dealing with non-differentiable activation functions like those in threshold networks. In contrast, this paper advocates a simpler method: randomly selecting hidden nodes and focusing solely on adjusting the output weights connecting the hidden and output layers. This approach is effective for a broad range of activation functions for additive nodes—any bounded, non-constant, piecewise continuous function g: R --> R—and for RBF nodes, any integrable, piecewise continuous function g: R --> R with the condition that the integral of R g(x)dx is not equal to zero. The proposed incremental method is not only efficient for SLFNs with continuous (including non-differentiable) activation functions but also extends to SLFNs with piecewise continuous activation functions like threshold functions. Unlike other methodologies, this new approach is fully automated, requiring no manual intervention or adjustment of control parameters by the user, making it a user-friendly solution for complex network learning tasks.

Boosting is known to be sensitive to label noise. We studied two approaches to improve AdaBoost’s robustness against labelling errors. One is to employ a label-noise robust classifier as a base learner, while the other is to modify the AdaBoost algorithm to be more robust. Empirical evaluation shows that a committee of robust classifiers, although converges faster than non label-noise aware AdaBoost, is still susceptible to label noise. However, pairing it with the new robust Boosting algorithm we propose here results in a more resilient algorithm under mislabelling.

To deal with credit card fraud, a detection model based on Class Weighted Support Vector Machine was established. Due to large-scale and high dimensions of data, Principal Component Analysis (PCA) was adopted firstly to screen out the main factors from a great deal of indicative attributes in order to reduce the training dimension of SVM effectively. Then according to the characteristics of credit card transactions data which are imbalance, an improved SVM--Imbalance Class Weighted SVM (ICW-SVM) was adopted. With the application and verification in real dataset from bank, it is demonstrated that this model is more suitable for solving credit card fraud detection problem with higher precision and effective than others.

Feature selection is an important component of many machine learning applications. Especially in many bioinformatics tasks, efficient and robust feature selection methods are desired to extract meaningful features and eliminate noisy ones. In this paper, we propose a new robust feature selection method with emphasizing joint `2,1-norm minimization on both loss function and regularization. The `2,1-norm based loss function is robust to outliers in data points and the `2,1norm regularization selects features across all data points with joint sparsity. An efficient algorithm is introduced with proved convergence. Our regression based objective makes the feature selection process more efficient. Our method has been applied into both genomic and proteomic biomarkers discovery. Extensive empirical studies are performed on six data sets to demonstrate the performance of our feature selection method.

Many learning tasks such as spam filtering and credit card fraud detection face an active adversary that tries to avoid detection. For learning problems that deal with an active adversary, it is important to model the adversary's attack strategy and develop robust learning models to mitigate the attack. These are the two objectives of this paper. We consider two attack models: a free-range attack model that permits arbitrary data corruption and a restrained attack model that anticipates more realistic attacks that a reasonable adversary would devise under penalties. We then develop optimal SVM learning strategies against the two attack models. The learning algorithms minimize the hinge loss while assuming the adversary is modifying data to maximize the loss. Experiments are performed on both artificial and real data sets. We demonstrate that optimal solutions may be overly pessimistic when the actual attacks are much weaker than expected. More important, we demonstrate that it is possible to develop a much more resilient SVM learning model while making loose assumptions on the data corruption models. When derived under the restrained attack model, our optimal SVM learning strategy provides more robust overall performance under a wide range of attack parameters.

This paper considers feature selection for data classification in the presence of a huge number of irrelevant features. We propose a new feature-selection algorithm that addresses several major issues with prior work, including problems with algorithm implementation, computational complexity, and solution accuracy. The key idea is to decompose an arbitrarily complex nonlinear problem into a set of locally linear ones through local learning, and then learn feature relevance globally within the large margin framework. The proposed algorithm is based on well-established machine learning and numerical analysis techniques, without making any assumptions about the underlying data distribution. It is capable of processing many thousands of features within minutes on a personal computer while maintaining a very high accuracy that is nearly insensitive to a growing number of irrelevant features. Theoretical analyses of the algorithm's sample complexity suggest that the algorithm has a logarithmical sample complexity with respect to the number of features. Experiments on 11 synthetic and real-world data sets demonstrate the viability of our formulation of the feature-selection problem for supervised learning and the effectiveness of our algorithm.

The objective of machine learning is to identify a model that yields good generalization performance. This involves repeatedly selecting a hypothesis class, searching the hypothesis class by minimizing a given objective function over the model’s parameter space, and evaluating the generalization performance of the resulting model. This search can be computationally intensive as training data continuously arrives, or as one needs to tune hyperparameters in the hypothesis class and the objective function. In this paper, we present a framework for exact incremental learning and adaptation of support vector machine (SVM) classifiers. The approach is general and allows one to learn and unlearn individual or multiple examples, adapt the current SVM to changes in regularization and kernel parameters, and evaluate generalization performance through exact leave-one-out error estimation. I. I NTRODUCTION SVM techniques for classification and regression provide powerful tools for learning models that generalize well even in sparse, high dimensional settings. Their success can be attributed to Vapnik’s seminal work in statistical learning theory [15] which provided key insights into the factors affecting generalization performance. SVM learning can be viewed as a practical implementation of Vapnik’s structural risk minimizationinduction principle which involves searching over hypothesis classes of varying capacity to find the model with the best generalization performance. SVM classifiers of the formf(x) = w ·Φ(x)+b are learned from the data{(xi, yi) ∈ R I m × {−1, 1} ∀ i ∈ {1, . . . , N}} by minimizing min w,b,ξ 1 2 ‖w‖ + C N ∑

An intrusion Detection System (IDS) provides alarts against intrusion attacks where a traditional firewall fails. Machine learning algorithms aim to detect anomalies using supervised and unsupervised approaches. Features selection techniques identify important features and remove irrelevant and redundant attributes to reduce the dimensionality of feature space. This paper presents a features selection framework for efficient network anomaly detection using different machine learning classifiers. The framework applies different strategies by using filter and wrapper features selection methodologies. The aim of this framework is to select the minimum number of features that achieve the highest accuracy. UNSW-NB15 dataset is used in the experimental results to evaluate the proposed framework. The results show that by using 18 features from one of the filter ranking methods and applying J48 as a classifier, an accuracy of 88% is achieved.

We present an approach for learning an anisotropic RBF kernel in a game theoretical setting where the value of the game is the degree of separation between positive and negative training examples. The method extends a previously proposed method (KOMD) to perform feature re-weighting and distance metric learning in a kernel-based classification setting. Experiments on several benchmark datasets demonstrate that our method generally outperforms stateof-the-art distance metric learning methods, including the Large Margin Nearest Neighbor Classification family of methods.

MOTIVATION
Subcellular localization is a key functional characteristic of proteins. A fully automatic and reliable prediction system for protein subcellular localization is needed, especially for the analysis of large-scale genome sequences.

RESULTS
In this paper, Support Vector Machine has been introduced to predict the subcellular localization of proteins from their amino acid compositions. The total prediction accuracies reach 91.4% for three subcellular locations in prokaryotic organisms and 79.4% for four locations in eukaryotic organisms. Predictions by our approach are robust to errors in the protein N-terminal sequences. This new approach provides superior prediction performance compared with existing algorithms based on amino acid composition and can be a complementary method to other existing methods based on sorting signals.

AVAILABILITY
A web server implementing the prediction method is available at http://www.bioinfo.tsinghua.edu.cn/SubLoc/.

SUPPLEMENTARY INFORMATION
Supplementary material is available at http://www.bioinfo.tsinghua.edu.cn/SubLoc/.

Multilabel classification is prevalent in many real-world applications where data instances may be associated with multiple labels simultaneously. In multilabel classification, exploiting label correlations is an essential but nontrivial task. Most of the existing multilabel learning algorithms are either ineffective or computationally demanding and less scalable in exploiting label correlations. In this paper, we propose a co-evolutionary multilabel hypernetwork (Co-MLHN) as an attempt to exploit label correlations in an effective and efficient way. To this end, we firstly convert the traditional hypernetwork into a multilabel hypernetwork (MLHN) where label correlations are explicitly represented. We then propose a co-evolutionary learning algorithm to learn an integrated classification model for all labels. The proposed Co-MLHN exploits arbitrary order label correlations and has linear computational complexity with respect to the number of labels. Empirical studies on a broad range of multilabel data sets demonstrate that Co-MLHN achieves competitive results against state-of-the-art multilabel learning algorithms, in terms of both classification performance and scalability with respect to the number of labels.


Question: Multi-level preference regression for addressing cold-start recommendation challenges.

This paper presents an advanced approach for collaborative filtering by incorporating graph information for improved predictions and consistency in large-scale applications. The core concept revolves around low rank matrix completion, which is crucial for collaborative filtering, as it assumes that variables exist in a subspace smaller than the ambient space. When additional information about variable relationships is available, such as through a graph, this data can significantly enhance prediction accuracy. To address the challenge of matrix completion with graph information, we propose a highly efficient optimization method based on a conjugate gradient algorithm and alternating minimization. This technique outperforms state-of-the-art gradient-descent methods by up to two orders of magnitude in speed, as demonstrated through optimizations involving over 55 million observations. Theoretically, our method extends the concept of weighted nuclear norm formulations and includes statistical consistency guarantees. We substantiate our findings through extensive testing on both synthetic and real-world datasets, showcasing the effectiveness of our approach in improving collaborative filtering outcomes.

When a user has watched, say, 70 romance movies and 30 action movies, then it is reasonable to expect the personalized list of recommended movies to be comprised of about 70% romance and 30% action movies as well. This important property is known as calibration, and recently received renewed attention in the context of fairness in machine learning. In the recommended list of items, calibration ensures that the various (past) areas of interest of a user are reflected with their corresponding proportions. Calibration is especially important in light of the fact that recommender systems optimized toward accuracy (e.g., ranking metrics) in the usual offline-setting can easily lead to recommendations where the lesser interests of a user get crowded out by the user's main interests-which we show empirically as well as in thought-experiments. This can be prevented by calibrated recommendations. To this end, we outline metrics for quantifying the degree of calibration, as well as a simple yet effective re-ranking algorithm for post-processing the output of recommender systems.

In fashion e-commerce platforms, product discovery is one of the key components of a good user experience. There are numerous ways using which people find the products they desire. Similar product recommendations is one of the popular modes using which users find products that resonate with their intent. Generally these recommendations are not personalized to a specific user. Traditionally, collaborative filtering based approaches have been popular in the literature for recommending non-personalized products given a query product. Also, there has been focus on personalizing the product listing for a given user. In this paper, we marry these approaches so that users will be recommended with personalized similar products. Our experimental results on a large fashion ecommerce platform (Myntra) show that we can improve the key metrics by applying personalization on similar product recommendations.

Recommender systems traditionally assume that user profiles and movie attributes are static. Temporal dynamics are purely reactive, that is, they are inferred after they are observed, e.g. after a user's taste has changed or based on hand-engineered temporal bias corrections for movies. We propose Recurrent Recommender Networks (RRN) that are able to predict future behavioral trajectories. This is achieved by endowing both users and movies with a Long Short-Term Memory (LSTM) autoregressive model that captures dynamics, in addition to a more traditional low-rank factorization. On multiple real-world datasets, our model offers excellent prediction accuracy and it is very compact, since we need not learn latent state but rather just the state transition function.

There are many applications in which it is desirable to order rather than classify instances. Here we consider the problem of learning how to order, given feedback in the form of preference judgments, i.e., statements to the effect that one instance should be ranked ahead of another. We outline a two-stage approach in which one first learns by conventional means a preference function, of the form PREF , which indicates whether it is advisable to rank before . New instances are then ordered so as to maximize agreements with the learned preference function. We show that the problem of finding the ordering that agrees best with a preference function is NP-complete, even under very restrictive assumptions. Nevertheless, we describe a simple greedy algorithm that is guaranteed to find a good approximation. We then discuss an on-line learning algorithm, based on the “Hedge” algorithm, for finding a good linear combination of ranking “experts.” We use the ordering algorithm combined with the on-line learning algorithm to find a combination of “search experts,” each of which is a domain-specific query expansion strategy for a WWW search engine, and present experimental results that demonstrate the merits of our approach.

The rapid development of Internet technologies in recent decades has imposed a heavy information burden on users. This has led to the popularity of recommender systems, which provide advice to users about items they may like to examine. Collaborative Filtering (CF) is the most promising technique in recommender systems, providing personalized recommendations to users based on their previously expressed preferences and those of other similar users. This paper introduces a CF framework based on Fuzzy Association Rules and Multiple-level Similarity (FARAMS). FARAMS extended existing techniques by using fuzzy association rule mining, and takes advantage of product similarities in taxonomies to address data sparseness and nontransitive associations. Experimental results show that FARAMS improves prediction quality, as compared to similar approaches.

Similarity-based algorithms, often referred to as memory-based collaborative filtering techniques, are one of the most successful methods in recommendation systems. When explicit ratings are available, similarity is usually defined using similarity functions, such as the Pearson correlation coefficient, cosine similarity or mean square difference. These metrics assume similarity is a symmetric criterion. Therefore, two users have equal impact on each other in recommending new items. In this paper, we introduce new weighting schemes that allow us to consider new features in finding similarities between users. These weighting schemes, first, transform symmetric similarity to asymmetric similarity by considering the number of ratings given by users on non-common items. Second, they take into account the habit effects of users are regarded on rating items by measuring the proximity of the number of repetitions for each rate on common rated items. Experiments on two datasets were implemented and compared to other similarity measures. The results show that adding weighted schemes to traditional similarity measures These authors contributed equally to this work as the first author. P. Pirasteh · D. Hwang Department of Computer Engineering, Yeungnam University, Gyeongsan, Republic of Korea P. Pirasteh e-mail: parivash63@gmail.com D. Hwang e-mail: dosamhwang@gmail.com J. E. Jung ( ) Department of Computer Engineering, Chung-Ang University, Seoul, Republic of Korea e-mail: ontology.society@gmail.com significantly improve the results obtained from traditional similarity measures.

Yelp aims to help people find great local businesses, e.g. restaurants. Automated software is currently used to recommend the most helpful and reliable reviews for the Yelp community, based on various measures of quality, reliability, and activity. However, this is not tailored to each customer. Our goal in this project is to apply machine learning to predict a customer’s star rating of a restaurant based on his/her reviews, as well as other customers’ reviews/ratings, to recommend other restaurants to the customer, as shown in Figure 1.

A recommendation is called explainable if it not only predicts a numerical rating for an item, but also generates explanations for users' preferences. Most existing methods for explainable recommendation apply topic models to analyze user reviews to provide descriptions along with the recommendations they produce. So far, such methods have neglected user opinions and influences from social relations as a source of information for recommendations, even though these are known to improve the rating prediction.
 In this paper, we propose a latent variable model, called social collaborative viewpoint regression (sCVR), for predicting item ratings based on user opinions and social relations. To this end, we use so-called viewpoints, represented as tuples of a concept, topic, and a sentiment label from both user reviews and trusted social relations. In addition, such viewpoints can be used as explanations. We apply a Gibbs EM sampler to infer posterior distributions of sCVR. Experiments conducted on three large benchmark datasets show the effectiveness of our proposed method for predicting item ratings and for generating explanations.

Matrix factorization is one of the most powerful techniques in collaborative filtering, which models the (user, item) interactions behind historical explicit or implicit feedbacks. However, plain matrix factorization may not be able to uncover the structure correlations among users and items well such as communities and taxonomies. As a response, we design a novel algorithm, i.e., hierarchical group matrix factorization (HGMF), in order to explore and model the structure correlations among users and items in a principled way. Specifically, we first define four types of correlations, including (user, item), (user, item group), (user group, item) and (user group, item group); we then extend plain matrix factorization with a hierarchical group structure; finally, we design a novel clustering algorithm to mine the hidden structure correlations. In the experiments, we study the effectiveness of our HGMF for both rating prediction and item recommendation, and find that it is better than some state-of-the-art methods on several real-world data sets.

Understanding and estimating satisfaction with search engines is an important aspect of evaluating retrieval performance. Research to date has modeled and predicted search satisfaction on a binary scale, i.e., the searchers are either satisfied or dissatisfied with their search outcome. However, users' search experience is a complex construct and there are different degrees of satisfaction. As such, binary classification of satisfaction may be limiting. To the best of our knowledge, we are the first to study the problem of understanding and predicting graded (multi-level) search satisfaction. We ex-amine sessions mined from search engine logs, where searcher satisfaction was also assessed on multi-point scale by human annotators. Leveraging these search log data, we observe rich and non-monotonous changes in search behavior in sessions with different degrees of satisfaction. The findings suggest that we should predict finer-grained satisfaction levels. To address this issue, we model search satisfaction using features indicating search outcome, search effort, and changes in both outcome and effort during a session. We show that our approach can predict subtle changes in search satisfaction more accurately than state-of-the-art methods, affording greater insight into search satisfaction. The strong performance of our models has implications for search providers seeking to accu-rately measure satisfaction with their services.


Question: Multi-level preference regression for addressing the cold-start problem in recommendations.

This paper introduces a novel framework, Collaborative Deep Learning (CDL), designed to enhance the performance of recommender systems, particularly in scenarios where user-item interaction data is sparse. Unlike traditional collaborative filtering (CF) methods, which rely solely on user ratings for recommendation, CDL leverages both user ratings and auxiliary information, such as item content, to overcome data sparsity. The approach builds upon recent advancements in deep learning, extending its application from independent and identically distributed (i.i.d.) inputs to non-i.i.d. inputs that are characteristic of CF-based systems. CDL is formulated as a hierarchical Bayesian model that simultaneously conducts deep representation learning for item content and collaborative filtering for the user-item ratings matrix. By integrating these two components, CDL aims to learn more effective latent representations, even in cases where auxiliary information is sparse. Through comprehensive experimentation on three real-world datasets from diverse domains, the proposed framework demonstrates its capability to significantly outperform existing state-of-the-art methods in recommendation accuracy.

Understanding and estimating satisfaction with search engines is an important aspect of evaluating retrieval performance. Research to date has modeled and predicted search satisfaction on a binary scale, i.e., the searchers are either satisfied or dissatisfied with their search outcome. However, users' search experience is a complex construct and there are different degrees of satisfaction. As such, binary classification of satisfaction may be limiting. To the best of our knowledge, we are the first to study the problem of understanding and predicting graded (multi-level) search satisfaction. We ex-amine sessions mined from search engine logs, where searcher satisfaction was also assessed on multi-point scale by human annotators. Leveraging these search log data, we observe rich and non-monotonous changes in search behavior in sessions with different degrees of satisfaction. The findings suggest that we should predict finer-grained satisfaction levels. To address this issue, we model search satisfaction using features indicating search outcome, search effort, and changes in both outcome and effort during a session. We show that our approach can predict subtle changes in search satisfaction more accurately than state-of-the-art methods, affording greater insight into search satisfaction. The strong performance of our models has implications for search providers seeking to accu-rately measure satisfaction with their services.

Integrating interactive faceted filtering with intelligent recommendation techniques has shown to be a promising means for increasing user control in Recommender Systems. In this paper, we extend the concept of blended recommending by automatically extracting meaningful facets from social media by means of Natural Language Processing. Concretely, we allow users to influence the recommendations by selecting facet values and weighting them based on information other users provided in their reviews. We conducted a user study with an interactive recommender implemented in the hotel domain. This evaluation shows that users are consequently able to find items fitting interests that are typically difficult to take into account when only structured content data is available. For instance, the extracted facets representing the opinions of hotel visitors make it possible to effectively search for hotels with comfortable beds or that are located in quiet surroundings without having to read the user reviews.

Recommending appropriate product items to the target user is becoming the key to ensure continuous success of E-commerce. Today, many E-commerce systems adopt various recommendation techniques, e.g., Collaborative Filtering (abbreviated as CF)-based technique, to realize product item recommendation. Overall, the present CF recommendation can perform very well, if the target user owns similar friends (user-based CF), or the product items purchased and preferred by target user own one or more similar product items (item-based CF). While due to the sparsity of big rating data in E-commerce, similar friends and similar product items may be both absent from the user-product purchase network, which lead to a big challenge to recommend appropriate product items to the target user. Considering the challenge, we put forward a Structural Balance Theory-based Recommendation (i.e., SBT-Rec) approach. In the concrete, (I) user-based recommendation: we look for target user's “enemy” (i.e., the users having opposite preference with target user); afterwards, we determine target user's “possible friends”, according to “enemy's enemy is a friend” rule of Structural Balance Theory, and recommend the product items preferred by “possible friends” of target user to the target user. (II) likewise, for the product items purchased and preferred by target user, we determine their “possibly similar product items” based on Structural Balance Theory and recommend them to the target user. At last, the feasibility of SBT-Rec is validated, through a set of experiments deployed on MovieLens-1M dataset.

In this work we present topic diversification, a novel method designed to balance and diversify personalized recommendation lists in order to reflect the user's complete spectrum of interests. Though being detrimental to average accuracy, we show that our method improves user satisfaction with recommendation lists, in particular for lists generated using the common item-based collaborative filtering algorithm.Our work builds upon prior research on recommender systems, looking at properties of recommendation lists as entities in their own right rather than specifically focusing on the accuracy of individual recommendations. We introduce the intra-list similarity metric to assess the topical diversity of recommendation lists and the topic diversification approach for decreasing the intra-list similarity. We evaluate our method using book recommendation data, including offline analysis on 361, !, 349 ratings and an online study involving more than 2, !, 100 subjects.

Collaborative tagging systems allow users to assign keywords—so called “tags”—to resources. Tags are used for navigation, finding resources and serendipitous browsing and thus provide an immediate benefit for users. These systems usually include tag recommendation mechanisms easing the process of finding good tags for a resource, but also consolidating the tag vocabulary across users. In practice, however, only very basic recommendation strategies are applied. In this paper we evaluate and compare several recommendation algorithms on large-scale real life datasets: an adaptation of user-based collaborative filtering, a graph-based recommender built on top of the FolkRank algorithm, and simple methods based on counting tag occurences. We show that both FolkRank and Collaborative Filtering provide better results than non-personalized baseline methods. Moreover, since methods based on counting tag occurrences are computationally cheap, and thus usually preferable for real time scenarios, we discuss simple approaches for improving the performance of such methods. We show, how a simple recommender based on counting tags from users and resources can perform almost as good as the best recommender.

With the development and popularity of social networks, an increasing number of consumers prefer to order tourism products online, and like to share their experiences on social networks. Searching for tourism destinations online is a difficult task on account of its more restrictive factors. Recommender system can help these users to dispose information overload. However, such a system is affected by the issue of low recommendation accuracy and the cold-start problem. In this paper, we propose a tourism destination recommender system that employs opinion-mining technology to refine user sentiment, and make use of temporal dynamics to represent user preference and destination popularity drifting over time. These elements are then fused with the SVD+ + method by combining user sentiment and temporal influence. Compared with several well-known recommendation approaches, our method achieves improved recommendation accuracy and quality. A series of experimental evaluations, using a publicly available dataset, demonstrates that the proposed recommender system outperforms the existing recommender systems.

We present a new ranking algorithm that combines the strengt hs of two previous methods: boosted tree classification, and LambdaR ank, which has been shown to be empirically optimal for a widely used inform ation retrieval measure. The algorithm is based on boosted regression trees , although the ideas apply to any weak learners, and it is significantly fast er in both train and test phases than the state of the art, for comparable accu racy. We also show how to find the optimal linear combination for any two ran kers, and we use this method to solve the line search problem exactly du ring boosting. In addition, we show that starting with a previously tra ined model, and boosting using its residuals, furnishes an effective techn ique for model adaptation, and we give results for a particularly pressing prob lem in Web Search training rankers for markets for which only small amounts o f labeled data are available, given a ranker trained on much more data from a larger market.

Recommender systems represent user preferences for the purpose of suggesting items to purchase or examine. They have become fundamental applications in electronic commerce and information access, providing suggestions that effectively prune large information spaces so that users are directed toward those items that best meet their needs and preferences. A variety of techniques have been proposed for performing recommendation, including content-based, collaborative, knowledge-based and other techniques. To improve performance, these methods have sometimes been combined in hybrid recommenders. This paper surveys the landscape of actual and possible hybrid recommenders, and introduces a novel hybrid, EntreeC, a system that combines knowledge-based recommendation and collaborative filtering to recommend restaurants. Further, we show that semantic ratings obtained from the knowledge-based part of the system enhance the effectiveness of collaborative filtering.

To help generate relevant suggestions for researchers, recommendation systems have started to leverage the latent interests in the publication profiles of the researchers themselves. While using such a publication citation network has been shown to enhance performance, the network is often sparse, making recommendation difficult. To alleviate this sparsity, we identify "potential citation papers" through the use of collaborative filtering. Also, as different logical sections of a paper have different significance, as a secondary contribution, we investigate which sections of papers can be leveraged to represent papers effectively.
 On a scholarly paper recommendation dataset, we show that recommendation accuracy significantly outperforms state-of-the-art recommendation baselines as measured by nDCG and MRR, when we discover potential citation papers using imputed similarities via collaborative filtering and represent candidate papers using both the full text and assigning more weight to the conclusion sections.

Recommender systems have traditionally recommended items to individual users, but there has recently been a proliferation of recommenders that address their recommendations to groups of users. The shift of focus from an individual to a group makes more of a difference than one might at first expect. This chapter discusses the most important new issues that arise, organizing them in terms of four subtasks that can or must be dealt with by a group recommender: 1. acquiring information about the user’s preferences; 2. generating recommendations; 3. explaining recommendations; and 4. helping users to settle on a final decision. For each issue, we discuss how it has been dealt with in existing group recommender systems and what open questions call for further research.