Question: Utilizing an ant colony optimization algorithm for the induction of decision trees.

This paper delves into the application of an Ant Colony Optimization (ACO) algorithm for the induction of decision trees, marking a departure from conventional techniques. The focus is on leveraging ACO, a nature-inspired heuristic optimization approach, to construct efficient decision trees for both classification and regression tasks. Inspired by the foraging behavior of ants, ACO algorithms have been successfully applied to various combinatorial optimization problems. In the context of decision tree induction, the ACO algorithm explores the space of possible decision trees, identifying the most promising paths through a pheromone-based system. This method facilitates the construction of trees that are not only accurate but also possess a desirable level of complexity, crucial for effective classification and regression analysis. By comparing the results with those obtained from standard decision tree induction methods, such as the widely-used C4.5 algorithm, the study highlights the potential of ACO in enhancing decision tree learning, particularly in scenarios where the data is complex and the need for robust, interpretable models is paramount.

There has long been a chasm between theoretical mod els of machine learning and practical machine learn ing algorithms For instance empirically successful algorithms such as C and backpropagation have not met the criteria of the PAC model and its vari ants Conversely the algorithms suggested by com putational learning theory are usually too limited in various ways to nd wide application The theoreti cal status of decision tree learning algorithms is a case in point while it has been proven that C and all reasonable variants of it fails to meet the PAC model criteria other recently proposed decision tree algo rithms that do have non trivial performance guaran tees unfortunately require membership queries Two recent developments have narrowed this gap between theory and practice not for the PAC model but for the related model known as weak learning or boosting First an algorithm called Adaboost was proposed that meets the formal criteria of the boosting model and is also competitive in practice Second the basic algorithms underlying the popular C and CART programs have also very recently been shown to meet the formal criteria of the boosting model Thus it seems plausible that the weak learning frame work may provide a setting for interaction between formal analysis and machine learning practice that is lacking in other theoretical models Our aim in this paper is to push this interaction further in light of these recent developments In par ticular we perform experiments suggested by the for mal results for Adaboost and C within the weak learning framework We concentrate on two particu larly intriguing issues First the theoretical boosting results for top down decision tree algorithms such as C suggest that a new splitting criterion may result in trees that are smaller and more accurate than those obtained using the usual information gain We con rm this suggestion experimentally Second a super cial interpretation of the theo retical results suggests that Adaboost should vastly outperform C This is not the case in practice and we argue through experimental results that the theory must be understood in terms of a measure of a boosting algorithm s behavior called its advantage sequence We compare the advantage sequences for C and Adaboost in a number of experiments We nd that these sequences have qualitatively dif ferent behavior that explains in large part the dis crepancies between empirical performance and the theoretical results Brie y we nd that although C and Adaboost are both boosting algorithms Adaboost creates successively harder ltered dis tributions while C creates successively easier ones in a sense that will be made precise

Clustering is a process that aims to group the similar records in one cluster and dissimilar records in different clusters. K-means is one of the most popular and well-known clustering technique for its simplicity and light weight. However, the main drawback of K-means clustering technique is that it requires a user (data miner) to estimate the number of clusters in advance. Another limitation of K-means is that it has a tendency to get stuck at local optima. In order to overcome these limitations many evolutionary algorithm based clustering techniques have been proposed since the 1990s and applied to various fields. In this paper, we present an up-to-date review of some major evolutionary algorithm based clustering techniques for the last twenty (20) years (1995-2015). A total of 63 ranked (i.e. based on citation reports and JCR/CORE rank) evolutionary algorithm based clustering approaches are reviewed. Maximum of the techniques do not require any user to define the number of clusters in advance. We present the limitations and advantages of some evolutionary algorithm based clustering techniques. We also present a thorough discussion and future research directions of evolutionary algorithm based clustering techniques.

The use of data mining techniques in banking domain is suitable due to the nature and sensitivity of bank data and due to the real time complex decision processes. The main concern for a bank's manager is to take good decisions in order to minimize the risks level associated to bank's activities. It is very important for a bank to have knowledge of causes which generate the financial crises or imbalances. Lending is one of the most risky activities in banking area and adequate methods to support the decision making process are necessary. In this paper the authors present a prototype decision support system based on data mining techniques used in lending process. The proposed system was designed to assist a customer who applies for a credit and it may represent an extension for e-banking activities.

Boosted decision trees are among the most popular learning techniques in use today. While exhibiting fast speeds at test time, relatively slow training renders them impractical for applications with real-time learning requirements. We propose a principled approach to overcome this drawback. We prove a bound on the error of a decision stump given its preliminary error on a subset of the training data; the bound may be used to prune unpromising features early in the training process. We propose a fast training algorithm that exploits this bound, yielding speedups of an order of magnitude at no cost in the final performance of the classifier. Our method is not a new variant of Boosting; rather, it is used in conjunction with existing Boosting algorithms and other sampling methods to achieve even greater speedups.

A combination of distributed computation, positive feedback and constructive greedy heuristic is proposed as a new approach to stochastic optimization and problem solving. Positive feedback accounts for rapid discovery of very good solutions, distributed computation avoids premature convergence, and greedy heuristic helps the procedure to find acceptable solutions in the early stages of the search process. An application of the proposed methodology to the classical travelling salesman problem shows that the system can rapidly provide very good, if not optimal, solutions. We report on many simulation results and discuss the working of the algorithm. Some hints about how this approach can be applied to a variety of optimization problems are also given.

This paper introduces a software tool named KEEL, which is a software tool to assess evolutionary algorithms for Data Mining problems of various kinds including as regression, classification, unsupervised learning, etc. It includes evolutionary learning algorithms based on different approaches: Pittsburgh, Michigan and IRL, as well as the integration of evolutionary learning techniques with different pre-processing techniques, allowing it to perform a complete analysis of any learning model in comparison to existing software tools. Moreover, KEEL has been designed with a double goal: research and educational.

In this article a method for color reduction based on ant colony algorithm is presented. Generally color reduction involves two steps: choosing a proper palette and mapping colors to this palette. This article is about the first step. Using ant colony algorithm, pixel clusters are formed based on their colors and neighborhood information to make the final palette. A comparison between the results of the proposed method and some other methods is presented. There are some parameters in the proposed method which can be set based on user needs and priorities. This increases the flexibility of the method. 2007 Elsevier B.V. All rights reserved.

Classification for very large datasets has many practical applications in data mining. Techniques such as discretization and dataset sampling can be used to scale up decision tree classifiers to large datasets. Unfortunately, both of these techniques can cause a significant loss in accuracy. We present a novel decision tree classifier called CLOUDS, which samples the splitting points for numeric attributes followed by an estimation step to narrow the search space of the best split. CLOUDS reduces computation and I/O complexity substantially compared to state of the art classifters, while maintaining the quality of the generated trees in terms of accuracy and tree size. We provide experimental results with a number of real and synthetic data~ets.

Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, ***, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.

Artificial bee colony algorithm (ABC), which is inspired by the foraging behavior of honey bee swarm, is a biological-inspired optimization. It shows more effective than genetic algorithm (GA), particle swarm optimization (PSO) and ant colony optimization (ACO). However, ABC is good at exploration but poor at exploitation, and its convergence speed is also an issue in some cases. For these insufficiencies, we propose an improved ABC algorithm called I-ABC. In I-ABC, the best-so-far solution, inertia weight and acceleration coefficients are introduced to modify the search process. Inertia weight and acceleration coefficients are defined as functions of the fitness. In addition, to further balance search processes, the modification forms of the employed bees and the onlooker ones are different in the second acceleration coefficient. Experiments show that, for most functions, the I-ABC has a faster convergence speed and ptimization better performances than each of ABC and the gbest-guided ABC (GABC). But I-ABC could not still substantially achieve the best solution for all optimization problems. In a few cases, it could not find better results than ABC or GABC. In order to inherit the bright sides of ABC, GABC and I-ABC, a high-efficiency hybrid ABC algorithm, which is called PS-ABC, is proposed. PS-ABC owns the abilities of prediction and selection. Results show that PS-ABC has a faster convergence speed like I-ABC and better search ability ods f than other relevant meth


Question: Generating decision trees through an ant colony optimization algorithm approach.

This document explores the transformative journey from data mining to knowledge discovery, presenting an extensive overview of the methodologies and techniques involved. It delves into the complex process of extracting valuable information from large datasets, highlighting the pivotal role of data mining in this endeavor. The document outlines various approaches to data mining, including classification, clustering, and association rule learning, and emphasizes the importance of preprocessing and postprocessing steps in ensuring the reliability and accuracy of mined data. Central to this narrative is the evolution of data mining techniques towards more sophisticated knowledge discovery frameworks, which not only unearth hidden patterns but also facilitate their interpretation and application. The document also discusses the challenges and limitations of traditional data mining methods, paving the way for advanced algorithms like ant colony optimization that promise to enhance the efficiency and effectiveness of knowledge discovery processes. By integrating insights from multiple disciplines such as statistics, machine learning, and pattern recognition, this overview underscores the interdisciplinary nature of data mining and its potential to revolutionize decision-making across various domains.

A reported weakness of C4.5 in domains with continuous attributes is addressed by modifying the formation and evaluation of tests on continuous attributes. An MDL-inspired penalty is applied to such tests, eliminating some of them from consideration and altering the relative desirability of all tests. Empirical trials show that the modi cations lead to smaller decision trees with higher predictive accuracies. Results also con rm that a new version of C4.5 incorporating these changes is superior to recent approaches that use global discretization and that construct small trees with multi-interval splits.

A new algorithm, called Hamming Clustering (HC), is proposed to extract a set of rules underlying a given classification problem. It is able to reconstruct the and-or expression associated with any Boolean function from a training set of

A combination of distributed computation, positive feedback and constructive greedy heuristic is proposed as a new approach to stochastic optimization and problem solving. Positive feedback accounts for rapid discovery of very good solutions, distributed computation avoids premature convergence, and greedy heuristic helps the procedure to find acceptable solutions in the early stages of the search process. An application of the proposed methodology to the classical travelling salesman problem shows that the system can rapidly provide very good, if not optimal, solutions. We report on many simulation results and discuss the working of the algorithm. Some hints about how this approach can be applied to a variety of optimization problems are also given.

-The huge amounts of data generated by healthcare transactions are too complex and voluminous to be processed and analyzed by traditional methods. Data mining provides the methodology and technology to transform these mounds of data into useful information for decision making. The Healthcare industry is generally “information rich”, which is not feasible to handle manually. These large amounts of data are very important in the field of data mining to extract useful information and generate relationships amongst the attributes. Kidney disease is a complex task which requires much experience and knowledge. Kidney disease is a silent killer in developed countries and one of the main contributors to disease burden in developing countries. In the health care industry the data mining is mainly used for predicting the diseases from the datasets. The Data mining classification techniques, namely Decision trees, ANN, Naive Bayes are analyzed on Kidney disease data set. Keywords--Data Mining, Kidney Disease, Decision tree, Naive Bayes, ANN, K-NN, SVM, Rough Set, Logistic Regression, Genetic Algorithms (GAs) / Evolutionary Programming (EP), Clustering

The modeling of trees is challenging due to their complex branching structures. Three different ways to generate trees are using real world data for reconstruction, interactive modeling methods and modeling with procedural or rule-based systems. Procedural content generation is the idea of using algorithms to automate content creation processes, and it is useful in plant modeling since it can generate a wide variety of plants that can adapt and react to the environment and changing conditions. This thesis focuses on and extends a procedural tree generation technique that uses a space colonization algorithm to model the tree branches’ competition for space, and shifts the previous works’ focus from realism to fantasy. The technique satisfied the idea of using interaction between the tree’s internal and external factors to determine its final shape, by letting the designer control the where and the how of the tree’s growth process. The implementation resulted in a tree generation application where the user’s imagination decides the limit of what can be produced, and if that limit is reached can the application be used to randomly generate a wide variety of trees and tree-like structures. A motivation for many researchers in the procedural content generation area is how it can be used to augment human imagination. The result of this thesis can be used for that, by stepping away from the restrictions of realism, and with ease let the user generate widely diverse trees, that are not necessarily realistic but, in most cases, adapts to the idea of a tree.

There has long been a chasm between theoretical mod els of machine learning and practical machine learn ing algorithms For instance empirically successful algorithms such as C and backpropagation have not met the criteria of the PAC model and its vari ants Conversely the algorithms suggested by com putational learning theory are usually too limited in various ways to nd wide application The theoreti cal status of decision tree learning algorithms is a case in point while it has been proven that C and all reasonable variants of it fails to meet the PAC model criteria other recently proposed decision tree algo rithms that do have non trivial performance guaran tees unfortunately require membership queries Two recent developments have narrowed this gap between theory and practice not for the PAC model but for the related model known as weak learning or boosting First an algorithm called Adaboost was proposed that meets the formal criteria of the boosting model and is also competitive in practice Second the basic algorithms underlying the popular C and CART programs have also very recently been shown to meet the formal criteria of the boosting model Thus it seems plausible that the weak learning frame work may provide a setting for interaction between formal analysis and machine learning practice that is lacking in other theoretical models Our aim in this paper is to push this interaction further in light of these recent developments In par ticular we perform experiments suggested by the for mal results for Adaboost and C within the weak learning framework We concentrate on two particu larly intriguing issues First the theoretical boosting results for top down decision tree algorithms such as C suggest that a new splitting criterion may result in trees that are smaller and more accurate than those obtained using the usual information gain We con rm this suggestion experimentally Second a super cial interpretation of the theo retical results suggests that Adaboost should vastly outperform C This is not the case in practice and we argue through experimental results that the theory must be understood in terms of a measure of a boosting algorithm s behavior called its advantage sequence We compare the advantage sequences for C and Adaboost in a number of experiments We nd that these sequences have qualitatively dif ferent behavior that explains in large part the dis crepancies between empirical performance and the theoretical results Brie y we nd that although C and Adaboost are both boosting algorithms Adaboost creates successively harder ltered dis tributions while C creates successively easier ones in a sense that will be made precise

Previous studies propose that associative classification has high classification accuracy and strong flexibility at handling unstructured data. However, it still suffers from the huge set of mined rules and sometimes biased classification or overfitting since the classification is based on only single high-confidence rule. In this study, we propose a new associative classification method, CMAR, i.e., Classification based on Multiple Association Rules. The method extends an efficient frequent pattern mining method, FP-growth, constructs a class distribution-associated FP-tree, and mines large database efficiently. Moreover, it applies a CR-tree structure to store and retrieve mined association rules efficiently, and prunes rules effectively based on confidence, correlation and database coverage. The classification is performed based on a weighted analysis using multiple strong association rules. Our extensive experiments on databases from UCI machine learning database repository show that CMAR is consistent, highly effective at classification of various kinds of databases and has better average classification accuracy in comparison with CBA and C4.5. Moreover, our performance study shows that the method is highly efficient and scalable in comparison with other reported associative classification methods.

The ability to predict performance of students is very crucial in our present education system. We can use data mining concepts for this purpose. ID3 algorithm is one of the famous algorithms present today to generate decision trees. But this algorithm has a shortcoming that it is inclined to attributes with many values. So , this research aims to overcome this shortcoming of the algorithm by using gain ratio(instead of information gain) as well as by giving weights to each attribute at every decision making point. Several other algorithms like J48 and Naive Bayes classification algorithm are also applied on the dataset. The WEKA tool was used for the analysis of J48 and Naive Bayes algorithms. The results are compared and presented. The dataset used in our study is taken from the School of Computing Sciences and Engineering (SCSE), VIT University. Keyworddata mining, educational data mining (EDM), decision tree, gain ratio, weighted ID3

In this paper we examine the application of the random forest classifier for the all relevant feature selection problem. To this end we first examine two recently proposed all relevant feature selection algorithms, both being a random forest wrappers, on a series of synthetic data sets with varying size. We show that reasonable accuracy of predictions can be achieved and that heuristic algorithms that were designed to handle the all relevant problem, have performance that is close to that of the reference ideal algorithm. Then, we apply one of the algorithms to four families of semi-synthetic data sets to assess how the properties of particular data set influence results of feature selection. Finally we test the procedure using a well-known gene expression data set. The relevance of nearly all previously established important genes was confirmed, moreover the relevance of several new ones is discovered.

Advance in information and communication technologies, force us to keep most of the information electronically, consequently, the security of information has become a fundamental issue. The traditional intrusion detection systems look for unusual or suspicious activity, such as patterns of network traffic that are likely indicators of unauthorized activity. However, normal operation often produces traffic that matches likely "attack signature", resulting in false alarms. One main drawback is the inability of detecting new attacks which do not have known signatures. In this paper particle swarm optimization (PSO) is used to implement a feature selection, and support vector machine (SVMs) with the one-versus-rest method serve as a fitness function of PSO for classification problems from the literature. Experimental result shows that our method allows us to recognize not only known attacks but also to detect suspicious activity that may be the result of a new, unknown attack. Our method simplifies features effectively and obtains a higher classification accuracy compared to other methods


Question: Utilizing an ant colony optimization algorithm to construct decision trees.

This paper serves as an extension and elaboration on Demšar's (2006) work, which outlined a series of non-parametric statistical tests and procedures for comparing the performance of classifiers across various data sets. While Demšar's paper provided a solid foundation, particularly for comparing a control method, it did not delve deeply into certain advanced topics. Our focus is on these more sophisticated statistical procedures that enable the comparison of n×n classifiers, offering a significant enhancement in power and precision. Additionally, we present a straightforward method for calculating adjusted and comparable p-values, which is crucial for conducting accurate multiple comparison procedures. This approach ensures that the statistical significance of differences between classifiers is accurately assessed, providing a more comprehensive and nuanced understanding of their relative performances.

Since the Hoeffding tree algorithm was proposed in the literature, decision trees became one of the most popular tools for mining data streams. The key point of constructing the decision tree is to determine the best attribute to split the considered node. Several methods to solve this problem were presented so far. However, they are either wrongly mathematically justified (e.g., in the Hoeffding tree algorithm) or time-consuming (e.g., in the McDiarmid tree algorithm). In this paper, we propose a new method which significantly outperforms the McDiarmid tree algorithm and has a solid mathematical basis. Our method ensures, with a high probability set by the user, that the best attribute chosen in the considered node using a finite data sample is the same as it would be in the case of the whole data stream.

Artificial Neural Network is among the most popular algorithm for supervised learning. However, Neural Networks have a well-known drawback of being a “Black Box” learner that is not comprehensible to the Users. This lack of transparency makes it unsuitable for many high risk tasks such as medical diagnosis that requires a rational justification for making a decision. Rule Extraction methods attempt to curb this limitation by extracting comprehensible rules from a trained Network. Many such extraction algorithms have been developed over the years with their respective strengths and weaknesses. They have been broadly categorized into three types based on their approach to use internal model of the Network. Eclectic Methods are hybrid algorithms that combine the other approaches to attain more performance. In this paper, we present an Eclectic method called HERETIC. Our algorithm uses Inductive Decision Tree learning combined with information of the neural network structure for extracting logical rules. Experiments and theoretical analysis show HERETIC to be better in terms of speed and performance.

A combination of distributed computation, positive feedback and constructive greedy heuristic is proposed as a new approach to stochastic optimization and problem solving. Positive feedback accounts for rapid discovery of very good solutions, distributed computation avoids premature convergence, and greedy heuristic helps the procedure to find acceptable solutions in the early stages of the search process. An application of the proposed methodology to the classical travelling salesman problem shows that the system can rapidly provide very good, if not optimal, solutions. We report on many simulation results and discuss the working of the algorithm. Some hints about how this approach can be applied to a variety of optimization problems are also given.

In this work, a proposed hybrid dragonfly algorithm (DA) with extreme learning machine (ELM) system for prediction problem is presented. ELM model is considered a promising method for data regression and classification problems. It has fast training advantage, but it always requires a huge number of nodes in the hidden layer. The usage of a large number of nodes in the hidden layer increases the test/evaluation time of ELM. Also, there is no guarantee of optimality of weights and biases settings on the hidden layer. DA is a recently promising optimization algorithm that mimics the moving behavior of moths. DA is exploited here to select less number of nodes in the hidden layer to speed up the performance of the ELM. It also is used to choose the optimal hidden layer weights and biases. A set of assessment indicators is used to evaluate the proposed and compared methods over ten regression data sets from the UCI repository. Results prove the capability of the proposed DA-ELM model in searching for optimal feature combinations in feature space to enhance ELM generalization ability and prediction accuracy. The proposed model was compared against the set of commonly used optimizers and regression systems. These optimizers are namely, particle swarm optimization (PSO) and genetic algorithm (GA). The proposed DA-ELM model proved an advance overall compared methods in both accuracy and generalization ability.

In this paper we introduce the concept of privacy preserving data mining. In our model, two parties owning conndential databases wish to run a data mining algorithm on the union of their databases, without revealing any unnecessary information. This problem has many practical and important applications, such as in medical research with conndential patient records. Data mining algorithms are usually complex, especially as the size of the input is measured in megabytes, if not gigabytes. A generic secure multi-party computation solution, based on evaluation of a circuit computing the algorithm on the entire input, is therefore of no practical use. We focus on the problem of decision tree learning and use ID3, a popular and widely used algorithm for this problem. We present a solution that is considerably more eecient than generic solutions. It demands very few rounds of communication and reasonable bandwidth. In our solution, each party performs by itself a computation of the same order as computing the ID3 algorithm for its own database. The results are then combined using eecient cryptographic protocols, whose overhead is only logarithmic in the number of transactions in the databases. We feel that our result is a substantial contribution, demonstrating that secure multi-party computation can be made practical, even for complex problems and large inputs.

The classification problem can be addressed by numerous techniques and algorithms which belong to different paradigms of machine learning. In this paper, we are interested in evolutionary algorithms, the so-called genetics-based machine learning algorithms. In particular, we will focus on evolutionary approaches that evolve a set of rules, i.e., evolutionary rule-based systems, applied to classification tasks, in order to provide a state of the art in this field. This paper has a double aim: to present a taxonomy of the genetics-based machine learning approaches for rule induction, and to develop an empirical analysis both for standard classification and for classification with imbalanced data sets. We also include a comparative study of the genetics-based machine learning (GBML) methods with some classical non-evolutionary algorithms, in order to observe the suitability and high potential of the search performed by evolutionary algorithms and the behavior of the GBML algorithms in contrast to the classical approaches, in terms of classification accuracy.

A new algorithm, called Hamming Clustering (HC), is proposed to extract a set of rules underlying a given classification problem. It is able to reconstruct the and-or expression associated with any Boolean function from a training set of

The main goal of the structural optimization is to minimize the weight of structures while satisfying all design requirements imposed by design codes. In this paper, the Artificial Bee Colony algorithm with an adaptive penalty function approach (ABC-AP) is proposed to minimize the weight of truss structures. The ABC algorithm is swarm intelligence based optimization technique inspired by the intelligent foraging behavior of honeybees. Five truss examples with fixed-geometry and up to 200 elements were studied to verify that the ABC algorithm is an effective optimization algorithm in the creation of an optimal design for truss structures. The results of the ABC-AP compared with results of other optimization algorithms from the literature show that this algorithm is a powerful search and optimization technique for structural design. © 2010 Elsevier B.V. All rights reserved.

Random forest is well known as one of the best learning methods. In spite of its great success, it also has certain drawbacks: the heuristic learning rule does not effectively minimize the global training loss; the model size is usually too large for many real applications. To address the issues, we propose two techniques, global refinement and global pruning, to improve a pre-trained random forest. The proposed global refinement jointly relearns the leaf nodes of all trees under a global objective function so that the complementary information between multiple trees is well exploited. In this way, the fitting power of the forest is significantly enhanced. The global pruning is developed to reduce the model size as well as the over-fitting risk. The refined model has better performance and smaller storage cost, as verified in extensive experiments.

The development of data-mining applications such as classification and clustering has been applied to large scale data. In this research, we present comparative study of different classification techniques using three data mining tools named WEKA, TANAGRA and MATLAB. The aim of this paper is to analyze the performance of different classification techniques for a set of large data. The algorithm or classifiers tested are Multilayer Perceptron, Bayes Network, J48graft (c4.5), Fuzzy Lattice Reasoning (FLR), NaiveBayes, JRip (RIPPER), Fuzzy Inference System (FIS), Adaptive Neuro-Fuzzy Inference Systems(ANFIS). A fundamental review on the selected technique is presented for introduction purposes. The diabetes data with a total instance of 768 and 9 attributes (8 for input and 1 for output) will be used to test and justify the differences between the classification methods or algorithms. Subsequently, the classification technique that has the potential to significantly improve the common or conventional methods will be suggested for use in large scale data, bioinformatics or other general applications.


Question: Utilizing an ant colony optimization algorithm to construct decision trees.

Introducing KEEL: An Integrated Software Tool for Evaluating Evolutionary Algorithms in Data Mining

This paper presents KEEL, an advanced software tool designed to evaluate and compare evolutionary algorithms for a wide range of data mining tasks, including regression, classification, and unsupervised learning. KEEL distinguishes itself by incorporating a variety of evolutionary learning methodologies, such as the Pittsburgh, Michigan, and IRL approaches. Additionally, it seamlessly integrates these evolutionary techniques with diverse pre-processing methods, enabling a comprehensive analysis of any learning model. This feature sets KEEL apart from existing software tools in the field.

KEEL was conceived with dual objectives in mind: to serve as a valuable resource for research and to facilitate educational endeavors. Its comprehensive framework not only supports rigorous scientific investigation but also provides an accessible platform for teaching and learning about evolutionary algorithms and their applications in data mining. By offering a rich suite of functionalities and a user-friendly interface, KEEL aims to become a cornerstone tool for both researchers and educators in the evolving landscape of data science and machine learning.

Abstract. In his original paper on random forests, Breiman proposed two different decision tree ensembles: one generated from “orthogonal” trees with thresholds on individual features in every split, and one from “oblique” trees separating the feature space by randomly oriented hyperplanes. In spite of a rising interest in the random forest framework, however, ensembles built from orthogonal trees (RF) have gained most, if not all, attention so far. In the present work we propose to employ “oblique” random forests (oRF) built from multivariate trees which explicitly learn optimal split directions at internal nodes using linear discriminative models, rather than using random coefficients as the original oRF. This oRF outperforms RF, as well as other classifiers, on nearly all data sets but those with discrete factorial features. Learned node models perform distinctively better than random splits. An oRF feature importance score shows to be preferable over standard RF feature importance scores such as Gini or permutation importance. The topology of the oRF decision space appears to be smoother and better adapted to the data, resulting in improved generalization performance. Overall, the oRF propose here may be preferred over standard RF on most learning tasks involving numerical and spectral data.

PurposeThe immense quantity of available unstructured text documents serve as one of the largest source of information. Text classification can be an essential task for many purposes in information retrieval, such as document organization, text filtering and sentiment analysis. Ensemble learning has been extensively studied to construct efficient text classification schemes with higher predictive performance and generalization ability. Providing diversity among the classification algorithms is a key issue in the ensemble design. Design/methodology/approachAn ensemble scheme based on hybrid supervised clustering is presented for text classification. In the presented scheme, supervised hybrid clustering, which is based on cuckoo search algorithm and k-means, is introduced to partition the data samples of each class into clusters so that training subsets with higher diversities can be provided. Each classifier is trained on the diversified training subsets and the predictions of individual classifiers are combined by the majority voting rule. The predictive performance of the proposed classifier ensemble is compared to conventional classification algorithms (such as Naïve Bayes, logistic regression, support vector machines and C4.5 algorithm) and ensemble learning methods (such as AdaBoost, Bagging and Random Subspace) using eleven text benchmarks. FindingThe experimental results indicate that the presented classifier ensemble outperforms the conventional classification algorithms and ensemble learning methods for text classification. Originality/valueThe presented ensemble scheme is the first to use supervised clustering to obtain diverse ensemble for text classification

Motivated by the need for unification of the field of data mining and the growing demand for formalized representation of outcomes of research, we address the task of constructing an ontology of data mining. The proposed ontology, named OntoDM, is based on a recent proposal of a general framework for data mining, and includes definitions of basic data mining entities, such as datatype and dataset, data mining task, data mining algorithm and components thereof (e.g., distance function), etc. It also allows for the definition of more complex entities, e.g., constraints in constraint-based data mining, sets of such constraints (inductive queries) and data mining scenarios (sequences of inductive queries). Unlike most existing approaches to constructing ontologies of data mining, OntoDM is a deep/heavy-weight ontology and follows best practices in ontology engineering, such as not allowing multiple inheritance of classes, using a predefined set of relations and using a top level ontology.

High-throughput sequencing has made it theoretically possible to obtain high-quality de novo assembled genome sequences but in practice DNA extracts are often contaminated with sequences from other organisms. Currently, there are few existing methods for rigorously decontaminating eukaryotic assemblies. Those that do exist filter sequences based on nucleotide similarity to contaminants and risk eliminating sequences from the target organism. We introduce a novel application of an established machine learning method, a decision tree, that can rigorously classify sequences. The major strength of the decision tree is that it can take any measured feature as input and does not require a priori identification of significant descriptors. We use the decision tree to classify de novo assembled sequences and compare the method to published protocols. A decision tree performs better than existing methods when classifying sequences in eukaryotic de novo assemblies. It is efficient, readily implemented, and accurately identifies target and contaminant sequences. Importantly, a decision tree can be used to classify sequences according to measured descriptors and has potentially many uses in distilling biological datasets.

Granting banking facility is one of the most important parts of the financial supplies for each bank. So this activity becomes more valuable economically and always has a degree of risk. These days several various developed Artificial Intelligent systems like Neural Network, Decision Tree, Logistic Regression Analysis, Linear Discriminant Analysis and etc, are used in the field of granting facilities that each of this system owns its advantages and disadvantages. But still studying and working are needed to improve the accuracy and performance of them. In this article among other AI methods, fuzzy expert system is selected. This system is based on data and also extracts rules by using data. Therefore the dependency to experts is omitted and interpretability of rules is obtained. Validity of these rules could be confirmed or rejected by banking affair experts. For investigating the performance of proposed system, this system and some other methods were performed on various datasets. Results show that the proposed algorithm obtained better performance among the others.

We provide an introduction to theoretical and practical aspects of Boosting and Ensemble learning, providing a useful reference for researchers in the field of Boosting as well as for those seeking to enter this fascinating area of research. We begin with a short background concerning the necessary learning theoretical foundations of weak learners and their linear combinations. We then point out the useful connection between Boosting and the Theory of Optimization, which facilitates the understanding of Boosting and later on enables us to move on to new Boosting algorithms, applicable to a broad spectrum of problems. In order to increase the relevance of the paper to practitioners, we have added remarks, pseudo code, “tricks of the trade”, and algorithmic considerations where appropriate. Finally, we illustrate the usefulness of Boosting algorithms by giving an overview of some existing applications. The main ideas are illustrated on the problem of binary classification, although several extensions are discussed. 1 A Brief History of Boosting The underlying idea of Boosting is to combine simple “rules” to form an ensemble such that the performance of the single ensemble member is improved, i.e. “boosted”. Let h1, h2, . . . , hT be a set of hypotheses, and consider the composite ensemble hypothesis

Classification is an important data mining problem. Although classification is a wellstudied problem, most of the current classification algorithms require that all or a portion of the the entire dataset remain permanently in memory. This limits their suitability for mining over large databases. We present a new decision-tree-based classification algorithm, called SPRINT that removes all of the memory restrictions, and is fast and scalable. The algorithm has also been designed to be easily parallelized, allowing many processors to work together to build a single consistent model. This parallelization, also presented here, exhibits excellent scalability as well. The combination of these characteristics makes the proposed algorithm an ideal tool for data mining.

WHAT IS DATA MINING? WHY DATA MINING? NEED FOR HUMAN DIRECTION OF DATA MINING CROSS-INDUSTRY STANDARD PROCESS: CRISP–DM CASE STUDY 1: ANALYZING AUTOMOBILE WARRANTY CLAIMS: EXAMPLE OF THE CRISP–DM INDUSTRY STANDARD PROCESS IN ACTION FALLACIES OF DATA MINING WHAT TASKS CAN DATA MINING ACCOMPLISH? CASE STUDY 2: PREDICTING ABNORMAL STOCK MARKET RETURNS USING NEURAL NETWORKS CASE STUDY 3: MINING ASSOCIATION RULES FROM LEGAL DATABASES CASE STUDY 4: PREDICTING CORPORATE BANKRUPTCIES USING DECISION TREES CASE STUDY 5: PROFILING THE TOURISM MARKET USING k-MEANS CLUSTERING ANALYSIS

We present two new parallel implementations of the tree-ensemble algorithms Random Forest (RF) and Extremely randomized trees (ERT) for emerging many-core platforms, e.g., contemporary graphics cards suitable for general-purpose computing (GPGPU). Random Forest and Extremely randomized trees are ensemble learners for classification and regression. They operate by constructing a multitude of decision trees at training time and outputting a prediction by comparing the outputs of the individual trees. Thanks to the inherent parallelism of the task, an obvious platform for its computation is to employ contemporary GPUs with a large number of processing cores. Previous parallel algorithms for Random Forests in the literature are either designed for traditional multi-core CPU platforms or early history GPUs with simpler hardware architecture and relatively few number of cores. The new parallel algorithms are designed for contemporary GPUs with a large number of cores and take into account aspects of the newer hardware architectures as memory hierarchy and thread scheduling. They are implemented using the C/C++ language and the CUDA interface for best possible performance on NVidia-based GPUs. An experimental study comparing with the most important previous solutions for CPU and GPU platforms shows significant improvement for the new implementations, often with several magnitudes.

Scalable data mining in large databases is one of today's challenges to database technologies. Thus, substantial effort is dedicated to a tight coupling of database and data mining systems leading to database primitives supporting data mining tasks. In order to support a wide range of tasks and to be of general usage these primitives should be rather building blocks than implementations of specific algorithms. In this paper, we describe primitives for building and applying decision tree classifiers. Based on the analysis of available algorithms and previous work in this area we have identified operations which are useful for a number of classification algorithms. We discuss the implementation of these primitives on top of a commercial DBMS and present experimental results demonstrating the performance benefit.


Question: Utilizing an ant colony optimization algorithm for the induction of decision trees.

This paper presents an innovative approach for optimizing the parameters of the particle swarm optimization (PSO) algorithm and proposes two enhancements to boost its performance after a specified number of iterations. To enhance local search capabilities and ensure convergence to the global optimum, time-varying acceleration coefficients (TVAC) are integrated into PSO, alongside a time-varying inertia weight factor. Building upon the concept of TVAC, two strategies are devised to elevate the PSO's performance. Firstly, a mutation component is introduced into the PSO with TVAC (MPSO-TVAC), where a slight disturbance is added to the velocity vector of a randomly chosen particle at a predetermined probability. Secondly, a novel PSO variant, the "self-organizing hierarchical particle swarm optimizer with TVAC (HPSO-TVAC)", is proposed. This method focuses on the "social" and "cognitive" aspects of the PSO strategy to determine the new velocity of each particle, with particles being reinitialized upon stagnation in the search space. To address the challenge of selecting an optimal mutation step size for diverse problems, a time-varying mutation step size is introduced. It is noteworthy that, for most benchmarks, the mutation probability exhibits minimal impact on the MPSO-TVAC method's performance. Conversely, the effect of reinitialization velocity on the HPSO-TVAC method's performance is scrutinized, with a time-varying reinitialization step size proving to be an effective parameter optimization strategy. The HPSO-TVAC method outshines all other methods under investigation for the majority of functions. However, it is observed that both the MPSO and HPSO strategies exhibit suboptimal performance when the acceleration coefficients are fixed at two.

This paper compares the accuracy of decision tree and Bayesian network algorithms for predicting the academic performance of undergraduate and postgraduate students at two very different academic institutes: Can Tho University (CTU), a large national university in Viet Nam; and the Asian Institute of Technology (AIT), a small international postgraduate institute in Thailand that draws students from 86 different countries. Although the diversity of these two student populations is very different, the data-mining tools were able to achieve similar levels of accuracy for predicting student performance: 73/71% for {fail, fair, good, very good} and 94/93% for {fail, pass} at the CTU/AIT respectively. These predictions are most useful for identifying and assisting failing students at CTU (64% accurate), and for selecting very good students for scholarships at the AIT (82% accurate). In this analysis, the decision tree was consistently 3-12% more accurate than the Bayesian network. The results of these case studies give insight into techniques for accurately predicting student performance, compare the accuracy of data mining algorithms, and demonstrate the maturity of open source tools.

In machine learning, a combination of classifiers, known as an ensemble classifier, often outperforms individual ones. While many ensemble approaches exist, it remains, however, a difficult task to find a suitable ensemble configuration for a particular dataset. This paper proposes a novel ensemble construction method that uses PSO generated weights to create ensemble of classifiers with better accuracy for intrusion detection. Local unimodal sampling (LUS) method is used as a meta-optimizer to find better behavioral parameters for PSO. For our empirical study, we took five random subsets from the well-known KDD99 dataset. Ensemble classifiers are created using the new approaches as well as the weighted majority algorithm (WMA) approach. Our experimental results suggest that the new approach can generate ensembles that outperform WMA in terms of classification accuracy . © 2015 Published by Elsevier B.V. US ultiple classifier ptimization SO 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 VM eighted majority voting (WMV) MA

The healthcare industry is a vast field with a plethora of data about patients,added to the huge medical records every passing day. In terms of science, this industry is ’information rich’ yet ’knowledge poor’. However, data mining with its various analytical tools and techniques plays a major role in reducing the use of cumbersome tests used on patients to detect a disease. The aim of this paper is to employ and analyze different data mining techniques for the prediction of heart disease in a patient through extraction of interesting patterns from the dataset using vital parameters. This paper strives to bring out the methodology and implementation of these techniques-Artificial Neural Networks, Decision Tree and Naive Bayes and stress upon the results and conclusion induced on the basis of accuracy and time complexity. By far, the observations reveal that Artificial Neural Networks outperformed Naive Bayes and Decision Tree.

This paper presents a method to employ particle swarms optim izers in a cooperative configuration. This is achieved by splitting the input vector into several sub-vectors, each w hich is optimized cooperatively in its own swarm. The applic ation of this technique to neural network training is investigate d, with promising results.

The cloud computing is the development of distributed computing, parallel computing and grid computing, or defined as the commercial implementation of these computer science concepts. One of the fundamental issues in this environment is related to task scheduling. Cloud task scheduling is an NP-hard optimization problem, and many meta-heuristic algorithms have been proposed to solve it. A good task scheduler should adapt its scheduling strategy to the changing environment and the types of tasks. This paper proposes a cloud task scheduling policy based on Load Balancing Ant Colony Optimization (LBACO) algorithm. The main contribution of our work is to balance the entire system load while trying to minimizing the make span of a given tasks set. The new scheduling strategy was simulated using the CloudSim toolkit package. Experiments results showed the proposed LBACO algorithm outperformed FCFS (First Come First Serve) and the basic ACO (Ant Colony Optimization).

Detection of malicious software (malware) using ma chine learning methods has been explored extensively to enable fas t detection of new released malware. The performance of these classifiers depen ds on the induction algorithms being used. In order to benefit from mul tiple different classifiers, and exploit their strengths we suggest using an ens emble method that will combine the results of the individual classifiers i nto one final result to achieve overall higher detection accuracy. In this paper we evaluate several combining methods using five different base inducers (C4.5 Dec ision Tree, Naïve Bayes, KNN, VFI and OneR) on five malware datasets. The mai n goal is to find the best combining method for the task of detecting mal icious files in terms of accuracy, AUC and execution time.

Many researchers have argued that Artificial Neural Networks (ANNs) can improve the performance of intrusion detection systems (IDS). One of the central areas in network intrusion detection is how to build effective systems that are able to distinguish normal from intrusive traffic. In this paper four different algorithms are used namely as Multilayer Perception, Radial Base Function, Logistic Regression and Voted Perception. All these neural based algorithms are implemented in WEKA data mining tool to evaluate the performance. For experimental work, NSL KDD dataset is used. Each neural based algorithm is tested with conducted dataset. It is shown that the Multilayer Perception neural network algorithm is providing more accurate results than other algorithms. Keywords— Data Mining, Intrusion Detection System, Neural network classifier algorithm, NSL KDD dataset

Rough set theory can be applied to rule induction. There are two different types of classification rules, positive and boundary rules, leading to different decisions and consequences. They can be distinguished not only from the syntax measures such as confidence, coverage and generality, but also the semantic measures such as decision-monotocity, cost and risk. The classification rules can be evaluated locally for each individual rule, or globally for a set of rules. Both the two types of classification rules can be generated from, and interpreted by, a decision-theoretic model, which is a probabilistic extension of the Pawlak rough set model. As an important concept of rough set theory, an attribute reduct is a subset of attributes that are jointly sufficient and individually necessary for preserving a particular property of the given information table. This paper addresses attribute reduction in decision-theoretic rough set models regarding different classification properties, such as: decisionmonotocity, confidence, coverage, generality and cost. It is important to note that many of these properties can be truthfully reflected by a single measure c in the Pawlak rough set model. On the other hand, they need to be considered separately in probabilistic models. A straightforward extension of the c measure is unable to evaluate these properties. This study provides a new insight into the problem of attribute reduction. Crown Copyright 2008 Published by Elsevier Inc. All rights reserved.

Nowadays it is very important to maintain a high level security to ensure safe and trusted communication of information between various organizations. But secured data communication over internet and any other network is always under threat of intrusions and misuses. So Intrusion Detection Systems have become a needful component in terms of computer and network security. There are various approaches being utilized in intrusion detections, but unfortunately any of the systems so far is not completely flawless. So, the quest of betterment continues. In this progression, here we present an Intrusion Detection System (IDS), by applying genetic algorithm (GA) to efficiently detect various types of network intrusions. Parameters and evolution processes for GA are discussed in details and implemented. This approach uses evolution theory to information evolution in order to filter the traffic data and thus reduce the complexity. To implement and measure the performance of our system we used the KDD99 benchmark dataset and obtained reasonable detection rate.

Artificial bee colony algorithm (ABC), which is inspired by the foraging behavior of honey bee swarm, is a biological-inspired optimization. It shows more effective than genetic algorithm (GA), particle swarm optimization (PSO) and ant colony optimization (ACO). However, ABC is good at exploration but poor at exploitation, and its convergence speed is also an issue in some cases. For these insufficiencies, we propose an improved ABC algorithm called I-ABC. In I-ABC, the best-so-far solution, inertia weight and acceleration coefficients are introduced to modify the search process. Inertia weight and acceleration coefficients are defined as functions of the fitness. In addition, to further balance search processes, the modification forms of the employed bees and the onlooker ones are different in the second acceleration coefficient. Experiments show that, for most functions, the I-ABC has a faster convergence speed and ptimization better performances than each of ABC and the gbest-guided ABC (GABC). But I-ABC could not still substantially achieve the best solution for all optimization problems. In a few cases, it could not find better results than ABC or GABC. In order to inherit the bright sides of ABC, GABC and I-ABC, a high-efficiency hybrid ABC algorithm, which is called PS-ABC, is proposed. PS-ABC owns the abilities of prediction and selection. Results show that PS-ABC has a faster convergence speed like I-ABC and better search ability ods f than other relevant meth


Question: A neural network method for reconstructing missing markers in motion capture data.

The Encoder-Recurrent-Decoder (ERD) Framework: A Novel Approach for Human Motion Analysis

We present the Encoder-Recurrent-Decoder (ERD) model, a recurrent neural network architecture designed for human body pose recognition and prediction in video and motion capture contexts. ERD integrates nonlinear encoder and decoder networks around recurrent layers, offering a versatile solution for motion generation, body pose classification, and future pose prediction. Our model demonstrates proficiency in handling diverse mocap datasets, spanning multiple subjects and activities, generating seamless motions while maintaining stability over extended periods. In body pose classification tasks, ERD surpasses frame-by-frame body part detectors by resolving left-right body part ambiguities. For video pose prediction, ERD forecasts body joint movements up to 400ms ahead, outperforming first-order motion models based on optical flow. ERD represents an advancement over prior Long Short Term Memory (LSTM) models, as it concurrently learns representations and their dynamics. Our empirical results underscore the significance of representation learning for both classification and prediction in space-time contexts. This feature distinguishes the spatio-temporal visual domain from one-dimensional media like text, speech, or handwriting, where simple, pre-defined representations have proven effective when coupled with recurrent units.

This ERD model showcases the potential of neural networks in addressing the complexities of human motion analysis, particularly in scenarios involving missing data or ambiguous body part identification. By outperforming traditional models in various tasks, ERD highlights the importance of a holistic approach to learning motion dynamics and spatial representations, essential for accurate human pose prediction and reconstruction.

This paper describes the representation, animation and data collection techniques that have been used to produce "realistic" computer generated half-tone animated sequences of a human face changing expression. It was determined that approximating the surface of a face with a polygonal skin containing approximately 250 polygons defined by about 400 vertices is sufficient to achieve a realistic face. Animation was accomplished using a cosine interpolation scheme to fill in the intermediate frames between expressions. This approach is good enough to produce realistic facial motion. The three-dimensional data used to describe the expressions of the face was obtained photogrammetrically using pairs of photographs.

MOTIVATION
Gene expression microarray experiments can generate data sets with multiple missing expression values. Unfortunately, many algorithms for gene expression analysis require a complete matrix of gene array values as input. For example, methods such as hierarchical clustering and K-means clustering are not robust to missing data, and may lose effectiveness even with a few missing values. Methods for imputing missing data are needed, therefore, to minimize the effect of incomplete data sets on analyses, and to increase the range of data sets to which these algorithms can be applied. In this report, we investigate automated methods for estimating missing data.

RESULTS
We present a comparative study of several methods for the estimation of missing values in gene microarray data. We implemented and evaluated three methods: a Singular Value Decomposition (SVD) based method (SVDimpute), weighted K-nearest neighbors (KNNimpute), and row average. We evaluated the methods using a variety of parameter settings and over different real data sets, and assessed the robustness of the imputation methods to the amount of missing data over the range of 1--20% missing values. We show that KNNimpute appears to provide a more robust and sensitive method for missing value estimation than SVDimpute, and both SVDimpute and KNNimpute surpass the commonly used row average method (as well as filling missing values with zeros). We report results of the comparative experiments and provide recommendations and tools for accurate estimation of missing microarray data under a variety of conditions.

A general-purpose, fully interconnected neural-net chip was used to perform computationally intensive tasks for handwritten digit recognition. The chip has nearly 3000 programmable connections, which can be set for template matching. The templates can be reprogrammed as needed during the recognition sequence. The recognition process proceeds in four major steps. First, the image is captured using a TV camera and a digital framegrab. This image is converted, using a digital computer, to either black or white pixels and scaled to fill a 16*16-pixel frame. Next, using the neural-net chip, the image is skeletonized, i.e. the image is thinned to a backbone one pixel wide. Then, the chip is programmed, and a feature map is created by template-matching stored primitive patterns on the chip with regions on the skeletonized image. Finally, recognition, based on the feature map, is achieved using any one of a variety of statistical and heuristic techniques on a digital computer. Best scores range between 90% and 99% correct classification, depending on the quality of the original handwritten digits.<<ETX>>

We describea markerlesscamera tracking systemfor augmentedreality thatoperatesin environmentswhich contain oneor more planes. This is a commonspecialcase, which we showsignificantlysimplifiestracking. The result is a practical, reliable, vision-basedtracker. Furthermore, thetrackedplaneimposesa natural referenceframe, sothat thealignmentof thereal andvirtual coordinatesystemsis rather simplerthan would be the casewith a general structure-and-motionsystem.Multiple planescan be tracked, and additional data such as 2D point tracks are easilyincorporated.

In facial animation, the accurate shape and motion of the lips of virtual humans is of paramount importance, since subtle nuances in mouth expression strongly influence the interpretation of speech and the conveyed emotion. Unfortunately, passive photometric reconstruction of expressive lip motions, such as a kiss or rolling lips, is fundamentally hard even with multi-view methods in controlled studios. To alleviate this problem, we present a novel approach for fully automatic reconstruction of detailed and expressive lip shapes along with the dense geometry of the entire face, from just monocular RGB video. To this end, we learn the difference between inaccurate lip shapes found by a state-of-the-art monocular facial performance capture approach, and the true 3D lip shapes reconstructed using a high-quality multi-view system in combination with applied lip tattoos that are easy to track. A robust gradient domain regressor is trained to infer accurate lip shapes from coarse monocular reconstructions, with the additional help of automatically extracted inner and outer 2D lip contours. We quantitatively and qualitatively show that our monocular approach reconstructs higher quality lip shapes, even for complex shapes like a kiss or lip rolling, than previous monocular approaches. Furthermore, we compare the performance of person-specific and multi-person generic regression strategies and show that our approach generalizes to new individuals and general scenes, enabling high-fidelity reconstruction even from commodity video footage.

Tianhe Yang Summary of " A Method for Registration of 3-D Shapes "

The knowledge discovery process is supported by data files information gathered from collected data sets, which often contain errors in the form of missing values. Data imputation is the activity aimed at estimating values for missing data items. This study focuses on the development of automated data imputation models, based on artificial neural networks for monotone patterns of missing values. The present work proposes a single imputation approach relying on a multilayer perceptron whose training is conducted with different learning rules, and a multiple imputation approach based on the combination of multilayer perceptron and k-nearest neighbours. Eighteen real and simulated databases were exposed to a perturbation experiment with random generation of monotone missing data pattern. An empirical test ean/mode model ultilayer perceptron egression model was accomplished on these data sets, including both approaches (single and multiple imputations), and three classical single imputation procedures – mean/mode imputation, regression and hot-deck – were also considered. Therefore, the experiments involved five imputation methods. The results, considering different performance measures, demonstrated that, in comparison with traditional tools, both proposals improve the automation level and data quality offering a satisfactory performance.

A computer vision system has been implemented that can recognize threedimensional objects from unknown viewpoints in single gray-scale images. Unlike most other approaches, the recognition is accomplished without any attempt to reconstruct depth information bottom-up from the visual input. Instead, three other mechanisms are used that can bridge the gap between the two-dimensional image and knowledge of three-dimensional objects. First, a process of perceptual organization is used to form groupings and structures in the image that are likely to be invariant over a wide range of viewpoints. Second, a probabilistic ranking method is used to reduce the size of the search space during model based matching. Finally, a process of spatial correspondence brings the projections of three-dimensional models into direct correspondence with the image by solving for unknown viewpoint and model parameters. A high level of robustness in the presence of occlusion and missing data can be achieved through full application of a viewpoint consistency constraint. It is argued that similar mechanisms and constraints form the basis for recognition in human vision. This paper has been published in Artificial Intelligence, 31, 3 (March 1987), pp. 355–395.

For improving the efficiency of 2D animation production, this paper presents a method to create in-between frames based on hand-drawn key-frames. The outlines of characters or objects on two key-frames are used as inputs. First, a skeleton linkage of the target object is automatically constructed by rasterizing each of input key-frames and then applying a pixel-based skeleton extraction method. Secondary, a pair of skeleton linkages having corresponding structure between the current key-frame and the next key-frame is constructed by applying the stroke matching algorithm. After these processes, motion transitions between the skeleton linkages are generated based on our simulation model. When the in-between frames are created only in the 2D plane, the outlines at in-between frames can be generated by a 2D deformation. In case that the in-between transitions are containing a rotation around an axis which is no perpendicular to the drawing plane, however, a 3D structure is required. For achieving such in-between transitions, our method constructs a 3D structure by inflating 2D mesh based on the input outlines. Finally, the contours from the view-point for the created 3D structure are projected onto the 2D plane during in-between transitions. In our method, we adopt the Photic Extremum Lines (PEL) to extract the 2D contours from the obtained 3D shape. In this way, we achieve the in-between creation containing spatial rotation such as hand-flipping, which has not been achieved by general ways of in-between creation method.

In order to control a reaching movement of the arm and body, several different computational problems must be solved. Some parallel methods that could be implemented in networks of neuron-like processors are described. Each method solves a different part of the overall task. First, a method is described for finding the torques necessary to follow a desired trajectory. The methods is more economical and more versatile than table look-up and requires very few sequential steps. Then a way of generating an internal representation of a desired trajectory is described. This method shows the trajectory one piece at a time by applying a large set of heuristic rules to a "motion blackboard" that represents the static and dynamic parameters of the state of the body at the current point in the trajectory. The computations are simplified by expressing the positions, orientations, and motions of parts of the body in terms of a single, non-accelerating, world-based frame of reference, rather than in terms of the joint-angles or an egocentric frame based on the body itself.


Question: A neural network method for reconstructing missing markers in motion capture data.

This paper introduces Structural-RNN, a novel methodology that integrates the strengths of spatio-temporal graphs with the sequence modeling capabilities of Deep Recurrent Neural Networks (RNNs). Despite their prowess in handling sequential data, RNN architectures often overlook the inherent high-level spatio-temporal structures that are pivotal in various computer vision tasks. To address this gap, we propose a method that amalgamates the intuitive structure of spatio-temporal graphs with the predictive power of RNNs. Our approach is designed to convert any spatio-temporal graph into a sophisticated RNN mixture that remains feedforward, fully differentiable, and trainable in a joint fashion. This transformation is achieved through a set of well-defined steps, making our method both generic and principled. We validate the effectiveness of Structural-RNN across a range of applications, from human motion analysis to object interaction modeling, where it outperforms state-of-the-art methods by a significant margin. We anticipate that this fusion of high-level spatio-temporal graphs and RNNs will pave the way for innovative problem-solving techniques in computer vision and beyond.

**Question**: A Genetic Algorithm for the Solution of the Economic Dispatch Problem with Valve-Point Effect

**Documents**: A novel evolutionary algorithm for solving multiobjective optimization problems
In this paper, a novel evolutionary algorithm, called the Multiobjective Evolutionary Algorithm based on Decomposition (MOEA/D), is proposed to solve multiobjective optimization problems. MOEA/D decomposes a multiobjective optimization problem into a number of single-objective optimization problems and then solves these subproblems simultaneously using a cooperative coevolutionary strategy. The subproblems are constructed by using a weighted Tchebycheff function. In each generation, MOEA/D selects a subproblem and updates the corresponding solution by using a local search operator. This process is repeated until all subproblems have been updated. The local search operator is based on a set of predefined neighborhoods, which are constructed using a clustering algorithm. The performance of MOEA/D is compared with that of several well-known multiobjective evolutionary algorithms on a set of test problems. The experimental results show that MOEA/D is capable of finding a set of well-distributed and high-quality Pareto optimal solutions for various types of multiobjective optimization problems.

**Similar Documents**: This paper presents a new evolutionary algorithm, the Multiobjective Evolutionary Algorithm based on Decomposition (MOEA/D), designed to tackle multiobjective optimization challenges. MOEA/D innovates by breaking down a multiobjective problem into several single-objective subproblems,

In this paper, we address the problem of extreme head pose estimation from intensity images, in a monocular setup. We introduce a novel fusion pipeline to integrate into a dedicated Kalman Filter the pose estimated from a tracking scheme in the prediction stage and the pose estimated from a detection scheme in the correction stage. To that end, the measurement covariance of the Kalman Filter is updated in every frame. The tracking scheme is performed using a set of keypoints extracted in the area of the head along with a simple 3D geometric model. The detection scheme, on the other hand, relies on the alignment of facial landmarks in each frame combined with 3D features extracted on a head mesh. The head pose in each scheme is estimated by minimizing the reprojection error from the 3D-2D correspondences. By combining both frameworks, we extend the applicability of head pose estimation from facial landmarks to cases where these features are no longer visible. We compared the proposed method to other related approaches, showing that it can achieve state-of-the-art performance. We also demonstrate that our approach is suitable for cases with extreme head rotations and (self-) occlusions, besides being suitable for real time applications.

In this paper, we present a novel approach for human action recognition with histograms of 3D joint locations (HOJ3D) as a compact representation of postures. We extract the 3D skeletal joint locations from Kinect depth maps using Shotton et al.'s method [6]. The HOJ3D computed from the action depth sequences are reprojected using LDA and then clustered into k posture visual words, which represent the prototypical poses of actions. The temporal evolutions of those visual words are modeled by discrete hidden Markov models (HMMs). In addition, due to the design of our spherical coordinate system and the robust 3D skeleton estimation from Kinect, our method demonstrates significant view invariance on our 3D action dataset. Our dataset is composed of 200 3D sequences of 10 indoor activities performed by 10 individuals in varied views. Our method is real-time and achieves superior results on the challenging 3D action dataset. We also tested our algorithm on the MSR Action 3D dataset and our algorithm outperforms Li et al. [25] on most of the cases.

We present a real-time face tracker in this paper The system has achieved a rate of 30% frameshecond using an HP-9000 workstation with a framegrabber and a Canon VC-CI camera. It can track a person 'sface while the person moves freely (e.g., walks, jumps, sits down and stands up) in a room. Three types of models have been employed in developing the system. First, we present a stochastic model to characterize skin-color distributions of human faces. The information provided by the model is sufJicient for tracking a human face in various poses and views. This model is adaptable to different people and different lighting conditions in real-time. Second, a motion model e's used to estimate image motion and to predict search window. Third, a camera model is used toprediet and to compensate for camera motion. The system can be applied to tele-conferencing and many HCI applications including lip-reading and gaze tracking. The principle in developing this system can be extended to other tracking problems such as tracking the human hand.

We consider the problem of reconstruction from incomplete point-clouds. To find a closed mesh the reconstruction is guided by a set of primitive shapes which has been detected on the input point-cloud (e.g. planes, cylinders etc.). With this guidance we not only continue the surrounding structure into the holes but also synthesize plausible edges and corners from the primitives’ intersections. To this end we give a surface energy functional that incorporates the primitive shapes in a guiding vector field. The discretized functional can be minimized with an efficient graphcut algorithm. A novel greedy optimization strategy is proposed to minimize the functional under the constraint that surface parts corresponding to a given primitive must be connected. From the primitive shapes our method can also reconstruct an idealized model that is suitable for use in a CAD system.

Active vision systems are usually limited to either partial or static scene reconstructions. In this paper, we propose to acquire the entire 3D shape of a dynamic scene. This is performed using a multiple projectors and cameras system, that allows to recover the entire shape of the object within a single scan at each frame. Like previous approaches, a static and simple pattern is used to avoid interferences of multiple patterns projected on the same object. In this paper, we extend the technique to capture a dense entire shape of a moving object with accuracy and high video frame rate. To achieve this, we mainly propose two additional steps; one is checking the consistency between the multiple cameras and projectors, and the other is an algorithm for light sectioning based on a plane parameter optimization. In addition, we also propose efficient noise reduction and mesh generation algorithm which are necessary for practical applications. In the experiments, we show that we can successfully reconstruct dense entire shapes of moving objects. Results are illustrated on real data from a system composed of six projectors and six cameras that was actually built.

Tianhe Yang Summary of " A Method for Registration of 3-D Shapes "

We propose a deep convolutional neural network for 3D human pose and camera estimation from monocular images that learns from 2D joint annotations. The proposed network follows the typical architecture, but contains an additional output layer which projects predicted 3D joints onto 2D, and enforces constraints on body part lengths in 3D. We further enforce pose constraints using an independently trained network that learns a prior distribution over 3D poses. We evaluate our approach on several benchmark datasets and compare against state-of-the-art approaches for 3D human pose estimation, achieving comparable performance. Additionally, we show that our approach significantly outperforms other methods in cases where 3D ground truth data is unavailable, and that our network exhibits good generalization properties.

A new method for estimating knee joint flexion/extension angles from segment acceleration and angular velocity data is described. The approach uses a combination of Kalman filters and biomechanical constraints based on anatomical knowledge. In contrast to many recently published methods, the proposed approach does not make use of the earth's magnetic field and hence is insensitive to the complex field distortions commonly found in modern buildings. The method was validated experimentally by calculating knee angle from measurements taken from two IMUs placed on adjacent body segments. In contrast to many previous studies which have validated their approach during relatively slow activities or over short durations, the performance of the algorithm was evaluated during both walking and running over 5 minute periods. Seven healthy subjects were tested at various speeds from 1 to 5 mile/h. Errors were estimated by comparing the results against data obtained simultaneously from a 10 camera motion tracking system (Qualysis). The average measurement error ranged from 0.7 degrees for slow walking (1 mph) to 3.4 degrees for running (5 mph). The joint constraint used in the IMU analysis was derived from the Qualysis data. Limitations of the method, its clinical application and its possible extension are discussed.

We propose the Encoder-Recurrent-Decoder (ERD) model for recognition and prediction of human body pose in videos and motion capture. The ERD model is a recurrent neural network that incorporates nonlinear encoder and decoder networks before and after recurrent layers. We test instantiations of ERD architectures in the tasks of motion capture (mocap) generation, body pose labeling and body pose forecasting in videos. Our model handles mocap training data across multiple subjects and activity domains, and synthesizes novel motions while avoiding drifting for long periods of time. For human pose labeling, ERD outperforms a per frame body part detector by resolving left-right body part confusions. For video pose forecasting, ERD predicts body joint displacements across a temporal horizon of 400ms and outperforms a first order motion model based on optical flow. ERDs extend previous Long Short Term Memory (LSTM) models in the literature to jointly learn representations and their dynamics. Our experiments show such representation learning is crucial for both labeling and prediction in space-time. We find this is a distinguishing feature between the spatio-temporal visual domain in comparison to 1D text, speech or handwriting, where straightforward hard coded representations have shown excellent results when directly combined with recurrent units [31].

This paper describes the representation, animation and data collection techniques that have been used to produce "realistic" computer generated half-tone animated sequences of a human face changing expression. It was determined that approximating the surface of a face with a polygonal skin containing approximately 250 polygons defined by about 400 vertices is sufficient to achieve a realistic face. Animation was accomplished using a cosine interpolation scheme to fill in the intermediate frames between expressions. This approach is good enough to produce realistic facial motion. The three-dimensional data used to describe the expressions of the face was obtained photogrammetrically using pairs of photographs.