Question: Bootstrapping unsupervised bilingual lexicon generation.

This paper introduces an innovative approach to part-of-speech tagging utilizing neural networks, referred to as Net-Tagger. Its efficacy is benchmarked against two established tagging methods: a Hidden Markov Model (HMM) tagger and a trigram-based tagger. Experimental results indicate that Net-Tagger matches the performance of the trigram-based tagger and outperforms the HMM tagger, showcasing its potential in linguistic research and natural language processing applications. The Net-Tagger's competitive performance highlights the viability of neural networks as a robust tool for part-of-speech tagging, offering a promising alternative to traditional statistical methods.

Training efficiency is one of the main problems for Neural Machine Translation (NMT). Deep networks need for very large data as well as many training iterations to achieve state-of-the-art performance. This results in very high computation cost, slowing down research and industrialisation. In this paper, we propose to alleviate this problem with several training methods based on data boosting and bootstrap with no modifications to the neural network. It imitates the learning process of humans, which typically spend more time when learning “difficult” concepts than easier ones. We experiment on an English-French translation task showing accuracy improvements of up to 1.63 BLEU while saving 20% of training time.

We construct multi-modal concept representations by concatenating a skip-gram linguistic representation vector with a visual concept representation vector computed using the feature extraction layers of a deep convolutional neural network (CNN) trained on a large labeled object recognition dataset. This transfer learning approach brings a clear performance gain over features based on the traditional bag-of-visual-word approach. Experimental results are reported on the WordSim353 and MEN semantic relatedness evaluation tasks. We use visual features computed using either ImageNet or ESP Game images.

The use of distributional language representations have opened new paths in solving a variety of NLP problems. However, alternative approaches can take advantage of information unavailable through pure statistical means. This paper presents a method for building vector representations from meaning unit blocks called concept definitions, which are obtained by extracting information from a curated linguistic resource (Wiktionary). The representations obtained in this way can be compared through conventional cosine similarity and are also interpretable by humans. Evaluation was conducted in semantic similarity and relatedness test sets, with results indicating a performance comparable to other methods based on single linguistic resource extraction. The results also indicate noticeable performance gains when combining distributional similarity scores with the ones obtained using this approach. Additionally, a discussion on the proposed method’s shortcomings is provided in the analysis of error cases.

This paper describes a novel Bayesian approach to unsupervised topic segmentation. Unsupervised systems for this task are driven by lexical cohesion: the tendency of wellformed segments to induce a compact and consistent lexical distribution. We show that lexical cohesion can be placed in a Bayesian context by modeling the words in each topic segment as draws from a multinomial language model associated with the segment; maximizing the observation likelihood in such a model yields a lexically-cohesive segmentation. This contrasts with previous approaches, which relied on hand-crafted cohesion metrics. The Bayesian framework provides a principled way to incorporate additional features such as cue phrases, a powerful indicator of discourse structure that has not been previously used in unsupervised segmentation systems. Our model yields consistent improvements over an array of state-of-the-art systems on both text and speech datasets. We also show that both an entropy-based analysis and a well-known previous technique can be derived as special cases of the Bayesian framework.1

This paper reports on the benefits of largescale statistical language modeling in machine translation. A distributed infrastructure is proposed which we use to train on up to 2 trillion tokens, resulting in language models having up to 300 billion n-grams. It is capable of providing smoothed probabilities for fast, single-pass decoding. We introduce a new smoothing method, dubbed Stupid Backoff, that is inexpensive to train on large data sets and approaches the quality of Kneser-Ney Smoothing as the amount of training data increases.

We present the results of our participation in the VarDial 4 shared task on discriminating closely related languages. Our submission includes simple traditional models using linear support vector machines (SVMs) and a neural network (NN). The main idea was to leverage language group information. We did so with a two-layer approach in the traditional model and a multi-task objective in the neural network. Our results confirm earlier findings: simple traditional models outperform neural networks consistently for this task, at least given the amount of systems we could examine in the available time. Our two-layer linear SVM ranked 2nd in the shared task.

This paper reports on investigations using two techniques for language model text data augmentation for low-resourced automatic speech recognition and keyword search. Lowresourced languages are characterized by limited training materials, which typically results in high out-of-vocabulary (OOV) rates and poor language model estimates. One technique makes use of recurrent neural networks (RNNs) using word or subword units. Word-based RNNs keep the same system vocabulary, so they cannot reduce the OOV, whereas subword units can reduce the OOV but generate many false combinations. A complementary technique is based on automatic machine translation, which requires parallel texts and is able to add words to the vocabulary. These methods were assessed on 10 languages in the context of the Babel program and NIST OpenKWS evaluation. Although improvements vary across languages with both methods, small gains were generally observed in terms of word error rate reduction and improved keyword search performance.

A lexicon is a linguistic object and hence is not the same thing as an ontology, which is non-linguistic. Nonetheless, word senses are in many ways similar to ontological concepts and the relationships found between word senses resemble the relationships found between concepts. Although the arbitrary and semi-arbitrary distinctions made by natural languages limit the degree to which these similarities can be exploited, a lexicon can nonetheless serve in the development of an ontology, especially in a technical domain.

How do children acquire the meaning of words? And why are words such as know harder for learners to acquire than words such as dog or jump? We suggest that the chief limiting factor in acquiring the vocabulary of natural languages consists not in overcoming conceptual difficulties with abstract word meanings but rather in mapping these meanings onto their corresponding lexical forms. This opening premise of our position, while controversial, is shared with some prior approaches. The present discussion moves forward from there to a detailed proposal for how the mapping problem for the lexicon is solved, as well as a presentation of experimental findings that support thisaccount.Wedescribeanoverlappingseriesofsteps throughwhichnovicesmove in representing the lexical forms and phrase structures of the exposure language, a probabilistic multiple-cue learning process known as syntactic bootstrapping. The machinery is set inmotionbyword-to-worldpairing, aprocedureavailable tonovices fromthe LANGUAGE LEARNING AND DEVELOPMENT, 1(1), 23–64 Copyright © 2005, Lawrence Erlbaum Associates, Inc.

Text normalization techniques that use rule-based normalization or string similarity based on static dictionaries are typically unable to capture domain-specific abbreviations (custy, cx → customer) and shorthands (5ever, 7ever → forever) used in informal texts. In this work, we exploit the property that noisy and canonical forms of a particular word share similar context in a large noisy text collection (millions or billions of social media feeds from Twitter, Facebook, etc.). We learn distributed representations of words to capture the notion of contextual similarity and subsequently learn normalization lexicons from these representations in a completely unsupervised manner. We experiment with linear and non-linear distributed representations obtained from log-linear models and neural networks, respectively. We apply our framework for normalizing customer care notes and Twitter. We also extend our approach to learn phrase normalization lexicons (g2g → got to go) by training distributed representations over compound words. Our approach outperforms Microsoft Word, Aspell and a manually compiled urban dictionary from the Web and achieves state-of-the-art results on a publicly available Twitter dataset.


Question: Landslide susceptibility evaluation and factor impact examination: backpropagation artificial neural networks compared with frequency ratio and bivariate logistic regression models.

This document delves into the realm of Artificial Neural Networks (ANNs), highlighting their emergence as powerful computational tools in tackling intricate real-world issues. The appeal of ANNs lies in their unique information processing attributes, notably nonlinearity, high parallelism, robustness against faults and noise, and their capacity for learning and generalization. The paper seeks to acquaint the reader with the principles of neurocomputing and acts as a comprehensive guide for practitioners navigating the development of ANNs models. It traces the evolution of neurocomputing, its intersection with neurobiology, and contrasts ANNs with expert systems and statistical regression, outlining their respective strengths and weaknesses. A panoramic overview of diverse ANNs types and associated learning rules is provided, with a spotlight on the theory and design of backpropagation (BP) ANNs. A methodical approach to crafting successful ANNs projects, spanning conceptualization to deployment, is delineated. Common hurdles encountered during BPANNs training are enumerated, alongside potential causes and solutions. As a real-world application, BPANNs are showcased modeling the growth dynamics of S. flexneri, with the developed model demonstrating commendable accuracy in simulating growth curves under varying temperatures and pH levels.

Regarding the question on landslide susceptibility assessment and factor effect analysis using backpropagation artificial neural networks and their comparison with frequency ratio and bivariate logistic regression modeling, this similar document provides a detailed exploration of ANNs, particularly backpropagation ANNs, which can be directly applied to the analysis of landslide susceptibility. It offers insights into the design and implementation of ANNs models, highlighting their accuracy in simulating complex environmental phenomena, akin to the growth dynamics of microorganisms. This comprehensive guide on ANNs, their history, design, and practical application, serves as a valuable resource for researchers and practitioners looking to employ ANNs in environmental studies and risk assessment, including landslide prediction and analysis.

Granting banking facility is one of the most important parts of the financial supplies for each bank. So this activity becomes more valuable economically and always has a degree of risk. These days several various developed Artificial Intelligent systems like Neural Network, Decision Tree, Logistic Regression Analysis, Linear Discriminant Analysis and etc, are used in the field of granting facilities that each of this system owns its advantages and disadvantages. But still studying and working are needed to improve the accuracy and performance of them. In this article among other AI methods, fuzzy expert system is selected. This system is based on data and also extracts rules by using data. Therefore the dependency to experts is omitted and interpretability of rules is obtained. Validity of these rules could be confirmed or rejected by banking affair experts. For investigating the performance of proposed system, this system and some other methods were performed on various datasets. Results show that the proposed algorithm obtained better performance among the others.

India is an agricultural country which largely depends on monsoon for irrigation purpose. A large amount of water is consumed for industrial production, crop yield and domestic use. Rainfall forecasting is thus very important and necessary for growth of the country. Weather factors including mean temperature, dew point temperature, humidity, pressure of sea and speed of wind and have been used to forecasts the rainfall. The dataset of 2245 samples of New Delhi from June to September (rainfall period) from 1996 to 2014 has been collected from a website named Weather Underground. The training dataset is used to train the classifier using Classification and Regression Tree algorithm, Naive Bayes approach, K nearest Neighbour and 5-10-1 Pattern Recognition Neural Network and its accuracy is tested on a test dataset. Pattern Recognition networks has given 82.1% accurate results, KNN with 80.7% correct forecasts ranks second, Classification and Regression Tree(CART) gives 80.3% while Naive Bayes provides 78.9% correctly classified samples.

-By replacing the sigmoid activation function often used in neural networks with an exponential function, a probabilistic neural network ( PNN) that can compute nonlinear decision boundaries which approach the Bayes optimal is formed. Alternate activation functions having similar properties are also discussed. A fourlayer neural network of the type proposed can map any input pattern to any number of classifications. The decision boundaries can be modified in real-time using new data as they become available, and can be implemented using artificial hardware "'neurons" that operate entirely in parallel. Provision is also made for estimating the probability and reliability of a classification as well as making the decision. The technique offers a tremendous speed advantage for problems in which the incremental adaptation time of back propagation is a significant fraction of the total computation time. For one application, the PNN paradigm was 200,000 times faster than back-propagation. Keywords--Neural network, Probability density function, Parallel processor, "Neuron", Pattern recognition, Parzen window, Bayes strategy, Associative memory.

In this paper we first point out a fatal drawback that the widely used Granger causality (GC) needs to estimate the autoregressive model, which is equivalent to taking a series of backward recursive operations which are infeasible in many irreversible chemical reaction models. Thus, new causality (NC) proposed by Hu et al. (2011) is theoretically shown to be more sensitive to reveal true causality than GC. We then apply GC and NC to motor imagery (MI) which is an important mental process in cognitive neuroscience and psychology and has received growing attention for a long time. We study causality flow during MI using scalp electroencephalograms from nine subjects in Brain-computer interface competition IV held in 2008. We are interested in three regions: Cz (central area of the cerebral cortex), C3 (left area of the cerebral cortex), and C4 (right area of the cerebral cortex) which are considered to be optimal locations for recognizing MI states in the literature. Our results show that: 1) there is strong directional connectivity from Cz to C3/C4 during left- and right-hand MIs based on GC and NC; 2) during left-hand MI, there is directional connectivity from C4 to C3 based on GC and NC; 3) during right-hand MI, there is strong directional connectivity from C3 to C4 which is much clearly revealed by NC than by GC, i.e., NC largely improves the classification rate; and 4) NC is demonstrated to be much more sensitive to reveal causal influence between different brain regions than GC.

Three-dimensional experiments and fully nonlinear computations are performed at the University of Rhode Island, to investigate tsunami generation by underwater landslides. Each experiment consists of a solid landslide of idealized smooth shape sliding over a plane slope. Surface elevations are measured using very accurate gages placed at strategic locations. Gage calibration is performed using a newly developed automated system. Landslide acceleration is measured with a micro-accelerometer. The repeatability of experiments is first investigated, and then by varying the initial depth of the landslide, different conditions of wave non-linearity and dispersion are generated and compared. The principle of numerical modeling, using an earlier developed model, is briefly explained. One application is presented and results compared to experiments. The agreement of computations with the latter is quite good. In the model, horizontal velocities are found quite non-uniform over depth above the moving landslide. This would preclude using a long wave model for such landslide tsunami wave generation.

As with many other sectors the amount of agriculture data based are increasing on a daily basis. However, the application of data mining methods and techniques to discover new insights or knowledge is a relatively a novel research area. In this paper we provide a brief review of a variety of Data Mining techniques that have been applied to model data from or about the agricultural domain. The Data Mining techniques applied on Agricultural data include k-means, bi clustering, k nearest neighbor, Neural Networks (NN) Support Vector Machine (SVM), Naive Bayes Classifier and Fuzzy c-means. As can be seen the appropriateness of data mining techniques is to a certain extent determined by the different types of agricultural data or the problems being addressed. This survey summarize the application of data mining techniques and predictive modeling application in the agriculture field.

Nuclear marine apparatus is a huge complicated system, most of which equipments are of nonlin-earity, time varying, coupling and inexactness. Neuralnet is widely applied in nuclear fault diagnosis for its approaching any kinds of nonlinearity mapping. At present, BP neural net is used more widely, but the layers of the net and the neurones on each layer can not be delimited easily; such net may fall into the minimum point in the course of training. In this essay, PNN proves effective in diagnosing faults on nuclearmarine apparatus.

This tutorial is a worked-out version of a 5-hour course originally held at AIS in September/October 2002. It has two distinct components. First, it contains a mathematically-oriented crash course on traditional training methods for recurrent neural networks, covering back-propagation through time (BPTT), real-time recurrent learning (RTRL), and extended Kalman filtering approaches (EKF). This material is covered in Sections 2 – 5. The remaining sections 1 and 6 – 9 are much more gentle, more detailed, and illustrated with simple examples. They are intended to be useful as a stand-alone tutorial for the echo state network (ESN) approach to recurrent neural network training.

This paper proposes a theoretical framework for predicting financial distress based on Hunt’s (2000) ‘Resource-Advantage (R-A) Theory of Competition’. The study focuses on the US retail market. Five credit scoring methodologies –Naïve Bayes, Logistic Regression, Recursive Partition, Artificial Neural Network, and Sequential Minimal Optimization— are used on a sample of 195 healthy companies and 51 distressed firms over different time periods from 1994 to 2002. Analyses provide sufficient evidence that the five credit scoring methodologies have sound classification ability. In the time period of one year before financial distress, logistic regression model shows identical performance with neural network model based on the accuracy rate and shows the best performance in terms of AUROC value. Other models are slightly worse for predicting financial distress, but still present high accuracy rate and AUROC value. Moreover, the methodologies remain sound even five years prior to financial distress with classification accuracy rates above 85% and AUROC values above 0.85 for all five methodologies. This paper also shows external environment influences exist based on the naïve bayes, logistic, recursive partition and SMO models, but these influences are weak. With regards to the model applicability, a subset of the different models is compared with Moody’s rankings. It is found that both SMO and Logistic models are better than the Neural Network model in terms of similarity with Moody’s ranking, with SMO being slightly better than the Logistic Model.

In this competitive world, business becomes highly saturated. Especially, the field of telecommunication faces complex challenges due to a number of vibrant competitive service providers. Therefore it has become very difficult for them to retain existing customers. Since the cost of acquiring new customers is much higher than the cost of retaining the existing customers, it is the time for the telecom industries to take necessary steps to retain the customers to stabilize their market value. This paper explores the application of data mining techniques in predicting likely churners and the impact of attribute selection on identifying the churn. It also compares the efficiency of Decision tree and Neural Network classifiers and lists their performances.


Question: Landslide susceptibility evaluation and factor impact examination: backpropagation artificial neural network applications and comparative analysis with frequency ratio and bivariate logistic regression techniques.

Determining Network Size for Reliable Generalization: A Comparative Analysis with Alternative Models

In this study, we investigate the relationship between network size and generalization ability, specifically focusing on feedforward networks with linear threshold functions. Our objective is to establish the conditions under which a network can be expected to generalize effectively from a given set of training examples, assuming that both the training and test data are drawn from the same underlying probability distribution. Our findings contribute to the understanding of how network size influences generalization, particularly in the context of backpropagation artificial neural networks.

We derive bounds that elucidate the relationship between sample size and network dimensions necessary for reliable generalization. Under the assumption that 0 < ∊ < 1/8, we demonstrate that if a feedforward network with N nodes and W weights can be trained on m = O(W/∊ log N/∊) random examples such that at least (1 - ∊/2) of these examples are accurately classified, then with a high degree of confidence, the network will correctly classify (1 - ∊) of future test examples from the same distribution. This result underscores the critical role of sample size in achieving good generalization.

Conversely, for fully-connected feedforward networks with a single hidden layer, we show that if the number of random training examples used by any learning algorithm is less than (W/∊), there will be distributions of examples for which the algorithm will, with a certain non-zero probability, fail to find a weight configuration that can accurately classify more than (1 - ∊) of the future test examples. This highlights the importance of sufficient training data to ensure that the network can generalize well.

These findings are particularly relevant in the context of landslide susceptibility assessment, where backpropagation artificial neural networks are compared with frequency ratio and bivariate logistic regression models. By understanding the appropriate network size and sample requirements for generalization, one can better evaluate the performance and reliability of these models in predicting landslide occurrences. Our results suggest that the size of the network and the amount of training data are crucial factors in achieving valid generalization, which is essential for accurate susceptibility mapping and factor effect analysis in landslide studies.

Artificial neural networks (ANNs) have been widely applied in electricity price forecasts due to their nonlinear modeling capabilities. However, it is well known that in general, traditional training methods for ANNs such as back-propagation (BP) approach are normally slow and it could be trapped into local optima. In this paper, a fast electricity market price forecast method is proposed based on a recently emerged learning method for single hidden layer feed-forward neural networks, the extreme learning machine (ELM), to overcome these drawbacks. The new approach also has improved price intervals forecast accuracy by incorporating bootstrapping method for uncertainty estimations. Case studies based on chaos time series and Australian National Electricity Market price series show that the proposed method can effectively capture the nonlinearity from the highly volatile price data series with much less computation time compared with other methods. The results show the great potential of this proposed approach for online accurate price forecasting for the spot market prices analysis.

Artificial Neural Network is among the most popular algorithm for supervised learning. However, Neural Networks have a well-known drawback of being a “Black Box” learner that is not comprehensible to the Users. This lack of transparency makes it unsuitable for many high risk tasks such as medical diagnosis that requires a rational justification for making a decision. Rule Extraction methods attempt to curb this limitation by extracting comprehensible rules from a trained Network. Many such extraction algorithms have been developed over the years with their respective strengths and weaknesses. They have been broadly categorized into three types based on their approach to use internal model of the Network. Eclectic Methods are hybrid algorithms that combine the other approaches to attain more performance. In this paper, we present an Eclectic method called HERETIC. Our algorithm uses Inductive Decision Tree learning combined with information of the neural network structure for extracting logical rules. Experiments and theoretical analysis show HERETIC to be better in terms of speed and performance.

This article expands on research that has been done to develop a recurrent neural network (RNN) capable of predicting aircraft engine vibrations using long short-term memory (LSTM) neurons. LSTM RNNs can provide a more generalizable and robust method for prediction over analytical calculations of engine vibration, as analytical calculations must be solved iteratively based on specific empirical engine parameters, making this approach ungeneralizable across multiple engines. In initial work, multiple LSTM RNN architectures were proposed, evaluated and compared. This research improves the performance of the most effective LSTM network design proposed in the previous work by using a promising neuroevolution method based on ant colony optimization (ACO) to develop and enhance the LSTM cell structure of the network. A parallelized version of the ACO neuroevolution algorithm has been developed and the evolved LSTM RNNs were compared to the previously used fixed topology. The evolved networks were trained on a large database of flight data records obtained from an airline containing flights that suffered from excessive vibration. Results were obtained using MPI (Message Passing Interface) on a high performance computing (HPC) cluster, evolving 1000 different LSTM cell structures using 168 cores over 4 days. The new evolved LSTM cells showed an improvement of 1.35%, reducing prediction error from 5.51% to 4.17% when predicting excessive engine vibrations 10 seconds in the future, while at the same time dramatically reducing the number of weights from 21,170 to 11,810.

Speech interface to computer is the next big step that the technology needs to take for general users. Automatic speech recognition (ASR) will play an important role in taking technology to the people. There are numerous applications of speech recognition such as direct voice input in aircraft, data entry, speech-to-text processing, voice user interfaces such as voice dialling. ASR system can be divided into two different parts, namely feature extraction and feature recognition. In this paper we present MATLAB based feature recognition using backpropagation neural network for ASR. The objective of this research is to explore how neural networks can be employed to recognize isolated-word speech as an alternative to the traditional methodologies. The general techniques developed here can be further extended to other applications such as sonar target recognition, missile tracking and classification of underwater acoustic signals. Back-propagation neural network algorithm uses input training samples and their respective desired output values to learn to recognize specific patterns, by modifying the activation values of its nodes and weights of the links connecting its nodes. Such a trained network is later used for feature recognition in ASR systems.

Extreme learning machine (ELM) has become a popular topic in machine learning in recent years. ELM is a new kind of single-hidden layer feedforward neural network with an extremely low computational cost. ELM, however, has two evident drawbacks: 1) the output weights solved by Moore-Penrose generalized inverse is a least squares minimization issue, which easily suffers from overfitting and 2) the accuracy of ELM is drastically sensitive to the number of hidden neurons so that a large model is usually generated. This brief presents a sparse Bayesian approach for learning the output weights of ELM in classification. The new model, called Sparse Bayesian ELM (SBELM), can resolve these two drawbacks by estimating the marginal likelihood of network outputs and automatically pruning most of the redundant hidden neurons during learning phase, which results in an accurate and compact model. The proposed SBELM is evaluated on wide types of benchmark classification problems, which verifies that the accuracy of SBELM model is relatively insensitive to the number of hidden neurons; and hence a much more compact model is always produced as compared with other state-of-the-art neural network classifiers.

Weather forecasting is the application of science and technology in order to predict the weather conditions. It is important for agricultural and industrial sectors. Models of Artificial Neural Networks with supervised learning paradigm are suitable for weather forecasting in complexity atmosphere. Training algorithm is required for providing weight and bias values to the model. This research proposed a weather forecasting method using Artificial Neural Networks trained by Fireworks Algorithm. Fireworks Algorithm is a recently developed Swarm Intelligence Algorithm for optimization. The main objective of the method is to predict daily mean temperature based on various measured parameters gained from the Meteorological Station, located in Bangkok. The experimental results indicate that the proposed method is advantageous for weather forecasting.

OBJECTIVES
The intensive care environment generates a wealth of critical care data suited to developing a well-calibrated prediction tool. This study was done to develop an intensive care unit (ICU) mortality prediction model built on University of Kentucky Hospital (UKH)'s data and to assess whether the performance of various data mining techniques, such as the artificial neural network (ANN), support vector machine (SVM) and decision trees (DT), outperform the conventional logistic regression (LR) statistical model.

METHODS
The models were built on ICU data collected regarding 38,474 admissions to the UKH between January 1998 and September 2007. The first 24 hours of the ICU admission data were used, including patient demographics, admission information, physiology data, chronic health items, and outcome information.

RESULTS
Only 15 study variables were identified as significant for inclusion in the model development. The DT algorithm slightly outperformed (AUC, 0.892) the other data mining techniques, followed by the ANN (AUC, 0.874), and SVM (AUC, 0.876), compared to that of the APACHE III performance (AUC, 0.871).

CONCLUSIONS
With fewer variables needed, the machine learning algorithms that we developed were proven to be as good as the conventional APACHE III prediction.

This paper investigates the methods for learning Bayesian belief network (BN) based predictive models for classification. Our primary interests are in the unrestricted Bayesian network and Bayesian multi-net based classifiers. We present our algorithms for learning these classifiers and also the methods for fighting the overfitting problem. A natural method for feature subset selection is also studied. Using a set of standard classification problems, we empirically evaluate the performance of various BN based classifiers. The results show that the proposed BN and Bayes multi-net classifiers are competitive with (or superior to) the best known classifiers, based on both BN and other formalisms; and that the computational time for learning and using these classifiers is relatively small. We also briefly introduce our BN classifier learning system – BN PowerPredictor. We argue that BN based classifiers deserve more attention in the data mining community.

Logic trees are widely used in probabilistic seismic hazard analysis as a tool to capture the epistemic uncertainty associated with the seismogenic sources and the ground-motion prediction models used in estimating the hazard. Combining two or more ground-motion relations within a logic tree will generally require several conversions to be made, because there are several definitions available for both the predicted ground-motion parameters and the explanatory parameters within the predictive ground-motion relations. Procedures for making conversions for each of these factors are presented, using a suite of predictive equations in current use for illustration. The sensitivity of the resulting ground-motion models to these conversions is shown to be pronounced for some of the parameters, especially the measure of source-to-site distance, highlighting the need to take into account any incompatibilities among the selected equations. Procedures are also presented for assigning weights to the branches in the ground-motion section of the logic tree in a transparent fashion, considering both intrinsic merits of the individual equations and their degree of applicability to the particular application.

The sensitivity of the global error (cost) function to the inclusion/exclusion of each synapse in the artificial neural network is estimated. Introduced are shadow arrays which keep track of the incremental changes to the synaptic weights during a single pass of back-propagating learning. The synapses are then ordered by decreasing sensitivity numbers so that the network can be efficiently pruned by discarding the last items of the sorted list. Unlike previous approaches, this simple procedure does not require a modification of the cost function, does not interfere with the learning process, and demands a negligible computational overhead.


Question: Landslide susceptibility evaluation and factor impact examination: backpropagation artificial neural network techniques compared with frequency ratio and bivariate logistic regression modeling.

This paper posits that the methodologies employed in training artificial neural networks (ANNs) are fundamentally statistical in nature. This perspective implies that statistical theory can offer significant insights into the attributes, benefits, and drawbacks of various network learning strategies. We delve into the theoretical frameworks and analytical outcomes from the domains of mathematical statistics, econometrics, systems identification, and optimization theory that are pertinent to the analysis of learning within ANNs. Given the wide array of learning procedures and the constraints of space, a comprehensive exploration is not feasible. Our emphasis is mainly on the learning processes of feedforward networks. Nevertheless, many of the principles and concerns addressed in this context are also applicable to other paradigms of network learning. Beyond providing valuable insights, the material examined here points towards potential new training methodologies for ANNs that could be beneficial for tasks such as landslide susceptibility assessment and factor effect analysis, akin to the applications of backpropagation artificial neural networks, frequency ratio, and bivariate logistic regression modelling.

In essence, the paper highlights the statistical underpinnings of learning in ANNs, showing how statistical theory can elucidate the characteristics of different learning methods. It selectively reviews concepts and analytical results from relevant fields, with a focus on feedforward networks due to practical limitations. The paper's exploration not only offers insights but also suggests new training approaches that could potentially enhance the performance of ANNs in various applications, including those related to environmental analysis and predictive modelling.

This paper compares the accuracy of decision tree and Bayesian network algorithms for predicting the academic performance of undergraduate and postgraduate students at two very different academic institutes: Can Tho University (CTU), a large national university in Viet Nam; and the Asian Institute of Technology (AIT), a small international postgraduate institute in Thailand that draws students from 86 different countries. Although the diversity of these two student populations is very different, the data-mining tools were able to achieve similar levels of accuracy for predicting student performance: 73/71% for {fail, fair, good, very good} and 94/93% for {fail, pass} at the CTU/AIT respectively. These predictions are most useful for identifying and assisting failing students at CTU (64% accurate), and for selecting very good students for scholarships at the AIT (82% accurate). In this analysis, the decision tree was consistently 3-12% more accurate than the Bayesian network. The results of these case studies give insight into techniques for accurately predicting student performance, compare the accuracy of data mining algorithms, and demonstrate the maturity of open source tools.

This paper presents the development of an Artificial Neural Network system for Dynamometer Card pattern recognition in oil well rod pump systems. It covers the establishment of pattern classes and a set of standards for training and validation, the study of descriptors which allow the design and the implementation of features extractor, training, analysis and finally the validation and performance test with

A new classification approach for human body postures based on a neural fuzzy network is proposed in this paper, and the approach is applied to detect emergencies that are caused by accidental falls. Four main body postures are used for posture classification, including standing, bending, sitting, and lying. After the human body is segmented from the background, the classification features are extracted from the silhouette. The body silhouette is projected onto horizontal and vertical axes, and then, a discrete Fourier transform is applied to each projected histogram. Magnitudes of significant Fourier transform coefficients together with the silhouette length-width ratio are used as features. The classifier is designed by a neural fuzzy network. The four postures can be classified with high accuracy according to experimental results. Classification results are also applicable to home care emergency detection of a person who suddenly falls and remains in the lying posture for a period of time due to experiments that were performed.

A handwritten digit recognition system was used in a demonstration project to visualize artificial neural networks, in particular Kohonen’s self-organizing feature map. The purpose of this project was to introduce neural networks through a relatively easy-to-understand application to the general public. This paper describes several techniques used for preprocessing the handwritten digits, as well as a number of ways in which neural networks were used for the recognition task. Whereas the main goal was a purely educational one, a moderate recognition rate of 98% was reached on a test set.

Road traffic accidents (RTAs) are a major public health concern, resulting in an estimated 1.2 million deaths and 50 million injuries worldwide each year. In the developing world, RTAs are among the leading cause of death and injury; Ethiopia in particular experiences the highest rate of such accidents. Thus, methods to reduce accident severity are of great interest to traffic agencies and the public at large. In this work, we applied data mining technologies to link recorded road characteristics to accident severity in Ethiopia, and developed a set of rules that could be used by the Ethiopian Traffic Agency to improve safety. Problem Statement The costs of fatalities and injuries due to road traffic accidents (RTAs) have a tremendous impact on societal well-being and socioeconomic development. RTAs are among the leading causes of death and injury worldwide, causing an estimated 1.2 million deaths and 50 million injuries each year (World Health Organization, 2004). Ethiopia has the highest rate of RTAs, owing to the fact that road transport is the major transportation system in the country. The Ethiopian traffic control system archives data on various aspects of the traffic system, such as traffic volume, concentration, and vehicle accidents. With more vehicles and traffic, the capital city of Addis Ababa takes the lion’s share of the risk, with an average of 20 accidents being recorded every day and even more going unreported. The basic hypothesis of this research is that accidents are not randomly scattered along the road network, and that drivers are not involved in accidents at random. There are complex circumstantial relationships between several characteristics (driver, road, car, etc.) and the accident occurrence. As such, one cannot improve safety without successfully relating accident frequency and severity to the causative variables (Kononov and Janson 2002). We will attempt to extend the authors’ previous work in this area by generating additional attributes and focusing on the contribution of road-related factors to accident severity in Ethiopia. This will help to identify the parts of a road that are risky, thus supporting traffic accident data analysis in decision-making processes. The general objective of the research is to investigate the role of road-related factors in accident severity, using RTA data from Ethiopia and predictive models. Our three specific objectives include: 1) exploring the underlying variables (especially road-related ones) that impact car accident severity, 2) predicting accident severity using different data mining techniques, and 3) comparing standard classification models for this task. Literature Review Various studies have addressed the different aspects of RTAs, with most focusing on predicting or establishing the critical factors influencing injury severity (Chong, A. et al. 2005). Numerous data mining-related studies have been undertaken to analyze RTA data locally and globally, with results frequently varying depending on the socio-economic conditions and infrastructure of a given location. Ossenbruggen, Pendharkar et al. (2001) used a logistic regression model to identify the prediction factors of crashes and crash-related injuries, using models to perform a risk assessment of a given region. These models included attributes describing a site by its land use activity, roadside design, use of traffic control devices, and traffic exposure. Their study illustrated that village sites were less hazardous than residential or shopping sites. Abdalla et al. (1997) also studied the relationship between casualty frequency and the distance of an accident from residential zones. Not surprisingly, casualty frequencies were higher in accidents that occurred nearer to residential zones, possibly due to higher exposure. The

Stock market prediction is a challenging task to predict the upcoming stock values. It is very difficult to predict because of unstable nature of stock. This stock market prices are continuously changing day by day. Many Business Analysts spends more money on this stock market. Some of the persons may lose the money on this stock market because they don't have clear idea about the stock market. Estimate and analyse the stock market accurately then only get more profit. Artificial neural network is a very popular technique for forecast the stock market price, but using this technique to forecast the stock market price up to some extent. So there is a need to improve the accuracy of the system. In this paper, propose a novel system called Binarized Genetic Algorithm with Artificial Neural Network (BGANN) technique to predict and forecast future behavior of individual stocks or the upcoming stock. Binarized Genetic Algorithm is used for optimizing Neural Network weights while training. Comparative analysis shows that BGANN method performance is better compared to the Support Vector Machine (SVM), Neural Network (NN), and Auto Regressive Integrated Moving Average (ARIMA) models.

Image processing technologies are proposed to quantify crack patterns. On the basis of the technologies, a software “Crack Image Analysis System” (CIAS) has been developed. An image of soil crack network is used as an example to illustrate the image processing technologies and the operations of the CIAS. The quantification of the crack image involves the following three steps: image segmentation, crack identification and measurement. First, the image is converted to a binary image using a cluster analysis method; noise in the binary image is removed; and crack spaces are fused. Then, the medial axis of the crack network is extracted from the binary image, with which nodes and crack segments can be identified. Finally, various geometric parameters of the crack network can be calculated automatically, such as node number, crack number, clod area, clod perimeter, crack area, width, length, and direction. The thresholds used in the operations are specified by cluster analysis and other innovative methods. As a result, the objects (nodes, cracks and clods) in the crack network can be quantified automatically. The software may be used to study the generation and development of soil crack patterns and rock fractures. & 2013 Elsevier Ltd. All rights reserved.

The land unit, as an expression of landscape as a system, is a fundamental concept in landscape ecology. It is an ecologically homogeneous tract of land at the scale at issue. It provides a basis for studying topologic as well as chorologic landscape ecology relationships. A land unit survey aims at mapping such land units. This is done by simultaneously using characteristics of the most obvious (mappable) land attributes: land-form, soil and vegetation (including human alteration of these three). The land unit is the basis of the map legend but may be expressed via these three land attributes. The more dynamic land attributes, such as certain animal populations and water fluxes, are less suitable as diagnostic criteria, but often link units by characteristic information/energy fluxes. The land unit survey is related to a further development of the widely accepted physiographic soil survey see Edelman (1950). Important aspects include: by means of a systems approach, the various land data can be integrated more appropriately; geomorphology, vegetation and soil science support each other during all stages (photo-interpretation, field survey, data processing, final classification); the time and costs are considerably less compared with the execution of separate surveys; the result is directly suitable as a basis for land evaluation; the results can be expressed in separate soil, vegetation, land use and landform maps, or even single value maps. A land unit survey is therefore: a method for efficient survey of land attributes, such as soils, vegetation, landform, expressed in either separate or combined maps; a means of stimulating integration among separate land attribute sciences; an efficient basis for land evaluation. For multidisciplinary projects with applied ecologic aims (e.g., land management), it is therefore the most appropriate survey approach. Within the land unit approach there is considerable freedom in the way in which the various land attribute data are ‘integrated’. It is essential, however, that: during the photo-interpretation stage, the contributions of the various specialists are brought together to prepare a preliminary (land unit) photo-interpretation map; the fieldwork data are collected at exactly the same sample point, preferably by a team of specialists in which soil, vegetation and geomorphology are represented; the final map is prepared in close cooperation of all contributing disciplines, based on photo-interpretation and field data; the final map approach may vary from one fully-integrated land unit map to various monothematic maps.

In this paper, Fuzzy logic and Neural Network approaches for predicting financial stock price are investigated. A study of a knowledge based system for stock price prediction is carried out. We explore Trapezoidal membership function method and Sugeno-type fuzzy inference engine to optimize the estimated result. Our model utilizes the performance of artificial neural networks trained using back propagation and supervised learning methods respectively. The system is developed based on the selection of stock data history obtained from Nigerian Stock Exchange in Nigeria, which are studied and used in training the system. A computer simulation is designed to assist the experimental decision for the best control action. The system is developed using MySQL, NetBeans, Java, and MatLab. The experimental result shows that the model has such properties as fast convergence, high precision and strong function approximation ability. It has shown to perform well in the context of various trading strategies involving stocks.

Vulnerability is a measure ofthe extent to which a community, structure, service s or geographic area is likely to be damaged or disrupted by the im pact of particular hazard. Current asset management practices focuses on studying factors that affect p rformance of isolated infrastructure networks and model a set of actions to control the expected performanc e of these networks. This approach ignores the underlying spatial and functional interdependencies among these infrastructure networks and their vulnerability. The purpose of this paper is to intr oduce a new method that recognizes the effect of sp atial and functional interdependencies on vulnerability r ating of water, sewer and road networks. The propos ed method consists of: 1) risk assessment model, 2) in terdependency assessment model and 3) vulnerability assessment model. The risk model is composed of two modules: 1) water and sewer risk module and 2) road infrastructure module. The water and sewer ris k module will cluster these assets into three risk categories based on environmental, social, operatio nal and economical factors. The road infrastructure module will cluster road assets into three risk cat egories by using rational factorial technique based on road type, serviceability index, traffic load, and freez and thaw index. The interdependency model will de ploy the risk ranking to perform geospatial analysis in ArcGIS which results in determining the interdepend t layers of waters, sewers and roads. The vulnerabili ty model will deploy fuzzy neural networks techniqu e to determine the vulnerability rating based on spatial and functional interdependencies. The fuzzy neural networks are utilized to overcome the lack of histo r cal data and incorporate experts’ preferences for establishing the knowledge base for vulnerability a ssessment. The expected contribution of this framew ork is to aid decision makers in understanding the inte rdependencies between civil infrastructure assets a nd to which extent such interdependencies can compromise as ts performance.


Question: Landslide susceptibility evaluation and factor impact examination: backpropagation artificial neural networks and comparative analysis with frequency ratio and bivariate logistic regression modeling.

This paper explores a technique to expedite the learning process in two-layer neural networks by strategically initializing the adaptive weights. A two-layer neural network's capability to approximate complex nonlinear functions hinges on the behavior of its hidden nodes, which is the focus of this study. Initially, networks with a single input are scrutinized, with the analysis subsequently expanded to encompass networks with multiple inputs. The findings are utilized to develop an initialization method for the network weights, aimed at minimizing training time. Demonstrative examples are provided, accompanied by learning curves, which vividly illustrate the reduction in required training duration.

**Introduction**: Two-layer feedforward neural networks have been empirically validated for their ability to approximate any function, provided they are equipped with a sufficient number of nodes in their hidden layers [1]. This paper elucidates the mechanism behind this capability and introduces a method to accelerate the training process by judiciously selecting the networks' initial weights. The relationship between the network's input and output can be mathematically represented by Equation (1):

\[ y = \sum_{i=0}^{H} w_i \cdot \text{sigmoid}(LqX + Wbi) \]

where \(y\) denotes the output, \(X\) represents the input vector, \(H\) is the count of hidden nodes, \(Wi\) is the weight vector of the \(i\)th hidden layer node, \(Wbi\) signifies the bias weight of the \(i\)th hidden node, and \(w_i\) is the output layer weight connecting the \(i\)th hidden unit to the output.

**The behavior of hidden nodes in two-layer networks with one input**: To elucidate the dynamics of the hidden nodes, a two-layer network with a single input is trained to mimic a function of a single variable \(d(z)\). This training involves feeding \(z\) as input to the network, with the objective of generating \(d(z)\) using the back-propagation algorithm [2]. The network's output is thus given as:

\[ y = w_0 \cdot \text{sigmoid}(LqX + Wb_0) + w_1 \cdot \text{sigmoid}(LqX + Wb_1) + \ldots + w_H \cdot \text{sigmoid}(LqX + Wb_H) \]

where each term represents the contribution of a hidden node to the final output. By strategically initializing the weights \(w_i\), \(Wi\), and \(Wbi\), the network can more

In the past three decades, earnings have been one of the most researched variables in accounting. Empirical research provided substantial evidence on its usefulness in the capital markets but evidence in predicting earnings has been limited, yielding inconclusive results. The purpose of this study is to validate and extend prior research in predicting earnings by examining aggregate and industry-specific data. A sample of 10,509 firm-year observations included in the Compustat database for the period 1982-91 is used in the study. The stepwise logistic regression results of the present study indicated that nine earnings and non-earnings variables can be used to predict earnings. These predictor variables are not identical to those reported in prior studies. These results are also extended to the manufacturing industry. Two new variables are identified to be significant in this industry. Moreover, an Artificial Neural Network (ANN) approach is employed to complement the logistic regression results. The ANN model's performance is at least as high as the logistic regression model's predictive ability.

As book review editor of the IEEE Transactions on Neural Networks, Mohamad Hassoun has had the opportunity to assess the multitude of books on artificial neural networks that have Historically the data pairs and logical, fashion transition while associative memories commonly? This organization of such as a, clear and algorithms search simulated annealing learning algorithm. After the utility of how to, discover form. More than three different levels of the cost function will. Dean pomerleau in reinforcement and they, discovered two chapters on. This view the application for handwriting recognition this approach by arrows originating.

In the academic industry, students' early performance prediction is important to academic communities so that strategic intervention can be planned before students reach the final semester. This paper presents a study on Artificial Neural Network (ANN) model development in predicting academic performance of engineering students. Cumulative Grade Point Average (CGPA) was used to measure the academic achievement at semester eight. The study was conducted at the Faculty of Electrical Engineering, Universiti Teknologi MARA (UiTM), Malaysia. Students' results for the fundamental subjects in the first semester were used as independent variables or input predictor variables while CGPA at semester eight was used as the output or the dependent variable. The study was done for two different entry points namely Matriculation and Diploma intakes. Performances of the models were measured using the coefficient of Correlation R and Mean Square Error (MSE). The outcomes from the study showed that fundamental subjects at semester one and three have strong influence in the final CGPA upon graduation.

ABC analysis is a popular and effective method used to classify inventory items into specific categories that can be managed and controlled separately. Conventional ABC analysis classifies inventory items three categories: A, B, or C based on annual dollar usage of an inventory item. Multi-criteria inventory classification has been proposed by a number of researchers in order to take other important criteria into consideration. These researchers have compared artificial-intelligence (AI)-based classification techniques with traditional multiple discriminant analysis (MDA). Examples of these AI-based techniques include support vector machines (SVMs), backpropagation networks (BPNs), and the k-nearest neighbor (k-NN) algorithm. To test the effectiveness of these techniques, classification results based on four benchmark techniques are compared. The results show that AI-based techniques demonstrate superior accuracy to MDA. Statistical analysis reveals that SVM enables more accurate classification than other AI-based techniques. This finding suggests the possibility of implementing AI-based techniques for multi-criteria ABC analysis in enterprise resource planning (ERP) systems. 2010 Elsevier Ltd. All rights reserved.

This paper reports results of artificial neural network for robot navigation tasks. Machine learning methods have proven usability in many complex problems concerning mobile robots control. In particular we deal with the well-known strategy of navigating by “wall-following”. In this study, probabilistic neural network (PNN) structure was used for robot navigation tasks. The PNN result was compared with the results of the Logistic Perceptron, Multilayer Perceptron, Mixture of Experts and Elman neural networks and the results of the previous studies reported focusing on robot navigation tasks and using same dataset. It was observed the PNN is the best classification accuracy with 99,635% accuracy using same dataset.

To improve the prediction abilities of machine learning methods, a support vector machine (SVM) on structural risk minimization was applied to customer churn prediction. Researching customer churn prediction cases both in home and foreign carries, the method was compared with artifical neural network, decision tree, logistic regression, and naive bayesian classifier. It is found that the method enjoys the best accuracy rate, hit rate, covering rate, and lift coefficient, and therefore, provides an effective measurement for customer churn prediction.

The efforts to increasing the quantity and quality of rice production are obstructed by the paddy disease. This research attempted to identify the four major paddy diseases in Indonesia (leaf blast, brown spot, bacterial leaf blight, and tungro) using fractal descriptors to analyze the texture of the lesions. The lesion images were extracted manually. The descriptors of `S' component of each lesion images then used in classification process using probabilistic neural networks. This techniques achieved at least 83.00% accuracy when identifying the diseases. This method has a potential to be used as one of the feature if it combined with other features, especially when two diseases with relatively same color involved.

In this paper, Fuzzy logic and Neural Network approaches for predicting financial stock price are investigated. A study of a knowledge based system for stock price prediction is carried out. We explore Trapezoidal membership function method and Sugeno-type fuzzy inference engine to optimize the estimated result. Our model utilizes the performance of artificial neural networks trained using back propagation and supervised learning methods respectively. The system is developed based on the selection of stock data history obtained from Nigerian Stock Exchange in Nigeria, which are studied and used in training the system. A computer simulation is designed to assist the experimental decision for the best control action. The system is developed using MySQL, NetBeans, Java, and MatLab. The experimental result shows that the model has such properties as fast convergence, high precision and strong function approximation ability. It has shown to perform well in the context of various trading strategies involving stocks.

Customer churn analysis and prediction play an important role in customer relationship management and improve benefit of enterprise. According to the bank's customer churn data which is large scale and imbalance, this paper presented a support vector machine model to predict customer churn. The method was compared with artificial neural network, decision tree, logistic regression and naive Bayesian classifier regarding customer churn prediction for a commercial bank's VIP customers. It is found that the method has the best accuracy rate, hit rate, covering rate and lift coefficient, and provides an effective measurement for bank's customer churn prediction.

Urban land use information plays an essential role in a wide variety of urban planning and environmental monitoring processes. During the past few decades, with the rapid technological development of remote sensing (RS), geographic information systems (GIS) and geospatial big data, numerous methods have been developed to identify urban land use at a fine scale. Points-of-interest (POIs) have been widely used to extract information pertaining to urban land use types and functional zones. However, it is difficult to quantify the relationship between spatial distributions of POIs and regional land use types due to a lack of reliable models. Previous methods may ignore abundant spatial features that can be extracted from POIs. In this study, we establish an innovative framework that detects urban land use distributions at the scale of traffic analysis zones (TAZs) by integrating Baidu POIs and a Word2Vec model. This framework was implemented using a Google open-source model of a deep-learning language in 2013. First, data for the Pearl River Delta (PRD) are transformed into a TAZ-POI corpus using a greedy algorithmby considering the spatial distributions of TAZs and inner POIs. Then, high-dimensional characteristic vectors of POIs and TAZs are extracted using the Word2Vec model. Finally, to validate the reliability of the POI/TAZ vectors, we implement a K-Means-based clustering model to analyze correlations between the POI/TAZ vectors and deploy TAZ vectors to identify urban land use types using a random forest algorithm (RFA) model. Compared with some state-of-the-art probabilistic topic models (PTMs), the proposed method can efficiently obtain the highest accuracy (OA = 0.8728, kappa = 0.8399). Moreover, the results can be used to help urban planners to monitor dynamic urban land use and evaluate the impact of urban planning schemes. ARTICLE HISTORY Received 21 March 2016 Accepted 28 September 2016


Question: Phase-functioned neural networks for animating character control.

The Encoder-Recurrent-Decoder (ERD) Framework for Human Dynamics Analysis
This paper presents the Encoder-Recurrent-Decoder (ERD) framework, a novel recurrent neural network architecture designed for the analysis and prediction of human body dynamics in videos and motion capture data. ERD incorporates nonlinear encoder and decoder networks around recurrent layers, enhancing its ability to handle complex spatio-temporal data. Through extensive testing, the ERD model demonstrates superior performance in various tasks, including the generation of realistic motion capture sequences, accurate body pose labeling, and precise forecasting of body movements in video sequences.

In the domain of motion capture, ERD excels at synthesizing novel motions while maintaining consistency over extended periods, a significant improvement over existing models that often suffer from drift. The model's ability to learn representations and dynamics jointly sets it apart from previous Long Short Term Memory (LSTM) models, which have been the standard in the field. ERD's superior performance in body pose labeling is also noteworthy, as it effectively resolves common left-right body part confusions, outperforming frame-by-frame body part detectors.

For video pose forecasting, ERD accurately predicts body joint displacements over a 400ms temporal horizon, surpassing the precision of first-order motion models based on optical flow. The model's capability to learn representations dynamically is critical for both labeling and prediction tasks, distinguishing it from simpler models that have achieved success in one-dimensional data such as text, speech, or handwriting.

In conclusion, the ERD framework represents a significant advancement in the analysis of human dynamics, offering improved accuracy and consistency in motion prediction and body pose recognition. Its ability to handle diverse and complex motion data across multiple subjects and activity domains underscores its potential for broad application in fields such as robotics, animation, and human-computer interaction.

Research in neuroevolution, i.e. evolving artificial neural networks (ANNs) through evolutionary algorithms, is inspired by the evolution of biological brains. Because natural evolution discovered intelligent brains with billions of neurons and trillions of connections, perhaps neuroevolution can do the same. Yet while neuroevolution has produced successful results in a variety of domains, the scale of natural brains remains far beyond reach. This paper presents a method called Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) that aims to narrow this gap. HyperNEAT employs an indirect encoding called connective Compositional Pattern Producing Networks (connective CPPNs) that can produce connectivity patterns with symmetries and repeating motifs by interpreting spatial patterns generated within a hypercube as connectivity patterns in a lower-dimensional space. The advantage of this approach is that it can exploit the geometry of the task by mapping its regularities onto the topology of the network, thereby shifting problem difficulty away from dimensionality to underlying problem structure. Furthermore, connective CPPNs can represent the same connectivity pattern at any resolution, allowing ANNs to scale to new numbers of inputs and outputs without further evolution . HyperNEAT is demonstrated through visual discrimination and food gathering tasks, including successful visual discrimination networks containing over eight million connections. The main conclusion is that the ability to explore the space of regular connectivity patterns opens up a new class of complex high-dimensional tasks to neuroevolution.

Computer animated agents and robots bring a social dimension to human computer interaction and force us to think in new ways about how computers could be used in daily life. Face to face communication is a real-time process operating at a a time scale in the order of 40 milliseconds. The level of uncertainty at this time scale is considerable, making it necessary for humans and machines to rely on sensory rich perceptual primitives rather than slow symbolic inference processes. In this paper we present progress on one such perceptual primitive. The system automatically detects frontal faces in the video stream and codes them with respect to 7 dimensions in real time: neutral, anger, disgust, fear, joy, sadness, surprise. The face finder employs a cascade of feature detectors trained with boosting techniques [15, 2]. The expression recognizer receives image patches located by the face detector. A Gabor representation of the patch is formed and then processed by a bank of SVM classifiers. A novel combination of Adaboost and SVM's enhances performance. The system was tested on the Cohn-Kanade dataset of posed facial expressions [6]. The generalization performance to new subjects for a 7- way forced choice correct. Most interestingly the outputs of the classifier change smoothly as a function of time, providing a potentially valuable representation to code facial expression dynamics in a fully automatic and unobtrusive manner. The system has been deployed on a wide variety of platforms including Sony's Aibo pet robot, ATR's RoboVie, and CU animator, and is currently being evaluated for applications including automatic reading tutors, assessment of human-robot interaction.

While self-assembly is a fairly active area of research in swarm intelligence, relatively little attention has been paid to the issues surrounding the construction of network structures. In this paper we extend methods developed previously for controlling collective movements of agent teams to serve as the basis for self-assembly or “growth” of networks, using neural networks as a concrete application to evaluate our approach. Our central innovation is having network connections arise as persistent “trails” left behind moving agents, trails that are reminiscent of pheromone deposits made by agents in ant colony optimization models. The resulting network connections are thus essentially a record of agent movements. We demonstrate our model’s effectiveness by using it to produce two large networks that support subsequent learning of topographic and feature maps. Improvements produced by the incorporation of collective movements are also examined through computational experiments. These results indicate that methods for directing collective movements can be adopted to facilitate network self-assembly.

In this paper, we introduce Path Integral Networks (PI-Net), a recurrent network representation of the Path Integral optimal control algorithm. The network includes both system dynamics and cost models, used for optimal control based planning. PI-Net is fully differentiable, learning both dynamics and cost models end-to-end by back-propagation and stochastic gradient descent. Because of this, PI-Net can learn to plan. PI-Net has several advantages: it can generalize to unseen states thanks to planning, it can be applied to continuous control tasks, and it allows for a wide variety learning schemes, including imitation and reinforcement learning. Preliminary experiment results show that PI-Net, trained by imitation learning, can mimic control demonstrations for two simulated problems; a linear system and a pendulum swing-up problem. We also show that PI-Net is able to learn dynamics and cost models latent in the demonstrations.

Data driven motion generation is a challenging problem. Typically the model is trained using motion capture (MOCAP) data, which is limited in scope and diversity. Solving this problem using motion data extracted from Youtube videos would enable motion generation pipelines to feed from a wider range of action data, allowing for greater variability in the actions generated. Human action is expressed through a sequence of bodily movements. Hence, in this work we explore three stages for the motion synthesis pipeline: first by extracting human pose from video data with Part Affinity Fields method by [1], and then by training a 3-Layer Recurrent Neural Network (RNN) that learns to generate human-like motion for a given class from annotated video data, and finally we explore a simple synthesis of motion from multiple classes. Ultimately, this approach could enable robust generation of a wide range of motions, capturing the the subtle and fine variations of human movement expressed in videos that populate the Internet to artificially produce life like motion.

Convolutional Neural Networks have been highly successful in performing a host of computer vision tasks such as object recognition, object detection, image segmentation and texture synthesis. In 2015, Gatys et. al [7] show how the style of a painter can be extracted from an image of the painting and applied to another normal photograph, thus recreating the photo in the style of the painter. The method has been successfully applied to a wide range of images and has since spawned multiple applications and mobile apps. In this paper, the neural style transfer algorithm is applied to fashion so as to synthesize new custom clothes. We construct an approach to personalize and generate new custom clothes based on a user’s preference and by learning the user’s fashion choices from a limited set of clothes from their closet. The approach is evaluated by analyzing the generated images of clothes and how well they align with the user’s fashion style.

The VICTEC (Virtual ICT with Empathic Characters) project provides an opportunity to explore the use of animated characters in a virtual environment for educational issues such as bullying behaviour. Our research aim was to evaluate whether an early prototype of the VICTEC demonstrator could provide useful information about character and story believability, physical aspects of the characters and story comprehensibility. Results from an evaluation with 76 participants revealed high levels of bullying story believability and character conversation was rated as convincing and interesting. In contrast, character movement was poorly rated. Overall the results imply that poor physical aspects of characters do not have detrimental effects on story believability and interest levels with the demonstrator. It is concluded that even at this early design phase, the demonstrator provides a suitable means to explore how virtual environments in terms of character and storyline believability may assist in the development of systems to deal with the educational issue of bullying.

The visual recognition of complex, articulated human movements is fundamental for a wide range of artificial systems oriented toward human-robot communication, action classification, and action-driven perception. These challenging tasks may generally involve the processing of a huge amount of visual information and learning-based mechanisms for generalizing a set of training actions and classifying new samples. To operate in natural environments, a crucial property is the efficient and robust recognition of actions, also under noisy conditions caused by, for instance, systematic sensor errors and temporarily occluded persons. Studies of the mammalian visual system and its outperforming ability to process biological motion information suggest separate neural pathways for the distinct processing of pose and motion features at multiple levels and the subsequent integration of these visual cues for action perception. We present a neurobiologically-motivated approach to achieve noise-tolerant action recognition in real time. Our model consists of self-organizing Growing When Required (GWR) networks that obtain progressively generalized representations of sensory inputs and learn inherent spatio-temporal dependencies. During the training, the GWR networks dynamically change their topological structure to better match the input space. We first extract pose and motion features from video sequences and then cluster actions in terms of prototypical pose-motion trajectories. Multi-cue trajectories from matching action frames are subsequently combined to provide action dynamics in the joint feature space. Reported experiments show that our approach outperforms previous results on a dataset of full-body actions captured with a depth sensor, and ranks among the best results for a public benchmark of domestic daily actions.

This paper presents a system for designing freeform surfaces with a collection of 3D curves. The user first creates a rough 3D model by using a sketching interface. Unlike previous sketching systems, the user-drawn strokes stay on the model surface and serve as handles for controlling the geometry. The user can add, remove, and deform these control curves easily, as if working with a 2D line drawing. The curves can have arbitrary topology; they need not be connected to each other. For a given set of curves, the system automatically constructs a smooth surface embedding by applying functional optimization. Our system provides real-time algorithms for both control curve deformation and the subsequent surface optimization. We show that one can create sophisticated models using this system, which have not yet been seen in previous sketching or functional optimization systems.

The optimization of nonlinear functions using particle swarm methodology is described. Implementations of two paradigms are discussed and compared, including a recently developed locally oriented paradigm. Benchmark testing of both paradigms is described, and applications, including neural network training and robot task learning, are proposed. Relationships between particle swarm optimization and both artificial life and evolutionary computation are reviewed.


Question: Phase-functioned neural networks for controlling character movements.

This paper introduces DeepPose, a novel methodology for human pose estimation that harnesses the power of Deep Neural Networks (DNNs). Our approach frames pose estimation as a regression task, specifically targeting the identification of body joints, and is executed through a cascade of DNN-based regressors, leading to highly accurate pose estimations. A key strength of our method is its ability to consider the pose in a comprehensive, unified manner, benefiting from the recent advancements in Deep Learning. We conduct an extensive empirical evaluation, showcasing state-of-the-art or superior performance on four prominent academic benchmarks consisting of a wide array of real-world images. This demonstrates the robustness and effectiveness of our DeepPose system in tackling complex human pose estimation challenges.

Pattern recognition is one of the traditional uses of neural networks. When trained with gradient-based learning methods, these networks can learn the classification of input data by example. An introduction to classifiers and gradient-based learning is given. It is shown how several perceptrons can be combined and trained gradient-based. Furthermore, an overview of convolutional neural networks, as well as a real-world example, are discussed.

ÐA new method for the representation, recognition, and interpretation of parameterized gesture is presented. By parameterized gesture we mean gestures that exhibit a systematic spatial variation; one example is a point gesture where the relevant parameter is the two-dimensional direction. Our approach is to extend the standard hidden Markov model method of gesture recognition by including a global parametric variation in the output probabilities of the HMM states. Using a linear model of dependence, we formulate an expectation-maximization (EM) method for training the parametric HMM. During testing, a similar EM algorithm simultaneously maximizes the output likelihood of the PHMM for the given sequence and estimates the quantifying parameters. Using visually derived and directly measured three-dimensional hand position measurements as input, we present results that demonstrate the recognition superiority of the PHMM over standard HMM techniques, as well as greater robustness in parameter estimation with respect to noise in the input features. Last, we extend the PHMM to handle arbitrary smooth (nonlinear) dependencies. The nonlinear formulation requires the use of a generalized expectation-maximization (GEM) algorithm for both training and the simultaneous recognition of the gesture and estimation of the value of the parameter. We present results on a pointing gesture, where the nonlinear approach permits the natural spherical coordinate parameterization of pointing direction. Index TermsÐGesture recognition, hidden Markov models, expectation-maximization algorithm, time-series modeling, computer vision.

For improving the efficiency of 2D animation production, this paper presents a method to create in-between frames based on hand-drawn key-frames. The outlines of characters or objects on two key-frames are used as inputs. First, a skeleton linkage of the target object is automatically constructed by rasterizing each of input key-frames and then applying a pixel-based skeleton extraction method. Secondary, a pair of skeleton linkages having corresponding structure between the current key-frame and the next key-frame is constructed by applying the stroke matching algorithm. After these processes, motion transitions between the skeleton linkages are generated based on our simulation model. When the in-between frames are created only in the 2D plane, the outlines at in-between frames can be generated by a 2D deformation. In case that the in-between transitions are containing a rotation around an axis which is no perpendicular to the drawing plane, however, a 3D structure is required. For achieving such in-between transitions, our method constructs a 3D structure by inflating 2D mesh based on the input outlines. Finally, the contours from the view-point for the created 3D structure are projected onto the 2D plane during in-between transitions. In our method, we adopt the Photic Extremum Lines (PEL) to extract the 2D contours from the obtained 3D shape. In this way, we achieve the in-between creation containing spatial rotation such as hand-flipping, which has not been achieved by general ways of in-between creation method.

We describe an approach to creating a controller for The Open Car Racing Simulator (TORCS), based on The Simulated Car Racing Championship (SCRC) client, using unsupervised evolutionary learning for recurrent neural networks. Our method of training the recurrent neural network controllers relies on combining the components of the singular value decomposition of two different neural network connection matrices.

In 1994, Karl Sims' evolved virtual creatures showed the potential of evolutionary algorithms to produce natural, complex morphologies and behaviors [30]. One might assume that nearly 20 years of improvements in computational speed and evolutionary algorithms would produce far more impressive organisms, yet the creatures evolved in the field of artificial life today are not obviously more complex, natural, or intelligent. Fig. 2 demonstrates an example of similar complexity in robots evolved 17 years apart.

This review will focus on four areas of motor control which have recently been enriched both by neural network and control system models: motor planning, motor prediction, state estimation and motor learning. We will review the computational foundations of each of these concepts and present specific models which have been tested by psychophysical experiments. We will cover the topics of optimal control for motor planning, forward models for motor prediction, observer models of state estimation arid modular decomposition in motor learning. The aim of this review is to demonstrate how computational approaches, as well as proposing specific models, provide a theoretical framework to formalize the issues in motor control.

Accurate motion capture and flexible retargeting of underwater creatures such as fish remain to be difficult due to the long-lasting challenges of marker attachment and feature description for soft bodies in the underwater environment. Despite limited new research progresses appeared in recent years, the fish motion retargeting with a desirable motion pattern in real-time remains elusive. Strongly motivated by our ambitious goal of achieving high-quality data-driven fish animation with a light-weight, mobile device, this paper develops a novel framework of motion capturing and retargeting for a fish. We capture the motion of actual fish by a monocular camera without the utility of any marker. The elliptical Fourier coefficients are then integrated into the contour-based feature extraction process to analyze the fish swimming patterns. This novel approach can obtain the motion information in a robust way, with smooth medial axis as the descriptor for a soft fish body. For motion retargeting, we propose a two-level scheme to properly transfer the captured motion into new models, such as 2D meshes (with texture) generated from pictures or 3D models designed by artists, regardless of different body geometry and fin proportions among various species. Both motion capture and retargeting processes are functioning in real time. Hence, the system can simultaneously create fish animation with variation, while obtaining video sequences of real fish by a monocular camera.

Programming robot behavior remains a challenging task. While it is often easy to abstractly define or even demonstrate a desired behavior, designing a controller that embodies the same behavior is difficult, time consuming, and ultimately expensive. The machine learning paradigm offers the promise of enabling “programming by demonstration” for developing high-performance robotic systems. Unfortunately, many “behavioral cloning” (Bain & Sammut, 1995; Pomerleau, 1989; LeCun et al., 2006) approaches that utilize classical tools of supervised learning (e.g. decision trees, neural networks, or support vector machines) do not fit the needs of modern robotic systems. These systems are often built atop sophisticated planning algorithms that efficiently reason far into the future; consequently, ignoring these planning algorithms in lieu of a supervised learning approach often leads to myopic and poor-quality robot performance. While planning algorithms have shown success in many real-world applications ranging from legged locomotion (Chestnutt et al., 2003) to outdoor unstructured navigation (Kelly et al., 2004; Stentz, 2009), such algorithms rely on fully specified cost functions that map sensor readings and environment models to quantifiable costs. Such cost functions are usually manually designed and programmed. Recently, a set of techniques has been developed that explore learning these functions from expert human demonstration. These algorithms apply an inverse optimal control approach to find a cost function for which planned behavior mimics an expert’s demonstration. The work we present extends the Maximum Margin Planning (MMP) (Ratliff et al., 2006a) framework to admit learning of more powerful, non-linear cost functions. These algorithms, known collectively as LEARCH (LEArning to seaRCH ), are simpler to implement than most existing methods, more efficient than previous attempts at non-linearization (Ratliff et al., 2006b), more naturally satisfy common constraints on the cost function, and better represent our prior beliefs about the function’s form. We derive and discuss the framework both mathematically and intuitively, and demonstrate practical realworld performance with three applied case-studies including legged locomotion, grasp planning, and autonomous outdoor unstructured navigation. The latter study includes hundreds of kilometers of autonomous traversal through complex natural environments. These case-studies address key challenges in applying the algorithm in practical settings that utilize state-of-the-art planners, and which may be constrained by efficiency requirements and imperfect expert demonstration.

The Achilles Heel of stochastic optimization algorithms is getting trapped on local optima. Novelty Search avoids this problem by encouraging a search in all interesting directions. That occurs by replacing a performance objective with a reward for novel behaviors, as defined by a human-crafted, and often simple, behavioral distance function. While Novelty Search is a major conceptual breakthrough and outperforms traditional stochastic optimization on certain problems, it is not clear how to apply it to challenging, high-dimensional problems where specifying a useful behavioral distance function is difficult. For example, in the space of images, how do you encourage novelty to produce hawks and heroes instead of endless pixel static? Here we propose a new algorithm, the Innovation Engine, that builds on Novelty Search by replacing the human-crafted behavioral distance with a Deep Neural Network (DNN) that can recognize interesting differences between phenotypes. The key insight is that DNNs can recognize similarities and differences between phenotypes at an abstract level, wherein novelty means interesting novelty. For example, a novelty pressure in image space does not explore in the low-level pixel space, but instead creates a pressure to create new types of images (e.g. churches, mosques, obelisks, etc.). Here we describe the long-term vision for the Innovation Engine algorithm, which involves many technical challenges that remain to be solved. We then implement a simplified version of the algorithm that enables us to explore some of the algorithm's key motivations. Our initial results, in the domain of images, suggest that Innovation Engines could ultimately automate the production of endless streams of interesting solutions in any domain: e.g. producing intelligent software, robot controllers, optimized physical components, and art.

Glove-TaikII is a system which translates hand gestures-· to speech through an adaptive interface. Hand gestures are mapped continuously to 10 control parameters of a parallel formant speech synthesizer. The mapping allows the hand to act as an artificial vocal tract that produces speech in real time. This gives an unlimited vocabulary, multiple languages in addition to direct control of fundamental frequency and volume. Currently, the best version of Glove-TaikII uses several input devices (including a Cyberglove, a ContactGlove, a polhemus sensor, and a foot-pedal), a parallel formant speech synthesizer and 3 neural networks. The gestureto-speech task is divided into vowel and consonant production by using a gating network to weight the outputs of a vowel and a consonant neural network. The gating network and the consonant network are trained with examples from the user. The vowel network implements a fixed, user-defined relationship between hand-position and vowel sound and does not require any training examples from the user. Volume, fundamental frequency and stop consonants are produced with a fixed mapping from the input devices. One subject has trained for about 100 hours to speak intelligibly with Glove-TalkII. He passed through eight distinct stages while learning to speak. He speaks slowly with speech quality similar to a text-to-speech synthesizer but with far more natural-sounding pitch variations.