Question: LSTM recurrent neural network structures for large-scale acoustic modeling.

This paper presents a groundbreaking connectionist system designed for the accurate recognition of unconstrained handwritten text, addressing the complexities inherent in segmenting cursive or overlapping characters and the necessity of leveraging surrounding context. Despite advancements in preprocessing techniques and language modeling, the core recognition algorithms have seen relatively little innovation, with many systems still reliant on hidden Markov models (HMMs), a decades-old approach known for its limitations. Our proposed solution diverges from this traditional path by introducing a specialized recurrent neural network architecture, tailored for sequence labeling tasks where segmentation is challenging and long-range bidirectional dependencies are crucial. Through extensive testing on two sizable databases of unconstrained handwriting, our system achieves impressive word recognition rates of 79.7% on online data and 74.1% on offline data, outperforming a leading HMM-based system. Furthermore, we highlight the system's resilience to varying lexicon sizes, quantify the impact of its hidden layers, and conduct a detailed analysis of its contextual utilization. Finally, we offer a comprehensive comparison between our network and HMMs, elucidating the factors contributing to the network's superior performance in recognizing unconstrained handwriting.

This chapter describes a general representation and algorithmic framework for speech recognition based on weighted finite-state transducers. These transducers provide a common and natural representation for major components of speech recognition systems, including hidden Markov models (HMMs), context-dependency models, pronunciation dictionaries, statistical grammars, and word or phone lattices. General algorithms for building and optimizing transducer models are presented, including composition for combining models, weighted determinization and minimization for optimizing time and space requirements, and a weight pushing algorithm for redistributing transition weights optimally for speech recognition. The application of these methods to large-vocabulary recognition tasks is explained in detail, and experimental results are given, in particular for the North American Business News (NAB) task, in which these methods were used to combine HMMs, full cross-word triphones, a lexicon of forty thousand words, and a large trigram grammar into a single weighted transducer that is only somewhat larger than the trigram word grammar and that runs NAB in real-time on a very simple decoder. Another example demonstrates that the same methods can be used to optimize lattices for second-pass recognition.

In this paper, we propose a novel method to adapt context-dependent deep neural network hidden Markov model (CD-DNN-HMM) with only limited number of parameters by taking into account the underlying factors that contribute to the distorted speech signal. We derive this factorized adaptation method from the perspectives of joint factor analysis and vector Taylor series expansion, respectively. Evaluated on Aurora 4, the proposed method can get 19.0% and 10.6% relative word error rate reduction on test set B and D with only 20 adaptation utterances, and can have decent improvement with as few as two adaptation utterances. We also show that the proposed method is better than feature discriminative linear regression (fDLR), an existing DNN adaptation method. Its small number of parameters and short training time offer an attractive solution to low-footprint speech applications.

This paper describes digital circuit architectures for automatic speech recognition (ASR) and voice activity detection (VAD) with improved accuracy, programmability, and scalability. Our ASR architecture is designed to minimize off-chip memory bandwidth, which is the main driver of system power consumption. A SIMD processor with 32 parallel execution units efficiently evaluates feed-forward deep neural networks (NNs) for ASR, limiting memory usage with a sparse quantized weight matrix format. We argue that VADs should prioritize accuracy over area and power, and introduce a VAD circuit that uses an NN to classify modulation frequency features with 22.3-<inline-formula> <tex-math notation="LaTeX">$\mu \text{W}$ </tex-math></inline-formula> power consumption. The 65-nm test chip is shown to perform a variety of ASR tasks in real time, with vocabularies ranging from 11 words to 145 000 words and full-chip power consumption ranging from 172 <inline-formula> <tex-math notation="LaTeX">$\mu \text{W}$ </tex-math></inline-formula> to 7.78 mW.

Systems based on deep neural networks have made a breakthrough in many different pattern recognition tasks. However, the use of these systems with traditional architectures seems not to work properly when the amount of training data is scarce. This is the case of the on-line signature verification task. In this paper, we propose a novel writer-independent on-line signature verification systems based on Recurrent Neural Networks (RNNs) with a Siamese architecture whose goal is to learn a dissimilarity metric from the pairs of signatures. To the best of our knowledge, this is the first time these recurrent Siamese networks are applied to the field of on-line signature verification, which provides our main motivation. We propose both Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) systems with a Siamese architecture. In addition, a bidirectional scheme (which is able to access both past and future context) is considered for both LSTM- and GRU-based systems. An exhaustive analysis of the system performance and also the time consumed during the training process for each recurrent Siamese network is carried out in order to compare the advantages and disadvantages for practical applications. For the experimental work, we use the BiosecurID database comprised of 400 users who contributed a total of 11,200 signatures in four separated acquisition sessions. Results achieved using our proposed recurrent Siamese networks have outperformed the state-of-the-art on-line signature verification systems using the same database.

Recent work in unsupervised feature learning and deep learning has shown that being able to train large models can dramatically improve performance. In this paper, we consider the problem of training a deep network with billions of parameters using tens of thousands of CPU cores. We have developed a software framework called DistBelief that can utilize computing clusters with thousands of machines to train large models. Within this framework, we have developed two algorithms for large-scale distributed training: (i) Downpour SGD, an asynchronous stochastic gradient descent procedure supporting a large number of model replicas, and (ii) Sandblaster, a framework that supports a variety of distributed batch optimization procedures, including a distributed implementation of L-BFGS. Downpour SGD and Sandblaster L-BFGS both increase the scale and speed of deep network training. We have successfully used our system to train a deep network 30x larger than previously reported in the literature, and achieves state-of-the-art performance on ImageNet, a visual object recognition task with 16 million images and 21k categories. We show that these same techniques dramatically accelerate the training of a more modestlysized deep network for a commercial speech recognition service. Although we focus on and report performance of these methods as applied to training large neural networks, the underlying algorithms are applicable to any gradient-based machine learning algorithm.

Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. We develop an un-gated unit, the statistical recurrent unit (SRU), that is able to learn long term dependencies in data by only keeping moving averages of statistics. The SRU’s architecture is simple, un-gated, and contains a comparable number of parameters to LSTMs; yet, SRUs perform favorably to more sophisticated LSTM and GRU alternatives, often outperforming one or both in various tasks. We show the efficacy of SRUs as compared to LSTMs and GRUs in an unbiased manner by optimizing respective architectures’ hyperparameters in a Bayesian optimization scheme for both synthetic and realworld tasks.

This paper describes the Arabic handwriting recognition systems proposed by A2iA to the NIST OpenHaRT2013 evaluation. These systems were based on an optical model using Long Short-Term Memory (LSTM) recurrent neural networks, trained to recognize the different forms of the Arabic characters directly from the image, without explicit feature extraction nor segmentation.Large vocabulary selection techniques and n-gram language modeling were used to provide a full paragraph recognition, without explicit word segmentation. Several recognition systems were also combined with the ROVER combination algorithm. The best system exceeded 80% of recognition rate.

Mainly for the sake of solving the lack of keyword-specific data, we propose one Keyword Spotting (KWS) system using Deep Neural Network (DNN) and Connectionist Temporal Classifier (CTC) on power-constrained small-footprint mobile devices, taking full advantage of general corpus from continuous speech recognition which is of great amount. DNN is to directly predict the posterior of phoneme units of any personally customized key-phrase, and CTC to produce a confidence score of the given phoneme sequence as responsive decision-making mechanism. The CTC-KWS has competitive performance in comparison with purely DNN based keyword specific KWS, but not increasing any computational complexity.

We present Deep Voice 3, a fully-convolutional attention-based neural textto-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural speech synthesis systems in naturalness while training an order of magnitude faster. We scale Deep Voice 3 to dataset sizes unprecedented for TTS, training on more than eight hundred hours of audio from over two thousand speakers. In addition, we identify common error modes of attention-based speech synthesis networks, demonstrate how to mitigate them, and compare several different waveform synthesis methods. We also describe how to scale inference to ten million queries per day on a single GPU server.

In this paper, we propose a Layer-RNN (L-RNN) module that is able to learn contextual information adaptively using within-layer recurrence. Our contributions are three-fold: (i) we propose a hybrid neural network architecture that interleaves traditional convolutional layers with L-RNN module for learning longrange dependencies at multiple levels; (ii) we show that a L-RNN module can be seamlessly inserted into any convolutional layer of a pre-trained CNN, and the entire network then fine-tuned, leading to a boost in performance; (iii) we report experiments on the CIFAR-10 classification task, showing that a network with interleaved convolutional layers and L-RNN modules, achieves comparable results (5.39% top1 error) using only 15 layers and fewer parameters to ResNet-164 (5.46%); and on the PASCAL VOC2012 semantic segmentation task, we show that the performance of a pre-trained FCN network can be boosted by 5% (mean IOU) by simply inserting Layer-RNNs.


Question: Large-scale acoustic modeling using long short-term memory recurrent neural network architectures.

This document explores the application of connectionist methods in the domain of speech recognition, specifically for the challenging task of recognizing Broadcast News. A key distinction between connectionist systems and traditional mixture-of-Gaussian models lies in their approach to estimation; connectionist models directly calculate posterior probabilities rather than mere likelihoods. This capability has facilitated the development of innovative strategies for confidence estimation, pronunciation modeling, and search algorithms. Additionally, we have explored a novel feature extraction method that employs modulation-filtered spectrograms, alongside techniques for integrating multiple data sources. All these advancements have been integrated into a comprehensive system designed for the transcription of Broadcast News. To demonstrate its efficacy, we present performance outcomes on the 1998 DARPA Hub-4E Broadcast News evaluation dataset.

**Question**: An Empirical Comparison of Convolutional Neural Networks for Sentence Classification

**Documents**: Convolutional Neural Networks for Sentence Classification
Convolutional Neural Networks (CNNs) have shown promising results in sentence classification tasks. In this paper, we explore multiple CNN architectures for sentence classification and compare their performance on six publicly available datasets. Our results show that simple CNNs perform comparably to or better than more complex models, and that the use of multiple kernel sizes can further improve performance. We also find that pre-trained word embeddings significantly improve the performance of CNNs for sentence classification.

**Similar Documents**: This study delves into the effectiveness of Convolutional Neural Networks (CNNs) for sentence classification tasks, an area where CNNs have demonstrated significant potential. We investigate various CNN architectures and evaluate their performance on six widely-used benchmark datasets. Our findings reveal that even basic CNN configurations can match or outperform more intricate models in sentence classification. Moreover, incorporating multiple kernel sizes in the CNN design leads to notable enhancements in performance. Another crucial observation is that the utilization of pre-trained word embeddings markedly boosts the capabilities of CNNs in sentence classification tasks, underscoring the importance of such embeddings in the model's overall effectiveness.

We present Deep Voice 3, a fully-convolutional attention-based neural textto-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural speech synthesis systems in naturalness while training an order of magnitude faster. We scale Deep Voice 3 to dataset sizes unprecedented for TTS, training on more than eight hundred hours of audio from over two thousand speakers. In addition, we identify common error modes of attention-based speech synthesis networks, demonstrate how to mitigate them, and compare several different waveform synthesis methods. We also describe how to scale inference to ten million queries per day on a single GPU server.

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 Englishto-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.2 after training for 4.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

The development of high-performance speech processing systems for low-resource languages is a challenging area. One approach to address the lack of resources is to make use of data from multiple languages. A popular direction in recent years is to use bottleneck features, or hybrid systems, trained on multilingual data for speech-to-text (STT) systems. This paper presents an investigation into the application of these multilingual approaches to spoken term detection. Experiments were run using the IARPA Babel limited language pack corpora (~10 hours/language) with 4 languages for initial multilingual system development and an additional held-out target language. STT gains achieved through using multilingual bottleneck features in a Tandem configuration are shown to also apply to keyword search (KWS). Further improvements in both STT and KWS were observed by incorporating language questions into the Tandem GMM-HMM decision trees for the training set languages. Adapted hybrid systems performed slightly worse on average than the adapted Tandem systems. A language independent acoustic model test on the target language showed that retraining or adapting of the acoustic models to the target language is currently minimally needed to achieve reasonable performance.

Recurrent neural network models with an attention mechanism have proven to be extremely effective on a wide variety of sequence-tosequence problems. However, the fact that soft attention mechanisms perform a pass over the entire input sequence when producing each element in the output sequence precludes their use in online settings and results in a quadratic time complexity. Based on the insight that the alignment between input and output sequence elements is monotonic in many problems of interest, we propose an end-to-end differentiable method for learning monotonic alignments which, at test time, enables computing attention online and in linear time. We validate our approach on sentence summarization, machine translation, and online speech recognition problems and achieve results competitive with existing sequence-tosequence models.

Up until recently Recurrent neural networks (RNNs) have been the standard go-to component when processing sequential data with neural networks. Issues relative to vanishing gradient have been partly addressed by Long short-term memory (LSTM) and gated recurrent unit (GRU), but in practice experiments show that very long terms dependencies (beyond 1000 time steps) are difficult to learn. We introduce IGLOO, a new neural network architecture which aims at being faster than both LSTM and GRU but also their respective CuDNN optimized versions when convergence happens and provide an alternative in the case when there is no convergence at all. IGLOO’s core idea is to use the relationships between patches sliced out of the features maps of convolution layers at different levels of granularity to build a representation for the sequence. We show that the model can deal with dependencies of more than 25,000 steps in a reasonable time frame. Beyond the well bench-marked copy-memory and addition problems which show good results, we also achieve best recorded accuracy on permuted MNIST (98.4%). IGLOO is also applied on the IMDB set and on the TUH Abnormal EEG Corpus.

We present a novel approach to speech processing based on the principle of pattern discovery. Our work represents a departure from traditional models of speech recognition, where the end goal is to classify speech into categories defined by a prespecified inventory of lexical units (i.e., phones or words). Instead, we attempt to discover such an inventory in an unsupervised manner by exploiting the structure of repeating patterns within the speech signal. We show how pattern discovery can be used to automatically acquire lexical entities directly from an untranscribed audio stream. Our approach to unsupervised word acquisition utilizes a segmental variant of a widely used dynamic programming technique, which allows us to find matching acoustic patterns between spoken utterances. By aggregating information about these matching patterns across audio streams, we demonstrate how to group similar acoustic sequences together to form clusters corresponding to lexical entities such as words and short multiword phrases. On a corpus of academic lecture material, we demonstrate that clusters found using this technique exhibit high purity and that many of the corresponding lexical identities are relevant to the underlying audio stream.

In this report, we describe three systems designed at BOSCH for rare audio sound events detection task of DCASE 2017 challenge. The first system is an end-to-end audio event segmentation using embeddings based on deep convolutional neural network (DCNN) and deep recurrent neural network (DRNN) trained on Mel-filter banks and spectogram features. Both system 2 and 3 contain two parts: audio event tagging and audio event segmentation. Audio event tagging selects the positive audio recordings (containing audio events), which are later processed by the audio segmentation part. Feature selection method has been deployed to select a subset of features in both systems. System 2 employs Dilated convolutional neural network on the selected features for audio tagging, and an audio-codebook approach to convert audio features to audio vectors (Audio2vec system) which are then passed to an LSTM network for audio events boundary prediction. System 3 is based on multiple instance learning problem using variational auto encoder (VAE) to perform audio event tagging and segmentation. Similar to system 2, here a LSTM network is used for audio segmentation. Finally, we have utilized models based on score-fusion among different systems to improve the final results.

Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved promising performance in sequential data modeling. The hidden layers in RNNs can be regarded as the memory units, which are helpful in storing information in sequential contexts. However, when dealing with high dimensional input data, such as video and text, the input-to-hidden linear transformation in RNNs brings high memory usage and huge computational cost. This makes the training of RNNs very difficult. To address this challenge, we propose a novel compact LSTM model, named as TR-LSTM, by utilizing the low-rank tensor ring decomposition (TRD) to reformulate the input-to-hidden transformation. Compared with other tensor decomposition methods, TRLSTM is more stable. In addition, TR-LSTM can complete an end-to-end training and also provide a fundamental building block for RNNs in handling large input data. Experiments on real-world action recognition datasets have demonstrated the promising performance of the proposed TR-LSTM compared with the tensortrain LSTM and other state-of-the-art competitors.

The Development Workflow There are two major stages within isolated word recognition: a training stage and a testing stage. Training involves “teaching” the system by building its dictionary, an acoustic model for each word that the system needs to recognize. In our example, the dictionary comprises the digits ‘zero’ to ‘nine’. In the testing stage we use acoustic models of these digits to recognize isolated words using a classification algorithm. The development workflow consists of three steps: Speech acquisition • Speech analysis • User interface development •

Traditional convolutional neural network (CNN) based query classification uses linear feature mapping in its convolution operation. The recurrent neural network (RNN), differs from a CNN in representing word sequence with their ordering information kept explicitly. We propose using a deep long-short-term-memory (DLSTM) based feature mapping to learn feature representation for CNN. The DLSTM, which is a stack of LSTM units, has different order of feature representations at different depth of LSTM unit. The bottom LSTM unit equipped with input and output gates, extracts the first order feature representation from current word. To extract higher order nonlinear feature representation, the LSTM unit at higher position gets input from two parts. First part is the lower LSTM unit’s memory cell from previous word. Second part is the lower LSTM unit’s hidden output from current word. In this way, the DLSTM captures the nonlinear nonconsecutive interaction within n-grams. Using an architecture that combines a stack of the DLSTM layers with a tradition CNN layer, we have observed new state-of-the-art query classification accuracy on benchmark data sets for query classification.


Question: Long-term short-term memory recurrent neural network structures for large-scale acoustic modeling.

This paper examines the efficacy of employing Pretrained Deep Neural Networks (PDNN) in enhancing Large Vocabulary Speech Recognition (LVSR) systems. Recent advancements in Artificial Neural Network Hidden Markov Model (ANN/HMM) hybrid systems for Automatic Speech Recognition (ASR) have been significantly bolstered by the introduction of Deep Belief Networks (DBN) for neural network pretraining. We present the outcomes of utilizing a DBN-pretrained context-dependent ANN/HMM system trained on exceptionally large datasets, significantly larger than those used in previous studies involving DBN-pretrained ANN/HMM systems. Specifically, our experiments were conducted on two vast datasets: 5870 hours of Voice Search and 1400 hours of YouTube audio. The results demonstrate that the DBN-pretrained ANN/HMM system outperforms the leading Gaussian Mixture Model Hidden Markov Model (GMM/HMM) baseline, which was trained on a considerably larger dataset. On the Voice Search dataset, the pretrained system achieved an absolute Word Error Rate (WER) reduction of 3.7% over the GMM/HMM baseline. Similarly, on the YouTube dataset, it surpassed the baseline by 4.7% absolute WER. Further improvements were attained through Maximum Mutual Information (MMI) fine-tuning and model combination using Segmental Conditional Random Fields (SCARF), leading to additional gains of 0.1% and 0.4% absolute on the Voice Search dataset, and 0.5% and 0.9% absolute on the YouTube dataset, respectively. These findings underscore the potential of PDNN in significantly enhancing LVSR performance on large-scale datasets.

In this paper, we propose a context-aware keyword spotting m odel employing a character-level recurrent neural network (RNN) for spoken term detection in co tinuous speech. The RNN is end-toend trained with connectionist temporal classification (CT C) to generate the probabilities of character and word-boundary labels. There is no need for the phonetic t ranscription, senone modeling, or system dictionary in training and testing. Also, keywords can easi ly be added and modified by editing the text based keyword list without retraining the RNN. Moreover, th e unidirectional RNN processes an infinitely long input audio streams without pre-segmentation and keyw ords are detected with low-latency before the utterance is finished. Experimental results show that th e proposed keyword spotter significantly outperforms the deep neural network (DNN) and hidden Markov m del (HMM) based keyword-filler model even with less computations.

We present our work on end-to-end training of acoustic models using the lattice-free maximum mutual information (LF-MMI) objective function in the context of hidden Markov models. By end-to-end training, we mean flat-start training of a single DNN in one stage without using any previously trained models, forced alignments, or building state-tying decision trees. We use full biphones to enable context-dependent modeling without trees, and show that our end-to-end LF-MMI approach can achieve comparable results to regular LF-MMI on well-known large vocabulary tasks. We also compare with other end-to-end methods such as CTC in character-based and lexicon-free settings and show 5 to 25 percent relative reduction in word error rates on different large vocabulary tasks while using significantly smaller models.

This paper investigates data augmentation based on label-preserving transformations for deep convolutional neural network (CNN) acoustic modeling to deal with limited training data. We show how stochastic feature mapping (SFM) can be carried out when training CNN models with log-Mel features as input and compare it with vocal tract length perturbation (VTLP). Furthermore, a two-stage data augmentation scheme with a stacked architecture is proposed to combine VTLP and SFM as complementary approaches. Improved performance has been observed in experiments conducted on the limited language pack (LLP) of Haitian Creole in the IARPA Babel program.

Recurrent neural networks (RNNs) have shown promising performance for language modeling. However, traditional training of RNNs, using back-propagation through time, often suffers from overfitting. One reason for this is that stochastic optimization (used for large training sets) does not provide good estimates of model uncertainty. This paper leverages recent advances in stochastic gradient Markov Chain Monte Carlo (also appropriate for large training sets) to learn weight uncertainty in RNNs. It yields a principled Bayesian learning algorithm, adding gradient noise during training (enhancing exploration of the model-parameter space) and model averaging when testing. Extensive experiments on various RNN models and across a broad range of applications demonstrate the superiority of the proposed approach relative to stochastic optimization.

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.

This paper describes digital circuit architectures for automatic speech recognition (ASR) and voice activity detection (VAD) with improved accuracy, programmability, and scalability. Our ASR architecture is designed to minimize off-chip memory bandwidth, which is the main driver of system power consumption. A SIMD processor with 32 parallel execution units efficiently evaluates feed-forward deep neural networks (NNs) for ASR, limiting memory usage with a sparse quantized weight matrix format. We argue that VADs should prioritize accuracy over area and power, and introduce a VAD circuit that uses an NN to classify modulation frequency features with 22.3-<inline-formula> <tex-math notation="LaTeX">$\mu \text{W}$ </tex-math></inline-formula> power consumption. The 65-nm test chip is shown to perform a variety of ASR tasks in real time, with vocabularies ranging from 11 words to 145 000 words and full-chip power consumption ranging from 172 <inline-formula> <tex-math notation="LaTeX">$\mu \text{W}$ </tex-math></inline-formula> to 7.78 mW.

Despite the enormous interest in emotion classification from speech, the impact of noise on emotion classification is not well understood. This is important because, due to the tremendous advancement of the smartphone technology, it can be a powerful medium for speech emotion recognition in the outside laboratory natural environment, which is likely to incorporate background noise in the speech. We capitalize on the current breakthrough of Recurrent Neural Network (RNN) and seek to investigate its performance for emotion classification from noisy speech. We particularly focus on the recently proposed Gated Recurrent Unit (GRU), which is yet to be explored for emotion recognition from speech. Experiments conducted with speech compounded with eight different types of noises reveal that GRU incurs an 18.16% smaller run-time while performing quite comparably to the Long Short-Term Memory (LSTM), which is the most popular Recurrent Neural Network proposed to date. This result is promising for any embedded platform in general and will initiate further studies to utilize GRU to its full potential for emotion recognition on smartphones.

Up until recently Recurrent neural networks (RNNs) have been the standard go-to component when processing sequential data with neural networks. Issues relative to vanishing gradient have been partly addressed by Long short-term memory (LSTM) and gated recurrent unit (GRU), but in practice experiments show that very long terms dependencies (beyond 1000 time steps) are difficult to learn. We introduce IGLOO, a new neural network architecture which aims at being faster than both LSTM and GRU but also their respective CuDNN optimized versions when convergence happens and provide an alternative in the case when there is no convergence at all. IGLOO’s core idea is to use the relationships between patches sliced out of the features maps of convolution layers at different levels of granularity to build a representation for the sequence. We show that the model can deal with dependencies of more than 25,000 steps in a reasonable time frame. Beyond the well bench-marked copy-memory and addition problems which show good results, we also achieve best recorded accuracy on permuted MNIST (98.4%). IGLOO is also applied on the IMDB set and on the TUH Abnormal EEG Corpus.

We present Deep Voice 3, a fully-convolutional attention-based neural textto-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural speech synthesis systems in naturalness while training an order of magnitude faster. We scale Deep Voice 3 to dataset sizes unprecedented for TTS, training on more than eight hundred hours of audio from over two thousand speakers. In addition, we identify common error modes of attention-based speech synthesis networks, demonstrate how to mitigate them, and compare several different waveform synthesis methods. We also describe how to scale inference to ten million queries per day on a single GPU server.

We present a state-of-the-art speech recognition system developed using end-toend deep learning. Our architecture is significantly simpler than traditional speech systems, which rely on laboriously engineered processing pipelines; these traditional systems also tend to perform poorly when used in noisy environments. In contrast, our system does not need hand-designed components to model background noise, reverberation, or speaker variation, but instead directly learns a function that is robust to such effects. We do not need a phoneme dictionary, nor even the concept of a “phoneme.” Key to our approach is a well-optimized RNN training system that uses multiple GPUs, as well as a set of novel data synthesis techniques that allow us to efficiently obtain a large amount of varied data for training. Our system, called Deep Speech, outperforms previously published results on the widely studied Switchboard Hub5’00, achieving 16.0% error on the full test set. Deep Speech also handles challenging noisy environments better than widely used, state-of-the-art commercial speech systems.


Question: An evolutionary approach to creating a tic-tac-toe game participant.

In an innovative experiment, multilayer feedforward neural networks were pitted against each other in a survival-of-the-fittest scenario, competing through their prowess in playing checkers. These networks, devoid of any expert knowledge beyond the piece differential, were tasked with evaluating alternative board positions using a minimax search strategy. At each evolutionary stage, networks were paired in competitive matches, with selection weeding out the underperformers. Offspring networks were generated through random variation of weights and bias terms from the surviving networks. After 250 generations of evolution, the most adept network was pitted against human opponents, resulting in a series of 90 games hosted on an internet website. Impressively, this evolved neural network managed to outplay two expert-level humans and held its own against a master, drawing the match. According to a standard rating system, the network was classified as "Class A." The experiment's success lay in the evolutionary process's ability to glean necessary information for competent gameplay, primarily from the end-game outcomes (victory, defeat, or stalemate), without the reliance on explicit expert knowledge. This approach starkly contrasts with the conventional practice of infusing expert insights into game-playing software.

The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

Addressing unresolved questions concerning computational thinking.

In this article we introduce the Arcade Learning Environment (ALE): both a challenge problem and a platform and methodology for evaluating the development of general, domain-independent AI technology. ALE provides an interface to hundreds of Atari 2600 game environments, each one different, interesting, and designed to be a challenge for human players. ALE presents significant research challenges for reinforcement learning, model learning, model-based planning, imitation learning, transfer learning, and intrinsic motivation. Most importantly, it provides a rigorous testbed for evaluating and comparing approaches to these problems. We illustrate the promise of ALE by developing and benchmarking domain-independent agents designed using well-established AI techniques for both reinforcement learning and planning. In doing so, we also propose an evaluation methodology made possible by ALE, reporting empirical results on over 55 different games. All of the software, including the benchmark agents, is publicly available.

Game designers can use computer-aided game design methods to model how players may experience the perceived difficulty of a game. We present methods to generate and analyze the difficulty of a wide variety of minimal action game variants throughout game space, where each point in this abstract design space represents a unique game variant. Focusing on a parameterized version of Flappy Bird, we predict hazard rates and difficulty curves using automatic playtesting, Monte Carlo simulation, a player model based on human motor skills (precision and actions per second), and survival analysis of score histograms. We demonstrate our techniques using simulated game play and actual game data from over 106 million play sessions of a popular online Flappy Bird variant, showing quantitative reasons why balancing a game for a wide range of player skill can be difficult. Some applications of our techniques include searching for a specific difficulty, game space visualization, computational creativity to find unique variants, and tuning game balance to adjust the difficulty curve even when game parameters are time varying, score dependent, or changing based on game progress.

The Turing Test (TT) is claimed by many to be a way to test for the presence, in computers, of such ``deep'' phenomena as thought and consciousness. Unfortunately, attempts to build computational systems able to pass TT (or at least restricted versions of this test) have devolved into shallow symbol manipulation designed to, by hook or by crook, trick. The human creators of such systems know all too well that they have merely tried to fool those people who interact with their systems into believing that these systems really have minds. And the problem is fundamental: the structure of the TT is such as to cultivate tricksters. A better test is one that insists on a certain restrictive epistemic relation between an artificial agent (or system) A, its output o, and the human architect H of A – a relation which, roughly speaking, obtains when H cannot account for how A produced o. We call this test the ``Lovelace Test'' in honor of Lady Lovelace, who believed that only when computers originate things should they be believed to have minds.

We present our digital fabrication technique for manufacturing active objects in 2D from a collection of smart particles. Given a passive model of the object to be formed, we envision submerging this original in a vat of smart particles, executing the new shape duplication algorithm described in this paper, and then brushing aside any extra modules to reveal both the original object and an exact copy, side-by-side. Extensions to the duplication algorithm can be used to create a magnified version of the original or multiple copies of the model object. Our novel duplication algorithm uses a distributed approach to identify the geometric specification of the object being duplicated and then forms the duplicate from spare modules in the vicinity of the original. This paper details the duplication algorithm and the features that make it robust to (1) an imperfect packing of the modules around the original object; (2) missing communication links between neighboring modules; and (3) missing modules in the vicinity of the duplicate object(s). We show that the algorithm requires O(1) storage space per module and that the algorithm exchanges O(n) messages per module. Finally, we present experimental results from 60 hardware trials and 150 simulations. These experiments demonstrate the algorithm working correctly and reliably despite broken communication links and missing modules.

We describe an approach to creating a controller for The Open Car Racing Simulator (TORCS), based on The Simulated Car Racing Championship (SCRC) client, using unsupervised evolutionary learning for recurrent neural networks. Our method of training the recurrent neural network controllers relies on combining the components of the singular value decomposition of two different neural network connection matrices.

Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development. The complexity of the robot's activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby play mat with objects that it can learn to manipulate. Experimental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learned. Finally, these various results are discussed in relation to more complex forms of behavioral organization and data coming from developmental psychology

By classifying programs according to their relationship to the environment in which they are executed, the paper identifies the sources of evolutionary pressure on computer applications and programs and shows why this results in a process of never ending maintenance activity. The resultant life cycle processes are then briefly discussed. The paper then introduces laws of Program Evolution that have been formulated following quantitative studies of the evolution of a number of different systems. Finally an example is provided of the application of Evolution Dynamics models to program release planning.

GUS is the first o f a series o f experimental computer systems that we intend to construct as part o f a program of research on language understanding. In large measure, these systems will fill the role o f periodic progress reports, summarizing what we have learned, assessing the mutual coherence o f the various lines o f investigation we have been following, and saggestin# where more emphasis is needed in future work. GUS (Genial Understander System) is intended to engage a sympathetic and highly cooperative human in an English dialog, directed towards a specific goal within a very restricted domain o f discourse. As a starting point, G US was restricted to the role o f a travel agent in a conversation with a client who wants to make a simple return trip to a single city in California. There is good reason for restricting the domain o f discourse for a computer system which is to engage in an English dialog. Specializing the subject matter that the system can talk about permiis it to achieve some measure o f realism without encompassing all the possibilities o f human knowledge or o f the English language. It also provides the user with specific motivation for participating in the conversation, thus narrowing the range o f expectations that GUS must have about the user's purposes. A system restricted in this way will be more able to guide the conversation within the boundaries o f its competence. 1. Motivation and Design Issues Within its limitations, ous is able to conduct a more-or-less realistic dialog. But the outward behavior of this first system is not what makes it interesting or significant. There are, after all, much more convenient ways to plan a trip and, unlike some other artificial intelligence programs, (;us does not offer services or furnish information that are otherwise difficult or impossible to obtain. The system is i nteresting because of the phenomena of natural dialog that it attempts to model tThis work was done by the language understander project at the Xerox Palo Alto Research center. Additional affiliations: D. A. Norman, University of California, San Diego; H. Thompso6, University of California, Berkeley; and T. Winograd, Stanford University. Artificial Intelligence 8 0977), 155-173 Copyright © 1977 by North-Holland Publishing Company 156 D . G . BOBI~OW ET AL. and because of the principles of program organization around which it was de, Signed. Among the hallmarks of natural dialogs are unexpected and seemingly unpredictable sequences of events. We describe some of the forms that these can take below. "We then go on to discuss the modular design which makes the system re!atively insensitive t o the vagaries of ordinary conversation. 1.1. Problems of natural dialog The simple dialog shown in Fig. 1 illustrates some of the language-understanding problems we attacked. (The parenthesized numbers are for reference in the text). The problems illustrated in this figure, and described in the paragraphs below, include allowing both the client and the system to take the initiative, understanding indirect answers to questions, resolving anaphora, understanding fragments of sentences offered as answers to questions, and interpreting the discourse in the light of known conversational patterns. 1.1.1. Mixed initiative A typical contribution to a dialog, in addition to its more obvious functions, conveys an expectation about how the other participant will respond. This is clearest in the ease of a question, but it is true of all dialog. If one of the participants has very particular expectations and states them strongly whenever he speaks, and ff the other always responds in such a way as to meet the expectations conveyed, then the initiative remains with the first participant throughout. The success of interactive computer systems can often be traced to the skill with which their designers were able to assure them such a dominating position in the interaction. In natural conversations between humans, however, each participant usually assumes the initiative from time to time. Either clear expectations are not stated or simply not honored. GUS attempts to retain the initiative, but not to the extent of jeopardizing the natural flow of the conversation. To this extent it is a mixed-initiative system (see Carbonell [5, 6]). This is exemplified in the dialogue at (1) where the client volunteers more information than GUS requested. In addition to his destination, the client gives the date on which he wants to travel. Line (3) illustrates a ease where the client takes control of the conversation. GUS had found a potentially acceptable flight and asked for the client's approval. Instead of either giving or denying it, the client replied with a question of his own. 1.1.2. Indirect answers It is by no means always clear what constitutes an answer to a question. Frequently the purported answer is at best only a basis on which to infer the information requested. For example, when ous asks "Whatt ime do you want to leave?" it is seeking information to constrain the selection of a flight, client's res onse t o • • P " . this question, a t (2), does constrain the flight selection, b u t only indirectly. In Artificial Intelligence8 (1977), 155--17a : -


Question: An evolutionary approach to playing tic-tac-toe.

This paper explores the effectiveness of co-evolution in teaching a computer to play checkers at an expert level, with minimal human intervention. Anaconda, a neural network, has been co-evolved to the point where, when combined with a minimax search algorithm, it can evaluate checkerboard positions and play at a level equivalent to human experts, as evidenced by its 2045 rating on an international online checkers platform. The neural network relies solely on the board's piece locations, types, and numbers as inputs, eschewing any additional features that might require human expertise. To validate Anaconda's expert rating, experiments were conducted involving a 10-game match against a "novice-level" version of Chinook, a world-renowned checkers program. Anaconda achieved 2 wins, 4 losses, and 4 draws in this match. Based on an estimated novice-level rating for Chinook, these results affirm Anaconda's status as an expert-level player, supporting the notion that co-evolution can produce a self-taught neural network capable of high-level game play. Copyright © 2002 Elsevier Science B.V. All rights reserved.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An Evolutionary Algorithm for the Economic Dispatch Problem with Valve-Point Effect
The economic dispatch problem with valve-point effect is a non-convex optimization problem that arises in power systems operation. This paper presents an evolutionary algorithm that effectively solves this problem. The proposed algorithm uses a combination of genetic operators and local search to navigate the complex search space. The algorithm is tested on several benchmark systems, and the results show that it outperforms other state-of-the-art algorithms in terms of solution quality and computational efficiency. The algorithm's ability to find near-optimal solutions in a reasonable amount of time makes it a practical choice for real-world economic dispatch problems.

**Similar Documents**: This paper introduces an advanced evolutionary algorithm designed to tackle the economic dispatch problem in power systems operation, specifically addressing the valve-point effect. This non-convex optimization challenge is efficiently addressed by the proposed algorithm, which integrates genetic operators with local search techniques to traverse the intricate solution landscape. Rigorous testing on a series of benchmark systems has been conducted, and the outcomes demonstrate that this algorithm surpasses contemporary state-of-the-art methods in both the quality of the solutions generated and computational efficiency. Its capacity to discover solutions that are close to optimal within a feasible time frame positions it as a viable option

The Achilles Heel of stochastic optimization algorithms is getting trapped on local optima. Novelty Search avoids this problem by encouraging a search in all interesting directions. That occurs by replacing a performance objective with a reward for novel behaviors, as defined by a human-crafted, and often simple, behavioral distance function. While Novelty Search is a major conceptual breakthrough and outperforms traditional stochastic optimization on certain problems, it is not clear how to apply it to challenging, high-dimensional problems where specifying a useful behavioral distance function is difficult. For example, in the space of images, how do you encourage novelty to produce hawks and heroes instead of endless pixel static? Here we propose a new algorithm, the Innovation Engine, that builds on Novelty Search by replacing the human-crafted behavioral distance with a Deep Neural Network (DNN) that can recognize interesting differences between phenotypes. The key insight is that DNNs can recognize similarities and differences between phenotypes at an abstract level, wherein novelty means interesting novelty. For example, a novelty pressure in image space does not explore in the low-level pixel space, but instead creates a pressure to create new types of images (e.g. churches, mosques, obelisks, etc.). Here we describe the long-term vision for the Innovation Engine algorithm, which involves many technical challenges that remain to be solved. We then implement a simplified version of the algorithm that enables us to explore some of the algorithm's key motivations. Our initial results, in the domain of images, suggest that Innovation Engines could ultimately automate the production of endless streams of interesting solutions in any domain: e.g. producing intelligent software, robot controllers, optimized physical components, and art.

The aim of this paper is to introduce the use of Tower Defence (TD) games in Computational Intelligence (CI) research. We show how TD games can provide an important test-bed for the often under-represented casual games research area. Additionally, the use of CI in the TD games has the potential to create a more interesting, interactive and ongoing game experience for casual gamers. We present a definition of the current state and development of TD games, and include a classification of TD game components. We then describe some potential ways CI can be used to augment the TD experience. Finally, a prototype TD game based on experience-driven procedural content generation is presented.

Page ii Complex Adaptive Systems John H. Holland, Christopher Langton, and Stewart W. Wilson, advisors Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence, MIT Press edition John H. Holland Toward a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life edited by Francisco J. Varela and Paul Bourgine Genetic Programming: On the Programming of Computers by Means of Natural Selection John R. Koza

Thompson sampling is one of oldest heuristic to address the e xploration / exploitation trade-off, but it is surprisingly unpopular in t he literature. We present here some empirical results using Thompson sampling on simu lated and real data, and show that it is highly competitive. And since this heuris tic is very easy to implement, we argue that it should be part of the standard bas eline to compare against.

Reinforcement Learning methods have been succesfully applied to various optimalization problems. Scaling this up to real world sized problems has however been more of a problem. In this research we apply Reinforcement Learning to the game of Tetris which has a very large state space. We not only try to learn policies for Standard Tetris but try to learn parameterized policies for Generalized Tetris, which varies from game to game in field size and the chance of a single block occuring. In comparison to a non-parameterized policy we find that a parameterized policy is able to outperform the non-parameterized policy. The increased complexity of learning such a policy using the Cross-Entropy method however reaches a limit at which the policy is too complex to learn and performance drops below the non-parameterized policy.

Appel and Jacobson1 presented a fast algorithm for generating every possible move in a given position in the game of Scrabble using a DAWG, a finite automaton derived from the trie of a large lexicon. This paper presents a faster algorithm that uses a GADDAG, a finite automaton that avoids the non-deterministic prefix generation of the DAWG algorithm by encoding a bidirectional path starting from each letter of each word in the lexicon. For a typical lexicon, the GADDAG is nearly five times larger than the DAWG, but generates moves more than twice as fast. This time/space trade-off is justified not only by the decreasing cost of computer memory, but also by the extensive use of move-generation in the analysis of board positions used by Gordon in the probabilistic search for the most appropriate play in a given position within realistic time constraints.

This paper focuses on cyber-security simulations in networks modeled as a Markov game with incomplete information and stochastic elements. The resulting game is an adversarial sequential decision making problem played with two agents, the attacker and defender. The two agents pit one reinforcement learning technique, like neural networks, Monte Carlo learning and Q-learning, against each other and examine their effectiveness against learning opponents. The results showed that Monte Carlo learning with the Softmax exploration strategy is most effective in performing the defender role and also for learning attacking strategies.

Blackjack is a popular card game played in many casinos. The objective of the game is to win money by obtaining a point total higher than the dealer’s without exceeding 21. Determining an optimal blackjack strategy proves to be a difficult challenge due to the stochastic nature of the game. This presents an interesting opportunity for machine learning algorithms. Supervised learning techniques may provide a viable solution, but do not take advantage of the inherent reward structure of the game. Reinforcement learning algorithms generally perform well in stochastic environments, and could utilize blackjack's reward structure. This paper explores reinforcement learning as a means of approximating an optimal blackjack strategy using the Q-learning algorithm.

This work introduces a novel, modular, layered web based platform for managing machine learning experiments on grid-based High Performance Computing infrastructures. The coupling of the communication services offered by the grid, with an administration layer and conventional web server programming, via a data synchronization utility, leads to the straightforward development of a web-based user interface that allows the monitoring and managing of diverse online distributed computing applications. It also introduces an experiment generation and monitoring tool particularly suitable for investigating machine learning in game playing. The platform is demonstrated with experiments for two different games.

The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.


Question: An evolutionary approach to creating a tic-tac-toe game participant.

Innovative Collective Strategy in Tic-tac-toe: Decentralized Coordination Among Players
Consider the classic game of Tic-tac-toe, traditionally played by two individuals aiming to align their symbols across a 3x3 grid. Now, imagine an unconventional scenario where a single player competes against a collective of nine agents, each controlling a single grid cell. In this novel setup, the team of agents must collaboratively strategize despite their individual perspectives and limited information. Our paper introduces a scalable solution that not only applies to this unique Tic-tac-toe variant but also extends to multiplayer configurations and other board games. Central to our approach is a decentralized decision-making model, which initially appears to introduce complexity. Contrary to expectations, we demonstrate that this decentralized framework enables a sophisticated level of decision-making through the synergy of simple components, paradoxically reducing overall system intricacy. This innovative method showcases the potential of decentralized coordination in enhancing strategic gameplay among multiple independent agents.

Games are considered important benchmark tasks of artificial intelligence research. Modern strategic board games can typically be played by three or more people, which makes them suitable test beds for investigating multi-player strategic decision making. Monte-Carlo Tree Search (MCTS) is a recently published family of algorithms that achieved successful results with classical, two-player, perfect-information games such as Go. In this paper we apply MCTS to the multi-player, non-deterministic board game Settlers of Catan. We implemented an agent that is able to play against computer-controlled and human players. We show that MCTS can be adapted successfully to multi-agent environments, and present two approaches of providing the agent with a limited amount of domain knowledge. Our results show that the agent has considerable playing strength when compared to existing heuristics for the game. We conclude that MCTS is suitable for implementing a strong Settlers of Catan player.

Recent advances in deep learning have allowed artificial agents to rival human-level performance on a wide range of complex tasks; however, the ability of these networks to learn generalizable strategies remains a pressing challenge. This critical limitation is due in part to two factors: the opaque information representation in deep neural networks and the complexity of the task environments in which they are typically deployed. Here we propose a novel Hierarchical Q-Network (HQN), motivated by theories of the hierarchical organization of the human prefrontal cortex, that attempts to identify lower dimensional patterns in the value landscape that can be exploited to construct an internal model of rules in simple environments. We draw on combinatorial games, where there exists a single optimal strategy for winning that generalizes across other features of the game, to probe the strategy generalization of the HQN and other reinforcement learning (RL) agents using variations of Wythoff’s game. Traditional RL approaches failed to reach satisfactory performance on variants of Wythoff’s Game; however, the HQN learned heuristic-like strategies that generalized across changes in board configuration. More importantly, the HQN allowed for transparent inspection of the agent’s internal model of the game following training. Our results show how a biologically inspired hierarchical learner can facilitate learning abstract rules to promote robust and flexible action policies in simplified training environments with clearly delineated optimal strategies. 1 ar X iv :1 80 1. 06 68 9v 1 [ cs .A I] 2 0 Ja n 20 18

In this paper we discuss how location-based mobile games can be designed for learning in modern technology enhanced public spaces. We start with the description of the design process and we identify the main challenges faced. We elaborate the case of the game Invisible City: Rebels vs. Spies, a game to be played in a city centre using mobile devices. Through this case we highlight the adaptation of an original party game into a mobile form, the issues we faced and the key aspects conductive to learning in a smart city. It is claimed that creating mobile city games for learning is a new challenge, as our city landscapes are augmented with an increasing number of layers of digital information in which a new generation of city games are played.

An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.

TALE-SPIN, the laat major AI attempt at atory generation, approached the problem of making up atoriea primarily from the perspective of an impartial world simulator. AUTHOR ia a program (under development) which generates atoriea as a creative reaaoner in purauit of her own narrative goals. It is thus intended to aimulate an author's mind aa ahe makes up a story, rather than the world as things happen in it. The four major forces driving the atory generation process, according to the AUTHOR model, are (1) author intentionality, (2) conceptual reformulation, (3) reminding, and (4) the opportunity enhancement metagoal. 1. World Simulation Based Story Generation TALE-SPIN [Meehan, 1976] is a program which made up Aesop-fable-like conceptual atoriea, which it then expressed in natural language. Each atory TALE-SPIN made up waa based on a model of the storyworld, including the characters in it and their goala, peraonalities, and interperaonal relatione. Storiea were apun by working out consequences of the model. Thus, central to TALE-SPIN waa the idea of world simulation. The three levels TALE-SPIN was most concerned with simulating were: 1. character intentionality-e.g., if Joe Bear was hungry, he would try to get something to eat 2. social conatrainta-e.g., if Joe Bear believed Irving Bird to be hia friend, he could aak an outright favor; if Joe Bear believed their relation to be a bit cooler, he would feel obliged to offer something in return (auch aa a worm) 3. phyaical cauaality-e.g., if Irving Bird ate a worm, the worm would cease to exist; after "fooling around", Aa oppoaed to Sheldon Klein's ayetem which only produced atory texts and which ia incapable, in principle, of understanding the atoriea it "makes up" [Klein et el., 1976]. characters felt "wiped out" World simulation, however, fails to reflect the proctaa that an author goes through in making up a atory-storyworlds are developed by authors aa needed, frequently as post hoc justification for eventa that the author has already decided ahe wants as part of the story. As illustration, let us consider the question of why, in the Star Wars aequel, "The Empire Strikes Back", Princess Leia and Solo go to visit Landeau. If one asks a reader (or viewer) this question, the anawer will be along the lines of, "They needed a place to hide from imperial stormtroopers until they got the hyperdrive fixed, and Landeau was an old buddy of Solo's, and ..." • …

Believable agents are autonomous agents that exhibit rich personalities. Interactive dramas take place in virtual worlds inhabited by characters (believable agents) with whom an audience interacts. In the course of this interaction, the audience experiences a story (lives a plot arc). This report presents the research philosophy behind the Oz Project, a research group at CMU that has spent the last ten years studying believable agents and interactive drama. The report then surveys current work from an Oz perspective. This research was partially supported by a grant from Intel Corporation. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of Intel Corporation.

This paper illustrates an artiicial developmental system that is a computationally ee-cient technique for the automatic generation of complex Artiicial Neural Networks (ANN). Artiicial developmental system can develop a graph grammar into a modular ANN made of a combination of more simple subnetworks. A genetic algorithm is used to evolve coded grammars that generates ANNs for controlling a six-legged robot locomotion. A mechanism for the automatic deenition of sub-neural networks is incorporated. Using this mechanism, the genetic algorithm can automatically decompose a problem into subproblems, generate a subANN for solving the subproblem, and instantiate copies of this subANN to build a higher level ANN that solves the problem. We report some simulation results showing that the same problem cannot be solved if the mechanism for automatic deenition of sub-networks is suppressed. We support our argumentation with pictures describing the steps of development, how ANN structures are evolved, and how the ANNs compute.matic deention of sub-neural networks.

Evaluating agent performance when outcomes are stochastic and agents use randomized strategies can be challenging when there is limited data available. The variance of sampled outcomes may make the simple approach of Monte Carlo sampling inadequate. This is the case for agents playing heads-up no-limit Texas hold’em poker, where manmachine competitions typically involve multiple days of consistent play by multiple players, but still can (and sometimes did) result in statistically insignificant conclusions. In this paper, we introduce AIVAT, a low variance, provably unbiased value assessment tool that exploits an arbitrary heuristic estimate of state value, as well as the explicit strategy of a subset of the agents. Unlike existing techniques which reduce the variance from chance events, or only consider game ending actions, AIVAT reduces the variance both from choices by nature and by players with a known strategy. The resulting estimator produces results that significantly outperform previous state of the art techniques. It was able to reduce the standard deviation of a Texas hold’em poker man-machine match by 85% and consequently requires 44 times fewer games to draw the same statistical conclusion. AIVAT enabled the first statistically significant AI victory against professional poker players in no-limit hold’em. Furthermore, the technique was powerful enough to produce statistically significant results versus individual players, not just an aggregate pool of the players. We also used AIVAT to analyze a short series of AI vs human poker tournaments, producing statistical significant results with as few as 28 matches.

In this paper, I will describe my intuitions about what makes computer games fun. More detailed descriptions of the experiments and the theory on which this paper is based are given by Malone (1980a, 1980b). My primary goal here is to provide a set of heuristics or guidelines for designers of instructional computer games. I have articulated and organized common sense principles to spark the creativity of instructional designers (see Banet, 1979, for an unstructured list of similar principles). To demonstrate the usefulness of these principles, I have included several applications to actual or proposed instructional games. Throughout the paper I emphasize games with educational uses, but I focus on what makes the games fun, not on what makes them educational.
 Though I will not emphasize the point in this paper, these same ideas can be applied to other educational environments and life situations. In a sense, the categories I will describe constitute a general taxonomy of intrinsic motivation—of what makes an activity fun or rewarding for its own sake rather than for the sake of some external reward (See Lepper and Greene, 1979).
 I think the essential characteristics of good computer games and other intrinsically enjoyable situations can be organized into three categories: challenge, fantasy, and curiosity.

This article provides a historical overview of educational computing research at MIT from the mid-1960s to the present day, focusing on physical interfaces. It discusses some of the results of this research: electronic toys that help children develop advanced modes of thinking through free-form play. In this historical context, the article then describes and discusses the author’s own research into tangible programming, culminating in the development of the Tangible Programming Bricks system—a platform for creating microworlds for children to explore computation and scientific thinking.


Question: An evolutionary approach to creating a tic-tac-toe game participant.

This paper presents a self-learning evolutionary approach applied to the game of chess, aiming to create a program capable of learning from its experiences and achieving expertise comparable to human players. The core methodology involves an evolutionary algorithm that enables the program to enhance its chess-playing skills through self-play. The program's evaluation of chessboard configurations integrates the positions of pieces, material values, positional advantages, and employs neural networks to analyze critical segments of the board. Throughout the evolutionary process, the program's skill level advanced significantly, with an improvement of nearly 400 rating points. When subjected to tournament-like conditions and pitted against Pocket Fritz 2.0, the evolved program demonstrated performance exceeding the master level, showcasing the effectiveness of the evolutionary learning strategy in complex strategic games.

One of the key steps in creating quality interactive drama is the ability to create quality interactive characters (or believable agents). Two important aspects of such characters will be that they appear emotional and that they can engage in social interactions. My basic approach to these problems has been to use a broad agent architecture and minimal amounts of modeling of other agent in the environment. This approach is based on an understanding of the artistic nature of the problem. To enable agent-builders (artists) to create emotional agents, I provide a general framework for building emotional agents, default emotion-processing rules, and discussion about how to create quality, emotional characters. My framework gets a lot of its power from being part of a broad agent architecture. The concept is simple: the agent will be emotionally richer if there are more things to have emotions about and more ways to express them. This reliance on breadth has also meant that I have been able to create simple emotion models that rely on perception and motivation instead of deep modeling of other agents and complex cognitive processing. To enable agent builders to create social behaviors for believable agents, I have designed a methodology that provides heuristics for incorporating personality into social behaviors and suggests how to model other agents in the environment. I propose an approach to modeling other agents that calls for limiting the amount of modeling of other agents to that which is sufficient to create the desired behavior. Using this technique, I have been able to build robust social behaviors that use surprisingly little representation. I have used this methodology to build a number of social behaviors, like negotiation and making friends. I have built three simulations containing seven agents to drive and test this work. I have also conducted user studies to demonstrate that these agents appear to be emotional and can engage in non-trivial social interactions while also being good characters with distinct personalities.

In interactive storytelling systems and other story-based computer games, a drama manager is a background agent that aims to bring about an enjoyable and coherent experience for the players. In this paper, we present a personalized drama manager that increases a player’s expected enjoyment without removing player agency. Our personalized drama manager models a player’s preference using data-driven techniques, predicts the probability the player transitioning to different story experiences, selects an objective experience that can maximize the player’s expected enjoyment, and guides the player to the selected story experience. Human study results show that our drama manager can significantly increase players’ enjoyment ratings in an interactive storytelling testbed, compared to drama managers in pre-

We report the most recent advances in The Painting Fool project, where we have integrated machine vision capabilities from the DARCI system into the automated painter, to enhance its abilities before, during and after the painting process. This has enabled new art projects, including a commission from an Artificial Intelligence company, and we report on this collaboration, which is one of the first instances in Computational Creativity research where creative software has been commissioned directly. The new projects have advanced The Painting Fool as an independent artist able to produce more diverse styles which break away from simulating natural media. The projects have also raised a philosophical question about whether software artists need to see in the same way as people, which we discuss briefly.

This article provides a historical overview of educational computing research at MIT from the mid-1960s to the present day, focusing on physical interfaces. It discusses some of the results of this research: electronic toys that help children develop advanced modes of thinking through free-form play. In this historical context, the article then describes and discusses the author’s own research into tangible programming, culminating in the development of the Tangible Programming Bricks system—a platform for creating microworlds for children to explore computation and scientific thinking.

This paper summarises the history of the terms ecology and ecosystem, before examining their application in the early and recent literature of A-Life agent-based software simulation. It investigates trends in A-Life that have led to a predominance of simulations incorporating artificial evolution acting on generic agents, but lacking a level of detail that would allow the emergence of phenomena relating to the transfer and transformation of energy and matter between the virtual abiotic environment and biota. Implications of these characteristics for the relevance of A-Life’s virtual ecosystem models to Ecology are discussed. We argue a position that the inclusion of low-level representations of energetics, matter and evolution, in concert with pattern-oriented modelling techniques from Ecology for model validation, will improve the relevance of A-Life models to Ecology. We also suggest two methods that may allows us to meet this goal: artificial evolution can be employed as a mechanism for automating pattern-oriented ecological modelling from the level of individual species up to that of the ecosystem, or it may be employed to explore general principles of ecosystem behaviour over evolutionary time periods.

Geometric crossover is a representation-independent definition of crossover based on the distance of the search space interpreted as a metric space. It generalizes the traditional crossover for binary strings and other important recombination operators for the most used representations. Using a distance tailored to the problem at hand, the abstract definition of crossover can be used to design new problem specific crossovers that embed problem knowledge in the search. In recent work, we have introduced the important notion of product geometric crossover that enables the construction of new geometric crossovers combining preexisting geometric crossovers in a simple way. In this paper, we use it to design an evolutionary algorithm to solve the Sudoku puzzle. The different types of constraints make Sudoku an interesting study case for crossover design. We conducted extensive experimental testing and found that, on medium and hard problems, the new geometric crossovers perform significantly better than hill-climbers and mutations alone.

Layout planning is a process of sizing and placing rooms (e.g. in a house) while attempting to optimize various criteria. Often there are conflicting criteria such as construction cost, minimizing the distance between related activities, and meeting the area requirements for these activities. The process of layout planning has mostly been done by hand, with a handful of attempts to automate the process. This thesis explores some of these past attempts and describes several new techniques for automating the layout planning process using evolutionary computation. These techniques are inspired by the existing methods, while adding some of their own innovations. Additional experiments are done to test the possibility of allowing polygonal exteriors with rectilinear interior walls. Several multi-objective approaches are used to evaluate and compare fitness. The evolutionary representation and requirements specification used provide great flexibility in problem scope and depth and is worthy of considering in future layout and design attempts. The system outlined in this thesis is capable of evolving a variety of floor plans conforming to functional and geometric specifications. Many of the resulting plans look reasonable even when compared to a professional floor plan. Additionally polygonal and multi-floor buildings were also generated.

Serious games receive increasing interest in the area of e-learning. Their development, however, is often still a demanding, specialized and arduous process, especially when regarding reasonable non-player character behaviour. Reinforcement learning and, since recently, also deep reinforcement learning have proven to automatically generate successful AI behaviour to a certain degree. These methods are computationally expensive and hardly scalable to various complex serious game scenarios. For this reason, we introduce a new approach of augmenting the application of deep reinforcement learning methods by interactively making use of domain experts' knowledge to guide the learning process. Thereby, we aim to create a synergistic combination of experts and emergent cognitive systems. We call this approach interactive deep reinforcement learning and point out important aspects regarding realization within a framework.

In this paper, the early design of our selforganized agent-based simulation model for exploration of synaptic connections that faithfully generates what is observed in natural situation is given. While we take inspiration from neuroscience, our intent is not to create a veridical model of processes in neurodevelopmental biology, nor to represent a real biological system. Instead, our goal is to design a simulation model that learns acting in the same way of human nervous system by using findings on human subjects using reflex methodologies in order to estimate unknown connections.

Matchmaking connects multiple players to participate in online player-versus-player games. Current matchmaking systems depend on a single core strategy: create fair games at all times. These systems pair similarly skilled players on the assumption that a fair game is best player experience. We will demonstrate, however, that this intuitive assumption sometimes fails and that matchmaking based on fairness is not optimal for engagement. In this paper, we propose an Engagement Optimized Matchmaking (EOMM) framework that maximizes overall player engagement. We prove that equal-skill based matchmaking is a special case of EOMM on a highly simplified assumption that rarely holds in reality. Our simulation on real data from a popular game made by Electronic Arts,Inc. (EA) supports our theoretical results, showing significant improvement in enhancing player engagement compared to existing matchmaking methods.


Question: An evolutionary approach to playing tic-tac-toe.

This review paper explores the field of machine learning as an artificial intelligence methodology, with a focus on the acquisition of structural descriptions through examples, particularly concerning the learning of single concept characteristics. It delves into techniques for identifying the most specific conjunctive generalizations (MSC-generalizations) that encompass all training instances of a given concept. The paper examines several key aspects of structural learning and introduces criteria for assessing learning methods. These criteria encompass: (i) the adequacy of the representation language, (ii) the generalization rules used, (iii) computational efficiency, and (iv) the flexibility and extensibility of the methods. A comparative analysis is conducted on selected learning algorithms developed by Buchanan and colleagues, Hayes-Roth, Vere, Winston, and the authors of the review, evaluating them against the aforementioned criteria. The paper concludes by outlining potential directions for future research in the domain of structural learning.

**Question**: A Deep Learning Approach for Automatic Detection of Epileptic Seizures in EEG Signals

**Documents**: A new method for the analysis of time series with application to geomagnetic data
The method described is based on a least squares fit of a series of sinusoidal functions to the data. The coefficients of the sinusoids are determined by an iterative procedure which is analogous to the Newton-Raphson method for finding the roots of a nonlinear equation. The method has the advantage that it can be applied to data which are not evenly spaced in time. It is also applicable to the case where the data are correlated. The method has been applied to geomagnetic data from a number of observatories. The results obtained are compared with those obtained by the method of least squares spectral analysis. The method described here is found to be more accurate and less sensitive to the choice of the number of sinusoids to be fitted.

**Similar Documents**: This paper introduces a novel technique for time series analysis, specifically designed to accommodate data that may not be uniformly spaced and may exhibit correlations. The approach is founded on a least squares fitting of sinusoidal functions to the dataset. The coefficients for these sinusoids are calculated through an iterative process akin to the Newton-Raphson technique for solving nonlinear equations. A key advantage of this method is its applicability to irregularly spaced and correlated datasets, unlike conventional techniques. The efficacy of this method has been validated through its application to geomagnetic data from various observatories. Comparative analysis with the least squares spectral analysis method reveals that the proposed technique offers superior accuracy and is less influenced by the selection of the number of sinusoids to fit

Card games are interesting for many reasons besides their connection with gambling. Bridge is being a game of imperfect information, it is a well defined, decision making game. The estimation of the number of tricks to be taken by one pair of bridge players is called Double Dummy Bridge Problem (DDBP). Artificial Neural Networks are Non – Linear mapping structures based on the function of the human brain. Feed Forward Neural Network is used to solve the DDBP in contract bridge. The learning methodology, supervised learning was used in Back – Propagation Network (BPN) for training and testing the bridge sample deal. In our study we compared back – Propagation algorithm and obtained that Resilient Back – Propagation algorithms by using Hyperbolic Tangent function and Resilient Back – Propagation algorithm produced better result than the other. Among various neural network architectures, in this study we used four network architectures viz., 26x4, 52, 104 and 52x4 for solving DDBP in contract bridge.

Game designs often center on the game mechanics— rules governing the logical evolution of the game. We seek to develop an intelligent system that generates computer games. As first steps towards this goal we present a composable and cross-domain representation for game mechanics that draws from AI planning action representations. We use a constraint solver to generate mechanics subject to design requirements on the form of those mechanics—what they do in the game. A planner takes a set of generated mechanics and tests whether those mechanics meet playability requirements—controlling how mechanics function in a game to affect player behavior. We demonstrate our system by modeling and generating mechanics in a role-playing game, platformer game, and combined role-playing-platformer game.

We present a game-based interface for acquiring common sense knowledge. In addition to being interactive and entertaining, our interface guides the knowledge acquisition process to learn about the most salient characteristics of a particular concept. We use statistical classification methods to discover the most informative characteristics in the Open Mind Common Sense knowledge base, and use these characteristics to play a game of 20 Questions with the user. Our interface also allows users to enter knowledge more quickly than a more traditional knowledge-acquisition interface. An evaluation showed that users enjoyed the game and that it increased the speed of knowledge acquisition.

Evolutionary computation (EC) is a kind of optimization methodology inspired by the mechanisms of biological evolution and behaviors of living organisms. In the literature, the terminology evolutionary algorithms is frequently treated the same as EC. This article focuses on making a survey of researches based on using ML techniques to enhance EC algorithms. In the framework of an ML-technique enhanced-EC algorithm (MLEC), the main idea is that the EC algorithm has stored ample data about the search space, problem features, and population information during the iterative search process, thus the ML technique is helpful in analyzing these data for enhancing the search performance. The paper presents a survey of five categories: ML for population initialization, ML for fitness evaluation and selection, ML for population reproduction and variation, ML for algorithm adaptation, and ML for local search.

Korf, R.E., Multi-player alpha-beta pruning (Research Note), Artificial Intelligence 48 (1991) 99-111. We consider the generalization of minimax search with alpha-beta pruning to non-cooperative, perfect-information games with more than two players. The minimax algorithm was generalized in [2] to the maxn algorithm applied to vectors of n-tuples representing the evaluations for each of the players. If we assume an upper bound on the sum of the evaluations for each player, and a lower bound on each individual evaluation, then shallow alpha-beta pruning is possible, but not deep pruning. In the best case, the asymptotic branching factor is reduced to (1 + 4bv'~b-"s~-3)/2. In the average case, however, pruning does not reduce the asymptotic branching factor. Thus, alpha-beta pruning is found to be effective only in the special case of two-player games. In addition, we show that it is an optimal directional algorithm for two players.

Good computer and video games like System Shock 2, Deus Ex, Pikmin, Rise of Nations, Neverwinter Nights, and Xenosaga: Episode 1 are learning machines. They get themselves learned and learned well, so that they get played long and hard by a great many people. This is how they and their designers survive and perpetuate themselves. If a game cannot be learned and even mastered at a certain level, it won't get played by enough people, and the company that makes it will go broke. Good learning in games is a capitalist-driven Darwinian process of selection of the fittest. Of course, game designers could have solved their learning problems by making games shorter and easier, by dumbing them down, so to speak. But most gamers don't want short and easy games. Thus, designers face and largely solve an intriguing educational dilemma, one also faced by schools and workplaces: how to get people, often young people, to learn and master something that is long and challenging--and enjoy it, to boot.

In this paper, the effect of Genetic algorithm (GA) on the enhancement of the texture and quality of plants visual L. Davis, The handbook of Genetic Algorithms. Genetic Algorithm is one of many methods that can be used to create a schedule. This method Davis, L.: Handbook on Genetic Algorithms. Van Nostrand. In this research, genetic algorithm approach has been applied for solving exam (18) Davis L., Handbook of Genetic Algorithms, New York, Van Nostrand.

In this paper we develop a story generator that leverages knowledge inherent in corpora without requiring extensive manual involvement. A key feature in our approach is the reliance on a story planner which we acquire automatically by recording events, their participants, and their precedence relationships in a training corpus. Contrary to previous work our system does not follow a generate-and-rank architecture. Instead, we employ evolutionary search techniques to explore the space of possible stories which we argue are well suited to the story generation task. Experiments on generating simple children’s stories show that our system outperforms previous data-driven approaches.

This paper introduces RoxyBot, one of the top-scoring agents in the First International Trading Agent Competition. A TAC agent simulates one vision of future travel agents: it represents a set of clients in simultaneous auctions, trading complementary (e.g., airline tickets and hotel reservations) and substitutable (e.g., symphony and theater tickets) goods. RoxyBot faced two key technical challenges in TAC: (i) allocation---assigning purchased goods to clients at the end of a game instance so as to maximize total client utility, and (ii) completion---determining the optimal quantity of each resource to buy and sell given client preferences, current holdings, and market prices. For the dimensions of TAC, an optimal solution to the allocation problem is tractable, and RoxyBot uses a search algorithm based on A* to produce optimal allocations. An optimal solution to the completion problem is also tractable, but in the interest of minimizing bidding cycle time, RoxyBot solves the completion problem using beam search, producing approximately optimal completions. RoxyBot's completer relies on an innovative data structure called a priceline.

This paper puts forward two useful methods for self-adaptation of the mutation distribution - the concepts of derandomization and cumulation. Principle shortcomings of the concept of mutative strategy parameter control and two levels of derandomization are reviewed. Basic demands on the self-adaptation of arbitrary (normal) mutation distributions are developed. Applying arbitrary, normal mutation distributions is equiv-alent to applying a general, linear problem encoding. The underlying objective of mutative strategy parameter control is roughly to favor previously selected mutation steps in the future. If this objective is pursued rigor-ously, a completely derandomized self-adaptation scheme results, which adapts arbitrary normal mutation distributions. This scheme, called covariance matrix adaptation (CMA), meets the previously stated demands. It can still be considerably improved by cumulation - utilizing an evolution path rather than single search steps. Simulations on various test functions reveal local and global search properties of the evolution strategy with and without covariance matrix adaptation. Their performances are comparable only on perfectly scaled functions. On badly scaled, non-separable functions usually a speed up factor of several orders of magnitude is ob-served. On moderately mis-scaled functions a speed up factor of three to ten can be expected.