Question: Enhancing Chinese word embeddings through the utilization of internal structure.

This paper introduces an innovative learning technique, back-propagation, designed for neural network architectures composed of units akin to neurons. Through an iterative process, back-propagation fine-tunes the weights of the network's connections, aiming to reduce the discrepancy between the network's actual output and the desired output. This adjustment process enables the emergence of internal 'hidden' units, which, despite not being directly involved in input or output, learn to represent critical features of the task domain. The network captures the underlying patterns of the task through the interactions of these hidden units, distinguishing back-propagation from previous, more rudimentary techniques, such as the perceptron-convergence method, by its capacity to generate and utilize new, meaningful features.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An efficient heuristic algorithm for economic dispatch with valve-point effects
In this paper, a novel heuristic algorithm, named Improved Differential Evolution (IDE), is proposed to solve the economic dispatch problem with valve-point effects. The proposed IDE algorithm introduces a mutation strategy and a local search mechanism to enhance the search capability and convergence speed of the original Differential Evolution (DE) algorithm. The mutation strategy allows the algorithm to explore the solution space more effectively, while the local search mechanism helps to refine the solutions found, leading to higher quality results. The effectiveness of the IDE algorithm is demonstrated by solving the economic dispatch problem with valve-point effects for systems with 10 and 30 generators. The results obtained by the IDE algorithm are compared with those obtained by other heuristic algorithms, such as Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and original DE algorithm. The comparison shows that the IDE algorithm outperforms other algorithms in terms of solution quality and convergence speed.

**Similar Documents**: This work presents a sophisticated heuristic approach, the Enhanced Differential Evolution (EDE) algorithm, to address the economic dispatch problem in the presence of valve-point effects. The EDE algorithm incorporates a mutation strategy and an enhanced local search mechanism to augment the search efficiency and accelerate the convergence rate of the standard Differential Evolution (DE) algorithm. The mutation strategy facilitates a more thorough exploration of the solution space, and the refined local search mechanism contributes to the optimization of discovered solutions, resulting in superior outcomes. The EDE algorithm's efficacy is showcased through its application to the economic dispatch problem with valve-point effects for power systems comprising 10 and 30 generators. Comparative analysis against conventional heuristic algorithms, including Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and

Word embedding has been proven to improve performance of sentiment classification systems. A new proposed word embedding called sentiment-specific word embedding exploits sentiment information, in order to reach better performance of sentiment analysis systems using word embedding. In this paper, we build Indonesian sentiment-specific word embedding and apply it for sentiment analysis. We compare performance of sentiment classification based on sentiment-specific word embedding with other popular feature representations such as bag of words, Term Frequency-Inverse Document Frequency (TF-IDF), and also generic word embedding. Although our sentiment-specific word embedding models achieve a higher score than the Word2Vec embedding, the lexical representations, bag of words and TF-IDF still gained better performance.

Recent methods for learning vector space representations of words have succeeded in capturing fine-grained semantic and syntactic regularities using vector arithmetic, but the origin of these regularities has remained opaque. We analyze and make explicit the model properties needed for such regularities to emerge in word vectors. The result is a new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods. Our model efficiently leverages statistical information by training only on the nonzero elements in a word-word cooccurrence matrix, rather than on the entire sparse matrix or on individual context windows in a large corpus. The model produces a vector space with meaningful substructure, as evidenced by its performance of 75% on a recent word analogy task. It also outperforms related models on similarity tasks and named entity recognition.

WordNet is perhaps the most important and widely used lexical resource for natural language processing systems up to now. WordNet: An Electronic Lexical Database, edited by Christiane Fellbaum, discusses the design of WordNet from both theoretical and historical perspectives, provides an up-to-date description of the lexical database, and presents a set of applications of WordNet. The book contains a foreword by George Miller, an introduction by Christiane Fellbaum, seven chapters from the Cognitive Sciences Laboratory of Princeton University, where WordNet was produced, and nine chapters contributed by scientists from elsewhere. Miller's foreword offers a fascinating account of the history of WordNet. He discusses the presuppositions of such a lexical database, how the top-level noun categories were determined, and the sources of the words in WordNet. He also writes about the evolution of WordNet from its original incarnation as a dictionary browser to a broad-coverage lexicon, and the involvement of different people during its various stages of development over a decade. It makes very interesting reading for casual and serious users of WordNet and anyone who is grateful for the existence of WordNet. The book is organized in three parts. Part I is about WordNet itself and consists of four chapters: "Nouns in WordNet" by George Miller, "Modifiers in WordNet" by Katherine Miller, "A semantic network of English verbs" by Christiane Fellbaum, and "Design and implementation of the WordNet lexical database and search software" by Randee Tengi. These chapters are essentially updated versions of four papers from Miller (1990). Compared with the earlier papers, the chapters in this book focus more on the underlying assumptions and rationales behind the design decisions. The description of the information contained in WordNet, however, is not as detailed as in Miller (1990). The main new additions in these chapters include an explanation of sense grouping in George Miller's chapter, a section about adverbs in Katherine Miller's chapter, observations about autohyponymy (one sense of a word being a hyponym of another sense of the same word) and autoantonymy (one sense of a word being an antonym of another sense of the same word) in Fellbaum's chapter, and Tengi's description of the Grinder, a program that converts the files the lexicographers work with to searchable lexical databases. The three papers in Part II are characterized as "extensions, enhancements and

Machine Reading Comprehension (MRC) has become enormously popular recently and has attracted a lot of attention. However, the existing reading comprehension datasets are mostly in English. In this paper, we introduce a SpanExtraction dataset for Chinese Machine Reading Comprehension to add language diversities in this area. The dataset is composed by near 20,000 real questions annotated by human on Wikipedia paragraphs. We also annotated a challenge set which contains the questions that need comprehensive understanding and multi-sentence inference throughout the context. With the release of the dataset, we hosted the Second Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC 2018). We hope the release of the dataset could further accelerate the machine reading comprehension research in Chinese language.1

This paper develops a model that addresses sentence embedding, a hot topic in current natural language processing research, using recurrent neural networks (RNN) with Long Short-Term Memory (LSTM) cells. The proposed LSTM-RNN model sequentially takes each word in a sentence, extracts its information, and embeds it into a semantic vector. Due to its ability to capture long term memory, the LSTM-RNN accumulates increasingly richer information as it goes through the sentence, and when it reaches the last word, the hidden layer of the network provides a semantic representation of the whole sentence. In this paper, the LSTM-RNN is trained in a weakly supervised manner on user click-through data logged by a commercial web search engine. Visualization and analysis are performed to understand how the embedding process works. The model is found to automatically attenuate the unimportant words and detect the salient keywords in the sentence. Furthermore, these detected keywords are found to automatically activate different cells of the LSTM-RNN, where words belonging to a similar topic activate the same cell. As a semantic representation of the sentence, the embedding vector can be used in many different applications. These automatic keyword detection and topic allocation abilities enabled by the LSTM-RNN allow the network to perform document retrieval, a difficult language processing task, where the similarity between the query and documents can be measured by the distance between their corresponding sentence embedding vectors computed by the LSTM-RNN. On a web search task, the LSTM-RNN embedding is shown to significantly outperform several existing state of the art methods. We emphasize that the proposed model generates sentence embedding vectors that are specially useful for web document retrieval tasks. A comparison with a well known general sentence embedding method, the Paragraph Vector, is performed. The results show that the proposed method in this paper significantly outperforms Paragraph Vector method for web document retrieval task.

In this work we present a novel approach for the utilization of observed relations between entity pairs in the task of triple argument prediction. The approach is based on representing observations in a shared, continuous vector space of structured relations and text. Results on a recent benchmark dataset demonstrate that the new model is superior to existing sparse feature models. In combination with state-of-the-art models, we achieve substantial improvements when observed relations are available.

We present a family of neural-network– inspired models for computing continuous word representations, specifically designed to exploit both monolingual and multilingual text. This framework allows us to perform unsupervised training of embeddings that exhibit higher accuracy on syntactic and semantic compositionality, as well as multilingual semantic similarity, compared to previous models trained in an unsupervised fashion. We also show that such multilingual embeddings, optimized for semantic similarity, can improve the performance of statistical machine translation with respect to how it handles words not present in the parallel data.

We study the topmost weight matrix of neural network language models. We show that this matrix constitutes a valid word embedding. When training language models, we recommend tying the input embedding and this output embedding. We analyze the resulting update rules and show that the tied embedding evolves in a more similar way to the output embedding than to the input embedding in the untied model. We also offer a new method of regularizing the output embedding. Our methods lead to a significant reduction in perplexity, as we are able to show on a variety of neural network language models. Finally, we show that weight tying can reduce the size of neural translation models to less than half of their original size without harming their performance.

We introduce a novel latent vector space model that jointly learns the latent representations of words, e-commerce products and a mapping between the two without the need for explicit annotations. The power of the model lies in its ability to directly model the discriminative relation between products and a particular word. We compare our method to existing latent vector space models (LSI, LDA and word2vec) and evaluate it as a feature in a learning to rank setting. Our latent vector space model achieves its enhanced performance as it learns better product representations. Furthermore, the mapping from words to products and the representations of words benefit directly from the errors propagated back from the product representations during parameter estimation. We provide an in-depth analysis of the performance of our model and analyze the structure of the learned representations.

This paper proposes a Hidden Markov Model (HMM) and an HMM-based chunk tagger, from which a named entity (NE) recognition (NER) system is built to recognize and classify names, times and numerical quantities. Through the HMM, our system is able to apply and integrate four types of internal and external evidences: 1) simple deterministic internal feature of the words, such as capitalization and digitalization; 2) internal semantic feature of important triggers; 3) internal gazetteer feature; 4) external macro context feature. In this way, the NER problem can be resolved effectively. Evaluation of our system on MUC-6 and MUC-7 English NE tasks achieves F-measures of 96.6% and 94.1% respectively. It shows that the performance is significantly better than reported by any other machine-learning system. Moreover, the performance is even consistently better than those based on handcrafted rules.


Question: Enhance Chinese word embeddings through the utilization of internal structure.

In recent years, the development of neural probabilistic language models has significantly advanced the field of natural language processing, particularly in enhancing the representation of words in a vector space. These models, which employ neural networks, are capable of capturing the complex relationships between words, thereby generating more accurate and contextually relevant word embeddings. By considering the internal structure of Chinese words, a novel approach can be introduced to refine and improve these embeddings specifically for the Chinese language.

Neural probabilistic language models are designed to predict the probability of a word given its surrounding context. This predictive capability is achieved through the use of hidden layers in the neural network, which learn to represent the meaning of words in a high-dimensional vector space. The vectors, or embeddings, produced by these models have proven to be effective in a variety of tasks, including machine translation, sentiment analysis, and text classification.

For the Chinese language, which is characterized by its logographic script and rich internal structure, traditional word embeddings might not fully capture the nuanced meanings and relationships between characters and words. By exploiting the internal structure of Chinese words, such as radicals and components, it is possible to create more sophisticated and informative embeddings. This can be achieved by augmenting neural probabilistic language models with additional layers or mechanisms that specifically target the unique features of Chinese language structure.

For instance, one could incorporate a character-level convolutional neural network (CNN) or a recurrent neural network (RNN) to analyze the internal components of Chinese words before feeding them into the main language model. This approach allows the model to learn not only the context in which a word is used but also the inherent meaning derived from its constituent characters. As a result, the embeddings generated by such an enhanced model would be more nuanced and capable of capturing the subtleties of the Chinese language.

Moreover, by integrating linguistic knowledge about Chinese grammar and syntax into the model, the embeddings can be further optimized to reflect the language's syntactic and semantic properties. This could involve using attention mechanisms to highlight the importance of certain characters or components within a word, or incorporating a hierarchical structure to model the relationships between different levels of linguistic units.

In conclusion, by leveraging the internal structure of Chinese words and integrating linguistic knowledge, neural probabilistic language models can be enhanced to produce more accurate and contextually rich word embeddings for the Chinese language. This advancement will not only improve the performance of natural language processing tasks specific to Chinese but also contribute to a deeper understanding of the language's complex structure and meaning.

This paper investigates neural characterbased morphological tagging for languages with complex morphology and large tag sets. Character-based approaches are attractive as they can handle rarelyand unseen words gracefully. We evaluate on 14 languages and observe consistent gains over a state-of-the-art morphological tagger across all languages except for English and French, where we match the state-of-the-art. We compare two architectures for computing characterbased word vectors using recurrent (RNN) and convolutional (CNN) nets. We show that the CNN based approach performs slightly worse and less consistently than the RNN based approach. Small but systematic gains are observed when combining the two architectures by ensembling.

We describe a simple neural language model that relies only on character-level inputs. Predictions are still made at the word-level. Our model employs a convolutional neural network (CNN) and a highway network over characters, whose output is given to a long shortterm memory (LSTM) recurrent neural network language model (RNN-LM). On the English Penn Treebank the model is on par with the existing state-ofthe-art despite having 60% fewer parameters. On languages with rich morphology (Czech, German, French, Spanish, Russian), the model consistently outperforms a Kneser-Ney baseline and word-level/morpheme-level LSTM baselines, again with far fewer parameters. Our results suggest that on many languages, character inputs are sufficient for language modeling.

Because of their superior ability to preserve sequence information over time, Long Short-Term Memory (LSTM) networks, a type of recurrent neural network with a more complex computational unit, have obtained strong results on a variety of sequence modeling tasks. The only underlying LSTM structure that has been explored so far is a linear chain. However, natural language exhibits syntactic properties that would naturally combine words to phrases. We introduce the Tree-LSTM, a generalization of LSTMs to tree-structured network topologies. TreeLSTMs outperform all existing systems and strong LSTM baselines on two tasks: predicting the semantic relatedness of two sentences (SemEval 2014, Task 1) and sentiment classification (Stanford Sentiment Treebank).

This paper proposes a novel approach to train word embeddings to capture antonyms. Word embeddings have shown to capture synonyms and analogies. Such word embeddings, however, cannot capture antonyms since they depend on the distributional hypothesis. Our approach utilizes supervised synonym and antonym information from thesauri, as well as distributional information from large-scale unlabelled text data. The evaluation results on the GRE antonym question task show that our model outperforms the state-of-the-art systems and it can answer the antonym questions in the F-score of 89%.

Previous studies have shown that semantically meaningful representations of words and text can be acquired through neural embedding models. In particular, paragraph vector (PV) models have shown impressive performance in some natural language processing tasks by estimating a document (topic) level language model. Integrating the PV models with traditional language model approaches to retrieval, however, produces unstable performance and limited improvements. In this paper, we formally discuss three intrinsic problems of the original PV model that restrict its performance in retrieval tasks. We also describe modifications to the model that make it more suitable for the IR task, and show their impact through experiments and case studies. The three issues we address are (1) the unregulated training process of PV is vulnerable to short document over-fitting that produces length bias in the final retrieval model; (2) the corpus-based negative sampling of PV leads to a weighting scheme for words that overly suppresses the importance of frequent words; and (3) the lack of word-context information makes PV unable to capture word substitution relationships.

In this paper, we addressed the Named Entity Recognition (NER) problem for morphologically rich languages by employing a semi-supervised learning approach based on neural networks. We adopted a fast unsupervised method for learning continuous vector representations of words, and used these representations along with language independent features to develop a NER system. We evaluated our system for the highly inflectional Turkish and Czech languages. We improved the state-of-the-art F-score obtained for Turkish without using gazetteers by 2.26% and for Czech by 1.53%. Unlike the previous state-of-the-art systems developed for these languages, our system does not make use of any language dependent features. Therefore, we believe it can easily be applied to other morphologically rich languages.

Languages with rich inflectional morphology exhibit lexical data sparsity, since the word used to express a given concept will vary with the syntactic context. For instance, each count noun in Czech has 12 forms (where English uses only singular and plural). Even in large corpora, we are unlikely to observe all inflections of a given lemma. This reduces the vocabulary coverage of methods that induce continuous representations for words from distributional corpus information. We solve this problem by exploiting existing morphological resources that can enumerate a word’s component morphemes. We present a latentvariable Gaussian graphical model that allows us to extrapolate continuous representations for words not observed in the training corpus, as well as smoothing the representations provided for the observed words. The latent variables represent embeddings of morphemes, which combine to create embeddings of words. Over several languages and training sizes, our model improves the embeddings for words, when evaluated on an analogy task, skip-gram predictive accuracy, and word similarity.

This study proposes a new neural machine translation (NMT) model based on the encoderdecoder model that incorporates named entity (NE) tags of source-language sentences. Conventional NMT models have two problems enumerated as follows: (i) they tend to have difficulty in translating words with multiple meanings because of the high ambiguity, and (ii) these models’ ability to translate compound words seems challenging because the encoder receives a word, a part of the compound word, at each time step. To alleviate these problems, the encoder of the proposed model encodes the input word on the basis of its NE tag at each time step, which could reduce the ambiguity of the input word. Furthermore, the encoder introduces a chunk-level LSTM layer over a word-level LSTM layer and hierarchically encodes a source-language sentence to capture a compound NE as a chunk on the basis of the NE tags. We evaluate the proposed model on an English-to-Japanese translation task with the ASPEC, and English-to-Bulgarian and English-to-Romanian translation tasks with the Europarl corpus. The evaluation results show that the proposed model achieves up to 3.11 point improvement in BLEU.

State-of-the-art sequence labeling systems traditionally require large amounts of taskspecific knowledge in the form of handcrafted features and data pre-processing. In this paper, we introduce a novel neutral network architecture that benefits from both wordand character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF. Our system is truly end-to-end, requiring no feature engineering or data preprocessing, thus making it applicable to a wide range of sequence labeling tasks. We evaluate our system on two data sets for two sequence labeling tasks — Penn Treebank WSJ corpus for part-of-speech (POS) tagging and CoNLL 2003 corpus for named entity recognition (NER). We obtain state-of-the-art performance on both datasets — 97.55% accuracy for POS tagging and 91.21% F1 for NER.

Prior work on computing semantic relatedness of words focused on representing their meaning in isolation, effectively disregarding inter-word affinities. We propose a large-scale data mining approach to learning word-word relatedness, where known pairs of related words impose constraints on the learning process. We learn for each word a low-dimensional representation, which strives to maximize the likelihood of a word given the contexts in which it appears. Our method, called CLEAR, is shown to significantly outperform previously published approaches. The proposed method is based on first principles, and is generic enough to exploit diverse types of text corpora, while having the flexibility to impose constraints on the derived word similarities. We also make publicly available a new labeled dataset for evaluating word relatedness algorithms, which we believe to be the largest such dataset to date.


Question: Enhancing Chinese word embeddings through the utilization of internal structure.

This paper introduces an innovative method for developing semantic representations, which expands the distributional hypothesis to encompass multilingual data and joint-space embeddings. Our models capitalize on parallel data to effectively align the embeddings of sentences with equivalent semantics, while simultaneously ensuring a clear separation between those of semantically divergent sentences. Notably, these models operate without the need for word alignments or syntactic information, and they have been successfully applied across a variety of languages. We further extend our approach to encompass the learning of semantic representations at the document level. The efficacy of our models is demonstrated through evaluation on two cross-lingual document classification tasks, where they surpass previous state-of-the-art methods. Through a combination of qualitative analysis and an examination of pivoting effects, we illustrate that our representations are semantically sound and capable of capturing cross-lingual semantic relationships even in the absence of parallel data.

Vector space word representations are useful for many natural language processing applications. The diversity of techniques for computing vector representations and the large number of evaluation benchmarks makes reliable comparison a tedious task both for researchers developing new vector space models and for those wishing to use them. We present a website and suite of offline tools that that facilitate evaluation of word vectors on standard lexical semantics benchmarks and permit exchange and archival by users who wish to find good vectors for their applications. The system is accessible at: www.wordvectors.org.

State-of-the-art sequence labeling systems traditionally require large amounts of taskspecific knowledge in the form of handcrafted features and data pre-processing. In this paper, we introduce a novel neutral network architecture that benefits from both wordand character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF. Our system is truly end-to-end, requiring no feature engineering or data preprocessing, thus making it applicable to a wide range of sequence labeling tasks. We evaluate our system on two data sets for two sequence labeling tasks — Penn Treebank WSJ corpus for part-of-speech (POS) tagging and CoNLL 2003 corpus for named entity recognition (NER). We obtain state-of-the-art performance on both datasets — 97.55% accuracy for POS tagging and 91.21% F1 for NER.

While continuous word embeddings are gaining popularity, current models are based solely on linear contexts. In this work, we generalize the skip-gram model with negative sampling introduced by Mikolov et al. to include arbitrary contexts. In particular, we perform experiments with dependency-based contexts, and show that they produce markedly different embeddings. The dependencybased embeddings are less topical and exhibit more functional similarity than the original skip-gram embeddings.

Vectorial representations of words have grown remarkably popular in natural language processing and machine translation. The recent surge in deep learning-inspired methods for producing distributed representations has been widely noted even outside these fields. Existing representations are typically trained on large monolingual corpora using context-based prediction models. In this paper, we propose extending pre-existing word representations by exploiting Wiktionary. This process results in a substantial extension of the original word vector representations, yielding a large multilingual dictionary of word embeddings. We believe that this resource can enable numerous monolingual and cross-lingual applications, as evidenced in a series of monolingual and cross-lingual semantic experiments that we have conducted.

Type-level word embeddings use the same set of parameters to represent all instances of a word regardless of its context, ignoring the inherent lexical ambiguity in language. Instead, we embed semantic concepts (or synsets) as defined in WordNet and represent a word token in a particular context by estimating a distribution over relevant semantic concepts. We use the new, context-sensitive embeddings in a model for predicting prepositional phrase (PP) attachments and jointly learn the concept embeddings and model parameters. We show that using context-sensitive embeddings improves the accuracy of the PP attachment model by 5.4% absolute points, which amounts to a 34.4% relative reduction in errors.

Characters are fundamental to literary analysis. Current approaches are heavily reliant on NER to identify characters, causing many to be overlooked. We propose a novel technique for character detection, achieving significant improvements over state of the art on multiple datasets.

This report documents the program and the outcomes of Dagstuhl Seminar 17042 “From Characters to Understanding Natural Language (C2NLU): Robust End-to-End Deep Learning for NLP”. The seminar brought together researchers from different fields, including natural language processing, computational linguistics, deep learning and general machine learning. 31 participants from 22 academic and industrial institutions discussed advantages and challenges of using characters, i.e., “raw text”, as input for deep learning models instead of language-specific tokens. Eight talks provided overviews of different topics, approaches and challenges in current natural language processing research. In five working groups, the participants discussed current natural language processing/understanding topics in the context of character-based modeling, namely, morphology, machine translation, representation learning, end-to-end systems and dialogue. In most of the discussions, the need for a more detailed model analysis was pointed out. Especially for character-based input, it is important to analyze what a deep learning model is able to learn about language – about tokens, morphology or syntax in general. For an efficient and effective understanding of language, it might furthermore be beneficial to share representations learned from multiple objectives to enable the models to focus on their specific understanding task instead of needing to learn syntactic regularities of language first. Therefore, benefits and challenges of transfer learning were an important topic of the working groups as well as of the panel discussion and the final plenary discussion. Seminar January 22–27, 2017 – http://www.dagstuhl.de/17042 1998 ACM Subject Classification I.2 Artificial Intelligence, I.2.7 Natural Language Processing

Almost all current dependency parsers classify based on millions of sparse indicator features. Not only do these features generalize poorly, but the cost of feature computation restricts parsing speed significantly. In this work, we propose a novel way of learning a neural network classifier for use in a greedy, transition-based dependency parser. Because this classifier learns and uses just a small number of dense features, it can work very fast, while achieving an about 2% improvement in unlabeled and labeled attachment scores on both English and Chinese datasets. Concretely, our parser is able to parse more than 1000 sentences per second at 92.2% unlabeled attachment score on the English Penn Treebank.

This paper presents a new approach for measuring semantic similarity/distance between words and concepts. It combines a lexical taxonomy structure with corpus statistical information so that the semantic distance between nodes in the semantic space constructed by the taxonomy can be better quantified with the computational evidence derived from a distributional analysis of corpus data. Specifically, the proposed measure is a combined approach that inherits the edge-based approach of the edge counting scheme, which is then enhanced by the node-based approach of the information content calculation. When tested on a common data set of word pair similarity ratings, the proposed approach outperforms other computational models. It gives the highest correlation value (r = 0.828) with a benchmark based on human similarity judgements, whereas an upper bound (r = 0.885) is observed when human subjects replicate the same task.

Most state-of-the-art named entity recognition (NER) systems rely on handcrafted features and on the output of other NLP tasks such as part-of-speech (POS) tagging and text chunking. In this work we propose a language-independent NER system that uses automatically learned features only. Our approach is based on the CharWNN deep neural network, which uses word-level and character-level representations (embeddings) to perform sequential classification. We perform an extensive number of experiments using two annotated corpora in two different languages: HAREM I corpus, which contains texts in Portuguese; and the SPA CoNLL2002 corpus, which contains texts in Spanish. Our experimental results shade light on the contribution of neural character embeddings for NER. Moreover, we demonstrate that the same neural network which has been successfully applied to POS tagging can also achieve state-of-the-art results for language-independet NER, using the same hyperparameters, and without any handcrafted features. For the HAREM I corpus, CharWNN outperforms the state-of-the-art system by 7.9 points in the F1-score for the total scenario (ten NE classes), and by 7.2 points in the F1 for the selective scenario (five NE classes).


Question: Enhance Chinese word embeddings through the exploitation of internal structure.

This paper presents an enhanced version of the Deep Boltzmann Machine (DBM) specifically tailored for deriving dense semantic representations from extensive, unstructured document collections. The primary challenge in training a DBM is addressed through strategic parameter tying, which facilitates an efficient pre-training process and a rapid initialization method for inference. This adaptation allows the model to be trained with the same ease as a conventional Restricted Boltzmann Machine. Our empirical results indicate that the proposed DBM variant outperforms the Replicated Softmax model in terms of log probability for unseen data. Moreover, the features generated by our model surpass those produced by LDA, Replicated Softmax, and DocNADE models when evaluated on document retrieval and classification tasks, showcasing its superior performance in these areas.

Word Sense Disambiguation models exist in many flavors. Even though supervised ones tend to perform best in terms of accuracy, they often lose ground to more flexible knowledge-based solutions, which do not require training by a word expert for every disambiguation target. To bridge this gap we adopt a different perspective and rely on sequence learning to frame the disambiguation problem: we propose and study in depth a series of end-to-end neural architectures directly tailored to the task, from bidirectional Long Short-Term Memory to encoder-decoder models. Our extensive evaluation over standard benchmarks and in multiple languages shows that sequence learning enables more versatile all-words models that consistently lead to state-of-the-art results, even against word experts with engineered features.

Neural language models learn word representations that capture rich linguistic and conceptual information. Here we investigate the embeddings learned by neural machine translation models. We show that translation-based embeddings outperform those learned by cutting-edge monolingual models at single-language tasks requiring knowledge of conceptual similarity and/or syntactic role. The findings suggest that, while monolingual models learn information about how concepts are related, neural-translation models better capture their true ontological status. It is well known that word representations can be learned from the distributional patterns in corpora. Originally, such representations were constructed by counting word co-occurrences, so that the features in one word’s representation corresponded to other words [11, 17]. Neural language models, an alternative means to learn word representations, use language data to optimise (latent) features with respect to a language modelling objective. The objective can be to predict either the next word given the initial words of a sentence [4, 14, 8], or simply a nearby word given a single cue word [13, 15]. The representations learned by neural models (sometimes called embeddings) generally outperform those acquired by co-occurrence counting models when applied to NLP tasks [3]. Despite these clear results, it is not well understood how the architecture of neural models affects the information encoded in their embeddings. Here, we explore this question by considering the embeddings learned by architectures with a very different objective function to monolingual language models: neural machine translation models. We show that translation-based embeddings outperform monolingual embeddings on two types of task: those that require knowledge of conceptual similarity (rather than simply association or relatedness), and those that require knowledge of syntactic role. We discuss what the findings indicate about the information content of different embeddings, and suggest how this content might emerge as a consequence of the translation objective. 1 Learning embeddings from language data Both neural language models and translation models learn real-valued embeddings (of specified dimension) for words in some pre-specified vocabulary, V , covering many or all words in their training corpus. At each training step, a ‘score’ for the current training example (or batch) is computed based on the embeddings in their current state. This score is compared to the model’s objective function, and the error is backpropagated to update both the model weights (affecting how the score is computed from the embeddings) and the embedding features. At the end of this process, the embeddings should encode information that enables the model to optimally satisfy its objective. 1.1 Monolingual models In the original neural language model [4] and subsequent variants [8], each training example consists of n subsequent words, of which the model is trained to predict the n-th word given the first n − 1 ar X iv :1 41 0. 07 18 v2 [ cs .C L ] 1 3 N ov 2 01 4 1 words. The model first represents the input as an ordered sequence of embeddings, which it transforms into a single fixed length ‘hidden’ representation by, e.g., concatenation and non-linear projection. Based on this representation, a probability distribution is computed over the vocabulary, from which the model can sample a guess at the next word. The model weights and embeddings are updated to maximise the probability of correct guesses for all sentences in the training corpus. More recent work has shown that high quality word embeddings can be learned via models with no nonlinear hidden layer [13, 15]. Given a single word in the corpus, these models simply predict which other words will occur nearby. For each word w in V , a list of training cases (w, c) : c ∈ V is extracted from the training corpus. For instance, in the skipgram approach [13], for each ‘cue word’ w the ‘context words’ c are sampled from windows either side of tokens of w in the corpus (with c more likely to be sampled if it occurs closer to w).1 For each w in V , the model initialises both a cue-embedding, representing the w when it occurs as a cue-word, and a context-embedding, used when w occurs as a context-word. For a cue word w, the model can use the corresponding cueembedding and all context-embeddings to compute a probability distribution over V that reflects the probability of a word occurring in the context of w. When a training example (w, c) is observed, the model updates both the cue-word embedding of w and the context-word embeddings in order to increase the conditional probability of c. 1.2 Translation-based embeddings Neural translation models generate an appropriate sentence in their target language St given a sentence Ss in their source language [see, e.g., 16, 6]. In doing so, they learn distinct sets of embeddings for the vocabularies Vs and Vt in the source and target languages respectively. Observing a training case (Ss, St), such a model represents Ss as an ordered sequence of embeddings of words from Vs. The sequence for Ss is then encoded into a single representation RS .2 Finally, by referencing the embeddings in Vt, RS and a representation of what has been generated thus far, the model decodes a sentence in the target language word by word. If at any stage the decoded word does not match the corresponding word in the training target St, the error is recorded. The weights and embeddings in the model, which together parameterise the encoding and decoding process, are updated based on the accumulated error once the sentence decoding is complete. Although neural translation models can differ in low-level architecture [7, 2], the translation objective exerts similar pressure on the embeddings in all cases. The source language embeddings must be such that the model can combine them to form single representations for ordered sequences of multiple words (which in turn must enable the decoding process). The target language embeddings must facilitate the process of decoding these representations into correct target-language sentences. 2 Comparing Mono-lingual and Translation-based Embeddings To learn translation-based embeddings, we trained both the RNN encoder-decoder [RNNenc, 7] and the RNN Search architectures [2] on a 300m word corpus of English-French sentence pairs. We conducted all experiments with the resulting (English) source embeddings from these models. For comparison, we trained a monolingual skipgram model [13] and its Glove variant [15] for the same number of epochs on the English half of the bilingual corpus. We also extracted embeddings from a full-sentence language model [CW, 8] trained for several months on a larger 1bn word corpus. As in previous studies [1, 5, 3], we evaluate embeddings by calculating pairwise (cosine) distances and correlating these distances with (gold-standard) human judgements. Table 1 shows the correlations of different model embeddings with three such gold-standard resources, WordSim-353 [1], MEN [5] and SimLex-999 [10]. Interestingly, translation embeddings perform best on SimLex-999, while the two sets of monolingual embeddings perform better on modelling the MEN and WordSim353. To interpret these results, it should be noted that SimLex-999 evaluation quantifies conceptual similarity (dog wolf ), whereas MEN and WordSim-353 (despite its name) quantify more general relatedness (dog collar) [10]. The results seem to indicate that translation-based embeddings better capture similarity, while monolingual embeddings better capture relatedness. 1 Subsequent variants use different algorithms for selecting the (w, c) from the training corpus [9, 12] Alternatively, subsequences (phrases) of Ss may be encoded at this stage in place of the whole sentence [2].

Named entity recognition (NER) in Chinese social media is important but difficult because of its informality and strong noise. Previous methods only focus on in-domain supervised learning which is limited by the rare annotated data. However, there are enough corpora in formal domains and massive in-domain unannotated texts which can be used to improve the task. We propose a unified model which can learn from out-of-domain corpora and in-domain unannotated texts. The unified model contains two major functions. One is for cross-domain learning and another for semi-supervised learning. Cross-domain learning function can learn out-of-domain information based on domain similarity. Semi-Supervised learning function can learn in-domain unannotated information by self-training. Both learning functions outperform existing methods for NER in Chinese social media. Finally, our unified model yields nearly 11% absolute improvement over previously published results.

This work presents an algorithm for the unsupervised learning, or induction, of a simple morphology of a natural language. A probabilistic maximum a posteriori model is utilized, which builds hierarchical representations for a set of morphs, which are morpheme-like units discovered from unannotated text corpora. The induced morph lexicon stores parameters related to both the “meaning” and “form” of the morphs it contains. These parameters affect the role of the morphs in words. The model is implemented in a task of unsupervised morpheme segmentation of Finnish and English words. Very good results are obtained for Finnish and almost as good results are obtained in the English task.

We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.

In recent years, collocation has been widely acknowledged as an essential characteristic to distinguish native speakers from non-native speakers. Research on academic writing has also shown that collocations are not only common but serve a particularly important discourse function within the academic community. In our study, we propose a machine learning approach to implementing an online collocation writing assistant. We use a data-driven classifier to provide collocation suggestions to improve word choices, based on the result of classification. The system generates and ranks suggestions to assist learners’ collocation usages in their academic writing with satisfactory results. *

Almost all current dependency parsers classify based on millions of sparse indicator features. Not only do these features generalize poorly, but the cost of feature computation restricts parsing speed significantly. In this work, we propose a novel way of learning a neural network classifier for use in a greedy, transition-based dependency parser. Because this classifier learns and uses just a small number of dense features, it can work very fast, while achieving an about 2% improvement in unlabeled and labeled attachment scores on both English and Chinese datasets. Concretely, our parser is able to parse more than 1000 sentences per second at 92.2% unlabeled attachment score on the English Penn Treebank.

This paper presents an analysis of existing methods for the intrinsic evaluation of word embeddings. We show that the main methodological premise of such evaluations is “interpretability” of word embeddings: a “good” embedding produces results that make sense in terms of traditional linguistic categories. This approach is not only of limited practical use, but also fails to do justice to the strengths of distributional meaning representations. We argue for a shift from abstract ratings of word embedding “quality” to exploration of their strengths and weaknesses.

State-of-the-art sequence labeling systems traditionally require large amounts of taskspecific knowledge in the form of handcrafted features and data pre-processing. In this paper, we introduce a novel neutral network architecture that benefits from both wordand character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF. Our system is truly end-to-end, requiring no feature engineering or data preprocessing, thus making it applicable to a wide range of sequence labeling tasks. We evaluate our system on two data sets for two sequence labeling tasks — Penn Treebank WSJ corpus for part-of-speech (POS) tagging and CoNLL 2003 corpus for named entity recognition (NER). We obtain state-of-the-art performance on both datasets — 97.55% accuracy for POS tagging and 91.21% F1 for NER.

One of the main obstacles to highperformance Word Sense Disambiguation (WSD) is the knowledge acquisition bottleneck. In this paper, we present a methodology to automatically extend WordNet with large amounts of semantic relations from an encyclopedic resource, namely Wikipedia. We show that, when provided with a vast amount of high-quality semantic relations, simple knowledge-lean disambiguation algorithms compete with state-of-the-art supervised WSD systems in a coarse-grained all-words setting and outperform them on gold-standard domain-specific datasets.


Question: A new evolutionary data mining technique and its application to predicting customer churn

This paper delves into the development of a scalable and efficient classifier, named SLIQ, specifically tailored for the demands of data mining large datasets. Unlike many traditional classification algorithms that are optimized for in-memory data, SLIQ is designed to overcome the limitations posed by large-scale data volumes. As a decision tree classifier, SLIQ is capable of processing both numeric and categorical attributes, offering a versatile solution for data mining tasks. A key innovation in SLIQ is its use of a pre-sorting technique during the tree-growing phase. This technique is seamlessly integrated with a breadth-first tree growth strategy, enabling the classification of datasets that reside on disk. This approach ensures that SLIQ can handle datasets of any size, regardless of the number of classes, attributes, or records. Additionally, SLIQ incorporates an advanced tree-pruning algorithm that is both cost-effective and efficient, leading to compact yet highly accurate decision trees. The amalgamation of these techniques makes SLIQ a powerful tool for data mining, capable of scaling seamlessly with the increasing size of datasets while maintaining high accuracy and efficiency.

Privacy is an important issue in data mining and knowledge discovery. In this paper, we propose to use the randomized response techniques to conduct the data mining computation. Specially, we present a method to build decision tree classifiers from the disguised data. We conduct experiments to compare the accuracy of our decision tree with the one built from the original undisguised data. Our results show that although the data are disguised, our method can still achieve fairly high accuracy. We also show how the parameter used in the randomized response techniques affects the accuracy of the results.

-Data Mining is a powerful tool for academic intervention. Mining in education environment is called Educational Data Mining. Educational Data Mining is concerned with developing new methods to discover knowledge from educational database and can used for decision making in educational system. In our work, we collected the student’s data from engineering institute that have different information about their previous and current academics records like students S.No., Name, Branch, 10, 12 , B.Tech passing percentage and final grade & then apply different classification algorithm using Data Mining tools (WEKA) for analysis the students academics performance for Training & placement department or company executives. This paper deals with a comparative study of various classification data mining algorithms for the performance analysis of the student’s academic records and check which algorithm is optimal for classifying students’ based on their final grade. This analysis also classifies the performance of Students into Excellent, Good and Average categories. Keywords– Data Mining, Discover knowledge, Technical Education, Educational Data, Mining, Classification, WEKA, Classifiers.

In recent years, Distributed Data Mining (DDM) has evolved in large space aiming at minimizing computation cost and memory overhead in processing huge geographically distributed data. There are two approaches of DDM -- homogeneous and heterogeneous classifier approach. This paper presents implementation of four homogeneous classifier techniques for DDM with different Electronic Health Records (EHRs) homogeneous datasets namely diabetes, hepatitis, hypothyroid and further analyzing results based on metric evaluation.

Previous studies propose that associative classification has high classification accuracy and strong flexibility at handling unstructured data. However, it still suffers from the huge set of mined rules and sometimes biased classification or overfitting since the classification is based on only single high-confidence rule. In this study, we propose a new associative classification method, CMAR, i.e., Classification based on Multiple Association Rules. The method extends an efficient frequent pattern mining method, FP-growth, constructs a class distribution-associated FP-tree, and mines large database efficiently. Moreover, it applies a CR-tree structure to store and retrieve mined association rules efficiently, and prunes rules effectively based on confidence, correlation and database coverage. The classification is performed based on a weighted analysis using multiple strong association rules. Our extensive experiments on databases from UCI machine learning database repository show that CMAR is consistent, highly effective at classification of various kinds of databases and has better average classification accuracy in comparison with CBA and C4.5. Moreover, our performance study shows that the method is highly efficient and scalable in comparison with other reported associative classification methods.

The explosion of process-related data in nowadays organizations has raised the interest to exploit the data in order to know in deep how the business processes are being carried out. To face this need, process mining has emerged as a way to analyze the behavior of an organization by extracting knowledge from process related data. In this paper, we present a case study of process mining in a real IT service management scenario. We describe an exploratory analysis of real life event logs generated between 2012 and 2015, in four different processes designed within an IT platform. More specifically, we analyze the way of handling the different requests and incidents registered in an organization.

We present an analysis of taxi flows in Manhattan (NYC) using a variety of data mining approaches. The methods presented here can aid in development of representative and accurate models of large-scale traffic flows with applications to many areas, including outlier detection and characterization.

Collaborative recommender systems allow personalization for e-commerce by exploiting similarities and dissimilarities among users' preferences. We investigate the use of association rule mining as an underlying technology for collaborative recommender systems. Association rules have been used with success in other domains. However, most currently existing association rule mining algorithms were designed with market basket analysis in mind. Such algorithms are ine cient for collaborative recommendation because they mine many rules that are not relevant to a given user. Also, it is necessary to specify the minimum support of the mined rules in advance, often leading to either too many or too few rules; this negatively impacts the performance of the overall system. We describe a collaborative recommendation technique based on a new algorithm speci cally designed to mine association rules for this purpose. Our algorithm does not require the minimum support to be speci ed in advance. Rather, a target range is given for the number of rules, and the algorithm adjusts the minimum support for each user in order to obtain a ruleset whose size is in the desired range. Rules are mined for a speci c target user, reducing the time required for the mining process. We employ associations between users as well as associations between items in making recommendations. Experimental evaluation of a system based on our algorithm reveals performance that is signi cantly better than that of traditional correlationbased approaches. Corresponding author. Present a liation: Department of Computer Science, Wellesley College, 106 Central Street, Wellesley, MA 02481 USA, e-mail: salvarez@wellesley.edu

Although data mining has been successfully implemented in the business world for some time now, its use in higher education is still relatively new, i.e. its use is intended for identification and extraction of new and potentially valuable knowledge from the data. Using data mining the aim was to develop a model which can derive the conclusion on students' academic success. Different methods and techniques of data mining were compared during the prediction of students' success, applying the data collected from the surveys conducted during the summer semester at the University of Tuzla, the Faculty of Economics, academic year 2010-2011, among first year students and the data taken during the enrollment. The success was evaluated with the passing grade at the exam. The impact of students' socio-demographic variables, achieved results from high school and from the entrance exam, and attitudes towards studying which can have an affect on success, were all investigated. In future investigations, with identifying and evaulating variables associated with process of studying, and with the sample increase, it would be possible to produce a model which would stand as a foundation for the development of decision support system in higher education.

NIMISHA PHILIP, SHERLY K.K Department of Computer Science & Engineering, Toc H Institute of Science & Technology email:nimishavellapally@gmail.com Department of Information Technology, Toc H Institute of Science & Technology, Ernakulam, Kerala, India shrly_shilu@yahoo.com Abstract — Globalization and increased use of the Internet for online shopping has resulted in a considerable proliferation of credit card transactions throughout the world.Higher acceptability and convenience of credit cards for purchases has not only given personal comfort to customers but also attracted a large number of attackers. As a result, credit card payment systems must be supported by efficient fraud detection capability for minimizing unwanted activities by adversaries. Most of the well known algorithms for fraud detection are based on supervised training Every cardholder has a certain shopping behavior, which establishes an activity profile for him. Existing FDS try to capture behavioral patterns as rules which are static .This becomes ineffective when cardholder develops new patterns of behavior Here, we propose a unsupervised method to dynamically profile behavior pattern of customer Then the incoming transactions are compared against the user profile to indicate the anomalies, based on which the corresponding warnings are outputted. A FP tree based pattern matching algorithm is used to evaluate how unusual the new transactions are.

This paper presents a review of the literature on the application of data mining techniques for the detection of insurance fraud. Academic literature were analyzed and classified into three types of insurance fraud (automobile insurance, crop insurance and healthcare insurance) and six classes of data mining techniques (classification, regression, clustering, prediction, outlier detection, and visualization). The findings of this review clearly show that automobile insurance fraud detection have also attracted a great deal of attention in recent years. The main data mining techniques used for insurance fraud detection are logistic models, Decision tree, the Naïve Bayes, and support vector machine.


Question: A new evolutionary data mining technique and its application to predicting customer churn

This paper presents an in-depth exploration of AdaBoost, a novel boosting algorithm designed to enhance the accuracy of learning algorithms by significantly reducing their error rates, particularly when these algorithms generate classifiers with performance slightly above random guessing. We further investigate the concept of "pseudo-loss," a technique that steers learning algorithms toward distinguishing the most challenging labels in multi-label classification tasks. To evaluate the efficacy of AdaBoost with and without pseudo-loss, we conducted comprehensive experiments on real-world learning problems. Our study comprised two main phases. In the initial phase, we contrasted the outcomes of boosting and Breiman's "bagging" technique when applied to aggregate a variety of classifiers, including decision trees and single attribute-value tests, across a set of established machine learning benchmarks. The second phase delved deeper into boosting's performance using a nearest-neighbor classifier on a specific Optical Character Recognition (OCR) task. These experiments not only shed light on the comparative advantages of AdaBoost over bagging but also highlight its potential in boosting the accuracy of classifiers in practical applications, particularly in scenarios where discriminative power is critical for success.

The explosion of process-related data in nowadays organizations has raised the interest to exploit the data in order to know in deep how the business processes are being carried out. To face this need, process mining has emerged as a way to analyze the behavior of an organization by extracting knowledge from process related data. In this paper, we present a case study of process mining in a real IT service management scenario. We describe an exploratory analysis of real life event logs generated between 2012 and 2015, in four different processes designed within an IT platform. More specifically, we analyze the way of handling the different requests and incidents registered in an organization.

Distributed Data Mining (DDM) has become one of the promising areas of Data Mining. DDM techniques include classifier approach and agent-approach. Classifier approach plays a vital role in mining distributed data, having homogeneous and heterogeneous approaches depend on data sites. Homogeneous classifier approach involves ensemble learning, distributed association rule mining, meta-learning and knowledge probing. Heterogeneous classifier approach involves collective principle component analysis, distributed clustering, collective decision tree and collective bayesian learning model. In this paper, classifier approach for DDM is summarized and an architectural model based on clustered-collaborative filtering for Electronic Health Records (EHR) is proposed.

Data Mining is the extraction of hidden predictive information from large databases. The process of discovering interesting, useful, nontrivial patterns from large spatial datasets is called spatial data mining. When time gets associated it becomes spatio temporal data mining. The study of spatio temporal data mining is of great concern for the study of mobile phone sensed data. Reality Mining is defined as the study of human social behavior based on mobile phone sensed data. It is based on the data collected by sensors in mobile phones, security cameras, RFID readers, etc. All allow for the measurement of human physical and social activity. In this paper Netviz, Gephi and Weka tools have been used to convert and analyze the Facebook. Further, analysis of a reality mining dataset is also presented.

Many current e-commerce systems provide personalization when their content is shown to users. In this sense, recommender systems make personalized suggestions and provide information of items available in the system. Nowadays, there is a vast amount of methods, including data mining techniques that can be employed for personalization in recommender systems. However, these methods are still quite vulnerable to some limitations and shortcomings related to recommender environment. In order to deal with some of them, in this work we implement a recommendation methodology in a recommender system for tourism, where classification based on association is applied. Classification based on association methods, also named associative classification methods, consist of an alternative data mining technique, which combines concepts from classification and association in order to allow association rules to be employed in a prediction context. The proposed methodology was evaluated in some case studies, where we could verify that it is able to shorten limitations presented in recommender systems and to enhance recommendation quality.

Mining frequent patterns has been a focused topic in data mining research in recent years, with the developmeht of numerous interesting algorithms for mining association, correlation, causality, sequential patterns, partial periodicity, constraint-based frequent pattern mining, associative classification, emerging patterns, etc. Most of the previous studies adopt an Apriori-like, candidate generation-and-test approach. However, based on our analysis, candidate generation and test may still be expensive, especially when encountering long and numerous patterns. A new methodology, called f r e q u e n t p a t t e r n g rowth , which mines frequent patterns without candidate generation, has been developed. The method adopts a divide-andconquer philosophy to project and partition databases based on the currently discovered frequent patterns and grow such patterns to longer ones in the projected databases. Moreover, efficient data structures have been developed for effective database compression and fast in-memory traversal. Such a methodology may eliminate or substantially reduce the number of candidate sets to be generated and also reduce the size of the database to be iteratively examined, and, therefore, lead to high performance. In this paper, we provide an overview of this approach and examine its methodology and implications for mining several kinds of frequent patterns, including association, frequent closed itemsets, max-patterns, sequential patterns, and constraint-based mining of frequent patterns. We show that frequent pattern growth is efficient at mining large databases and its further development may lead to scalable mining of many other kinds of patterns as well.

Educational Data Mining (EDM) is an evolving field exploring pedagogical data by applying different machine learning techniques/tools. It can be considered as interdisciplinary research field which provides intrinsic knowledge of teaching and learning process for effective education. The main objective of any educational institution is to provide quality education to its students. One way to achieve highest level of quality in higher education system is by discovering knowledge that predicts teachers’ performance. This study presents an efficient system model for evaluation and prediction of teachers’ performance in higher institutions of learning using data mining technologies. To achieve the objectives of this work, a two-layered classifier system was designed; it consists of an Artificial Neural Network (ANN) and Decision Tree. The classifier system was tested successfully using case study data from a Nigerian University in the South West of Nigeria. The data consists of academic qualifications for teachers as well as their experiences and grades of students in courses they taught among others. The attribute selected were evaluated using two feature selection methods in order to get a subset of the attributes that would make for a compact and accurate predictive model. The WEKA machine learning tool was used for the mining. The results show that, among the six attributes used, Working Experience, and Rank are rated the best two attributes that contributed mostly to the performance of teachers in this study. Also, considering the time taken to build the models and performance accuracy level, C4.5 decision tree outperformed the other two algorithms (ID3 and MLP) with good performance of 83.5% accuracy level and acceptable kappa statistics of 0.743. It does mean that C4.5 decision tree is best algorithm suitable for predicting teachers’ performance in relation to the other two algorithms in this work. General Terms Data Mining

Quite a large number of data mining techniques employed in financial fraud detection (FFD) are seen to be classification techniques. In this paper, we developed an algorithm to find the features of classification techniques (or method) that so much place it (classification techniques) in the heart of researchers in their various efforts in the study of insurance frauds detection. We also got to know the characteristics of insurance frauds data that made data mining classification techniques so much attracted to it (insurance data).

We investigate the problem of generating fast approximate a nswers to queries posed to large sparse binary data sets. We focus in particular on probabilistic mode l-based approaches to this problem and develop a number of techniques that are significantly more accurate t han a baseline independence model. In particular, we introduce two techniques for building probabil ist c models from frequent itemsets: the itemset maximum entropy method, and the itemset inclusion-exclusi on model. In the maximum entropy method we treat itemsets as constraints on the distribution of the q uery variables and use the maximum entropy principle to build a joint probability model for the query at tributes online. In the inclusion-exclusion model itemsets and their frequencies are stored in a data structur e alled an ADtree that supports an efficient implementation of the inclusion-exclusion principle in orde r to answer the query. We empirically compare these two itemset-based models to direct querying of the ori ginal data, querying of samples of the original data, as well as other probabilistic models such as the indep endence model, the Chow-Liu tree model, and the Bernoulli mixture model. These models are able to handle high-dimensionality (hundreds or thousands of attributes), whereas most other work on this topic has foc used on relatively low-dimensional OLAP problems. Experimental results on both simulated and realwor d transaction data sets illustrate various fundamental tradeoffs between approximation error, model complexity, and the online time required to compute a query answer.

Data mining [Chen et al. 1996] is the process of extracting interesting (non-trivial, implicit, previously unknown and potentially useful) information or patterns from large information repositories such as: relational database, data warehouses, XML repository, etc. Also data mining is known as one of the core processes of Knowledge Discovery in Database (KDD). Many people take data mining as a synonym for another popular term, Knowledge Discovery in Database (KDD). Alternatively other people treat Data Mining as the core process of KDD. The KDD processes are shown in Figure 1 [Han and Kamber 2000]. Usually there are three processes. One is called preprocessing, which is executed before data mining techniques are applied to the right data. The preprocessing includes data cleaning, integration, selection and transformation. The main process of KDD is the data mining process, in this process different algorithms are applied to produce hidden knowledge. After that comes another process called postprocessing, which evaluates the mining result according to users’ requirements and domain knowledge. Regarding the evaluation results, the knowledge can be presented if the result is satisfactory, otherwise we have to run some or all of those processes again until we get the satisfactory result. The actually processes work as follows. First we need to clean and integrate the databases. Since the data source may come from different databases, which may have some inconsistences and duplications, we must clean the data source by removing those noises or make some compromises. Suppose we have two different databases, different words are used to refer the same thing in their schema. When we try to integrate the two sources we can only choose one of them, if we know that they denote the same thing. And also real world data tend to be incomplete and noisy due to the manual input mistakes. The integrated data sources can be stored in a database, data warehouse or other repositories. As not all the data in the database are related to our mining task, the second process is to select task related data from the integrated resources and transform them into a format that is ready to be mined. Suppose we want to find which items are often purchased together in a supermarket, while the database that records the purchase history may contains customer ID, items bought, transaction time, prices, number of each items and so on, but for this specific task we only need items bought. After selection of relevant data, the database that we are going to apply our data mining techniques to will be much smaller, consequently the whole process will be

We are given a large database of customer transactions, where each transaction consists of customer-id, transaction time, and the items bought in the transaction. We introduce the problem of mining sequential patterns over such databases. We present three algorithms to solve this problem, and empirically evaluate their performance using synthetic data. Two of the proposed algorithms, AprioriSome and AprioriAll, have comparable performance, albeit AprioriSome performs a little better when the minimum number of customers that must support a sequential pattern is low. Scale-up experiments show that both AprioriSome and AprioriAll scale linearly with the number of customer transactions. They also have excellent scale-up properties with respect to the number of transactions per customer and the number of items in a transaction.


Question: A new evolutionary data mining technique and its application to predicting customer churn

This document explores a novel technique for constructing decision trees from datasets for classification purposes, inspired by Rissanen's pioneering work on the Minimum Description Length Principle (MDLP). MDLP advocates for the selection of the theory that yields the most compact representation of data when inferring models from a given dataset. This approach is grounded in the principle that the optimal model should not only fit the data well but also provide the shortest possible description of both the model and the data it represents. The concept of stochastic complexity, introduced by Rissanen, plays a crucial role in evaluating the effectiveness of different models. This paper builds upon the foundational work of Boulton and Wallace (1968, 1973a, 1973b), Georgeff and Wallace (1984), and Hart (1987), who have contributed significantly to the field of model selection and data compression. The methodology proposed here aims to find the decision tree that minimizes the combined length of the tree description and the encoded data, effectively implementing the MDLP in a practical context for classification tasks. This approach not only ensures that the model is a good fit for the data but also guards against overfitting by penalizing overly complex models. The application of MDLP in decision tree construction represents a significant advancement in the field of data mining, particularly in scenarios where the goal is to find the simplest yet most accurate representation of the data.

The prediction of survival of Coronary heart disease (CHD) has been a challenging research problem for medical society. The goal of this paper is to develop data mining algorithms for predicting survival of CHD patients based on 1000 cases .We carry out a clinical observation and a 6-month follow up to include 1000 CHD cases. The survival information of each case is obtained via follow up. Based on the data, we employed three popular data mining algorithms to develop the prediction models using the 502 cases. We also used 10-fold cross-validation methods to measure the unbiased estimate of the three prediction models for performance comparison purposes. The results indicated that the SVM is the best predictor with 92.1 % accuracy on the holdout sample artificial neural networks came out to be the second with91.0% accuracy and the decision trees models came out to be the worst of the three with 89.6% accuracy. The comparative study of multiple prediction models for survival of CHD patients along with a 10-fold cross-validation provided us with an insight into the relative prediction ability of different data.

Educational Data Mining (EDM) is an emerging multidisciplinary research area, in which methods and techniques for exploring data originating from various educational information systems have been developed. EDM is both a learning science, as well as a rich application area for data mining, due to the growing availability of educational data. EDM contributes to the study of how students learn, and the settings in which they learn. It enables data-driven decision making for improving the current educational practice and learning material. We present a brief overview of EDM and introduce four selected EDM papers representing a crosscut of different application areas for data mining in education.

We describe our first-place solution to the ECML/PKDD discovery challenge on taxi destination prediction. The task consisted in predicting the destination of a taxi based on the beginning of its trajectory, represented as a variable-length sequence of GPS points, and diverse associated meta-information, such as the departure time, the driver id and client information. Contrary to most published competitor approaches, we used an almost fully automated approach based on neural networks and we ranked first out of 381 teams. The architectures we tried use multi-layer perceptrons, bidirectional recurrent neural networks and models inspired from recently introduced memory networks. Our approach could easily be adapted to other applications in which the goal is to predict a fixed-length output from a variable-length sequence. Random order, this does not reflect the weights of contributions. ar X iv :1 50 8. 00 02 1v 2 [ cs .L G ] 2 1 Se p 20 15 2 Artificial Neural Networks Applied to Taxi Destination Prediction

In recent years, uncertain data management applications have grown in importance because of the large number of hardware applications which measure data approximately. For example, sensors are typically expected to have considerable noise in their readings because of inaccuracies in data retrieval, transmission, and power failures. In many cases, the estimated error of the underlying data stream is available. This information is very useful for the mining process, since it can be used in order to improve the quality of the underlying results. In this paper we will propose a method for clustering uncertain data streams. We use a very general model of the uncertainty in which we assume that only a few statistical measures of the uncertainty are available. We will show that the use of even modest uncertainty information during the mining process is sufficient to greatly improve the quality of the underlying results. We show that our approach is more effective than a purely deterministic method such as the CluStream approach. We will test the approach on a variety of real and synthetic data sets and illustrate the advantages of the method in terms of effectiveness and efficiency.

Cloud computing is the one of the admired paradigms of current era, which facilitates the users with on demand services and pay as you use services. It has tremendous applications in almost every sphere such as education, gaming, social networking, transportation, medical, business, stock market, pattern matching, etc. Stock market is such an industry where lots of data is generated and benefits are reaped on the basis of accurate prediction. So prediction is a vital part of stock market. Therefore, an attempt is being made to predict the stock market based on the given data set of stock market along with some features; using the techniques available for predictive data mining. Machine learning is one of the upcoming trends of data mining; hence few machine learning algorithms have been used such as Decision tree, Linear model, Random forest and further their results have been compared using the classification evaluation parameters such as H, AUC, ROC, TPR, FPR, etc. Random forest have been consider as the most effective model as it yield the highest accuracy of 54.12% whereas decision tree and linear model gives the accuracy of 51.87% and 52.83% respectively.

Prepaid customers in mobile telecommunications are not bound by a contract and can therefore change operators (‘churn’) at their convenience and without notification. This makes the task of predicting prepaid churn both challenging and financially rewarding. This paper presents an explorative, real world study of prepaid churn modeling by varying the experimental setup on three dimensions: data, outcome definition and population sample. Firstly, we add Customer Experience Management (CEM) data to data traditionally used for prepaid churn prediction. Secondly, we vary the outcome definition of prepaid churn. Thirdly, we alter the sample of the customers included, based on their inactivity period at the time of recording. While adding CEM parameters did not influence the predictability of churn, one variation on the sample and especially a particular change in the outcome definition had a substantial

In Web Usage Mining (WUM), Classification technique is useful in web user prediction, where user’s browsing behavior is predicted. In Web prediction, classification is done first to predict the next Web pages set that a user may visit based on history of previously visited pages. Predicting user’s behavior effectively is important in various applications like recommendation systems, smart phones etc. Currently various prediction systems are available based on Markov model, Association Rule Mining (ARM), classification techniques, etc. In Markov prediction model it cannot predict for a session that was not observed previously in the training set and prediction time is also constant. In ARM, Apriori algorithm requires more time to generate item sets. This paper presents review of various prediction models used to predict web users browsing behavior. Key Words— WUM, classification, Browsing behavior, prediction, ARM, session

The increased availability of data created as part of the sof tware development process allows us to apply novel analysis techniques on the data and use the results to guide t he process’s optimization. In this paper we describe variou s data sources and discuss the principles and techniques of data mi ning as applied on software engineering data. Data that can b e mined is generated by most parts of the development process: requi rements elicitation, development analysis, testing, debu gging, and maintenance. Based on this classification we survey the mini ng approaches that have been used and categorize them accord ing to the corresponding parts of the development process and th e task they assist. Thus the survey provides researchers wit h a concise overview of data mining techniques applied to softw are engineering data, and aids practitioners on the selecti on of appropriate data mining techniques for their work.

This paper presents a framework for process mining in critical care. The framework uses the CRISP-DM model, extended to incorporate temporal and multidimensional aspects (CRISP-TDM), combined with the Patient Journey Modeling Architecture (PaJMa), to provide a structured approach to knowledge discovery of new condition onset pathophysiologies in physiological data streams. The approach is based on temporal abstraction and mining of physiological data streams to develop process flow mappings that can be used to update patient journeys; instantiated in critical care within clinical practice guidelines. We demonstrate the framework within the neonatal intensive care setting, where we are performing clinical research in relation to pathophysiology within physiological streams of patients diagnosed with late onset neonatal sepsis. We present an instantiation of the framework for late onset neonatal sepsis, using CRISP-TDM for the process mining model and PaJMa for the knowledge representation.

Previous studies propose that associative classification has high classification accuracy and strong flexibility at handling unstructured data. However, it still suffers from the huge set of mined rules and sometimes biased classification or overfitting since the classification is based on only single high-confidence rule. In this study, we propose a new associative classification method, CMAR, i.e., Classification based on Multiple Association Rules. The method extends an efficient frequent pattern mining method, FP-growth, constructs a class distribution-associated FP-tree, and mines large database efficiently. Moreover, it applies a CR-tree structure to store and retrieve mined association rules efficiently, and prunes rules effectively based on confidence, correlation and database coverage. The classification is performed based on a weighted analysis using multiple strong association rules. Our extensive experiments on databases from UCI machine learning database repository show that CMAR is consistent, highly effective at classification of various kinds of databases and has better average classification accuracy in comparison with CBA and C4.5. Moreover, our performance study shows that the method is highly efficient and scalable in comparison with other reported associative classification methods.