Question: A review of bio-inspired computing algorithms and their scope of applications.

This paper offers a comprehensive examination of the utilization of genetic algorithms in addressing production and operations management dilemmas. Operations managers and academic researchers have increasingly turned to genetic algorithms to swiftly and effectively tackle complex real-world issues. As these algorithms demonstrate significant potential in problem-solving, a critical review of the problems they have been applied to, as well as the specific designs of the genetic algorithms used, becomes imperative for advancing future research. This study meticulously categorizes the reviewed literature based on the nature of the problems addressed. It delves into the fundamental architecture of each genetic algorithm, identifies the limitations and gaps in current research, and proposes recommendations for future investigations. By doing so, it aims to provide a solid foundation for scholars and practitioners seeking to harness the power of genetic algorithms in operations management.

**Question**: Evolutionary Algorithms: A Comprehensive Review

**Documents**: A review of evolutionary algorithms for optimization
Evolutionary algorithms are a family of population-based metaheuristic optimization techniques that mimic natural evolutionary processes. They have been successfully applied to a wide range of optimization problems across various fields. This paper reviews the development, main variants, and applications of evolutionary algorithms. We discuss the basic principles of genetic algorithms, genetic programming, evolutionary programming, and evolutionary strategies. The review also covers the recent advancements in hybrid evolutionary algorithms that combine traditional optimization methods with evolutionary mechanisms. We analyze the strengths and weaknesses of evolutionary algorithms and compare their performance with other optimization techniques. The review concludes with a discussion on the future trends and challenges in evolutionary optimization.

**Similar Documents**: This document provides an extensive review of evolutionary algorithms, encompassing their evolution, principal variants, and application in optimization problems across diverse disciplines. Evolutionary algorithms, inspired by natural evolution, are population-based metaheuristics that have proven effective in addressing a broad spectrum of optimization challenges. The review delves into the foundational concepts of genetic algorithms, genetic programming, evolutionary programming, and evolutionary strategies. It also explores the latest developments in hybrid evolutionary algorithms, which synergize traditional optimization methods with evolutionary principles. The document critically evaluates the advantages and drawbacks of evolutionary algorithms, juxtaposing their efficacy against other optimization methodologies. Lastly, it anticipates future directions and identifies key challenges in the realm of evolutionary optimization.

Data analysis plays an indispensable role for understanding various phenomena. Cluster analysis, primitive exploration with little or no prior knowledge, consists of research developed across a wide variety of communities. The diversity, on one hand, equips us with many tools. On the other hand, the profusion of options causes confusion. We survey clustering algorithms for data sets appearing in statistics, computer science, and machine learning, and illustrate their applications in some benchmark data sets, the traveling salesman problem, and bioinformatics, a new field attracting intensive efforts. Several tightly related topics, proximity measure, and cluster validation, are also discussed.

Biomimetic, or biologically inspired, design uses analogous biological phenomena to develop solutions for engineering problems. Several instances of biomimetic design result from personal observations of biological phenomena. However, many engineers’ knowledge of biology may be limited, thus reducing the potential of biologically inspired solutions. Our approach to biomimetic design takes advantage of the large amount of biological knowledge already available in books, journals, and so forth, by performing keyword searches on these existing natural-language sources. Because of the ambiguity and imprecision of natural language, challenges inherent to natural language processing were encountered. One challenge of retrieving relevant cross-domain information involves differences in domain vocabularies, or lexicons. A keyword meaningful to biologists may not occur to engineers. For an example problem that involved cleaning, that is, removing dirt, a biochemist suggested the keyword “defend.” Defend is not an obvious keyword to most engineers for this problem, nor are the words defend and “clean0remove” directly related within lexical references. However, previous work showed that biological phenomena retrieved by the keyword defend provided useful stimuli and produced successful concepts for the clean0remove problem. In this paper, we describe a method to systematically bridge the disparate biology and engineering domains using natural language analysis. For the clean0remove example, we were able to algorithmically generate several biologically meaningful keywords, including defend, that are not obviously related to the engineering problem. We developed a method to organize and rank the set of biologically meaningful keywords identified, and confirmed that we could achieve similar results for two other examples in encapsulation and microassembly. Although we specifically address cross-domain information retrieval from biology, the bridging process presented in this paper is not limited to biology, and can be used for any other domain given the availability of appropriate domain-specific knowledge sources and references.

Harmony search (HS) algorithm was applied to solving Sudoku puzzle. The HS is an evolutionary algorithm which mimics musicians’ behaviors such as random play, memory-based play, and pitch-adjusted play when they perform improvisation. Sudoku puzzles in this study were formulated as an optimization problem with number-uniqueness penalties. HS could successfully solve the optimization problem after 285 function evaluations, taking 9 seconds. Also, sensitivity analysis of HS parameters was performed to obtain a better idea of algorithm parameter values.

The purpose of this paper is to evaluate the performance of a new multiobjective algorithm called Vector Evaluated Population Based Incremental Learning (VEPBIL). The new algorithm was applied in solving a real world application named Reinsurance Contract Optimization (RCO), which is a multiobjective problem consisting of maximizing two conflicting functions: expected return and risk. The VEPBIL was tested on two instances of the problem composed by 7 and 15 layers of real anonymized data. In order to evaluate the algorithm, metrics such as hyper volume, number of solutions and coverage were used. A comparisons against Vector Evaluated Differential evolution (VEDE) is also carried out. The comparison has shown that VEPBIL can dominate about 70% and 50% of solutions from VEDE using 7 and 15 layers respectively, whereas VEDE dominates about 10% and 30% of solutions in the way around.

In this work, a proposed hybrid dragonfly algorithm (DA) with extreme learning machine (ELM) system for prediction problem is presented. ELM model is considered a promising method for data regression and classification problems. It has fast training advantage, but it always requires a huge number of nodes in the hidden layer. The usage of a large number of nodes in the hidden layer increases the test/evaluation time of ELM. Also, there is no guarantee of optimality of weights and biases settings on the hidden layer. DA is a recently promising optimization algorithm that mimics the moving behavior of moths. DA is exploited here to select less number of nodes in the hidden layer to speed up the performance of the ELM. It also is used to choose the optimal hidden layer weights and biases. A set of assessment indicators is used to evaluate the proposed and compared methods over ten regression data sets from the UCI repository. Results prove the capability of the proposed DA-ELM model in searching for optimal feature combinations in feature space to enhance ELM generalization ability and prediction accuracy. The proposed model was compared against the set of commonly used optimizers and regression systems. These optimizers are namely, particle swarm optimization (PSO) and genetic algorithm (GA). The proposed DA-ELM model proved an advance overall compared methods in both accuracy and generalization ability.

In this paper we describe JCLEC, a Java software system for the development of evolutionary computation applications. This system has been designed as a framework, applying design patterns to maximize its reusability and adaptability to new paradigms with a minimum of programming effort. JCLEC architecture comprises three main modules: the core contains all abstract type definitions and their implementation; experiments runner is a scripting environment to run algorithms in batch mode; finally, GenLab is a graphical user interface that allows users to configure an algorithm, to execute it interactively and to visualize the results obtained. The use of JCLEC system is illustrated though the analysis of one case study: the resolution of the 0/1 knapsack problem by means of evolutionary algorithms.

Artificial bee colony algorithm is a new population-based evolutionary method based on the intelligent behavior of honey bee swarm. It has shown more effective than other biological-inspired algorithms. However, there are still insufficiencies in ABC algorithm, which is good at exploration but poor at exploitation and its convergence speed is also an issue in some cases. For these insufficiencies, we propose a novel artificial bee colony algorithm (NABC) for numerical optimization problems in this paper to improve the exploitation capability by incorporating the current best solution into the search procedure. Experiments are conducted on a set of unimodal/multimodal benchmark functions. The experiments results of NABC have been compared with Gbest-guided artificial bee colony algorithm (G-ABC), improved artificial bee colony algorithm (I-ABC), Elitist artificial bee colony algorithm (E-ABC). The results show that NABC is superior to those algorithms in most of the tested functions.

An experimental comparison of evolutionary algorithms and gradient-based methods for the optimal control problem is carried out. The problem is solved separately by Particle swarm optimization, Grey wolf optimizer, Fast gradient descent method, Marquardt method and Adam method. The simulation is performed on a jet aircraft model. The results of each algorithm performance are compared according to the best found value of the fitness function, the mean value and the standard deviation.

The problem of controlling locomotion is an area in which neuroscience and robotics can fruitfully interact. In this article, I will review research carried out on locomotor central pattern generators (CPGs), i.e. neural circuits capable of producing coordinated patterns of high-dimensional rhythmic output signals while receiving only simple, low-dimensional, input signals. The review will first cover neurobiological observations concerning locomotor CPGs and their numerical modelling, with a special focus on vertebrates. It will then cover how CPG models implemented as neural networks or systems of coupled oscillators can be used in robotics for controlling the locomotion of articulated robots. The review also presents how robots can be used as scientific tools to obtain a better understanding of the functioning of biological CPGs. Finally, various methods for designing CPGs to control specific modes of locomotion will be briefly reviewed. In this process, I will discuss different types of CPG models, the pros and cons of using CPGs with robots, and the pros and cons of using robots as scientific tools. Open research topics both in biology and in robotics will also be discussed.

Conventionally, engineers have employed rigid materials to fabricate precise, predictable robotic systems, which are easily modelled as rigid members connected at discrete joints. Natural systems, however, often match or exceed the performance of robotic systems with deformable bodies. Cephalopods, for example, achieve amazing feats of manipulation and locomotion without a skeleton; even vertebrates such as humans achieve dynamic gaits by storing elastic energy in their compliant bones and soft tissues. Inspired by nature, engineers have begun to explore the design and control of soft-bodied robots composed of compliant materials. This Review discusses recent developments in the emerging field of soft robotics.


Question: A survey of bio-inspired computing algorithms and their potential applications.

This paper introduces the Social Spider Optimization (SSO) algorithm, a novel swarm optimization method inspired by the cooperative behavior of social spiders. SSO is grounded in swarm intelligence, a field that models the collective actions of insects or animals to address complex optimization challenges. The proposed SSO algorithm simulates the social dynamics and cooperation within spider colonies, where individuals (spiders) interact according to the biological principles governing the colony. Distinctively, SSO distinguishes between male and female spiders, each guided by unique evolutionary operators that reflect the varied cooperative behaviors observed within the spider colony. The efficacy and robustness of the SSO algorithm are demonstrated through comparisons with established evolutionary techniques. This assessment is conducted using a set of standard benchmark functions frequently employed in the evaluation of evolutionary algorithms. The results highlight the superior performance of SSO in locating the global optimum across multiple benchmark functions, showcasing its potential as a powerful optimization tool.

This paper presents a variant of particle swarm optimizers (PSOs) that we call the comprehensive learning particle swarm optimizer (CLPSO), which uses a novel learning strategy whereby all other particles' historical best information is used to update a particle's velocity. This strategy enables the diversity of the swarm to be preserved to discourage premature convergence. Experiments were conducted (using codes available from http://www.ntu.edu.sg/home/epnsugan) on multimodal test functions such as Rosenbrock, Griewank, Rastrigin, Ackley, and Schwefel and composition functions both with and without coordinate rotation. The results demonstrate good performance of the CLPSO in solving multimodal problems when compared with eight other recent variants of the PSO.

Algorithmic composition is the partial or total automation of the process of music composition by using computers. Since the 1950s, different computational techniques related to Artificial Intelligence have been used for algorithmic composition, including grammatical representations, probabilistic methods, neural networks, symbolic rule-based systems, constraint programming and evolutionary algorithms. This survey aims to be a comprehensive account of research on algorithmic composition, presenting a thorough view of the field for researchers in Artificial Intelligence.

The partitional clustering concept started with K-means algorithm which was published in 1957. Since then many classical partitional clustering algorithms have been reported based on gradient descent approach. The 1990 kick started a new era in cluster analysis with the application of nature inspired metaheuristics. After initial formulation nearly two decades have passed and researchers have developed numerous new algorithms in this field. This paper embodies an up-to-date review of all major nature inspired metaheuristic algorithms employed till date for partitional clustering. Further, key issues involved during formulation of various metaheuristics as a clustering problem and major application areas are discussed. & 2014 Published by Elsevier B.V.

The mechanisms underlying life machinery are still not completely understood. Something is known, something is “probably” known, other things are still unknown. Scientists all over the world are working very hard to clarify the processes regulating the cell life cycle and bioinformaticians try to support them by developing specialized automated tools. Within the plethora of applications devoted to the study of life mechanisms, tools for the analysis and comparison of biological networks are catching the attention of many researchers. It is interesting to investigate why.

The No Free Lunch (NFL) theorems for search and optimization are reviewed and their implications for the design of metaheurist ics are discussed. The theorems state that any two search or optimization algorith ms are equivalent when their performance is averaged across all possible problems and even over subsets of problems fulfilling certain constraints. The NFL results show that if there is no assumption regarding the relation between visited and unse en arch points, efficient search and optimization is impossible. There is no wel l performing universal metaheuristic, but the heuristics must be tailored to the pr oblem class at hand using prior knowledge. In practice, it is not likely that the pr econditions of the NFL theorems are fulfilled for a problem class and thus differenc es between algorithms exist. Therefore, tailored algorithms can exploit structu re nderlying the optimization problem. Given full knowledge about the problem class, it i in theory possible to construct an optimal algorithm.

The use of gesture as a natural interface serves as a motivating force for research in modeling, analyzing and recognition of gestures. In particular, human computer intelligent interaction needs vision-based gesture recognition, which involves many interdisciplinary studies. A survey on recent vision-based gesture recognition approaches is given in this paper. We shall review methods of static hand posture and temporal gesture recognition. Several application systems of gesture recognition are also described in this paper. We conclude with some thoughts about future research directions.

The classification problem can be addressed by numerous techniques and algorithms which belong to different paradigms of machine learning. In this paper, we are interested in evolutionary algorithms, the so-called genetics-based machine learning algorithms. In particular, we will focus on evolutionary approaches that evolve a set of rules, i.e., evolutionary rule-based systems, applied to classification tasks, in order to provide a state of the art in this field. This paper has a double aim: to present a taxonomy of the genetics-based machine learning approaches for rule induction, and to develop an empirical analysis both for standard classification and for classification with imbalanced data sets. We also include a comparative study of the genetics-based machine learning (GBML) methods with some classical non-evolutionary algorithms, in order to observe the suitability and high potential of the search performed by evolutionary algorithms and the behavior of the GBML algorithms in contrast to the classical approaches, in terms of classification accuracy.

0957-4174/$ see front matter 2011 Elsevier Ltd. A doi:10.1016/j.eswa.2011.02.075 ⇑ Corresponding author. Tel.: +4773597119. E-mail addresses: hanhong@stud.ntnu.no (H (Y. Wang), Kesheng.wang@ntnu.no (K. Wang), (Y. Chen). One of the most studied problems in the financial investment expert system is the intractability of portfolios. The non-linear constrained portfolio optimization problem with multi-objective functions cannot be efficiently solved using traditionally approaches. This paper presents a meta-heuristic approach to portfolio optimization problem using Particle Swarm Optimization (PSO) technique. The model is tested on various restricted and unrestricted risky investment portfolios and a comparative study with Genetic Algorithms is implemented. The PSO model demonstrates high computational efficiency in constructing optimal risky portfolios. Preliminary results show that the approach is very promising and achieves results comparable or superior with the state of the art solvers. 2011 Elsevier Ltd. All rights reserved.

This chapter describes the physiological computing paradigm where electrophysiological changes from the human nervous system are used to interface with a computer system in real time. Physiological computing systems are categorized into five categories: muscle interfaces, brain-computer interfaces, biofeedback, biocybernetic adaptation and ambulatory monitoring. The differences and similarities of each system are described. The chapter also discusses a number of fundamental issues for the design of physiological computing system, these include: the inference between physiology and behaviour, how the system represents behaviour, the concept of the biocybernetic control loop and ethical issues.

1 School of Electrical Engineering and Computer Science, Ohio University, Athens, Ohio, United States of America, 2 Bioinformatics and Research Computing, Whitehead Institute, Cambridge, Massachusetts, United States of America, 3 Department of Biological Sciences and School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States of America, 4 European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, United Kingdom, 5 School of Informatics and Computing, Indiana University, Bloomington, Indiana, United States of America, 6 School of Computer Science and Engineering, The University of New South Wales, Sydney, New South Wales, Australia, 7 The Genome Analysis Centre, Norwich Research Park, Norwich, United Kingdom


Question: A review of bio-inspired computing algorithms and their potential applications.

Inspired by the echolocation capabilities of micro bats, this paper introduces a novel bat-inspired optimization algorithm that integrates principles from evolutionary computation and swarm intelligence. The primary objective is to tackle both mono and multiobjective optimization challenges associated with brushless DC wheel motor design problems. Specifically, the mono-objective problem involves 5 design parameters and 6 constraints, while the multiobjective variant includes 2 objectives, 5 design parameters, and 5 constraints. The proposed algorithm's effectiveness is demonstrated by comparing its performance against other optimization techniques documented in recent literature, showcasing its capability to efficiently solve highly nonlinear problems prevalent in the field of electromagnetics. This innovative approach not only highlights the potential of bio-inspired algorithms but also expands the scope of applications for such computational techniques in engineering and design optimization.

As both CPUs and GPUs become employed in a wide range of applications, it has been acknowledged that both of these Processing Units (PUs) have their unique features and strengths and hence, CPU-GPU collaboration is inevitable to achieve high-performance computing. This has motivated a significant amount of research on heterogeneous computing techniques, along with the design of CPU-GPU fused chips and petascale heterogeneous supercomputers. In this article, we survey Heterogeneous Computing Techniques (HCTs) such as workload partitioning that enable utilizing both CPUs and GPUs to improve performance and/or energy efficiency. We review heterogeneous computing approaches at runtime, algorithm, programming, compiler, and application levels. Further, we review both discrete and fused CPU-GPU systems and discuss benchmark suites designed for evaluating Heterogeneous Computing Systems (HCSs). We believe that this article will provide insights into the workings and scope of applications of HCTs to researchers and motivate them to further harness the computational powers of CPUs and GPUs to achieve the goal of exascale performance.

Ant colony optimization and artificial potential field were used respectively as global path planning and local path planning methods in this paper. Some modifications were made to accommodate ant colony optimization to path planning. Pheromone generated by ant colony optimization was also utilized to prevent artificial potential field from getting local minimum. Simulation results showed that the hybrid algorithm could satisfy the real-time demand. The comparison between ant colony optimization and genetic algorithm was also made in this paper.

The biological immune system is an autonomic system for self-protection, which has evolved over millions of years probably through extensive redesigning, testing, tuning and optimization process. The powerful information processing capabilities of the immune system, such as feature extraction, pattern recognition, learning, memory, and its distributive nature provide rich metaphors for its artificial counterpart. Our study focuses on building an autonomic defense system, using some immunological metaphors for information gathering, analyzing, decision making and launching threat and attack responses. This on-going research effort is not to mimic the nature but to explore and learn valuable lessons useful for self-adaptive cyber defense systems.

Name of founding work in the area. Adaptation is key to survival and evolution. Evolution implicitly optimizes organisims. AI wants to mimic biological optimization { Survival of the ttest { Exploration and exploitation { Niche nding { Robust across changing environments (Mammals v. Dinos) { Self-regulation,-repair and-reproduction 2 Artiicial Inteligence Some deenitions { "Making computers do what they do in the movies" { "Making computers do what humans (currently) do best" { "Giving computers common sense; letting them make simple deci-sions" (do as I want, not what I say) { "Anything too new to be pidgeonholed" Adaptation and modiication is root of intelligence Some (Non-GA) branches of AI: { Expert Systems (Rule based deduction)

AS-PSO-2Opt is a new enhancement of the AS-PSO method. In the classical AS-PSO, the Ant heuristic is used to optimize the tour length of a Traveling Salesman Problem, TSP, and PSO is applied to optimize three parameters of ACO, (α, β, ρ). The AS-PSO-2Opt consider a post processing resuming path redundancy, helping to improve local solutions and to decrease the probability of falling in local minimum. Applied to TSP, the method allowed retrieving a valuable path solution and a set of fitted parameters for ACO. The performance of the AS-PSO-2Opt is tested on nine different TSP test benches. Experimental results based on a statistical analysis showed that the new proposal performs better than key state of art methods using Genetic algorithm, Neural Network and ACO algorithm. The AS-PSO-2Opt performs better than close related methods such as PSO-ACO-3Opt [9] and ACO with ABC [19] for various test benches.

In this paper, a black hole optimization algorithm (BH) is utilized to solve the optimal power flow problem considering the generation fuel cost, reduction of voltage deviation and improvement of voltage stability as an objective functions. The black hole algorithm simulate the black hole phenomenon which relay on tow operations, the star absorption and star sucking. The IEEE 30-Bus and IEEE 57-Bus systems are used to illustrate performance of the proposed algorithm and results are compared with those in literatures.

It is broadly recognised by global companies that supply chain management is one of the major core competencies for an organisation to compete in the marketplace. Organisational strategies are mainly concentrated on improvement of customer service levels as well as reduction of operational costs in order to maintain profit margins. Therefore supply chain performance has attracted researchers’ attention. A variety of soft computing techniques including fuzzy logic and genetic algorithms have been employed to improve effectiveness and efficiency in various aspects of supply chain management. Meanwhile, an increasing number of papers have been published to address related issues. The aim of this paper is to summarise the findings by a systematic review of existing research papers concerning the application of soft computing techniques to supply chain management. Some areas in supply chain management that have rarely been exposed in existing papers, such as customer relationship management and reverse logistics, are therefore suggested for future research. 2009 Elsevier B.V. All rights reserved.

In this work, a proposed hybrid dragonfly algorithm (DA) with extreme learning machine (ELM) system for prediction problem is presented. ELM model is considered a promising method for data regression and classification problems. It has fast training advantage, but it always requires a huge number of nodes in the hidden layer. The usage of a large number of nodes in the hidden layer increases the test/evaluation time of ELM. Also, there is no guarantee of optimality of weights and biases settings on the hidden layer. DA is a recently promising optimization algorithm that mimics the moving behavior of moths. DA is exploited here to select less number of nodes in the hidden layer to speed up the performance of the ELM. It also is used to choose the optimal hidden layer weights and biases. A set of assessment indicators is used to evaluate the proposed and compared methods over ten regression data sets from the UCI repository. Results prove the capability of the proposed DA-ELM model in searching for optimal feature combinations in feature space to enhance ELM generalization ability and prediction accuracy. The proposed model was compared against the set of commonly used optimizers and regression systems. These optimizers are namely, particle swarm optimization (PSO) and genetic algorithm (GA). The proposed DA-ELM model proved an advance overall compared methods in both accuracy and generalization ability.

In this paper, we present an enhanced parallel cat swarm optimization (EPCSO) method for solving numerical optimization problems. The parallel cat swarm optimization (PCSO) method is an optimization algorithm designed to solve numerical optimization problems under the conditions of a small population size and a few iteration numbers. The Taguchi method is widely used in the industry for optimizing the product and the process conditions. By adopting the Taguchi method into the tracing mode process of the PCSO method, we propose the EPCSO method with better accuracy and less computational time. In this paper, five test functions are used to evaluate the accuracy of the proposed EPCSO method. The experimental results show that the proposed EPCSO method gets higher accuracies than the existing PSO-based methods and requires less computational time than the PCSO method. We also apply the proposed method to solve the aircraft schedule recovery problem. The experimental results show that the proposed EPCSO method can provide the optimum recovered aircraft schedule in a very short time. The proposed EPCSO method gets the same recovery schedule having the same total delay time, the same delayed flight numbers and the same number of long delay flights as the Liu, Chen, and Chou method (2009). The optimal solutions can be found by the proposed EPCSO method in a very short time. 2011 Elsevier Ltd. All rights reserved.

This paper presents a variant of particle swarm optimizers (PSOs) that we call the comprehensive learning particle swarm optimizer (CLPSO), which uses a novel learning strategy whereby all other particles' historical best information is used to update a particle's velocity. This strategy enables the diversity of the swarm to be preserved to discourage premature convergence. Experiments were conducted (using codes available from http://www.ntu.edu.sg/home/epnsugan) on multimodal test functions such as Rosenbrock, Griewank, Rastrigin, Ackley, and Schwefel and composition functions both with and without coordinate rotation. The results demonstrate good performance of the CLPSO in solving multimodal problems when compared with eight other recent variants of the PSO.


Question: Learn multi-prototype entity mention embedding to connect text and knowledge.

This paper introduces a novel approach for entity linking that harnesses the power of convolutional neural networks to discern semantic similarity between entity mentions and potential target entities. A central issue in entity linking is the accurate interpretation of mentions, which may refer to distinct entities depending on the context. Our model employs convolutional neural networks to analyze the context surrounding a mention and establish a semantic link with a suggested entity. These networks operate at various levels of detail to capture diverse thematic nuances, and their extensive parameterization allows them to identify n-grams that are indicative of different topics. To further enhance performance, we integrate these networks with a sparse linear model, resulting in a system that surpasses state-of-the-art results on multiple entity linking datasets. Our approach outperforms previous methods, as demonstrated by comparisons with systems developed by Durrett and Klein (2014) and Nguyen et al. (2014). This method provides a robust framework for understanding the nuanced connections between entity mentions and their referents, thereby advancing the field of entity linking.

This paper proposes a neural generative model, namely Table2Seq, to generate a natural language sentence based on a table. Specifically, the model maps a table to continuous vectors and then generates a natural language sentence by leveraging the semantics of a table. Since rare words, e.g., entities and values, usually appear in a table, we develop a flexible copying mechanism that selectively replicates contents from the table to the output sequence. We conduct extensive experiments to demonstrate the effectiveness of our Table2Seq model and the utility of the designed copying mechanism. On the WIKIBIO and SIMPLEQUESTIONS datasets, the Table2Seq model improves the state-of-the-art results from 34.70 to 40.26 and from 33.32 to 39.12 in terms of BLEU-4 scores, respectively. Moreover, we construct an open-domain dataset WIKITABLETEXT that includes 13 318 descriptive sentences for 4962 tables. Our Table2Seq model achieves a BLEU-4 score of 38.23 on WIKITABLETEXT outperforming template-based and language model based approaches. Furthermore, through experiments on 1 M table-query pairs from a search engine, our Table2Seq model considering the structured part of a table, i.e., table attributes and table cells, as additional information outperforms a sequence-to-sequence model considering only the sequential part of a table, i.e., table caption.

We present an unsupervised explainable vector embedding technique, called EVE, which is built upon the structure of Wikipedia. The proposed model defines the dimensions of a semantic vector representing a concept using human-readable labels, thereby it is readily interpretable. Specifically, each vector is constructed using the Wikipedia category graph structure together with the Wikipedia article link structure. To test the effectiveness of the proposed model, we consider its usefulness in three fundamental tasks: 1) intruder detection—to evaluate its ability to identify a non-coherent vector from a list of coherent vectors, 2) ability to cluster—to evaluate its tendency to group related vectors together while keeping unrelated vectors in separate clusters, and 3) sorting relevant items first—to evaluate its ability to rank vectors (items) relevant to the query in the top order of the result. For each task, we also propose a strategy to generate a task-specific human-interpretable explanation from the model. These demonstrate the overall effectiveness of the explainable embeddings generated by EVE. Finally, we compare EVE with the Word2Vec, FastText, and GloVe embedding techniques across the three tasks, and report improvements over the state-of-the-art.

We present our winning solution for the WSDM Cup 2017 triple scoring task. We devise an ensemble of four base scorers, so as to leverage the power of both text and knowledge bases for that task. Then we further refine the outputs of the ensemble by trigger word detection, achieving even better predictive accuracy. The code is available at https://github.com/wsdm-cup-2017/bokchoy.

Existing knowledge-based question answering systems often rely on small annotated training data. While shallow methods like relation extraction are robust to data scarcity, they are less expressive than the deep meaning representation methods like semantic parsing, thereby failing at answering questions involving multiple constraints. Here we alleviate this problem by empowering a relation extraction method with additional evidence from Wikipedia. We first present a neural network based relation extractor to retrieve the candidate answers from Freebase, and then infer over Wikipedia to validate these answers. Experiments on the WebQuestions question answering dataset show that our method achieves an F1 of 53.3%, a substantial improvement over the state-of-the-art.

Distantly supervised approaches have become popular in recent years as they allow training relation extractors without textbound annotation, using instead known relations from a knowledge base and a large textual corpus from an appropriate domain. While state of the art distant supervision approaches use off-theshelf named entity recognition and classification (NERC) systems to identify relation arguments, discrepancies in domain or genre between the data used for NERC training and the intended domain for the relation extractor can lead to low performance. This is particularly problematic for “non-standard” named entities such as album which would fall into the MISC category. We propose to ameliorate this issue by jointly training the named entity classifier and the relation extractor using imitation learning which reduces structured prediction learning to classification learning. We further experiment with Web features different features and compare against using two off-the-shelf supervised NERC systems, Stanford NER and FIGER, for named entity classification. Our experiments show that imitation learning improves average precision by 4 points over an one-stage classification model, while removing Web features results in a 6 points reduction. Compared to using FIGER and Stanford NER, average precision is 10 points and 19 points higher with our imitation learning approach.

Traditional relation extraction predicts relations within some fixed and finite target schema. Machine learning approaches to this task require either manual annotation or, in the case of distant supervision, existing structured sources of the same schema. The need for existing datasets can be avoided by using a universal schema: the union of all involved schemas (surface form predicates as in OpenIE, and relations in the schemas of preexisting databases). This schema has an almost unlimited set of relations (due to surface forms), and supports integration with existing structured data (through the relation types of existing databases). To populate a database of such schema we present matrix factorization models that learn latent feature vectors for entity tuples and relations. We show that such latent models achieve substantially higher accuracy than a traditional classification approach. More importantly, by operating simultaneously on relations observed in text and in pre-existing structured DBs such as Freebase, we are able to reason about unstructured and structured data in mutually-supporting ways. By doing so our approach outperforms stateof-the-art distant supervision.

As part of the information fusion task we wish to automatically fuse information derived from the text extraction process with data from a structured knowledge base. This process will involve resolving, aggregating, integrating and abstracting information via the methodologies of Knowledge Representation and Reasoning into a single comprehensive description of an individual or event. This report surveys the key principles underlying research in the field of Knowledge Representation and Reasoning. It represents an initial step in deciding upon a Knowledge Representation and Reasoning system for our information fusion task. APPROVED FOR PUBLIC RELEASE

In this work we present a novel approach for the utilization of observed relations between entity pairs in the task of triple argument prediction. The approach is based on representing observations in a shared, continuous vector space of structured relations and text. Results on a recent benchmark dataset demonstrate that the new model is superior to existing sparse feature models. In combination with state-of-the-art models, we achieve substantial improvements when observed relations are available.

The reading comprehension task, that asks questions about a given evidence document, is a central problem in natural language understanding. Recent formulations of this task have typically focused on answer selection from a set of candidates pre-defined manually or through the use of an external NLP pipeline. However, Rajpurkar et al. (2016) recently released the SQUAD dataset in which the answers can be arbitrary strings from the supplied text. In this paper, we focus on this answer extraction task, presenting a novel model architecture that efficiently builds fixed length representations of all spans in the evidence document with a recurrent network. We show that scoring explicit span representations significantly improves performance over other approaches that factor the prediction into separate predictions about words or start and end markers. Our approach improves upon the best published results of Wang & Jiang (2016) by 5% and decreases the error of Rajpurkar et al.’s baseline by > 50%.

Machine reading calls for programs that read and understand text, but most current work only attempts to extract facts from redundant web-scale corpora. In this paper, we focus on a new reading comprehension task that requires complex reasoning over a single document. The input is a paragraph describing a biological process, and the goal is to answer questions that require an understanding of the relations between entities and events in the process. To answer the questions, we first predict a rich structure representing the process in the paragraph. Then, we map the question to a formal query, which is executed against the predicted structure. We demonstrate that answering questions via predicted structures substantially improves accuracy over baselines that use shallower representations.


Question: Learn multi-prototype entity mention embedding to bridge text and knowledge.

This document introduces an innovative method for relation extraction from unstructured text, designed to integrate information from both the textual content and existing knowledge bases. The proposed model is grounded in dual scoring functions, which facilitate the learning of low-dimensional embeddings for words, entities, and relationships sourced from a knowledge base. Through extensive experimentation on New York Times articles that are linked with Freebase relations, we substantiate the efficacy of our approach. Specifically, it is demonstrated that by leveraging the extensive information contained within a sizable portion of Freebase data—comprising 4 million entities and 23,000 relationships—the model significantly outperforms conventional methods that solely depend on textual features. This achievement underscores the model's capability to effectively harness the supplementary insights provided by knowledge bases to enhance relation extraction accuracy.

Many applications that process social data, such as tweets, must extract entities from tweets (e.g., “Obama” and “Hawaii” in “Obama went to Hawaii”), link them to entities in a knowledge base (e.g., Wikipedia), classify tweets into a set of predefined topics, and assign descriptive tags to tweets. Few solutions exist today to solve these problems for social data, and they are limited in important ways. Further, even though several industrial systems such as OpenCalais have been deployed to solve these problems for text data, little if any has been published about them, and it is unclear if any of the systems has been tailored for social media. In this paper we describe in depth an end-to-end industrial system that solves these problems for social data. The system has been developed and used heavily in the past three years, first at Kosmix, a startup, and later at WalmartLabs. We show how our system uses a Wikipedia-based global “real-time” knowledge base that is well suited for social data, how we interleave the tasks in a synergistic fashion, how we generate and use contexts and social signals to improve task accuracy, and how we scale the system to the entire Twitter firehose. We describe experiments that show that our system outperforms current approaches. Finally we describe applications of the system at Kosmix and WalmartLabs, and lessons learned.

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while ‘multi-sense’ methods have been proposed and tested on artificial wordsimilarity tasks, we don’t know if they improve real natural language understanding tasks. In this paper we introduce a multisense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language un-

We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel form of semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in state-of-the-art-performance.

Large-scale knowledge graphs such as those in the Linked Data cloud are typically represented as subject-predicate-object triples. However, many facts about the world involve more than two entities. While n-ary relations can be converted to triples in a number of ways, unfortunately, the structurally different choices made in different knowledge sources significantly impede our ability to connect them. They also make it impossible to query the data concisely and without prior knowledge of each individual source. We present FrameBase, a wide-coverage knowledge-base schema that uses linguistic frames to seamlessly represent and query n-ary relations from other knowledge bases, at different levels of granularity connected by logical entailment. It also opens possibilities to draw on natural language processing techniques for querying and data mining.

Distant supervision has attracted recent interest for training information extraction systems because it does not require any human annotation but rather employs existing knowledge bases to heuristically label a training corpus. However, previous work has failed to address the problem of false negative training examples mislabeled due to the incompleteness of knowledge bases. To tackle this problem, we propose a simple yet novel framework that combines a passage retrieval model using coarse features into a state-of-the-art relation extractor using multi-instance learning with fine features. We adapt the information retrieval technique of pseudorelevance feedback to expand knowledge bases, assuming entity pairs in top-ranked passages are more likely to express a relation. Our proposed technique significantly improves the quality of distantly supervised relation extraction, boosting recall from 47.7% to 61.2% with a consistently high level of precision of around 93% in the experiments.

Today, the practice of returning entities from a knowledge base in response to search queries has become widespread. One of the distinctive characteristics of entities is that they are typed, i.e., assigned to some hierarchically organized type system (type taxonomy). The primary objective of this paper is to gain a better understanding of how entity type information can be utilized in entity retrieval. We perform this investigation in an idealized "oracle" setting, assuming that we know the distribution of target types of the relevant entities for a given query. We perform a thorough analysis of three main aspects: (i) the choice of type taxonomy, (ii) the representation of hierarchical type information, and (iii) the combination of type-based and term-based similarity in the retrieval model. Using a standard entity search test collection based on DBpedia, we find that type information proves most useful when using large type taxonomies that provide very specific types. We provide further insights on the extensional coverage of entities and on the utility of target types.

BACKGROUND
While representation learning techniques have shown great promise in application to a number of different NLP tasks, they have had little impact on the problem of ontology matching. Unlike past work that has focused on feature engineering, we present a novel representation learning approach that is tailored to the ontology matching task. Our approach is based on embedding ontological terms in a high-dimensional Euclidean space. This embedding is derived on the basis of a novel phrase retrofitting strategy through which semantic similarity information becomes inscribed onto fields of pre-trained word vectors. The resulting framework also incorporates a novel outlier detection mechanism based on a denoising autoencoder that is shown to improve performance.

RESULTS
An ontology matching system derived using the proposed framework achieved an F-score of 94% on an alignment scenario involving the Adult Mouse Anatomical Dictionary and the Foundational Model of Anatomy ontology (FMA) as targets. This compares favorably with the best performing systems on the Ontology Alignment Evaluation Initiative anatomy challenge. We performed additional experiments on aligning FMA to NCI Thesaurus and to SNOMED CT based on a reference alignment extracted from the UMLS Metathesaurus. Our system obtained overall F-scores of 93.2% and 89.2% for these experiments, thus achieving state-of-the-art results.

CONCLUSIONS
Our proposed representation learning approach leverages terminological embeddings to capture semantic similarity. Our results provide evidence that the approach produces embeddings that are especially well tailored to the ontology matching task, demonstrating a novel pathway for the problem.

Common-sense or background knowledge is required to understand natural language, but in most neural natural language understanding (NLU) systems, the requisite background knowledge is indirectly acquired from static corpora. We develop a new reading architecture for the dynamic integration of explicit background knowledge in NLU models. A new task-agnostic reading module provides refined word representations to a task-specific NLU architecture by processing background knowledge in the form of free-text statements, together with the taskspecific inputs. Strong performance on the tasks of document question answering (DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness and flexibility of our approach. Analysis shows that our models learn to exploit knowledge selectively and in a semantically appropriate way.

Knowledge can often be represented using entities connected by relations. For example, the fact that tennis ball is round can be represented as “TennisBall HasShape Round”, where a “TennisBall” is one entity, “HasShape” is a relation and “Round” is another entity. A knowledge base is a way to store such structured information, a knowledge base stores triples of the “an entity-relation-an entity” form, and a real world knowledge base often has millions or billions of such triples. There are several well-known knowledge bases including FreeBase [1], WordNet [2], YAGO [3], etc. They are important in fields like reasoning and question answering; for instance if one asks “what is the shape of a tennis ball”, we can search the knowledge base for the triple as “TennisBall HasShape Round” and output “round” as the answer.

How can we enable computers to automatically answer questions like “Who created the character Harry Potter”? Carefully built knowledge bases provide rich sources of facts. However, it remains a challenge to answer factoid questions raised in natural language due to numerous expressions of one question. In particular, we focus on the most common questions — ones that can be answered with a single fact in the knowledge base. We propose CFO, a Conditional Focused neuralnetwork-based approach to answering factoid questions with knowledge bases. Our approach first zooms in a question to find more probable candidate subject mentions, and infers the final answers with a unified conditional probabilistic framework. Powered by deep recurrent neural networks and neural embeddings, our proposed CFO achieves an accuracy of 75.7% on a dataset of 108k questions – the largest public one to date. It outperforms the current state of the art by an absolute margin of 11.8%.


Question: Learn multi-prototype entity mention embedding to connect text and knowledge.

This paper explores a novel approach, Visual Knowledge Extraction (VisKE), for validating statements about the world by integrating textual and visual data. A key challenge in knowledge extraction is determining the truthfulness of relationships between entities. Traditionally, methods have relied solely on textual analysis for this purpose. VisKE, however, introduces the concept of visual verification for relation phrases, offering a comprehensive system for assessing their validity. The system evaluates the accuracy of verb-based relationships between common nouns by analyzing both textual descriptions and associated images. It focuses on the spatial consistency between the entities and the action they are involved in, without requiring any explicit human supervision. This capability facilitates large-scale validation of relation phrases, with VisKE already having verified over 12,000 such phrases. Beyond enriching textual knowledge bases by enhancing recall, VisKE also contributes to question-answering systems by augmenting their reasoning capabilities in open-domain scenarios. This dual approach of textual and visual analysis significantly enhances the robustness of knowledge extraction and question-answering systems.

One of the main obstacles to highperformance Word Sense Disambiguation (WSD) is the knowledge acquisition bottleneck. In this paper, we present a methodology to automatically extend WordNet with large amounts of semantic relations from an encyclopedic resource, namely Wikipedia. We show that, when provided with a vast amount of high-quality semantic relations, simple knowledge-lean disambiguation algorithms compete with state-of-the-art supervised WSD systems in a coarse-grained all-words setting and outperform them on gold-standard domain-specific datasets.

Traditional relation extraction predicts relations within some fixed and finite target schema. Machine learning approaches to this task require either manual annotation or, in the case of distant supervision, existing structured sources of the same schema. The need for existing datasets can be avoided by using a universal schema: the union of all involved schemas (surface form predicates as in OpenIE, and relations in the schemas of preexisting databases). This schema has an almost unlimited set of relations (due to surface forms), and supports integration with existing structured data (through the relation types of existing databases). To populate a database of such schema we present matrix factorization models that learn latent feature vectors for entity tuples and relations. We show that such latent models achieve substantially higher accuracy than a traditional classification approach. More importantly, by operating simultaneously on relations observed in text and in pre-existing structured DBs such as Freebase, we are able to reason about unstructured and structured data in mutually-supporting ways. By doing so our approach outperforms stateof-the-art distant supervision.

Link Prediction using Knowledge graph embedding projects symbolic entities and relations into low dimensional vector space, thereby learning the semantic relations between entities. Among various embedding models, there is a series of translation-based models such as TransE [1], TransH [2], and TransR[3]. This paper proposes modifications in the TransR model to address the issue of skewed data which is common in real-world knowledge graphs. The enhancements enable the model to smartly generate corrupted triplets during negative sampling, which significantly improves the training time and performance of TransR. The proposed approach can be applied to other translationbased models.

Identifying discourse relations that are not overtly marked with discourse connectives remains a challenging problem. The absence of explicit clues indicates a need for the combination of world knowledge and weak contextual clues, which can hardly be learned from a small amount of manually annotated data. In this paper, we address this problem by augmenting the input text with external knowledge and context and by adopting a neural network model that can effectively handle the augmented text. Experiments show that external knowledge did improve the classification accuracy. On the other hand, contextual information provided no significant gain for implicit discourse relations while it worked for explicit ones.

Question answering (QA) systems often consist of several components such as Named Entity Disambiguation (NED), Relation Extraction (RE), and Query Generation (QG). In this paper, we focus on the QG process of a QA pipeline on a large-scale Knowledge Base (KB), with noisy annotations and complex sentence structures. We therefore propose SQG, a SPARQL Query Generator with modular architecture, enabling easy integration with other components for the construction of a fully functional QA pipeline. SQG can be used on large open-domain KBs and handle noisy inputs by discovering a minimal subgraph based on uncertain inputs, that it receives from the NED and RE components. This ability allows SQG to consider a set of candidate entities/relations, as opposed to the most probable ones, which leads to a significant boost in the performance of the QG component. The captured subgraph covers multiple candidate walks, which correspond to SPARQL queries. To enhance the accuracy, we present a ranking model based on Tree-LSTM that takes into account the syntactical structure of the question and the tree representation of the candidate queries to find the one representing the correct intention behind the question. SQG outperforms the baseline systems and achieves a macro F1-measure of 75% on the LC-QuAD dataset.

With the advent of the Internet, large amount of digital text is generated everyday in the form of news articles, research publications, blogs, question answering forums and social media. It is important to develop techniques for extracting information automatically from these documents, as lot of important information is hidden within them. This extracted information can be used to improve access and management of knowledge hidden in large text corpora. Several applications such as Question Answering, Information Retrieval would benefit from this information. Entities like persons and organizations, form the most basic unit of the information. Occurrences of entities in a sentence are often linked through well-defined relations; e.g., occurrences of person and organization in a sentence may be linked through relations such as employed at. The task of Relation Extraction (RE) is to identify such relations automatically. In this paper, we survey several important supervised, semi-supervised and unsupervised RE techniques. We also cover the paradigms of Open Information Extraction (OIE) and Distant Supervision. Finally, we describe some of the recent trends in the RE techniques and possible future research directions. This survey would be useful for three kinds of readers i) Newcomers in the field who want to quickly learn about RE; ii) Researchers who want to know how the various RE techniques evolved over time and what are possible future research directions and iii) Practitioners who just need to know which RE technique works best in various settings.

We present the components of the ColumbiaGWU contribution to the 2016 TAC KBP BeSt Evaluation.

Knowledge graphs are typical large-scale multi-relational structures, which comprise a large amount of fact triplets. Nonetheless, existing knowledge graphs are still sparse and far from being complete. To refine the knowledge graphs, representation learning is widely used to embed fact triplets into low-dimensional spaces. Many existing knowledge graph embedding models either focus on learning rich features from entities but fail to extract good features of relations, or employ sophisticated models that have rather high time and memory-space complexities. In this paper, we propose a novel knowledge graph embedding model, CombinE. It exploits entity features from two complementary perspectives via the plus and minus combinations. We start with the plus combination, where we use shared features of entity pairs participating in a relation to convey its relation features. To also allow differences of each pairs of entities participating in a relation, we also use the minus combination, where we concentrate on individual entity features, and regard relations as a channel to offset the divergence and preserve the prominence between head and tail entities. Compared with the state-of-the-art models, our experimental results demonstrate that CombinE outperforms existing ones and has low time and memory-space complexities.

We present a question answering system over DBpedia, filling the gap between user information needs expressed in natural language and a structured query interface expressed in SPARQL over the underlying knowledge base (KB). Given the KB, our goal is to comprehend a natural language query and provide corresponding accurate answers. Focusing on solving the non-aggregation questions, in this paper, we construct a subgraph of the knowledge base from the detected entities and propose a graph traversal method to solve both the semantic item mapping problem and the disambiguation problem in a joint way. Compared with existing work, we simplify the process of query intention understanding and pay more attention to the answer path ranking. We evaluate our method on a non-aggregation question dataset and further on a complete dataset. Experimental results show that our method achieves best performance compared with several state-of-the-art systems.

In this paper, we describe a new method for the problem of named entity classification for specialized or technical domains, using distant supervision. Our approach relies on a simple observation: in some specialized domains, named entities are almost unambiguous. Thus, given a seed list of names of entities, it is cheap and easy to obtain positive examples from unlabeled texts using a simple string match. Those positive examples can then be used to train a named entity classifier, by using the PU learning paradigm, which is learning from positive and unlabeled examples. We introduce a new convex formulation to solve this problem, and apply our technique in order to extract named entities from financial reports corresponding to healthcare companies.


Question: Learn multi-prototype entity mention embedding to bridge text and knowledge.

This paper introduces a novel latent factor model designed to address the complexities of highly multi-relational data, a common feature in social networks, movie preferences, and knowledge bases. Multi-relational data, characterized by multiple relationships between entities, poses significant challenges in modeling, particularly when the number of relations increases significantly. Existing methodologies often struggle to cope with this complexity. Our proposed model aims to tackle this issue by effectively handling large multi-relational datasets, potentially encompassing thousands of distinct relations. The core of our approach is a bilinear structure, which not only captures various levels of interaction within the data but also shares sparse latent factors across different relations, enhancing the model's efficiency and scalability. We validate the effectiveness of our method through performance evaluations on standard tensor-factorization datasets, where our results either match or surpass state-of-the-art benchmarks. Additionally, a Natural Language Processing (NLP) application underscores our model's scalability and its capability to learn concise, semantically rich representations of verbs, showcasing its versatility in real-world applications.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design

**Similar Documents**: This paper presents a cutting-edge hybrid algorithm that merges the strengths of genetic algorithms (GA) and particle swarm optimization (PSO) to address the intricate task of designing recurrent networks. Unlike conventional approaches that rely solely on GA or PSO, this novel method, termed HGAPSO (Hybrid Genetic Algorithm-Particle Swarm Optimization), leverages the synergistic benefits of both techniques to create superior recurrent network designs. HGAPSO introduces an innovative elite strategy, where the most successful individuals within a population are selected, enhanced through a process inspired by PSO, and then reintroduced into the next generation. This mechanism not only preserves the best genetic traits but also promotes the evolution of these traits through simulated maturation. The application of HGAPSO is demonstrated in the design of recurrent neural networks for temporal sequence production and recurrent fuzzy networks for dynamic plant control. Comparative studies with traditional GA and PSO methods highlight the enhanced performance and adaptability of HGAPSO in recurrent network design, underscoring its potential as a powerful tool in the field.

Research on question answering with knowledge base has recently seen an increasing use of deep architectures. In this extended abstract, we study the application of the neural machine translation paradigm for question parsing. We employ a sequence-to-sequence model to learn graph patterns in the SPARQL graph query language and their compositions. Instead of inducing the programs through question-answer pairs, we expect a semi-supervised approach, where alignments between questions and queries are built through templates. We argue that the coverage of language utterances can be expanded using late notable works in natural language generation.

Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs. In this work, we propose holographic embeddings (HOLE) to learn compositional vector space representations of entire knowledge graphs. The proposed method is related to holographic models of associative memory in that it employs circular correlation to create compositional representations. By using correlation as the compositional operator, H OLE can capture rich interactions but simultaneously remains efficient to compute, easy to train, and scalable to very large datasets. Experimentally, we show that holographic embeddings are able to outperform state-ofthe-art methods for link prediction on knowledge graphs and relational learning benchmark datasets.

Relations are expressed in many domains such as newswire, weblogs and phone conversations. Trained on a source domain, a relation extractor’s performance degrades when applied to target domains other than the source. A common yet labor-intensive method for domain adaptation is to construct a target-domainspecific labeled dataset for adapting the extractor. In response, we present an unsupervised domain adaptation method which only requires labels from the source domain. Our method is a joint model consisting of a CNN-based relation classifier and a domain-adversarial classifier. The two components are optimized jointly to learn a domain-independent representation for prediction on the target domain. Our model outperforms the state-of-the-art on all three test domains of ACE 2005.

Multi-task learning in text classification leverages implicit correlations among related tasks to extract common features and yield performance gains. However, most previous works treat labels of each task as independent and meaningless onehot vectors, which cause a loss of potential information and makes it difficult for these models to jointly learn three or more tasks. In this paper, we propose Multi-Task Label Embedding to convert labels in text classification into semantic vectors, thereby turning the original tasks into vector matching tasks. We implement unsupervised, supervised and semisupervised models of Multi-Task Label Embedding, all utilizing semantic correlations among tasks and making it particularly convenient to scale and transfer as more tasks are involved. Extensive experiments on five benchmark datasets for text classification show that our models can effectively improve performances of related tasks with semantic representations of labels and additional information from each other.

Answer selection and knowledge base question answering (KBQA) are two important tasks of question answering (QA) systems. Existing methods solve these two tasks separately, which requires large number of repetitive work and neglects the rich correlation information between tasks. In this paper, we tackle answer selection and KBQA tasks simultaneously via multi-task learning (MTL), motivated by the following motivations. First, both answer selection and KBQA can be regarded as a ranking problem, with one at text-level while the other at knowledge-level. Second, these two tasks can benefit each other: answer selection can incorporate the external knowledge from knowledge base (KB), while KBQA can be improved by learning contextual information from answer selection. To fulfill the goal of jointly learning these two tasks, we propose a novel multi-task learning scheme that utilizes multi-view attention learned from various perspectives to enable these tasks to interact with each other as well as learn more comprehensive sentence representations. The experiments conducted on several real-world datasets demonstrate the effectiveness of the proposed method, and the performance of answer selection and KBQA is improved. Also, the multi-view attention scheme is proved to be effective in assembling attentive information from different representational perspectives.

Traditional relation extraction predicts relations within some fixed and finite target schema. Machine learning approaches to this task require either manual annotation or, in the case of distant supervision, existing structured sources of the same schema. The need for existing datasets can be avoided by using a universal schema: the union of all involved schemas (surface form predicates as in OpenIE, and relations in the schemas of preexisting databases). This schema has an almost unlimited set of relations (due to surface forms), and supports integration with existing structured data (through the relation types of existing databases). To populate a database of such schema we present matrix factorization models that learn latent feature vectors for entity tuples and relations. We show that such latent models achieve substantially higher accuracy than a traditional classification approach. More importantly, by operating simultaneously on relations observed in text and in pre-existing structured DBs such as Freebase, we are able to reason about unstructured and structured data in mutually-supporting ways. By doing so our approach outperforms stateof-the-art distant supervision.

Entity Linking (EL) and Word Sense Disambiguation (WSD) both address the lexical ambiguity of language. But while the two tasks are pretty similar, they differ in a fundamental respect: in EL the textual mention can be linked to a named entity which may or may not contain the exact mention, while in WSD there is a perfect match between the word form (better, its lemma) and a suitable word sense. In this paper we present Babelfy, a unified graph-based approach to EL and WSD based on a loose identification of candidate meanings coupled with a densest subgraph heuristic which selects high-coherence semantic interpretations. Our experiments show state-of-the-art performances on both tasks on 6 different datasets, including a multilingual setting. Babelfy is online at http://babelfy.org

Identifying discourse relations that are not overtly marked with discourse connectives remains a challenging problem. The absence of explicit clues indicates a need for the combination of world knowledge and weak contextual clues, which can hardly be learned from a small amount of manually annotated data. In this paper, we address this problem by augmenting the input text with external knowledge and context and by adopting a neural network model that can effectively handle the augmented text. Experiments show that external knowledge did improve the classification accuracy. On the other hand, contextual information provided no significant gain for implicit discourse relations while it worked for explicit ones.

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while ‘multi-sense’ methods have been proposed and tested on artificial wordsimilarity tasks, we don’t know if they improve real natural language understanding tasks. In this paper we introduce a multisense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language un-

We consider the task of predicting lexical entailment using distributional vectors. We focus experiments on one previous classifier which was shown to only learn to detect prototypicality of a word pair. Analysis shows that the model single-mindedly learns to detect Hearst Patterns, which are well known to be predictive of lexical relations. We present a new model which exploits this Hearst Detector functionality, matching or outperforming prior work on multiple data sets.


Question: Learn multi-prototype entity mention embedding to bridge text and knowledge.

This paper introduces a novel method for embedding structured data from Knowledge Bases (KBs) into continuous vector spaces, preserving and enhancing the inherent information. With the proliferation of KBs, such as WordNet, OpenCyc, Freebase, and DBpedia, each built on distinct symbolic frameworks, there is a need to make their data more accessible and interoperable for various AI applications, including natural language processing, vision, and collaborative filtering. Our proposed approach utilizes a unique neural network architecture to transform symbolic representations into a more flexible vector space format. This transformation ensures that the original knowledge is retained and enriched within the new format. By learning these embeddings, data from any KB can be seamlessly integrated into contemporary machine learning algorithms for enhanced prediction and information retrieval capabilities. To demonstrate the efficacy of our method, we apply it to WordNet and Freebase, showcasing its adaptability to knowledge extraction from raw text. This work bridges the gap between symbolic and vector-based representations, paving the way for more sophisticated AI applications that leverage the rich structured knowledge found in KBs.

In recent years, many deep-learning based models are proposed for text classification. This kind of models well fits the training set from the statistical point of view. However, it lacks the capacity of utilizing instance-level information from individual instances in the training set. In this work, we propose to enhance neural network models by allowing them to leverage information from k-nearest neighbor (kNN) of the input text. Our model employs a neural network that encodes texts into text embeddings. Moreover, we also utilize k-nearest neighbor of the input text as an external memory, and utilize it to capture instance-level information from the training set. The final prediction is made based on features from both the neural network encoder and the kNN memory. Experimental results on several standard benchmark datasets show that our model outperforms the baseline model on all the datasets, and it even beats a very deep neural network model (with 29 layers) in several datasets. Our model also shows superior performance when training instances are scarce, and when the training set is severely unbalanced. Our model also leverages techniques such as semi-supervised training and transfer learning quite well.

Neural Tensor Network for Learning Event Embeddings Event Representation E = (O1, P, O2, T), where P is the action, O1 is the actor, O2 is the object and T is the timestamp (T is mainly used for aligning stock data with news data). For example, the event “Sep 3, 2013 Microsoft agrees to buy Nokia’s mobile phone business for $7.2 billion.” is modeled as: (Actor = Microsoft, Action = buy, Object = Nokia’s mobile phone business, Time = Sep 3, 2013) Event Embedding

We address the problem common space learning approach that maps all related multimodal information into a common space for multimodal data. To establish a fine-grained common space, the aligned relevant local information of different modalities is used to learn a common subspace where the projected fragmented information is further integrated according to intra-modal semantic relationships. Specifically, we propose a novel multimodal LSTM with an attention alignment mechanism, namely attention alignment multimodal LSTM (AAM-LSTM), which mainly includes attentional alignment recurrent network (AA-R) and hierarchical multimodal LSTM (HM-LSTM). Different from the traditional methods which operate on the full modal data directly, the proposed model exploits the inter-modal and intra-modal semantic relationships of local information, to jointly establish a uniform representation of multi-modal data. Specifically, AA-R automatically captures semantic-aligned local information to learn common subspace without the need of supervised labels, then HM-LSTM leverages the potential relationships of these local information to learn a fine-grained common space. The experimental results on Filker30K, Filker8K, and Filker30K entities verify the performance and effectiveness of our model, which compares favorably with the state-of-the-art methods. In particular, the experiment of phrase localization on AA-R with Filker30K entities shows the expected accurate attention alignment. Moreover, from the experiment results of image-sentence retrieval tasks, it can be concluded that the proposed AAM-LSTM outperforms benchmark algorithms by a large margin.

In this paper, we study a novel approach for named entity recognition (NER) and mention detection (MD) in natural language processing. Instead of treating NER as a sequence labeling problem, we propose a new local detection approach, which relies on the recent fixed-size ordinally forgetting encoding (FOFE) method to fully encode each sentence fragment and its left/right contexts into a fixedsize representation. Subsequently, a simple feedforward neural network (FFNN) is learned to either reject or predict entity label for each individual text fragment. The proposed method has been evaluated in several popular NER and MD tasks, including CoNLL 2003 NER task and TAC-KBP2015 and TAC-KBP2016 Tri-lingual Entity Discovery and Linking (EDL) tasks. Our method has yielded pretty strong performance in all of these examined tasks. This local detection approach has shown many advantages over the traditional sequence labeling methods.

We propose CORE, a novel matrix factorization model that leverages contextual information for open relation extraction. Our model is based on factorization machines and integrates facts from various sources, such as knowledge bases or open information extractors, as well as the context in which these facts have been observed. We argue that integrating contextual information—such as metadata about extraction sources, lexical context, or type information—significantly improves prediction performance. Open information extractors, for example, may produce extractions that are unspecific or ambiguous when taken out of context. Our experimental study on a large real-world dataset indicates that CORE has significantly better prediction performance than state-ofthe-art approaches when contextual information is available.

We explore the use of Wikipedia as external knowledge to improve named entity recognition (NER). Our method retrieves the corresponding Wikipedia entry for each candidate word sequence and extracts a category label from the first sentence of the entry, which can be thought of as a definition part. These category labels are used as features in a CRF-based NE tagger. We demonstrate using the CoNLL 2003 dataset that the Wikipedia category labels extracted by such a simple method actually improve the accuracy of NER.

This tutorial gives an overview on state-of-the-art methods for the automatic construction of large knowledge bases and harnessing them for data and text analytics. It covers both big-data methods for building knowledge bases and knowledge bases being assets for big-data applications. The tutorial also points out challenges and research opportunities. 1. MOTIVATION AND SCOPE Comphrehensive machine-readable knowledge bases (KB’s) have been pursued since the seminal projects Cyc [19, 20] and WordNet [12]. In contrast to these manually created KB’s, great advances have recently been made on automating the building and curation of large KB’s [1, 16], using information extraction (IE) techniques and harnessing highquality Web sources like Wikipedia. Prominent endeavors of this kind include academic research projects such as DBpedia [3], KnowItAll [10], NELL [5] and YAGO [29], as well as industrial ones such as Freebase. These projects provide automatically constructed KB’s of facts about named entities, their semantic classes, and their mutual relationships. They contain millions of entities and billions of facts about them. Moreover, several KB’s are interlinked at the entity level, forming the backbone of the Web of Linked Data [14]. Such world knowledge in turn enables cognitive applications and knowledge-centric services like disambiguating natural-language text, entity linking, text summarization, deep question answering, and semantic search and analytics over entities and relations in Web and enterprise data (e.g., [2, 6, 8, 13]). Prominent examples of how KB’s can be harnessed include the Google Knowledge Graph [27] and the IBM Watson question answering system [17]. This tutorial presents state-of-the-art methods, recent advances, research opportunities, and open challenges along this avenue of knowledge harvesting and its applications. Particular emphasis is on the twofold role of KB’s for bigdata analytics: using scalable distributed algorithms for harvesting knowledge from Web and text sources, and leveraging entity-centric knowledge for deeper interpretation of and This work is licensed under the Creative Commons AttributionNonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/. Obtain permission prior to any use beyond those covered by the license. Contact copyright holder by emailing info@vldb.org. Articles from this volume were invited to present their results at the 40th International Conference on Very Large Data Bases, September 1st 5th 2014, Hangzhou, China. Proceedings of the VLDB Endowment, Vol. 7, No. 13 Copyright 2014 VLDB Endowment 2150-8097/14/08. better intelligence with big data. The following sections outline the structure of the tutorial. An extensive bibliography on this theme is given in [30]. 2. BUILDING KNOWLEDGE BASES Digital Knowledge: Today’s KB’s represent their data mostly in RDF-style SPO (subject-predicate-object) triples. We introduce this data model and the most salient KB projects, which include KnowItAll [10, 11], BabelNet [22], ConceptNet [28], DBpedia [3, 18], DeepDive [24], Freebase [4], ImageNet [7], NELL [5], Wikidata [31], WikiNet [21], WikiTaxonomy [26], and YAGO [29, 15]. We briefly discuss industrial projects like the Google Knowledge Graph and related work at Google [9, 13, 25], the EntityCube and Probase projects at Microsoft Research [23, 32], and IBM’s Watson project [17]. Harvesting Knowledge on Entities and Classes: Every entity in a KB (e.g., Steve Jobs) belongs to one or multiple classes (e.g., computer pioneer, entrepreneur). These classes are organized into a taxonomy, where more special classes are subsumed by more general classes (e.g., person). We discuss two families of methods to harvest such information: Wikipedia-based approaches that analyze the category system, and Web-based approaches that use techniques like set expansion. 3. HARVESTING FACTS AT WEB SCALE Harvesting Relational Facts: Relational facts express properties of and relationships between entities. There is a large spectrum of methods to extract such facts from Web documents. We give an overview on methods from pattern matching (e.g., regular expressions), computational linguistics (e.g., dependency parsing), statistical learning (e.g., factor graphs and MLN’s), and logical consistency reasoning (e.g., weighted MaxSat or ILP solvers). We also discuss to what extent these approaches scale to handle big data. Open Information Extraction: Alternatively to methods that operate on a pre-specified set of relations and entities, open information extraction harvests arbitrary SPO triples from natural language documents. It aggressively taps into noun phrases as entity candidates and verbal phrases as prototypic patterns for relations. We discuss recent methods that follow this direction. Some methods along these lines make clever use of big-data techniques like frequent sequence mining and map-reduce computation. Temporal and Multilingual Knowledge: Properly interpreting entities and facts in a KB often requires additional meta-information like entity names in different languages and the temporal scope of facts. We discuss tech-

Extracting keyphrases from documents automatically is an important and interesting task since keyphrases provide a quick summarization for documents. Although lots of efforts have been made on keyphrase extraction, most of the existing methods (the co-occurrence-based methods and the statistic-based methods) do not take semantics into full consideration. The co-occurrence-based methods heavily depend on the co-occurrence relations between two words in the input document, which may ignore many semantic relations. The statistic-based methods exploit the external text corpus to enrich the document, which introduce more unrelated relations inevitably. In this paper, we propose a novel approach to extract keyphrases using knowledge graphs, based on which we could detect the latent relations of two keyterms (i.e., noun words and named entities) without introducing many noises. Extensive experiments over real data show that our method outperforms the state-of-the-art methods including the graph-based co-occurrence methods and statistic-based clustering methods.

While there has been much research on automatically constructing structured Knowledge Bases (KBs), most of it has focused on generating facts to populate a KB. However, a useful KB must go beyond facts. For example, glosses (short natural language definitions) have been found to be very useful in tasks such as Word Sense Disambiguation. However, the important problem of Automatic Gloss Finding, i.e., assigning glosses to entities in an initially gloss-free KB, is relatively unexplored. We address that gap in this paper. In particular, we propose GLOFIN, a hierarchical semi-supervised learning algorithm for this problem which makes effective use of limited amounts of supervision and available ontological constraints. To the best of our knowledge, GLOFIN is the first system for this task. Through extensive experiments on real-world datasets, we demonstrate GLOFIN's effectiveness. It is encouraging to see that GLOFIN outperforms other state-of-the-art SSL algorithms, especially in low supervision settings. We also demonstrate GLOFIN's robustness to noise through experiments on a wide variety of KBs, ranging from user contributed (e.g., Freebase) to automatically constructed (e.g., NELL). To facilitate further research in this area, we have made the datasets and code used in this paper publicly available.

Our inspiration comes from Nell (Never Ending Language Learning), a computer program running at Carnegie Mellon University to extract structured information from unstructured web pages. We consider the problem of semi-supervised learning approach to extract category instances (e.g. country(USA), city(New York)) from web pages, starting with a handful of labeled training examples of each category or relation, plus hundreds of millions of unlabeled web documents. Semisupervised approaches using a small number of labeled examples together with many unlabeled examples are often unreliable as they frequently produce an internally consistent, but nevertheless, incorrect set of extractions. We believe that this problem can be overcome by simultaneously learning independent classifiers in a new approach named Coupled Bayesian Sets algorithm, based on Bayesian Sets, for many different categories and relations (in the presence of an ontology defining constraints that couple the training of these classifiers). Experimental results show that simultaneously learning a coupled collection of classifiers for random 11 categories resulted in much more accurate extractions than training classifiers through original Bayesian Sets algorithm, Naive Bayes, BaS-all and Coupled Pattern Learner (the category extractor used in NELL).