Question: Spectral network embedding: a rapid and scalable technique through sparsity.

This paper presents a comprehensive study on the task of embedding large-scale information networks into lower-dimensional vector spaces, a crucial technique for various applications such as data visualization, node classification, and link prediction. Conventional graph embedding techniques often fall short when applied to real-world networks, characterized by their vast size with millions of nodes. To address this challenge, we introduce a novel method termed "LINE" (Large-scale Information Network Embedding), designed to be universally applicable to different network types, including undirected, directed, and weighted graphs. The method's core innovation lies in its objective function, meticulously crafted to preserve both the local and global structures of the network. To enhance the scalability and efficiency of the inference process, we propose an edge-sampling algorithm that overcomes the limitations of traditional stochastic gradient descent techniques. Through extensive empirical evaluations, the effectiveness of LINE is substantiated across a diverse range of real-world networks, encompassing language, social, and citation networks. Notably, LINE demonstrates exceptional efficiency, capable of learning embeddings for networks with millions of vertices and billions of edges within mere hours on standard single-machine configurations. To facilitate further research and application, the source code for LINE is publicly accessible online\footnote{\url{https://github.com/tangjianpku/LINE}}.

The proposed method, LINE, represents a significant advancement in the field of network embedding by offering a scalable and versatile solution for large-scale information networks. Its ability to preserve both local and global network structures through a carefully designed objective function, combined with the efficiency gains from the edge-sampling algorithm, enables it to outperform traditional graph embedding methods in terms of both effectiveness and computational efficiency. This makes LINE a powerful tool for analyzing and understanding complex networks in various domains.

Neuroengineering is faced with unique challenges in repairing or replacing complex neural systems that are composed of many interacting parts. These interactions form intricate patterns over large spatiotemporal scales and produce emergent behaviors that are difficult to predict from individual elements. Network science provides a particularly appropriate framework in which to study and intervene in such systems by treating neural elements (cells, volumes) as nodes in a graph and neural interactions (synapses, white matter tracts) as edges in that graph. Here, we review the emerging discipline of network neuroscience, which uses and develops tools from graph theory to better understand and manipulate neural systems from micro- to macroscales. We present examples of how human brain imaging data are being modeled with network analysis and underscore potential pitfalls. We then highlight current computational and theoretical frontiers and emphasize their utility in informing diagnosis and monitoring, brain-machine interfaces, and brain stimulation. A flexible and rapidly evolving enterprise, network neuroscience provides a set of powerful approaches and fundamental insights that are critical for the neuroengineer's tool kit.

With the rapid expansion of mobile phone networks in developing countries, large-scale graph machine learning has gained sudden relevance in the study of global poverty. Recent applications range from humanitarian response and poverty estimation to urban planning and epidemic containment. Yet the vast majority of computational tools and algorithms used in these applications do not account for the multi-view nature of social networks: people are related in myriad ways, but most graph learning models treat relations as binary. In this paper, we develop a graph-based convolutional network for learning on multi-view networks. We show that this method outperforms state-of-the-art semi-supervised learning algorithms on three different prediction tasks using mobile phone datasets from three different developing countries. We also show that, while designed specifically for use in poverty research, the algorithm also outperforms existing benchmarks on a broader set of learning tasks on multi-view networks, including node labelling in citation networks.

In this paper, we proposeGPSP, a novel Graph Partition and Space Projection based approach, to learn the representation of a heterogeneous network that consists of multiple types of nodes and links. Concretely, we first partition the heterogeneous network into homogeneous and bipartite subnetworks. Then, the projective relations hidden in bipartite subnetworks are extracted by learning the projective embedding vectors. Finally, we concatenate the projective vectors from bipartite subnetworks with the ones learned from homogeneous subnetworks to form the final representation of the heterogeneous network. Extensive experiments are conducted on a real-life dataset. The results demonstrate that GPSP outperforms the state-of-the-art baselines in two key network mining tasks: node classification and clustering2.

Most objects and data in the real world are of multiple types, interconnected, forming complex, heterogeneous but often semi-structured information networks. However, most network science researchers are focused on homogeneous networks, without distinguishing different types of objects and links in the networks. We view interconnected, multityped data, including the typical relational database data, as heterogeneous information networks, study how to leverage the rich semantic meaning of structural types of objects and links in the networks, and develop a structural analysis approach on mining semi-structured, multi-typed heterogeneous information networks. In this article, we summarize a set of methodologies that can effectively and efficiently mine useful knowledge from such information networks, and point out some promising research directions.

In exploratory tensor mining, a common problem is how to analyze a set of variables across a set of subjects whose observations do not align naturally. For example, when modeling medical features across a set of patients, the number and duration of treatments may vary widely in time, meaning there is no meaningful way to align their clinical records across time points for analysis purposes. To handle such data, the state-of-the-art tensor model is the so-called PARAFAC2, which yields interpretable and robust output and can naturally handle sparse data. However, its main limitation up to now has been the lack of efficient algorithms that can handle large-scale datasets.
 In this work, we fill this gap by developing a scalable method to compute the PARAFAC2 decomposition of large and sparse datasets, called SPARTan. Our method exploits special structure within PARAFAC2, leading to a novel algorithmic reformulation that is both faster (in absolute time) and more memory-efficient than prior work. We evaluate SPARTan on both synthetic and real datasets, showing 22X performance gains over the best previous implementation and also handling larger problem instances for which the baseline fails. Furthermore, we are able to apply SPARTan to the mining of temporally-evolving phenotypes on data taken from real and medically complex pediatric patients. The clinical meaningfulness of the phenotypes identified in this process, as well as their temporal evolution over time for several patients, have been endorsed by clinical experts.

A scientific framework for simulations of large-scale networks, such as is required for the analysis of critical infrastructure interaction and interdependencies, is needed for applications on exascale computers. Such a framework must be able to manage heterogeneous physics and unstructured topology, and must be reusable. To this end we have developed DMNetwork, a class in PETSc that provides data and topology management and migration for network problems, along with multiphysics solvers to exploit the problem structure. It eases the application development cycle by providing the necessary infrastructure through simple abstractions to define and query the network. This paper presents the design of the DMNetwork, illustrates its user interface, and demonstrates its ability to solve large network problems through the numerical simulation of a water pipe network with more than 2 billion variables on extreme-scale computers using up to 30,000 processor cores.

Random Projection is another class of methods used for low-rank matrix approximation. A random projection algorithm projects datapoints from a high-dimensional space R n onto a lower-dimensional subspace R r (r ⌧ n) using a random matrix S 2 R r⇥n. The key idea of random mapping comes from the Johnson-Lindenstrauss lemma[7] (we will explain later in detail) that says " if points in a vector space are projected onto a randomly selected subspace of suitably high dimension, then the distances between points are approximately preserved ". Random projection methods are computationally efficient and sufficiently accurate in practice for dimen-sionality reduction of high-dimensional datasets. Moreover, since complexity of many geometric algorithms depends significantly on the dimension, applying random projection as pre-processing is a common task in many data mining applications. As opposed to column sampling methods, that need to access data for approximating the low-rank subspace, random projections are data oblivious, as their computation involves only a random matrix S. We start with the basic concepts and definitions required to understand random projections techniques. Definition 1 (Column Space). Consider a matrix A 2 R n⇥d (n > d). Notice that as one ranges over all vectors x 2 R d , Ax ranges over all linear combinations of the columns of A and therefore defines a d-dimensional subspace of R n , which we refer to as the column space of A and denote it by C(A). Definition 2 (` 2-Subspace Embedding). A matrix S 2 R r⇥n provides a subspace embedding for C(A) if kSAxk 2 2 = (1 ± ")kAxk 2 2 , 8x 2 R d. Such matrix S provides a low distortion embedding, and is called a (1 ± ") ` 2-subspace embedding. Using an`2-subspace embedding, one can work with SA 2 R r⇥d instead of A 2 R n⇥d. Typically r ⌧ n, so we are working with a smaller matrix that reduces the time/space complexity of many algorithms. However note that definitely r needs to be larger than d if we are talking about the whole subspace R d [11]. Note that subspace embedding does not depend on a particular basis for C(A), that means if we have a matrix U that is an orthonormal basis for C(A), then Ux gives the same subspace as Ax. Therefore if S is an embedding for A, it will be an embedding for U too. Let's consider …

Kernel approximation via nonlinear random feature maps is w idely used in speeding up kernel machines. There are two main challenges for the conventional kernel ap proximation methods. First, before performing kernel approximation, a good kernel has to be chosen. Pickin g a good kernel is a very challenging problem in itself. Second, high-dimensional maps are often required i n order to achieve good performance. This leads to high computational cost in both generating the nonlinear ma ps, nd in the subsequent learning and prediction process. In this work, we propose to optimize the nonlinear m ps directly with respect to the classification objective in a data-dependent fashion. The proposed approa ch achieves kernel approximation and kernel learning in a joint framework. This leads to much more compact maps wit hout hurting the performance. As a by-product, the same framework can also be used to achieve more compact ke rnel maps to approximate a known kernel. We also introduce Circulant Nonlinear Maps, which uses a circu lant-structured projection matrix to speed up the nonlinear maps for high-dimensional data.

Neural networks have been shown to be an effective tool for learning algorithms over graph-structured data. However, graph representation techniques—that convert graphs to real-valued vectors for use with neural networks—are still in their infancy. Recent works have proposed several approaches (e.g., graph convolutional networks), but these methods have difficulty scaling and generalizing to graphs with different sizes and shapes. We present GRAPH2SEQ, a new technique that represents vertices of graphs as infinite time-series. By not limiting the representation to a fixed dimension, GRAPH2SEQ scales naturally to graphs of arbitrary sizes and shapes. GRAPH2SEQ is also reversible, allowing full recovery of the graph structure from the sequences. By analyzing a formal computational model for graph representation, we show that an unbounded sequence is necessary for scalability. Our experimental results with GRAPH2SEQ show strong generalization and new state-of-the-art performance on a variety of graph combinatorial optimization problems.

Hyperspectral imaging is an important visual modality with growing interest and range of applications. The latter, however, is hindered by the fact that existing devices are limited in either spatial, spectral, and/or temporal resolution, while yet being both complicated and expensive. We present a low cost and fast method to recover high quality hyperspectral images directly from RGB. Our approach first leverages hyperspectral prior in order to create a sparse dictionary of hyperspectral signatures and their corresponding RGB projections. Describing novel RGB images via the latter then facilitates reconstruction of the hyperspectral image via the former. A novel, larger-than-ever database of hyperspectral images serves as a hyperspectral prior. This database further allows for evaluation of our methodology at an unprecedented scale, and is provided for the benefit of the research community. Our approach is fast, accurate, and provides high resolution hyperspectral cubes despite using RGB-only input.


Question: Spectral network embedding: a rapid and scalable technique through sparsity.

This paper delves into the application of supervised learning on molecular structures, a domain with significant implications for chemistry, pharmaceuticals, and materials science. Several neural network models, designed to be invariant to molecular symmetries, have been proposed and show promise. These models are capable of learning a message passing algorithm and an aggregation technique to evaluate functions across the entire input graph. The current challenge is to refine and optimize these models through extensive testing on chemical prediction benchmarks until their limits are reached or the benchmarks are satisfactorily addressed. To this end, we unify the existing models under a comprehensive framework termed Message Passing Neural Networks (MPNNs), while also exploring new variations within this framework. Our results, obtained using MPNNs, set new benchmarks in the prediction of molecular properties. These outcomes are so robust that we posit future efforts should concentrate on datasets featuring larger molecules or more precise ground truth labels, pushing the boundaries of what these models can achieve.

In this work we address the problem of fast and scalable learning of neuro-symbolic representations for general biological knowledge. Based on a recently published comprehensive biological knowledge graph (Alshahrani, 2017) that was used for demonstrating neurosymbolic representation learning, we show how to train fast (under 1 minute) log-linear neural embeddings of the entities. We utilize these representations as inputs for machine learning classifiers to enable important tasks such as biological link prediction. Classifiers are trained by concatenating learned entity embeddings to represent entity relations, and training classifiers on the concatenated embeddings to discern true relations from automatically generated negative examples. Our simple embedding methodology greatly improves on classification error compared to previously published state-of-the-art results, yielding a maximum increase of +0.28 F-measure and +0.22 ROC AUC scores for the most difficult biological link prediction problem. Finally, our embedding approach is orders of magnitude faster to train (≤ 1 minute vs. hours), much more economical in terms of embedding dimensions (d = 50 vs. d = 512), and naturally encodes the directionality of the asymmetric biological relations, that can be controlled by the order with which we concatenate the embeddings.

Neural networks have been shown to be an effective tool for learning algorithms over graph-structured data. However, graph representation techniques—that convert graphs to real-valued vectors for use with neural networks—are still in their infancy. Recent works have proposed several approaches (e.g., graph convolutional networks), but these methods have difficulty scaling and generalizing to graphs with different sizes and shapes. We present GRAPH2SEQ, a new technique that represents vertices of graphs as infinite time-series. By not limiting the representation to a fixed dimension, GRAPH2SEQ scales naturally to graphs of arbitrary sizes and shapes. GRAPH2SEQ is also reversible, allowing full recovery of the graph structure from the sequences. By analyzing a formal computational model for graph representation, we show that an unbounded sequence is necessary for scalability. Our experimental results with GRAPH2SEQ show strong generalization and new state-of-the-art performance on a variety of graph combinatorial optimization problems.

We study Nyström type subsampling approaches to large scale kernel methods, and prove learning bounds in the statistical learning setting, where random sampling and high probability estimates are considered. In particular, we prove that these approaches can achieve optimal learning bounds, provided the subsampling level is suitably chosen. These results suggest a simple incremental variant of Nyström Kernel Regularized Least Squares, where the subsampling level implements a form of computational regularization, in the sense that it controls at the same time regularization and computations. Extensive experimental analysis shows that the considered approach achieves state of the art performances on benchmark large scale datasets.

Many shape analysis methods treat the geometry of an object as a metric space that can be captured by the Laplace-Beltrami operator. In this paper, we propose to adapt a classical operator from quantum mechanics to the field of shape analysis where we suggest to integrate a scalar function through a unified elliptical Hamiltonian operator. We study the addition of a potential function to the Laplacian as a generator for dual spaces in which shape processing is performed. Then, we evaluate the resulting spectral basis for different applications such as mesh compression and shape matching. The suggested operator is shown to produce better functional spaces to operate with, as demonstrated by the proposed framework that outperforms existing spectral methods, for example, when applied to shape matching benchmarks.

Hyperspectral Unmixing (HU) has received increasing attention in the past decades due to its ability of unveiling information latent in hyperspectral data. Unfortunately, most existing methods fail to take advantage of the spatial information in data. To overcome this limitation, we propose a Structured Sparse regularized Nonnegative Matrix Factorization (SS-NMF) method from the following two aspects. First, we incorporate a graph Laplacian to encode the manifold structures embedded in the hyperspectral data space. In this way, the highly similar neighboring pixels can be grouped together. Second, the lasso penalty is employed in SS-NMF for the fact that pixels in the same manifold structure are sparsely mixed by a common set of relevant bases. These two factors act as a new structured sparse constraint. With this constraint, our method can learn a compact space, where highly similar pixels are grouped to share correlated sparse representations. Experiments on real hyperspectral data sets with different noise levels demonstrate that our method outperforms the state-of-the-art methods significantly. Index Terms Hyperspectral Unmixing (HU), Hyperspectral Image Analysis, Structured Sparse NMF (SS-NMF), Mixed Pixel, Nonnegative Matrix Factorization (NMF).

We consider extending decision support facilities toward large sophisticated networks, upon which multidimensional attributes are associated with network entities, thereby forming the so-called multidimensional networks. Data warehouses and OLAP (Online Analytical Processing) technology have proven to be effective tools for decision support on relational data. However, they are not well-equipped to handle the new yet important multidimensional networks. In this paper, we introduce Graph Cube, a new data warehousing model that supports OLAP queries effectively on large multidimensional networks. By taking account of both attribute aggregation and structure summarization of the networks, Graph Cube goes beyond the traditional data cube model involved solely with numeric value based group-by's, thus resulting in a more insightful and structure-enriched aggregate network within every possible multidimensional space. Besides traditional cuboid queries, a new class of OLAP queries, crossboid, is introduced that is uniquely useful in multidimensional networks and has not been studied before. We implement Graph Cube by combining special characteristics of multidimensional networks with the existing well-studied data cube techniques. We perform extensive experimental studies on a series of real world data sets and Graph Cube is shown to be a powerful and efficient tool for decision support on large multidimensional networks.

Vector embeddings of words have been shown to encode meaningful semantic relationships that enable solving of complex analogies. This vector embedding concept has been extended successfully to many different domains and in this paper we both create and visualize vector representations of an unstructured collection of online communities based on user participation. Further, we quantitatively and qualitatively show that these representations allow solving of semantically meaningful community analogies and also other more general types of relationships. These results could help improve community recommendation engines and also serve as a tool for sociological studies of community relatedness.

Effectively showing the relationships between objects in a dataset is one of the main tasks in information visualization. Typically there is a well-defined notion of distance between pairs of objects, and traditional approaches such as principal component analysis or multi-dimensional scaling are used to place the objects as points in 2D space, so that similar objects are close to each other. In another typical setting, the dataset is visualized as a network graph, where related nodes are connected by links. More recently, datasets are also visualized as maps, where in addition to nodes and links, there is an explicit representation of groups and clusters. We consider these three Techniques, characterized by a progressive increase of the amount of encoded information: node diagrams, node-link diagrams and node-link-group diagrams. We assess these three types of diagrams with a controlled experiment that covers nine different tasks falling broadly in three categories: node-based tasks, network-based tasks and group-based tasks. Our findings indicate that adding links, or links and group representations, does not negatively impact performance (time and accuracy) of node-based tasks. Similarly, adding group representations does not negatively impact the performance of network-based tasks. Node-link-group diagrams outperform the others on group-based tasks. These conclusions contradict results in other studies, in similar but subtly different settings. Taken together, however, such results can have significant implications for the design of standard and domain snecific visualizations tools.

How can we describe a large, dynamic graph over time? Is it random? If not, what are the most apparent deviations from randomness – a dense block of actors that persists over time, or perhaps a star with many satellite nodes that appears with some fixed periodicity? In practice, these deviations indicate patterns – for example, research collaborations forming and fading away over the years. Which patterns exist in real-world dynamic graphs, and how can we find and rank their importance? These are exactly the problems we focus on. Our main contributions are (a) formulation: we show how to formalize this problem as minimizing an information theoretic encoding cost, (b) algorithm: we propose TIMECRUNCH, an effective and scalable method for finding coherent, temporal patterns in dynamic graphs and (c) practicality: we apply our method to several large, diverse real-world datasets with up to 36 million edges and introduce our auxiliary ECOVIZ framework for visualizing and interacting with dynamic graphs which have been summarized by TIMECRUNCH. We show that TIMECRUNCH is able to compress these graphs by summarizing important temporal structures and finds patterns that agree with intuition.

Random Projection is another class of methods used for low-rank matrix approximation. A random projection algorithm projects datapoints from a high-dimensional space R n onto a lower-dimensional subspace R r (r ⌧ n) using a random matrix S 2 R r⇥n. The key idea of random mapping comes from the Johnson-Lindenstrauss lemma[7] (we will explain later in detail) that says " if points in a vector space are projected onto a randomly selected subspace of suitably high dimension, then the distances between points are approximately preserved ". Random projection methods are computationally efficient and sufficiently accurate in practice for dimen-sionality reduction of high-dimensional datasets. Moreover, since complexity of many geometric algorithms depends significantly on the dimension, applying random projection as pre-processing is a common task in many data mining applications. As opposed to column sampling methods, that need to access data for approximating the low-rank subspace, random projections are data oblivious, as their computation involves only a random matrix S. We start with the basic concepts and definitions required to understand random projections techniques. Definition 1 (Column Space). Consider a matrix A 2 R n⇥d (n > d). Notice that as one ranges over all vectors x 2 R d , Ax ranges over all linear combinations of the columns of A and therefore defines a d-dimensional subspace of R n , which we refer to as the column space of A and denote it by C(A). Definition 2 (` 2-Subspace Embedding). A matrix S 2 R r⇥n provides a subspace embedding for C(A) if kSAxk 2 2 = (1 ± ")kAxk 2 2 , 8x 2 R d. Such matrix S provides a low distortion embedding, and is called a (1 ± ") ` 2-subspace embedding. Using an`2-subspace embedding, one can work with SA 2 R r⇥d instead of A 2 R n⇥d. Typically r ⌧ n, so we are working with a smaller matrix that reduces the time/space complexity of many algorithms. However note that definitely r needs to be larger than d if we are talking about the whole subspace R d [11]. Note that subspace embedding does not depend on a particular basis for C(A), that means if we have a matrix U that is an orthonormal basis for C(A), then Ux gives the same subspace as Ax. Therefore if S is an embedding for A, it will be an embedding for U too. Let's consider …


Question: Spectral network embedding: a rapid and efficient technique through sparsity.

This work introduces a novel, efficient technique for semi-supervised learning on data with graph structures, utilizing a variant of convolutional neural networks (CNNs) specifically tailored for graph data. The architecture of our convolutional method is grounded in a localized, first-order approximation of spectral graph convolutions, providing a solid theoretical foundation for our approach. Notably, our model exhibits linear scalability with respect to the number of edges in the graph, making it highly scalable and applicable to large datasets. It effectively learns hidden layer representations that encapsulate both the local structure of the graph and the features associated with individual nodes. Through a series of experiments on citation networks and a knowledge graph dataset, we substantiate the superior performance of our method compared to existing related techniques, showcasing its efficacy in semi-supervised learning tasks on graph-structured data.

This survey provides a brief introduction to compressed sensing as well as several major algorithms to solve it and its various applications to communications systems. We firstly review linear simultaneous equations as ill-posed inverse problems, since the idea of compressed sensing could be best understood in the context of the linear equations. Then, we consider the problem of compressed sensing as an underdetermined linear system with a prior information that the true solution is sparse, and explain the sparse signal recovery based on 1 optimization, which plays the central role in compressed sensing, with some intuitive explanations on the optimization problem. Moreover, we introduce some important properties of the sensing matrix in order to establish the guarantee of the exact recovery of sparse signals from the underdetermined system. After summarizing several major algorithms to obtain a sparse solution focusing on the 1 optimization and the greedy approaches, we introduce applications of compressed sensing to communications systems, such as wireless channel estimation, wireless sensor network, network tomography, cognitive radio, array signal processing, multiple access scheme, and networked control. key words: compressed sensing, sparse signal, compressible signal, 1norm, underdetermined system

Complex biological, technological, and sociological networks can be of very different sizes and connectivities, making it difficult to compare their structures. Here we present an approach to systematically study similarity in the local structure of networks, based on the significance profile (SP) of small subgraphs in the network compared to randomized networks. We find several superfamilies of previously unrelated networks with very similar SPs. One superfamily, including transcription networks of microorganisms, represents "rate-limited" information-processing networks strongly constrained by the response time of their components. A distinct superfamily includes protein signaling, developmental genetic networks, and neuronal wiring. Additional superfamilies include power grids, protein-structure networks and geometric networks, World Wide Web links and social networks, and word-adjacency networks from different languages.

Network embedding is an important method to learn low-dimensional representations of vertexes in network, whose goal is to capture and preserve the highly non-linear network structures. Here, we propose an Adversarial autoencoders based Network Embedding method (ANE for short), which utilizes the rencently proposed adversarial autoencoders to perform variational inference by matching the aggregated posterior of low-dimensional representations of vertexes with an arbitraray prior distribution. This framework introduces adversarial regularization to autoencoders. And it is able to attaches the latent representations of similar vertexes to each other and thus prevents the manifold fracturing problem that is typically encountered in the embeddings learnt by the autoencoders. Experiments demonstrate the effictiveness of ANE on link prediction and multi-label classification on three real-world information networks.

Given a large graph, how can we calculate the relevance between nodes fast and accurately? Random walk with restart (RWR) provides a good measure for this purpose and has been applied to diverse data mining applications including ranking, community detection, link prediction, and anomaly detection. Since calculating RWR from scratch takes a long time, various preprocessing methods, most of which are related to inverting adjacency matrices, have been proposed to speed up the calculation. However, these methods do not scale to large graphs because they usually produce large dense matrices that do not fit into memory. In addition, the existing methods are inappropriate when graphs dynamically change because the expensive preprocessing task needs to be computed repeatedly.
 In this article, we propose Bear, a fast, scalable, and accurate method for computing RWR on large graphs. Bear has two versions: a preprocessing method BearS for static graphs and an incremental update method BearD for dynamic graphs. BearS consists of the preprocessing step and the query step. In the preprocessing step, BearS reorders the adjacency matrix of a given graph so that it contains a large and easy-to-invert submatrix, and precomputes several matrices including the Schur complement of the submatrix. In the query step, BearS quickly computes the RWR scores for a given query node using a block elimination approach with the matrices computed in the preprocessing step. For dynamic graphs, BearD efficiently updates the changed parts in the preprocessed matrices of BearS based on the observation that only small parts of the preprocessed matrices change when few edges are inserted or deleted. Through extensive experiments, we show that BearS significantly outperforms other state-of-the-art methods in terms of preprocessing and query speed, space efficiency, and accuracy. We also show that BearD quickly updates the preprocessed matrices and immediately computes queries when the graph changes.

In this paper, we present a deep extension of Sparse Subspace Clustering, termed Deep Sparse Subspace Clustering (DSSC). Regularized by the unit sphere distribution assumption for the learned deep features, DSSC can infer a new data affinity matrix by simultaneously satisfying the sparsity principle of SSC and the nonlinearity given by neural networks. One of the appealing advantages brought by DSSC is: when original real-world data do not meet the class-specific linear subspace distribution assumption, DSSC can employ neural networks to make the assumption valid with its hierarchical nonlinear transformations. To the best of our knowledge, this is among the first deep learning based subspace clustering methods. Extensive experiments are conducted on four real-world datasets to show the proposed DSSC is significantly superior to 12 existing methods for subspace clustering.

Next generation analytics will be all about graphs, though performance has been a fundamental challenge for large scale graph processing. In this paper, we present an industrial graph processing engine for exploring various large scale linked data, which exhibits superior performance due to the several innovations. This engine organizes a graph as a set of edge-sets, compatible with the traditional edge-centric sharding for graphs, but becomes more amenable for large scale processing. Each time only a portion of the sets are needed for computation and the data access patterns can be highly predictable for prefetch for many graph computing algorithms. Due to the sparsity of large scale graph structure, this engine differentiates logical edge-sets from the edge-sets physically stored on the disk, where multiple logical edge-sets can be organized into a same physical edge-set to increase the data locality. Besides, in contrast to existing solution, the data structures utilized for the physical edge-sets can vary from one to another. Such heterogeneous edge-set representation explores the best graph processing performance according to local data access patterns. We conduct experiments on a representative set of property graphs on multiple platforms, where the proposed system outperform the baseline systems consistently.

In the big data era, k-means clustering has been widely adopted as a basic processing tool in various contexts. However, its computational cost could be prohibitively high when the data size and the cluster number are large. The processing bottleneck of k-means lies in the operation of seeking the closest centroid in each iteration. In this paper, a novel solution towards the scalability issue of k-means is presented. In the proposal, k-means is supported by an approximate k-nearest neighbors graph. In the k-means iteration, each data sample is only compared to clusters that its nearest neighbors reside. Since the number of nearest neighbors we consider is much less than k, the processing cost in this step becomes minor and irrelevant to k. The processing bottleneck is therefore broken. The most interesting thing is that k-nearest neighbor graph is constructed by calling the fast k-means itself. Compared with existing fast k-means variants, the proposed algorithm achieves hundreds to thousands times speed-up while maintaining high clustering quality. As it is tested on 10 million 512-dimensional data, it takes only 5.2 hours to produce 1 million clusters. In contrast, it would take 3 years for traditional k-means to fulfill the same scale of clustering.

We provide a new unifying view, including all existing prope r probabilistic sparse approximations for Gaussian process regression. Our approach relies on exp ressing theeffective priorwhich the methods are using. This allows new insights to be gained, and highlights the relationship between existing methods. It also allows for a clear theoretically j ustified ranking of the closeness of the known approximations to the corresponding full GPs. Finall y we point directly to designs of new better sparse approximations, combining the best of the exi sting strategies, within attractive computational constraints.

Network scaling algorithms such as the Pathfinder algorithm are used to prunemany different kinds of networks, including citation networks, randomnetworks, and social networks. However, this algorithm suffers from run time problems for large networks and online processing due to its O(n4) time complexity. In this article, we introduce a new alternative, the MST-Pathfinder algorithm, which will allow us to prune the original network to get its PFNET(∞, n −1) in justO(n2 · logn) time.Theunderlying idea comes from the fact that the union (superposition) of all the Minimum Spanning Trees extracted from a given network is equivalent to the PFNET resulting from the Pathfinder algorithmparameterized by a specific set of values (r = ∞ and q = n −1), those usually considered in many different applications. Although this property is well-known in the literature, it seems that no algorithm based on it has been proposed, up to now, to decrease the high computational cost of the original Pathfinder algorithm.We also present a mathematical proof of the correctness of this new alternative and test its good efficiency in two different case studies: one dedicated to the post-processing of large random graphs, and the other one to a real world case in which medium networks obtained by a cocitation analysis of the scientific domains in different countries are pruned.

Greedy approximation algorithms have been frequently used to obtain sparse solutions to learning problems. In this paper, we present a general greedy algorithm for solving a class of convex optimization problems. We derive a bound on the rate of approximation for this algorithm, and show that our algorithm includes a number of earlier studies as special cases.


Question: Spectral network embedding: an efficient and scalable technique through sparsity.

This paper delves into the core functionalities of the ArnetMiner system, a pioneering platform designed for the extraction and mining of academic social networks. ArnetMiner's focus encompasses four primary areas: 1) The automated extraction of detailed researcher profiles from the internet; 2) The seamless integration of publication data from various digital libraries into the network; 3) The comprehensive modeling of the academic network as a whole; and 4) The provision of specialized search services tailored to the academic community. To date, ArnetMiner has successfully extracted 448,470 researcher profiles through a sophisticated tagging methodology. The system integrates publication data from online databases, employing a probabilistic framework to address the complexities of name ambiguity. Moreover, a unified modeling strategy is proposed to effectively capture the topical dimensions of papers, authors, and publication venues. Based on these models, ArnetMiner offers advanced search functionalities, such as expertise search and people association search. Herein, we detail the system's architecture and key features, along with an empirical assessment of the methods employed.

**Question**: An Effective Method for Reducing the Number of Support Vectors in SVM Classification

**Documents**: A Novel Method for Reducing the Number of Support Vectors in SVM Classification
In this paper, we propose a novel method for reducing the number of support vectors (SVs) in support vector machine (SVM) classification. The proposed method is based on the idea of merging SVs with similar properties. The merging process is carried out by clustering the SVs using the k-means algorithm. After clustering, the SVs in each cluster are replaced by a single SV which is the centroid of the cluster. The proposed method is evaluated on several benchmark datasets and compared with other methods for reducing the number of SVs. The experimental results show that the proposed method can significantly reduce the number of SVs while maintaining the classification accuracy.

**Similar Documents**: This work introduces an innovative technique aimed at minimizing the support vector (SV) count in support vector machine (SVM) classification tasks. The core concept of the proposed method revolves around consolidating SVs that exhibit comparable characteristics. This consolidation is achieved through a clustering phase, utilizing the k-means algorithm to categorize the SVs. Following the clustering, each cluster's SVs are supplanted by a singular SV, precisely the cluster's centroid. The efficacy of the proposed technique is assessed across a range of benchmark datasets, with comparisons made to alternative methods for SV reduction. Empirical findings reveal that

Neural networks have been shown to be an effective tool for learning algorithms over graph-structured data. However, graph representation techniques—that convert graphs to real-valued vectors for use with neural networks—are still in their infancy. Recent works have proposed several approaches (e.g., graph convolutional networks), but these methods have difficulty scaling and generalizing to graphs with different sizes and shapes. We present GRAPH2SEQ, a new technique that represents vertices of graphs as infinite time-series. By not limiting the representation to a fixed dimension, GRAPH2SEQ scales naturally to graphs of arbitrary sizes and shapes. GRAPH2SEQ is also reversible, allowing full recovery of the graph structure from the sequences. By analyzing a formal computational model for graph representation, we show that an unbounded sequence is necessary for scalability. Our experimental results with GRAPH2SEQ show strong generalization and new state-of-the-art performance on a variety of graph combinatorial optimization problems.

This paper proposes a new dimensionality reduction algorithm named branching embedding (BE). It converts a dendrogram to a two-dimensional scatter plot, and visualizes the inherent structures of the original high-dimensional data. Since the conversion part is not computationally demanding, the BE algorithm would be beneficial for the case where hierarchical clustering is already performed. Numerical experiments revealed that the outputs of the algorithm moderately preserve the original hierarchical structures.

In the big data era, k-means clustering has been widely adopted as a basic processing tool in various contexts. However, its computational cost could be prohibitively high when the data size and the cluster number are large. The processing bottleneck of k-means lies in the operation of seeking the closest centroid in each iteration. In this paper, a novel solution towards the scalability issue of k-means is presented. In the proposal, k-means is supported by an approximate k-nearest neighbors graph. In the k-means iteration, each data sample is only compared to clusters that its nearest neighbors reside. Since the number of nearest neighbors we consider is much less than k, the processing cost in this step becomes minor and irrelevant to k. The processing bottleneck is therefore broken. The most interesting thing is that k-nearest neighbor graph is constructed by calling the fast k-means itself. Compared with existing fast k-means variants, the proposed algorithm achieves hundreds to thousands times speed-up while maintaining high clustering quality. As it is tested on 10 million 512-dimensional data, it takes only 5.2 hours to produce 1 million clusters. In contrast, it would take 3 years for traditional k-means to fulfill the same scale of clustering.

With ever-increasing amounts of graph data from disparate sources, there has been a strong need for exploiting significant graph patterns with user-specified objective functions. Most objective functions are not antimonotonic, which could fail all of frequency-centric graph mining algorithms. In this paper, we give the first comprehensive study on general mining method aiming to find most significant patterns directly. Our new mining framework, called LEAP (Descending Leap Mine), is developed to exploit the correlation between structural similarity and significance similarity in a way that the most significant pattern could be identified quickly by searching dissimilar graph patterns. Two novel concepts, structural leap search and frequency descending mining, are proposed to support leap search in graph pattern space. Our new mining method revealed that the widely adopted branch-and-bound search in data mining literature is indeed not the best, thus sketching a new picture on scalable graph pattern discovery. Empirical results show that LEAP achieves orders of magnitude speedup in comparison with the state-of-the-art method. Furthermore, graph classifiers built on mined patterns outperform the up-to-date graph kernel method in terms of efficiency and accuracy, demonstrating the high promise of such patterns.

We address the problem of social network de-anonymization when relationships between people are described by scale-free graphs. In particular, we propose a rigorous, asymptotic mathematical analysis of the network de-anonymization problem while capturing the impact of power-law node degree distribution, which is a fundamental and quite ubiquitous feature of many complex systems such as social networks. By applying bootstrap percolation and a novel graph slicing technique, we prove that large inhomogeneities in the node degree lead to a dramatic reduction of the initial set of nodes that must be known a priori (the seeds) in order to successfully identify all other users. We characterize the size of this set when seeds are selected using different criteria, and we show that their number can be as small as n% for any small ε > 0. Our results are validated through simulation experiments on real social network graphs.

This paper addresses the nearest neighbor search problem under inner product similarity and introduces a compact code-based approach. The idea is to approximate a vector using the composition of several elements selected from a source dictionary and to represent this vector by a short code composed of the indices of the selected elements. The inner product between a query vector and a database vector is efficiently estimated from the query vector and the short code of the database vector. We show the superior performance of the proposed group M -selection algorithm that selects M elements from M source dictionaries for vector approximation in terms of search accuracy and efficiency for compact codes of the same length via theoretical and empirical analysis. Experimental results on large-scale datasets (1M and 1B SIFT features, 1M linear models and Netflix) demonstrate the superiority of the proposed approach.

Many areas of science depend on exploratory data analysis and visualization. The need to analyze large amounts of multivariate data raises the fundamental problem of dimensionality reduction: how to discover compact representations of high-dimensional data. Here, we introduce locally linear embedding (LLE), an unsupervised learning algorithm that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional inputs. Unlike clustering methods for local dimensionality reduction, LLE maps its inputs into a single global coordinate system of lower dimensionality, and its optimizations do not involve local minima. By exploiting the local symmetries of linear reconstructions, LLE is able to learn the global structure of nonlinear manifolds, such as those generated by images of faces or documents of text.

In this paper, we proposeGPSP, a novel Graph Partition and Space Projection based approach, to learn the representation of a heterogeneous network that consists of multiple types of nodes and links. Concretely, we first partition the heterogeneous network into homogeneous and bipartite subnetworks. Then, the projective relations hidden in bipartite subnetworks are extracted by learning the projective embedding vectors. Finally, we concatenate the projective vectors from bipartite subnetworks with the ones learned from homogeneous subnetworks to form the final representation of the heterogeneous network. Extensive experiments are conducted on a real-life dataset. The results demonstrate that GPSP outperforms the state-of-the-art baselines in two key network mining tasks: node classification and clustering2.

Neural networks and word embeddings are powerful tools to capture latent factors. These tools can provide effective measures of similarities between users or items in the context of sparse data. We propose a novel approach that relies on neural networks and word embeddings to the problem of matching a learner looking for mentoring, and a tutor that is willing to provide this mentoring. Tutors and learners can issue multiple offers/requests on different topics. The approach matches over the whole array of topics specified by learners and tutors. Its performance for tutor-learner matching is compared with the state of the art. It yields similar results in terms of precision, but improves the recall.

Compressed representations have become effective to store and access large Web and social graphs, in order to support various graph querying and mining tasks. The existing representations exploit various typical patterns in those networks and provide basic navigation support. In this paper we obtain unprecedented results by finding “dense subgraph” patterns and combining them with techniques such as node orderings and compact data structures. On those representations we support out-neighbor and out/in-neighbor queries, as well as mining queries based on dense subgraphs. First, we propose a compression scheme for Web graphs that reduces edges by representing dense subgraphs with “virtual nodes”; over this scheme we apply node orderings and other compression techniques. With this approach we match the best current compression ratios that support out-neighbor queries (i.e., nodes pointed from a given node), using 1.0–1.8 bits per edge (bpe) on large Web graphs, and retrieving each neighbor of a node in 0.6–1.0 microseconds (μsec). When supporting both outand in-neighbor queries, instead, our technique generally offers the best time when using little space. If the reduced graph, instead, is represented with a compact data structure that supports bidirectional navigation, we obtain the most compact Web graph representations (0.9–1.5 bpe) that support out/in-neighbor navigation, yet the time per neighbor extracted raises to around 5–20 μsec. We also propose a compact data structure that represents dense subgraphs without using virtual nodes. It allows us to recover out/in-neighbors and answer other more complex queries on the dense subgraphs identified. This structure is not competitive on Web graphs, but on social networks it achieves 4–13 bpe and 8–12 μsec per out/in-neighbor retrieved, which improves upon all existing representations.


Question: Spectral network embedding: a rapid and scalable technique through sparsity.

This paper presents SimRank, a novel method for assessing the similarity of objects based on their structural context within a network of relationships. Unlike domain-specific similarity measures, SimRank is a versatile approach that can be applied in any domain where objects are connected through relationships. It calculates similarity by evaluating how related objects are to other similar objects, encapsulated in the phrase "two objects are similar if they are connected to similar objects." This similarity measure is founded on a straightforward graph-theoretic model, making it a general and adaptable solution. SimRank can be seamlessly integrated with other domain-specific similarity measures to enhance its effectiveness. The paper also discusses efficient techniques for computing SimRank scores, ensuring computational feasibility. Experimental results from two distinct application domains demonstrate the practicality and utility of the SimRank approach, highlighting its potential for broad application in various fields.

**Question**: A Comparative Study of Hybrid Optimization Algorithms for the Vehicle Routing Problem

**Documents**: An Improved Ant Colony Optimization Algorithm for the Vehicle Routing Problem
The Vehicle Routing Problem (VRP) is a classic combinatorial optimization problem with numerous practical applications. In this paper, an improved Ant Colony Optimization (ACO) algorithm is proposed to solve the VRP. The proposed algorithm introduces a new pheromone updating strategy and a local search procedure to enhance the performance of ACO. The pheromone updating strategy is designed to balance the exploration and exploitation of the algorithm, while the local search procedure helps to avoid getting trapped in local optima. The effectiveness of the proposed algorithm is evaluated on several benchmark instances of the VRP from the literature. The results show that the proposed algorithm outperforms the original ACO algorithm and some other well-known algorithms for the VRP.

**Similar Documents**: This paper introduces an enhanced version of the Ant Colony Optimization (ACO) algorithm tailored for the Vehicle Routing Problem (VRP), a significant combinatorial optimization challenge with widespread real-world relevance. The proposed ACO variant incorporates a novel pheromone updating mechanism and a local search enhancement to significantly boost the algorithm's performance. The pheromone updating strategy is carefully crafted to strike a balance between exploration and exploitation, ensuring the algorithm can effectively navigate the solution space. The inclusion of a local search procedure is aimed at preventing the algorithm from prematurely settling on suboptimal solutions, a common pitfall in optimization problems. The superior performance of the proposed algorithm is validated through rigorous testing on a range of benchmark VRP instances, demonstrating its superiority over the original ACO algorithm and several other established VRP

Relationships between entities in datasets are often of multiple nature, like geographical distance, social relationships, or common interests among people in a social network, for example. This information can naturally be modeled by a set of weighted and undirected graphs that form a global multi-layer graph, where the common vertex set represents the entities and the edges on different layers capture the similarities of the entities in term of the different modalities. In this paper, we address the problem of analyzing multi-layer graphs and propose methods for clustering the vertices by efficiently merging the information provided by the multiple modalities. To this end, we propose to combine the characteristics of individual graph layers using tools from subspace analysis on a Grassmann manifold. The resulting combination can then be viewed as a low dimensional representation of the original data which preserves the most important information from diverse relationships between entities. As an illustrative application of our framework, we use our algorithm in clustering methods and test its performance on several synthetic and real world datasets where it is shown to be superior to baseline schemes and competitive to state-of-the-art techniques. Our generic framework further extends to numerous analysis and learning problems that involve different types of information on graphs.

Recent successes in word embedding and document embedding have motivated researchers to explore similar representations for networks and to use such representations for tasks such as edge prediction, node label prediction, and community detection. Existing methods are largely focused on finding distributed representations for unsigned networks and are unable to discover embeddings that respect polarities inherent in edges. We propose SIGNet, a fast scalable embedding method suitable for signed networks. Our proposed objective function aims to carefully model the social structure implicit in signed networks by reinforcing the principles of social balance theory. Our method builds upon the traditional word2vec family of embedding approaches but we propose a new targeted node sampling strategy to maintain structural balance in higher-order neighborhoods. We demonstrate the superiority of SIGNet over state-of-the-art methods proposed for both signed and unsigned networks on several real world datasets from different domains. In particular, SIGNet offers an approach to generate a richer vocabulary of features of signed networks to support representation and reasoning.

Just as semantic hashing (Salakhutdinov and Hinton 2009) can accelerate information retrieval, binary valued embeddings can significantly reduce latency in the retrieval of graphical data. We introduce a simple but effective model for learning such binary vectors for nodes in a graph. By imagining the embeddings as independent coin flips of varying bias, continuous optimization techniques can be applied to the approximate expected loss. Embeddings optimized in this fashion consistently outperform the quantization of both spectral graph embeddings and various learned real-valued embeddings, on both ranking and pre-ranking tasks for a variety of datasets.

We explore in this paper efficient algorithmic solutions to robust subspace segmentation. We propose the SSQP, namely Subspace Segmentation via Quadratic Programming, to partition data drawn from multiple subspaces into multiple clusters. The basic idea of SSQP is to express each datum as the linear combination of other data regularized by an overall term targeting zero reconstruction coefficients over vectors from different subspaces. The derived coefficient matrix by solving a quadratic programming problem is taken as an affinity matrix, upon which spectral clustering is applied to obtain the ultimate segmentation result. Similar to sparse subspace clustering (SCC) and low-rank representation (LRR), SSQP is robust to data noises as validated by experiments on toy data. Experiments on Hopkins 155 database show that SSQP can achieve competitive accuracy as SCC and LRR in segmenting affine subspaces, while experimental results on the Extended Yale Face Database B demonstrate SSQP’s superiority over SCC and LRR. Beyond segmentation accuracy, all experiments show that SSQP is much faster than both SSC and LRR in the practice of subspace segmentation.

Network embedding is an important method to learn low-dimensional representations of vertexes in network, whose goal is to capture and preserve the highly non-linear network structures. Here, we propose an Adversarial autoencoders based Network Embedding method (ANE for short), which utilizes the rencently proposed adversarial autoencoders to perform variational inference by matching the aggregated posterior of low-dimensional representations of vertexes with an arbitraray prior distribution. This framework introduces adversarial regularization to autoencoders. And it is able to attaches the latent representations of similar vertexes to each other and thus prevents the manifold fracturing problem that is typically encountered in the embeddings learnt by the autoencoders. Experiments demonstrate the effictiveness of ANE on link prediction and multi-label classification on three real-world information networks.

Convolutional neural networks (CNNs) have recently exhibited an excellent performance in hyperspectral image classification tasks. However, the straightforward CNNbased network architecture still finds obstacles when effectively exploiting the relationships between hyperspectral imaging (HSI) features in the spectral–spatial domain, which is a key factor to deal with the high level of complexity present in remotely sensed HSI data. Despite the fact that deeper architectures try to mitigate these limitations, they also find challenges with the convergence of the network parameters, which eventually limit the classification performance under highly demanding scenarios. In this paper, we propose a new CNN architecture based on spectral–spatial capsule networks in order to achieve a highly accurate classification of HSIs while significantly reducing the network design complexity. Specifically, based on Hinton’s capsule networks, we develop a CNN model extension that redefines the concept of capsule units to become spectral– spatial units specialized in classifying remotely sensed HSI data. The proposed model is composed by several building blocks, called spectral–spatial capsules, which are able to learn HSI spectral–spatial features considering their corresponding spatial positions in the scene, their associated spectral signatures, and also their possible transformations. Our experiments, conducted Manuscript received June 9, 2018; revised August 22, 2018; accepted September 19, 2018. This work was supported in part by the Ministerio de Educación (Resolución de 26 de diciembre de 2014 y de 19 de noviembre de 2015, de la Secretaría de Estado de Educación, Formación Profesional y Universidades, por la que se convocan ayudas para la formación de profesorado universitario, de los subprogramas de Formación y de Movilidad incluidos en el Programa Estatal de Promoción del Talento y su Empleabilidad, and en el marco del Plan Estatal de Investigación Científica y Técnica y de Innovación 2013–2016), in part by the Junta de Extremadura (decreto 297/2014, ayudas para la realización de actividades de investigación y desarrollo tecnológico, and de divulgación y de transferencia de conocimiento por los Grupos de Investigación de Extremadura) under Grant GR15005, in part by the Generalitat Valenciana under Contract APOSTD/2017/007, in part by the Spanish Ministry of Economy under Project ESP2016-79503-C2-2P and Project TIN2015-63646-C5-5-R, in part by the National Natural Science Foundation of China under Grant 61771496, in part by the National Key Research and Development Program of China under Grant 2017YFB0502900, and in part by the Guangdong Provincial Natural Science Foundation under Grant 2016A030313254. (Corresponding author: Jun Li.) M. E. Paoletti, J. M. Haut, J. Plaza, and A. Plaza are with the Hyperspectral Computing Laboratory, Department of Technology of Computers and Communications, Escuela Politécnica, University of Extremadura, 10003 Cáceres, Spain (e-mail: mpaoletti@unex.es; juanmariohaut@unex.es; jplaza@unex.es; aplaza@unex.es). R. Fernandez-Beltran and F. Pla are with the Institute of New Imaging Technologies, Universitat Jaume I, 12071 Castellón de la Plana, Spain (e-mail: rufernan@uji.es; pla@uji.es). J. Li is with the Guangdong Provincial Key Laboratory of Urbanization and Geosimulation, Center of Integrated Geographic Information Analysis, School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China (e-mail: lijun48@mail.sysu.edu.cn). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TGRS.2018.2871782 using five well-known HSI data sets and several state-of-theart classification methods, reveal that our HSI classification approach based on spectral–spatial capsules is able to provide competitive advantages in terms of both classification accuracy and computational time.

Inverse problems correspond to a certain type of optimization problems formulated over appropriate input distributions. Recently, there has been a growing interest in understanding the computational hardness of these optimization problems, not only in the worst case, but in an average-complexity sense under this same input distribution.In this revised note, we are interested in studying another aspect of hardness, related to the ability to learn how to solve a problem by simply observing a collection of previously solved instances. These ‘planted solutions’ are used to supervise the training of an appropriate predictive model that parametrizes a broad class of algorithms, with the hope that the resulting model will provide good accuracy-complexity tradeoffs in the average sense.We illustrate this setup on the Quadratic Assignment Problem, a fundamental problem in Network Science. We observe that data-driven models based on Graph Neural Networks offer intriguingly good performance, even in regimes where standard relaxation based techniques appear to suffer.

We present an algorithm for graph isomorphism and subgraph isomorphism suited for dealing with large graphs. A first version of the algorithm has been presented in a previous paper, where we examined its performance for the isomorphism of small and medium size graphs. The algorithm is improved here to reduce its spatial complexity and to achieve a better performance on large graphs; its features are analyzed in detail with special reference to time and memory requirements. The results of a testing performed on a publicly available database of synthetically generated graphs and on graphs relative to a real application dealing with technical drawings are presented, confirming the effectiveness of the approach, especially when working with large graphs.

Manifold learning is an effective methodology for extracting nonlinear structures from high-dimensional data with many applications in image analysis, computer vision, text data analysis and bioinformatics. The focus of this paper is on developing algorithms for reducing the computational complexity of manifold learning algorithms, in particular, we consider the case when the number of features is much larger than the number of data points. To handle the large number of features, we propose a preprocessing method, distance preserving dimension reduction (DPDR). It produces t-dimensional representations of the high-dimensional data, wheret is the rank of the original dataset. It exactly preserves the Euclidean L2-norm distances as well as cosine similarity measures between data points in the original space. With the original data projected to the t-dimensional space, manifold learning algorithms can be executed to obtain lower dimensional parameterizations with substantial reduction in computational cost. Our experimental results illustrate that DPDR significantly reduces computing time of manifold learning algorithms and produces low-dimensional parameterizations as accurate as those obtained from the original datasets.

Social networks produce an enormous quantity of data. Facebook consists of over 400 million active users sharing over 5 billion pieces of information each month. Analyzing this vast quantity of unstructured data presents challenges for software and hardware. We present GraphCT, a Graph Characterization Toolkit for massive graphs representing social network data. On a 128-processor Cray XMT, GraphCT estimates the betweenness centrality of an artificially generated (R-MAT) 537 million vertex, 8.6 billion edge graph in 55 minutes and a real-world graph (Kwak, et al.) with 61.6 million vertices and 1.47 billion edges in 105 minutes. We use GraphCT to analyze public data from Twitter, a microblogging network. Twitter's message connections appear primarily tree-structured as a news dissemination system. Within the public data, however, are clusters of conversations. Using GraphCT, we can rank actors within these conversations and help analysts focus attention on a much smaller data subset.


Question: The Text-To-Onto framework for ontology extraction and maintenance.

Transitioning from Manual to Semi-Automatic Semantic Annotation: An Overview of Ontology-Driven Text Annotation Systems

Semantic Annotation, a foundational component of intelligent content, plays a pivotal role in enhancing the capabilities of content-oriented intelligent applications, particularly in the realm of the Semantic Web. This paper delves into our endeavors in the domain of ontology-based semantic annotation, situated within the context of a knowledge portal application. Our exploration begins with an analysis of both effective and ineffective manual semantic annotation practices, gleaned from our involvement in the Knowledge Annotation (KA) initiative. These insights have served as a catalyst for the development of an annotation tool that is not only user-friendly but also backed by a robust knowledge base. Building upon this foundation, we are currently enhancing the tool with advanced features for semi-automatic information extraction, thereby facilitating a more streamlined annotation process. To address the dynamic nature of semantic content, we also outline our approach to evolving ontologies, which are designed to adapt and grow alongside the expanding field of semantic annotation. This paper was originally presented at the COLING-2000 Workshop on Semantic Annotation and Intelligent Content, held at the Centre Universitaire in Luxembourg on the 5th and 6th of August, 2000.

The integration of information of different kinds, such as spatial and alphanumeric at different levels of detail, is a challenge. While a solution is not reached, it is widely recognized that the need to integrate information is so pressing that it does not matter if detail is lost, as long as integration is achieved. This paper shows the potential for information retrieval at different levels of granularity inside the framework of information systems based on ontologies. Ontologies are theories that use a specific vocabulary to describe entities, classes, properties and functions related to a certain view of the world. The use of an ontology, translated into an active information system component, leads to ontology-driven information systems and, in the specific case of GIS, leads to what we call ontology-driven geographic information systems.

Manually querying search engines in order to accumulate a large bodyof factual information is a tedious, error-prone process of piecemealsearch. Search engines retrieve and rank potentially relevantdocuments for human perusal, but do not extract facts, assessconfidence, or fuse information from multiple documents. This paperintroduces KnowItAll, a system that aims to automate the tedious process ofextracting large collections of facts from the web in an autonomous,domain-independent, and scalable manner.The paper describes preliminary experiments in which an instance of KnowItAll, running for four days on a single machine, was able to automatically extract 54,753 facts. KnowItAll associates a probability with each fact enabling it to trade off precision and recall. The paper analyzes KnowItAll's architecture and reports on lessons learned for the design of large-scale information extraction systems.

Extracting entities and relations for types of interest from text is important for understanding massive text corpora. Traditionally, systems of entity relation extraction have relied on human-annotated corpora for training and adopted an incremental pipeline. Such systems require additional human expertise to be ported to a new domain, and are vulnerable to errors cascading down the pipeline. In this paper, we investigate joint extraction of typed entities and relations with labeled data heuristically obtained from knowledge bases (i.e., distant supervision). As our algorithm for type labeling via distant supervision is context-agnostic, noisy training data poses unique challenges for the task. We propose a novel domainindependent framework, called COTYPE, that runs a data-driven text segmentation algorithm to extract entity mentions, and jointly embeds entity mentions, relation mentions, text features and type labels into two low-dimensional spaces (for entity and relation mentions respectively), where, in each space, objects whose types are close will also have similar representations. COTYPE, then using these learned embeddings, estimates the types of test (unlinkable) mentions. We formulate a joint optimization problem to learn embeddings from text corpora and knowledge bases, adopting a novel partial-label loss function for noisy labeled data and introducing an object "translation" function to capture the cross-constraints of entities and relations on each other. Experiments on three public datasets demonstrate the effectiveness of COTYPE across different domains (e.g., news, biomedical), with an average of 25% improvement in F1 score compared to the next best method.

We present the architecture and technologies underpinning the OBDA system Ontop and taking full advantage of storing data in relational databases. We discuss the theoretical foundations of Ontop: the tree-witness query rewriting, T -mappings and optimisations based on database integrity constraints and SQL features. We analyse the performance of Ontop in a series of experiments and demonstrate that, for standard ontologies, queries and data stored in relational databases, Ontop is fast, efficient and produces SQL rewritings of high quality.

This paper presents a large-scale system for the recognition and semantic disambiguation of named entities based on information extracted from a large encyclopedic collection and Web search results. It describes in detail the disambiguation paradigm employed and the information extraction process from Wikipedia. Through a process of maximizing the agreement between the contextual information extracted from Wikipedia and the context of a document, as well as the agreement among the category tags associated with the candidate entities, the implemented system shows high disambiguation accuracy on both news stories and Wikipedia articles.

Background: Automatic identification of gene and protein names from biomedical publications can help curators and researchers to keep up with the findings published in the scientific literature. As of today, this is a challenging task related to information retrieval, and in the realm of Big Data Analytics. Objectives: To investigate the feasibility of using word embed-dings (i.e. distributed word representations) from Deep Learning algorithms together with terms from the Cardiovascular Disease Ontology (CVDO) as a step to identifying omics information encoded in the biomedical literature. Methods: Word embeddings were generated using the neural language models CBOW and Skip-gram with an input of more than 14 million PubMed citations (titles and abstracts) corresponding to articles published between 2000 and 2016. Then the abstracts of selected papers from the sysVASC systematic review were manually annotated with gene/protein names. We set up two experiments that used the word embeddings to produce term variants for gene/protein names: the first experiment used the terms manually annotated from the papers; the second experiment enriched/expanded the annotated terms using terms from the human-readable labels of key classes (gene/proteins) from the CVDO ontology. CVDO is formalised in the W3C Web Ontology Language (OWL) and contains 172,121 UniProt Knowledgebase protein classes related to human and 86,792 UniProtKB protein classes related to mouse. The hypothesis is that by enriching the original annotated terms, a better context is provided, and therefore, it is easier to obtain suitable (full and/or partial) term variants for gene/protein names from word embed-dings. Results: From the papers manually annotated, a list of 107 terms (gene/protein names) was acquired. As part of the word embeddings generated from CBOW and Skip-gram, a lexicon with more than 9 million terms was created. Using the cosine similarity metric, a list of the 12 top-ranked terms was generated from word embeddings for query terms present in the generated lexicon. Domain experts evaluated a total of 1968 pairs of terms and classified the retrieved terms as: TV (term variant); PTV (partial term variant); and NTV (non term variant, meaning none of the previous two categories). In experiment I, Skip-gram finds the double amount of (full and/or partial) term variants for gene/protein names as compared with CBOW. Using Skip-gram, the weighted Cohen's Kappa inter-annotator agreement for two domain experts was 0.80 for the first experiment and 0.74 for the second experiment. In the first experiment, suitable (full and/or partial) term variants were found …

This article presents an ontology-based approach to designing and developing new representation IR system instead of conventional keyword-based approach. Such representation improves the precision and recall of document retrieval. Experiments carried out on the ontology-based approach and keyword-based approach demonstrates the effectiveness of the proposed approach.

Taxonomy learning is an important task for knowledge acquisition, sharing, and classification as well as application development and utilization in various domains. To reduce human effort to build a taxonomy from scratch and improve the quality of the learned taxonomy, we propose a new taxonomy learning approach, named TaxoFinder. TaxoFinder takes three steps to automatically build a taxonomy. First, it identifies domain-specific concepts from a domain text corpus. Second, it builds a graph representing how such concepts are associated together based on their co-occurrences. As the key method in TaxoFinder, we propose a method for measuring associative strengths among the concepts, which quantify how strongly they are associated in the graph, using similarities between sentences and spatial distances between sentences. Lastly, TaxoFinder induces a taxonomy from the graph using a graph analytic algorithm. TaxoFinder aims to build a taxonomy in such a way that it maximizes the overall associative strengths among the concepts in the graph to build a taxonomy. We evaluate TaxoFinder using gold-standard evaluation on three different domains: emergency management for mass gatherings, autism research, and disease domains. In our evaluation, we compare TaxoFinder with a state-of-the-art subsumption method and show that TaxoFinder is an effective approach significantly outperforming the subsumption method.

In this work, we study the problem of concept mining, which serves as the first step in transforming unstructured text into structured information, and supports downstream analytical tasks such as information extraction, organization, recommendation and search. Previous work mainly relies on statistical signals, existing knowledge bases, or predefined linguistic patterns. In this work, we propose a novel approach that mines concepts based on their occurrence contexts, by learning embedding vector representations that summarize the context information for each possible candidates, and use these embeddings to evaluate the concept's global quality and their fitness to each local context. Experiments over several real-world corpora demonstrate the superior performance of our method. A publicly available implementation is provided at https://github.com/kleeeeea/ECON.

Summary form only given. The paper describes a novel automatic procedure for the generation of multiple-choice tests from electronic documents. In addition to employing various NLP techniques including term extraction and shallow parsing, the system makes use of language resources such as corpora and ontologies. The system operates in a fully automatic mode and also a semiautomatic environment where the user is offered the option to post-edit the generated test items. The results from the conducted evaluation suggest that the new procedure is very effective saving time and labour considerably and that the test items produced with the help of the program are not of inferior quality to those produced manually.


Question: The Text-To-Onto framework for ontology extraction and maintenance.

This paper outlines a technique for the automated extraction of hyponymy relationships from extensive text corpora without relying on pre-existing knowledge. The methodology is designed to be universally applicable and robust across diverse text genres. A series of lexicosyntactic patterns, which are both prevalent and unambiguous indicators of the desired lexical relation, are identified. These patterns are easily detectable and transcend genre boundaries. The paper details a process for their discovery and suggests that this approach can be generalized to other lexical relations as well. An implementation of a portion of the algorithm is discussed, with the outcomes serving to enhance and critically evaluate the structure of a substantial manually constructed thesaurus. Potential expansions and practical implications for information retrieval and related domains are also explored.

Many Knowledge Bases (KBs) are now readily available and encompass colossal quantities of information thanks to either a long-term funding effort (e.g. WordNet, OpenCyc) or a collaborative process (e.g. Freebase, DBpedia). However, each of them is based on a different rigid symbolic framework which makes it hard to use their data in other systems. It is unfortunate because such rich structured knowledge might lead to a huge leap forward in many other areas of AI like natural language processing (word-sense disambiguation, natural language understanding, ...), vision (scene classification, image semantic annotation, ...) or collaborative filtering. In this paper, we present a learning process based on an innovative neural network architecture designed to embed any of these symbolic representations into a more flexible continuous vector space in which the original knowledge is kept and enhanced. These learnt embeddings would allow data from any KB to be easily used in recent machine learning methods for prediction and information retrieval. We illustrate our method on WordNet and Freebase and also present a way to adapt it to knowledge extraction from raw text.

We describe our method for benchmarking Semantic Web knowledge base systems with respect to use in large OWL applications. We present the Lehigh University Benchmark (LUBM) as an example of how to design such benchmarks. The LUBM features an on-tology for the university domain, synthetic OWL data scalable to an arbitrary size, fourteen extensional queries representing a variety of properties, and several performance metrics. The LUBM can be used to evaluate systems with different reasoning capabilities and storage mechanisms. We demonstrate this with an evaluation of two memory-based systems and two systems with persistent storage.

We combine search in triple stores with full-text search into what we call \emph{semantic full-text search}. We provide a fully functional web application that allows the incremental construction of complex queries on the English Wikipedia combined with the facts from Freebase. The user is guided by context-sensitive suggestions of matching words, instances, classes, and relations after each keystroke. We also provide a powerful API, which may be used for research tasks or as a back end, e.g., for a question answering system. Our web application and public API are available under \url{http://broccoli.cs.uni-freiburg.de}.

With the continued demand for Islamic knowledge, which is mainly based on the Quran as a source of knowledge and wisdom, systems that facilitate an easy search of the content of the Quran remain a considerable challenge. Although in recent years there have been tools for Quran search, most of these tools are based on keyword search, meaning that the user needs to know the correct keywords before being able to retrieve the content of al-Quran. In this paper, we propose a system that supports the end user in querying and exploring the Quran ontology. The system comprises user query reformulation against the Quran ontology stored and annotated in the knowledge base. The Quran ontology consists of noun concepts identified in al-Quran, and the relationship that exists between these concepts. The user writes a query in the natural language and the proposed system reformulates the query to match the content found in the knowledge base in order to retrieve the relevant answer. The answer is represented by the Quranic verse related to the user query.

Many e-recruitment tools for recruiting candidates for jobs have significantly spread in recent years. Companies often receive resumes from candidates using e-recruitment tools or via job portal for each job posting and manually short list qualified applicants. The existing e-recruitment tools have been mainly used for the storage of applicant contact data. In this paper, we present EXPERT, an intelligent tool for screening candidates for recruitment using ontology mapping. EXPERT has three phases in screening candidates for recruitment. In first phase, the system collects candidates’ resumes and constructs ontology document for the features of the candidates. Job openings/job requirements are represented as ontology in the second phase and in third phase, EXPERT maps the job requirement ontology onto the candidate ontology document and retrieves the eligible candidates. Experiment results show that this model improves the accuracy of matching candidates with job requirement.

A business process may be modeled in different ways by different modelers even when utilizing the same modeling language. An appropriate method for solving ambiguity issues in process models caused by the use of synonyms, homonyms or different abstraction levels for process element names is the use of ontologybased descriptions of process models. So-called semantic business process models promise to support business process interoperability and interconnectivity. But, for (semi-) automatic process interoperability and interconnectivity two problems need to be solved. How can similar terms for process element names be automatically discovered and how can semantic business process composition be facilitated. In this paper we will present solutions for these problems based upon an OWL DL-based description of Petri nets.

Populating large-scale structured databases from unstructured sources is a critical and challenging task in data analytics. As automated feature engineering methods grow increasingly prevalent, constructing sufficiently large labeled training sets has become the primary hurdle in building machine learning information extraction systems. In light of this, we have taken a new approach called data programming [7]. Rather than hand-labeling data, in the data programming paradigm, users generate large amounts of noisy training labels by programmatically encoding domain heuristics as simple rules. Using this approach over more traditional distant supervision methods and fully supervised approaches using labeled data, we have been able to construct knowledge base systems more rapidly and with higher quality. Since the ability to quickly prototype, evaluate, and debug these rules is a key component of this paradigm, we introduce DDLite, an interactive development framework for data programming. This paper reports feedback collected from DDLite users across a diverse set of entity extraction tasks. We share observations from several DDLite hackathons in which 10 biomedical researchers prototyped information extraction pipelines for chemicals, diseases, and anatomical named entities. Initial results were promising, with the disease tagging team obtaining an F1 score within 10 points of the state-of-the-art in only a single day-long hackathon's work. Our key insights concern the challenges of writing diverse rule sets for generating labels, and exploring training data. These findings motivate several areas of active data programming research.

Software development in highly variable domains constrained by tight regulations and with many business concepts involved results in hard to deliver and maintain applications, due to the complexity of dealing with the large number of concepts provided by the different parties and system involved in the process. One way to tackle these problems is thru combining software product lines and model-driven software development supported by ontologies. Software product lines and model-driven approaches would promote reuse on the software artifacts and, if supported by an ontological layer, those artifacts would be domain-validated. We intend to create a new conceptual framework for software development with domain validated models in highly variable domains. To define such a framework we will propose a model that relates several dimensions and areas of software development thru time and abstraction levels. This model would guarantee to the software house traceability of components, domain validated artifacts, easy to maintain and reusable components, due to the relations and mappings we propose to establish in the conceptual framework, between the software artifacts and the ontology.

In this paper we introduce SAMOD, a.k.a. Simplified Agile Methodology for Ontology Development, a novel agile methodology for the development of ontologies by means of small steps of an iterative workflow that focuses on creating well-developed and documented models starting from exemplar domain descriptions. RASH: https://w3id.org/people/essepuntato/papers/samod-owled2016.html

With the advent of the Internet, large amount of digital text is generated everyday in the form of news articles, research publications, blogs, question answering forums and social media. It is important to develop techniques for extracting information automatically from these documents, as lot of important information is hidden within them. This extracted information can be used to improve access and management of knowledge hidden in large text corpora. Several applications such as Question Answering, Information Retrieval would benefit from this information. Entities like persons and organizations, form the most basic unit of the information. Occurrences of entities in a sentence are often linked through well-defined relations; e.g., occurrences of person and organization in a sentence may be linked through relations such as employed at. The task of Relation Extraction (RE) is to identify such relations automatically. In this paper, we survey several important supervised, semi-supervised and unsupervised RE techniques. We also cover the paradigms of Open Information Extraction (OIE) and Distant Supervision. Finally, we describe some of the recent trends in the RE techniques and possible future research directions. This survey would be useful for three kinds of readers i) Newcomers in the field who want to quickly learn about RE; ii) Researchers who want to know how the various RE techniques evolved over time and what are possible future research directions and iii) Practitioners who just need to know which RE technique works best in various settings.