Question: Transitioning from Softmax to Sparsemax: A sparse model for attention and multi-label classification problems.

This paper presents an innovative attention-based model for generating image captions, drawing inspiration from advancements in machine translation and object detection. The model is designed to autonomously describe image content, learning to focus on significant elements while generating descriptive text. Two training methods are explored: a deterministic approach utilizing standard backpropagation techniques and a stochastic method that maximizes a variational lower bound. Visualizations demonstrate the model's ability to selectively concentrate on salient objects, correlating with the generation of relevant words in the output sequence. The effectiveness of the attention mechanism is substantiated by achieving state-of-the-art results on three prominent benchmark datasets: Flickr9k, Flickr30k, and MS COCO.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An improved harmony search algorithm for optimal reactive power dispatch
A new harmony search algorithm is proposed to solve the reactive power dispatch problem in this paper. The algorithm uses a stochastic random search instead of a gradient-based search so as to avoid getting trapped in a local minimum. To improve the efficiency of the basic harmony search algorithm, an improved strategy is proposed, which includes introducing a new parameter called “pitch adjusting rate” to fine-tune solutions and a new method to generate a new harmony. The effectiveness of the proposed algorithm is verified by testing on two standard IEEE systems. The simulation results show that the proposed algorithm can obtain better solutions compared with some other optimization algorithms.

**Similar Documents**: This paper introduces an enhanced harmony search algorithm tailored for the optimal reactive power dispatch problem. The algorithm eschews gradient-based searches in favor of a stochastic random search to prevent entrapment in local minima, thereby enhancing the exploration of the solution space. To further optimize the basic harmony search algorithm, an innovative strategy is implemented, featuring the introduction of a novel parameter known as the "pitch adjusting rate" for refining solutions, and a distinct approach to generating new harmony vectors. The efficacy of the proposed algorithm is corroborated through testing on two benchmark IEEE systems. Comparative analysis with other optimization techniques reveals the algorithm's superiority in yielding superior solutions.

Regularized training of an autoencoder typically results in hidden unit biases that take on large negative values. We show that negative biases are a natural result of using a hidden layer whose responsibility is to both represent the input data and act as a selection mechanism that ensures sparsity of the representation. We then show that negative biases impede the learning of data distributions whose intrinsic dimensionality is high. We also propose a new activation function that decouples the two roles of the hidden layer and that allows us to learn representations on data with very high intrinsic dimensionality, where standard autoencoders typically fail. Since the decoupled activation function acts like an implicit regularizer, the model can be trained by minimizing the reconstruction error of training data, without requiring any additional regularization.

We consider the problem of finding a matching between two sets of features, given complex relations among them, going beyond pairwise. Each feature set is modeled by a hypergraph where the complex relations are represented by hyper-edges. A match between the feature sets is then modeled as a hypergraph matching problem. We derive the hyper-graph matching problem in a probabilistic setting represented by a convex optimization. First, we formalize a soft matching criterion that emerges from a probabilistic interpretation of the problem input and output, as opposed to previous methods that treat soft matching as a mere relaxation of the hard matching problem. Second, the model induces an algebraic relation between the hyper-edge weight matrix and the desired vertex-to-vertex probabilistic matching. Third, the model explains some of the graph matching normalization proposed in the past on a heuristic basis such as doubly stochastic normalizations of the edge weights. A key benefit of the model is that the global optimum of the matching criteria can be found via an iterative successive projection algorithm. The algorithm reduces to the well known Sinkhorn [15] row/column matrix normalization procedure in the special case when the two graphs have the same number of vertices and a complete matching is desired. Another benefit of our model is the straight-forward scalability from graphs to hyper-graphs.

Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information. To learn these distributions, we propose an energy-based max-margin objective. We show that the resulting approach captures uniquely expressive semantic information, and outperforms alternatives, such as word2vec skip-grams, and Gaussian embeddings, on benchmark datasets such as word similarity and entailment.

Text analytics applications using machine learning techniques have grown in importance with ever increasing amount of data being generated from web-scale applications, social media and digital repositories. Apart from being large in size, these generated data are often unstructured and are heavily sparse in nature. The performance of these applications on current systems is hampered by hard to predict branches and low compute-per-byte ratio. This paper proposes a set of fine-grained accelerators that improve the performance and energy-envelope of these applications by an order of magnitude.

We address therating-inference problem, wherein rather than simply decide whether a review is “thumbs up” or “thumbs down”, as in previous sentiment analysis work, one must determine an author’s evaluation with respect to a multi-point scale (e.g., one to five “stars”). This task represents an interesting twist on standard multi-class text categorization because there are several different degrees of similarity between class labels; for example, “three stars” is intuitively closer to “four stars” than to “one star”. We first evaluate human performance at the task. Then, we apply a metaalgorithm, based on a metric labelingformulation of the problem, that alters a given n-ary classifier’s output in an explicit attempt to ensure that similar items receive similar labels. We show that the meta-algorithm can provide significant improvements over both multi-class and regression versions of SVMs when we employ a novel similarity measure appropriate to the problem. Publication info: Proceedings of the ACL, 2005.

Traditional activation functions such as hyperbolic tangent and logistic sigmoid have seen frequent use historically in artificial neural networks. However, nowadays, in practice, they have fallen out of favor, undoubtedly due to the gap in performance observed in recognition and classification tasks when compared to their well-known counterparts such as rectified linear or maxout. In this paper, we introduce a simple, new type of activation function for multilayer feed-forward architectures. Unlike other approaches where new activation functions have been designed by discarding many of the mainstays of traditional activation function design, our proposed function relies on them and therefore shares most of the properties found in traditional activation functions. Nevertheless, our activation function differs from traditional activation functions on two major points: its asymptote and global extremum. Defining a function which enjoys the property of having a global maximum and minimum, turned out to be critical during our design-process since we believe it is one of the main reasons behind the gap observed in performance between traditional activation functions and their recently introduced counterparts. We evaluate the effectiveness of the proposed activation function on four commonly used datasets, namely, MNIST, CIFAR-10, CIFAR-100, and the Pang and Lee’s movie review. Experimental results demonstrate that the proposed function can effectively be applied across various datasets where our accuracy, given the same network topology, is competitive with the state-of-the-art. In particular, the proposed activation function outperforms the state-of-the-art methods on the MNIST dataset.

We consider the problem of sparse canonical correlation analysis (CCA), i.e., the search for two linear combi nations, one for each multivariate, that yield maximum correlation using a specified number of variables. We propose an efficient numeri cal approximation based on a direct greedy approach which bound s the correlation at each stage. The method is specifically des igned to cope with large data sets and its computational complexit y depends only on the sparsity levels. We analyze the algorith m’s performance through the tradeoff between correlation and parsimony. The results of numerical simulation suggest that a significant portion of the correlation may be captured using a relatively small number of variables. In addition, we exami ne the use of sparse CCA as a regularization method when the number of available samples is small compared to the dimensions of t he multivariates. I. I NTRODUCTION Canonical correlation analysis (CCA), introduced by Harol d Hotelling [1], is a standard technique in multivariate data n lysis for extracting common features from a pair of data sourc es [2], [3]. Each of these data sources generates a random vecto r that we call a multivariate. Unlike classical dimensionali ty reduction methods which address one multivariate, CCA take s into account the statistical relations between samples fro m two spaces of possibly different dimensions and structure. In particular, it searches for two linear combinations, one fo r each multivariate, in order to maximize their correlation. It is used in different disciplines as a stand-alone tool or as a preprocessing step for other statistical methods. Further more, CCA is a generalized framework which includes numerous classical methods in statistics, e.g., Principal Componen t Analysis (PCA), Partial Least Squares (PLS) and Multiple Linear Regression (MLR) [4]. CCA has recently regained attention with the advent of kernel CCA and its application to independent component analysis [5], [6]. The last decade has witnessed a growing interest in the search for sparse representations of signals and sparse numerical methods. Thus, we consider the problem of sparse CCA, i.e., the search for linear combinations with maximal correlation using a small number of variables. The quest for sparsity can be motivated through various reasonings. First is the ability to interpret and visualize the results. A small number of variables allows us to get the “big picture”, while sacrificing some of the small details. Moreover, spars e representations enable the use of computationally efficien t The first two authors contributed equally to this manuscript . This work was supported in part by an AFOSR MURI under Grant FA9550-06-1-0 324. numerical methods, compression techniques, as well as nois e reduction algorithms. The second motivation for sparsity i s regularization and stability. One of the main vulnerabilit ies of CCA is its sensitivity to a small number of observations. Thu s, regularized methods such as ridge CCA [7] must be used. In this context, sparse CCA is a subset selection scheme which allows us to reduce the dimensions of the vectors and obtain a stable solution. To the best of our knowledge the first reference to sparse CCA appeared in [2] where backward and stepwise subset selection were proposed. This discussion was of qualitativ e nature and no specific numerical algorithm was proposed. Recently, increasing demands for multidimensional data pr ocessing and decreasing computational cost has caused the topic to rise to prominence once again [8]–[13]. The main disadvantages with these current solutions is that there is no direct control over the sparsity and it is difficult (and nonintuitive) to select their optimal hyperparameters. In add ition, the computational complexity of most of these methods is too high for practical applications with high dimensional data sets. Sparse CCA has also been implicitly addressed in [9], [14] an d is intimately related to the recent results on sparse PCA [9] , [15]–[17]. Indeed, our proposed solution is an extension of the results in [17] to CCA. The main contribution of this work is twofold. First, we derive CCA algorithms with direct control over the sparsity in each of the multivariates and examine their performance. Our computationally efficient methods are specifically aime d at understanding the relations between two data sets of larg e dimensions. We adopt a forward (or backward) greedy approach which is based on sequentially picking (or dropping) variables. At each stage, we bound the optimal CCA solution and bypass the need to resolve the full problem. Moreover, the computational complexity of the forward greedy method does not depend on the dimensions of the data but only on the sparsity parameters. Numerical simulation results show th at a significant portion of the correlation can be efficiently cap tured using a relatively low number of non-zero coefficients. Our second contribution is investigation of sparse CCA as a regularization method. Using empirical simulations we examin e the use of the different algorithms when the dimensions of the multivariates are larger than (or of the same order of) the number of samples and demonstrate the advantage of sparse CCA. In this context, one of the advantages of the greedy approach is that it generates the full sparsity path i n a single run and allows for efficient parameter tuning using

Learning a classifier from ambiguously labeled face images is challenging since training images are not always explicitly-labeled. For instance, face images of two persons in a news photo are not explicitly labeled by their names in the caption. We propose a Matrix Completion for Ambiguity Resolution (MCar) method for predicting the actual labels from ambiguously labeled images. This step is followed by learning a standard supervised classifier from the disambiguated labels to classify new images. To prevent the majority labels from dominating the result of MCar, we generalize MCar to a weighted MCar (WMCar) that handles label imbalance. Since WMCar outputs a soft labeling vector of reduced ambiguity for each instance, we can iteratively refine it by feeding it as the input to WMCar. Nevertheless, such an iterative implementation can be affected by the noisy soft labeling vectors, and thus the performance may degrade. Our proposed Iterative Candidate Elimination (ICE) procedure makes the iterative ambiguity resolution possible by gradually eliminating a portion of least likely candidates in ambiguously labeled faces. We further extend MCar to incorporate the labeling constraints among instances when such prior knowledge is available. Compared to existing methods, our approach demonstrates improvements on several ambiguously labeled datasets.

Many criteria can be used to evaluate the performance of supervised learning. Different criteria are appropriate in different settings, and it is not always clear which criteria to use. A further complication is that learning methods that perform well on one criterion may not perform well on other criteria. For example, SVMs and boosting are designed to optimize accuracy, whereas neural nets typically optimize squared error or cross entropy. We conducted an empirical study using a variety of learning methods (SVMs, neural nets, k-nearest neighbor, bagged and boosted trees, and boosted stumps) to compare nine boolean classification performance metrics: Accuracy, Lift, F-Score, Area under the ROC Curve, Average Precision, Precision/Recall Break-Even Point, Squared Error, Cross Entropy, and Probability Calibration. Multidimensional scaling (MDS) shows that these metrics span a low dimensional manifold. The three metrics that are appropriate when predictions are interpreted as probabilities: squared error, cross entropy, and calibration, lay in one part of metric space far away from metrics that depend on the relative order of the predicted values: ROC area, average precision, break-even point, and lift. In between them fall two metrics that depend on comparing predictions to a threshold: accuracy and F-score. As expected, maximum margin methods such as SVMs and boosted trees have excellent performance on metrics like accuracy, but perform poorly on probability metrics such as squared error. What was not expected was that the margin methods have excellent performance on ordering metrics such as ROC area and average precision. We introduce a new metric, SAR, that combines squared error, accuracy, and ROC area into one metric. MDS and correlation analysis shows that SAR is centrally located and correlates well with other metrics, suggesting that it is a good general purpose metric to use when more specific criteria are not known.

We introduce an autoregressive attention mechanism for parallelizable characterlevel sequence modeling. We use this method to augment a neural model consisting of blocks of causal convolutional layers connected by highway network skip connections. We denote the models with and without the proposed attention mechanism respectively as Highway Causal Convolution (Causal Conv) and Autoregressive-attention Causal Convolution (ARA-Conv). The autoregressive attention mechanism crucially maintains causality in the decoder, allowing for parallel implementation. We demonstrate that these models, compared to their recurrent counterparts, enable fast and accurate learning in character-level NLP tasks. In particular, these models outperform recurrent neural network models in natural language correction and language modeling tasks, and run in a fraction of the time.


Question: Transitioning from Softmax to Sparsemax: A Sparse Representation for Attention and Multi-Label Classification Tasks

Introducing RCV1-v2: An Enhanced Benchmark for Text Categorization Research

Reuters Corpus Volume I (RCV1-v2) represents a significant advancement in text categorization research, offering a refined collection of over 800,000 manually classified news articles generously provided by Reuters, Ltd. for scholarly inquiry. The dataset's utility for text categorization studies hinges on a thorough grasp of the practical constraints that governed its creation. To this end, we have conducted detailed interviews with Reuters staff and meticulously reviewed internal documentation to elucidate the coding guidelines and quality assurance protocols that shaped RCV1. This investigation has shed light on the intended meaning of the hierarchical category structures and identified necessary corrections to eliminate erroneous data points. We designate the original dataset as RCV1-v1 and the revised, error-corrected version as RCV1-v2. To assess the utility and characteristics of RCV1-v2, we have benchmarked a range of established supervised learning algorithms on this dataset. These experiments not only illustrate the dataset's unique features but also suggest promising avenues for future research and establish baseline performance metrics for forthcoming studies. Comprehensive experimental outcomes, categorized by individual categories, along with the corrected category assignments and taxonomy, are accessible through online appendices, providing researchers with a valuable resource for advancing the field of text categorization.

Neural networks have achieved state-ofthe-art performance on several structuredoutput prediction tasks, trained in a fully supervised fashion. However, annotated examples in structured domains are often costly to obtain, which thus limits the applications of neural networks. In this work, we propose Maximum Margin Reward Networks, a neural networkbased framework that aims to learn from both explicit (full structures) and implicit supervision signals (delayed feedback on the correctness of the predicted structure). On named entity recognition and semantic parsing, our model outperforms previous systems on the benchmark datasets, CoNLL-2003 and WebQuestionsSP.

We present Local Naive Bayes Nearest Neighbor, an improvement to the NBNN image classification algorithm that increases classification accuracy and improves its ability to scale to large numbers of object classes. The key observation is that only the classes represented in the local neighborhood of a descriptor contribute significantly and reliably to their posterior probability estimates. Instead of maintaining a separate search structure for each class's training descriptors, we merge all of the reference data together into one search structure, allowing quick identification of a descriptor's local neighborhood. We show an increase in classification accuracy when we ignore adjustments to the more distant classes and show that the run time grows with the log of the number of classes rather than linearly in the number of classes as did the original. Local NBNN gives a 100 times speed-up over the original NBNN on the Caltech 256 dataset. We also provide the first head-to-head comparison of NBNN against spatial pyramid methods using a common set of input features. We show that local NBNN outperforms all previous NBNN based methods and the original spatial pyramid model. However, we find that local NBNN, while competitive with, does not beat state-of-the-art spatial pyramid methods that use local soft assignment and max-pooling.

In this paper we propose a method to use multiple generative models with latent variables for classi cation tasks. The standard approach to use generative models for classi cation is to train a separate model for each class. A novel data point is then classi ed by the model that attributes the highest probability. The algorithm we propose modi es the parameters of the models to improve the classi cation accuracy. Our approach is made computationally tractable by assuming that each of the models is deterministic, by which we mean that a data-point is associated to only a single latent state. The resulting algorithm is a variant of the support vector machine learning algorithm and in a limiting case the method is similar to the standard perceptron learning algorithm. We apply the method to two types of latent variable models. The rst has a discrete latent state space and the second, principal component analysis, has a continuous latent state space. We compare the e6ectiveness of both approaches on a handwritten digit recognition problem and on a satellite image recognition problem. c © 2003 Elsevier B.V. All rights reserved.

We introduce a new neural architecture to learn the conditional probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. Such problems cannot be trivially addressed by existent approaches such as sequence-to-sequence [1] and Neural Turing Machines [2], because the number of target classes in each step of the output depends on the length of the input, which is variable. Problems such as sorting variable sized sequences, and various combinatorial optimization problems belong to this class. Our model solves the problem of variable size output dictionaries using a recently proposed mechanism of neural attention. It differs from the previous attention attempts in that, instead of using attention to blend hidden units of an encoder to a context vector at each decoder step, it uses attention as a pointer to select a member of the input sequence as the output. We call this architecture a Pointer Net (Ptr-Net). We show Ptr-Nets can be used to learn approximate solutions to three challenging geometric problems – finding planar convex hulls, computing Delaunay triangulations, and the planar Travelling Salesman Problem – using training examples alone. Ptr-Nets not only improve over sequence-to-sequence with input attention, but also allow us to generalize to variable size output dictionaries. We show that the learnt models generalize beyond the maximum lengths they were trained on. We hope our results on these tasks will encourage a broader exploration of neural learning for discrete problems.

The choice of activation functions in deep networks has a significant effect on the training dynamics and task performance. Currently, the most successful and widely-used activation function is the Rectified Linear Unit (ReLU). Although various alternatives to ReLU have been proposed, none have managed to replace it due to inconsistent gains. In this work, we propose a new activation function, named Swish, which is simply f(x) = x · sigmoid(x). Our experiments show that Swish tends to work better than ReLU on deeper models across a number of challenging datasets. For example, simply replacing ReLUs with Swish units improves top-1 classification accuracy on ImageNet by 0.9% for Mobile NASNetA and 0.6% for Inception-ResNet-v2. The simplicity of Swish and its similarity to ReLU make it easy for practitioners to replace ReLUs with Swish units in any neural network.

Many learning tasks such as spam filtering and credit card fraud detection face an active adversary that tries to avoid detection. For learning problems that deal with an active adversary, it is important to model the adversary's attack strategy and develop robust learning models to mitigate the attack. These are the two objectives of this paper. We consider two attack models: a free-range attack model that permits arbitrary data corruption and a restrained attack model that anticipates more realistic attacks that a reasonable adversary would devise under penalties. We then develop optimal SVM learning strategies against the two attack models. The learning algorithms minimize the hinge loss while assuming the adversary is modifying data to maximize the loss. Experiments are performed on both artificial and real data sets. We demonstrate that optimal solutions may be overly pessimistic when the actual attacks are much weaker than expected. More important, we demonstrate that it is possible to develop a much more resilient SVM learning model while making loose assumptions on the data corruption models. When derived under the restrained attack model, our optimal SVM learning strategy provides more robust overall performance under a wide range of attack parameters.

In this paper, we propose an end-to-end trainable regression approach for human pose estimation from still images. We use the proposed Soft-argmax function to convert feature maps directly to joint coordinates, resulting in a fully differentiable framework. Our method is able to learn heat maps representations indirectly, without additional steps of artificial ground truth generation. Consequently, contextual information can be included to the pose predictions in a seamless way. We evaluated our method on two very challenging datasets, the Leeds Sports Poses (LSP) and the MPII Human Pose datasets, reaching the best performance among all the existing regression methods and comparable results to the state-of-the-art detection based approaches.

Recently, maxout networks have brought significant improvements to various speech recognition and computer vision tasks. In this paper we introduce two new types of generalized maxout units, which we call p-norm and soft-maxout. We investigate their performance in Large Vocabulary Continuous Speech Recognition (LVCSR) tasks in various languages with 10 hours and 60 hours of data, and find that the p-norm generalization of maxout consistently performs well. Because, in our training setup, we sometimes see instability during training when training unbounded-output nonlinearities such as these, we also present a method to control that instability. This is the “normalization layer”, which is a nonlinearity that scales down all dimensions of its input in order to stop the average squared output from exceeding one. The performance of our proposed nonlinearities are compared with maxout, rectified linear units (ReLU), tanh units, and also with a discriminatively trained SGMM/HMM system, and our p-norm units with p equal to 2 are found to perform best.

Stacked denoising autoencoders (SDAs) have been successfully used to learn new representations for domain adaptation. Recently, they have attained record accuracy on standard benchmark tasks of sentiment analysis across different text domains. SDAs learn robust data representations by reconstruction, recovering original features from data that are artificially corrupted with noise. In this paper, we propose marginalized SDA (mSDA) that addresses two crucial limitations of SDAs: high computational cost and lack of scalability to high-dimensional features. In contrast to SDAs, our approach of mSDA marginalizes noise and thus does not require stochastic gradient descent or other optimization algorithms to learn parameters — in fact, they are computed in closed-form. Consequently, mSDA, which can be implemented in only 20 lines of MATLAB, significantly speeds up SDAs by two orders of magnitude. Furthermore, the representations learnt by mSDA are as effective as the traditional SDAs, attaining almost identical accuracies in benchmark tasks.

Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information. To learn these distributions, we propose an energy-based max-margin objective. We show that the resulting approach captures uniquely expressive semantic information, and outperforms alternatives, such as word2vec skip-grams, and Gaussian embeddings, on benchmark datasets such as word similarity and entailment.


Question: Transitioning from Softmax to Sparsemax: A sparse model for attention mechanisms and multi-label classification tasks.

This paper presents a novel framework for enabling machines to comprehend and answer questions based on natural language documents. The challenge of teaching reading comprehension to machines has long persisted, primarily due to the absence of large-scale training and evaluation datasets that could accurately measure a system's ability to understand and respond to questions about the content it has processed. Our methodology addresses this gap by generating extensive supervised reading comprehension datasets, which, in turn, facilitates the creation and training of deep neural network models specifically designed for reading and answering complex questions. These models leverage attention mechanisms to focus on relevant parts of the text, thereby improving their comprehension and response accuracy with minimal reliance on pre-existing knowledge of language structure. This approach not only enhances the machine's ability to read and understand real-world documents but also sets a new standard for evaluating machine reading systems by providing a comprehensive and scalable dataset for supervised learning.

Multimedia content is dominating today's Web information. The nature of multimedia user-item interactions is 1/0 binary implicit feedback (e.g., photo likes, video views, song downloads, etc.), which can be collected at a larger scale with a much lower cost than explicit feedback (e.g., product ratings). However, the majority of existing collaborative filtering (CF) systems are not well-designed for multimedia recommendation, since they ignore the implicitness in users' interactions with multimedia content. We argue that, in multimedia recommendation, there exists item- and component-level implicitness which blurs the underlying users' preferences. The item-level implicitness means that users' preferences on items (e.g. photos, videos, songs, etc.) are unknown, while the component-level implicitness means that inside each item users' preferences on different components (e.g. regions in an image, frames of a video, etc.) are unknown. For example, a 'view'' on a video does not provide any specific information about how the user likes the video (i.e.item-level) and which parts of the video the user is interested in (i.e.component-level). In this paper, we introduce a novel attention mechanism in CF to address the challenging item- and component-level implicit feedback in multimedia recommendation, dubbed Attentive Collaborative Filtering (ACF). Specifically, our attention model is a neural network that consists of two attention modules: the component-level attention module, starting from any content feature extraction network (e.g. CNN for images/videos), which learns to select informative components of multimedia items, and the item-level attention module, which learns to score the item preferences. ACF can be seamlessly incorporated into classic CF models with implicit feedback, such as BPR and SVD++, and efficiently trained using SGD. Through extensive experiments on two real-world multimedia Web services: Vine and Pinterest, we show that ACF significantly outperforms state-of-the-art CF methods.

A large body of research in supervised learning deals with the analysis of singlelabel data, where training examples are associated with a single label λ from a set of disjoint labels L. However, training examples in several application domains are often associated with a set of labels Y ⊆ L. Such data are called multi-label. Textual data, such as documents and web pages, are frequently annotated with more than a single label. For example, a news article concerning the reactions of the Christian church to the release of the “Da Vinci Code” film can be labeled as both religion and movies. The categorization of textual data is perhaps the dominant multi-label application. Recently, the issue of learning from multi-label data has attracted significant attention from a lot of researchers, motivated from an increasing number of new applications, such as semantic annotation of images [1, 2, 3] and video [4, 5], functional genomics [6, 7, 8, 9, 10], music categorization into emotions [11, 12, 13, 14] and directed marketing [15]. Table 1 presents a variety of applications that are discussed in the literature. This chapter reviews past and recent work on the rapidly evolving research area of multi-label data mining. Section 2 defines the two major tasks in learning from multi-label data and presents a significant number of learning methods. Section 3 discusses dimensionality reduction methods for multi-label data. Sections 4 and 5 discuss two important research challenges, which, if successfully met, can significantly expand the real-world applications of multi-label learning methods: a) exploiting label structure and b) scaling up to domains with large number of labels. Section 6 introduces benchmark multi-label datasets and their statistics, while Section 7 presents the most frequently used evaluation measures for multi-label learn-

We address therating-inference problem, wherein rather than simply decide whether a review is “thumbs up” or “thumbs down”, as in previous sentiment analysis work, one must determine an author’s evaluation with respect to a multi-point scale (e.g., one to five “stars”). This task represents an interesting twist on standard multi-class text categorization because there are several different degrees of similarity between class labels; for example, “three stars” is intuitively closer to “four stars” than to “one star”. We first evaluate human performance at the task. Then, we apply a metaalgorithm, based on a metric labelingformulation of the problem, that alters a given n-ary classifier’s output in an explicit attempt to ensure that similar items receive similar labels. We show that the meta-algorithm can provide significant improvements over both multi-class and regression versions of SVMs when we employ a novel similarity measure appropriate to the problem. Publication info: Proceedings of the ACL, 2005.

Abstract. Multinomial naive Bayes (MNB) is a popular method for document classification due to its computational efficiency and relatively good predictive performance. It has recently been established that predictive performance can be improved further by appropriate data transformations [1, 2]. In this paper we present another transformation that is designed to combat a potential problem with the application of MNB to unbalanced datasets. We propose an appropriate correction by adjusting attribute priors. This correction can be implemented as another data normalization step, and we show that it can significantly improve the area under the ROC curve. We also show that the modified version of MNB is very closely related to the simple centroid-based classifier and compare the two methods empirically.

Classifying products precisely and efficiently is a major challenge in modern e-commerce. The high traffic of new products uploaded daily and the dynamic nature of the categories raise the need for machine learning models that can reduce the cost and time of human editors. In this paper, we propose a decision level fusion approach for multi-modal product classification based on text and image neural network classifiers. We train input specific state-of-the-art deep neural networks for each input source, show the potential of forging them together into a multi-modal architecture and train a novel policy network that learns to choose between them. Finally, we demonstrate that our multi-modal network improves classification accuracy over both networks on a real-world largescale product classification dataset that we collected from Walmart.com. While we focus on image-text fusion that characterizes e-commerce businesses, our algorithms can be easily applied to other modalities such as audio, video, physical sensors, etc.

We develop a fully discriminative learning approach for supervised Latent Dirichlet Allocation (LDA) model, which maximizes the posterior probability of the prediction variable given the input document. Different from traditional variational learning or Gibbs sampling approaches, the proposed learning method applies (i) the mirror descent algorithm for exact maximum a posterior inference and (ii) back propagation with stochastic gradient descent for model parameter estimation, leading to scalable learning of the model in an end-to-end discriminative manner. As a byproduct, we also apply this technique to develop a new learning method for the traditional unsupervised LDA model. Experimental results on two real-world regression and classification tasks show that the proposed methods significantly outperform the previous supervised/unsupervised LDA learning methods.

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while ‘multi-sense’ methods have been proposed and tested on artificial wordsimilarity tasks, we don’t know if they improve real natural language understanding tasks. In this paper we introduce a multisense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language un-

In multilabel learning, each instance in the training set is associated with a set of labels and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e., backpropagation for multilabel learning, is proposed. It is derived from the popular backpropagation algorithm through employing a novel error function capturing the characteristics of multilabel learning, i.e., the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two real-world multilabel learning problems, i.e., functional genomics and text categorization, show that the performance of BP-MLL is superior to that of some well-established multilabel learning algorithms

Helpdesk is a key component of any large IT organization, where users can log a ticket about any issue they face related to IT infrastructure, administrative services, human resource services, etc. Normally, users have to assign appropriate set of labels to a ticket so that it could be routed to right domain expert who can help resolve the issue. In practice, the number of labels are very large and organized in form of a tree. It is non-trivial to describe the issue completely and attach appropriate labels unless one knows the cause of the problem and the related labels. Sometimes domain experts discuss the issue with the users and change the ticket labels accordingly, without modifying the ticket description. This results in inconsistent and badly labeled data, making it hard for supervised algorithms to learn from. In this paper, we propose a novel approach of creating a conversational helpdesk system, which will ask relevant questions to the user, for identification of the right category and will then raise a ticket on users' behalf. We use attention based seq2seq model to assign the hierarchical categories to tickets. We use a slot filling model to help us decide what questions to ask to the user, if the top-k model predictions are not consistent. We also present a novel approach to generate training data for the slot filling model automatically based on attention in the hierarchical classification model. We demonstrate via a simulated user that the proposed approach can give us a significant gain in accuracy on ticket-data without asking too many questions to users. Finally, we also show that our seq2seq model is as versatile as other approaches on publicly available datasets, as state of the art approaches.

In this paper a doubly attentive transformer machine translation model (DATNMT) is presented in which a doubly-attentive transformer decoder normally joins spatial visual features obtained via pretrained convolutional neural networks, conquering any gap between image captioning and translation. In this framework, the transformer decoder figures out how to take care of source-language words and parts of an image freely by methods for two separate attention components in an Enhanced Multi-Head Attention Layer of doubly attentive transformer, as it generates words in the target language. We find that the proposed model can effectively exploit not just the scarce multimodal machine translation data, but also large general-domain textonly machine translation corpora, or imagetext image captioning corpora. The experimental results show that the proposed doublyattentive transformer-decoder performs better than a single-decoder transformer model, and gives the state-of-the-art results in the EnglishGerman multimodal machine translation task.


Question: A trust-aware method for identifying and filtering review spam.

This paper presents a Trust-Aware Collaborative Filtering technique designed to improve the performance and reliability of Recommender Systems. Traditionally, Recommender Systems facilitate users in discovering desired resources by leveraging the collective experiences and opinions of similar individuals, replacing the need for expert annotations with a decentralized, user-driven data collection process. However, this collaborative mechanism introduces a critical challenge: ensuring the quality of the recommendations. The introduction of a "web of trust," which involves quantifying the trustworthiness among users, significantly enhances the system's capabilities. By incorporating trust metrics, the filtering process benefits from the reputations of users, which are calculated through trust propagation. This not only enriches the filtering algorithm but also addresses the computational limitations associated with conventional similarity assessment methods. Experimental results obtained from the Epinions.com dataset highlight the efficacy of trust propagation in enhancing Recommender System coverage without compromising the accuracy of predictions. Notably, the system demonstrates substantial improvements in recommendation quality for new users who have provided limited feedback.

a r t i c l e i n f o Collaborative Filtering (CF) is the most popular recommendation technique but still suffers from data sparsi-ty, user and item cold-start problems, resulting in poor recommendation accuracy and reduced coverage. This study incorporates additional information from the users' social trust network and the items' semantic domain knowledge to alleviate these problems. It proposes an innovative Trust–Semantic Fusion (TSF)-based recommendation approach within the CF framework. Experiments demonstrate that the TSF approach significantly outperforms existing recommendation algorithms in terms of recommendation accuracy and coverage when dealing with the above problems. A business-to-business recommender system case study validates the applicability of the TSF approach. Recommender systems are considered the most popular forms of web personalization and have become a promising and important research topic in information sciences and decision support systems Recommender systems are used to either predict whether a particular user will like a particular item or to identify a set of k items that will be of interest to a certain user, and have been used in different web-based applications including e-business, e-learning and e-tourism [8,22,31]. Currently, Collaborative Filtering (CF) is probably the most known and commonly used recommendation approach in recommender systems. CF works by collecting user ratings for items in a given domain and computing similarities between users or between items in order to produce recommendations [1,31]. CF can be further divided into user-based and item-based CF approaches. In user-based CF approach, a user will receive recommendations of items that similar users liked. In item-based CF approach, a user will receive recommendations of items that are similar to the ones that the user liked in the past [1]. Despite their popularity and success, the CF-based approaches still suffer from some major limitations ; these include data sparsity, cold-start user and cold-start item problems [1,3,36,37]. The data sparsity problem occurs when the number of available items increases and the number of ratings in the rating matrix is insufficient for generating accurate predictions. When the ratings obtained are very small compared to the number of ratings that are needed to be predicted, a recommender system becomes unable to locate similar neighbors and produces poor recommendations. The cold-start (CS) user problem, which is also known as the new user problem, affects users who have none, or a small number of ratings. When the number of rated items is small for the CS user, …

The growing popularity of IP telephony systems has made them attractive targets for spammers. Voice call spam, also known as Spam over Internet Telephony (SPIT), is potentially a more serious problem than email spam because of the real time processing requirements of voice packets. We explore a novel mechanism that uses duration of calls between users to combat SPIT. CallRank, the scheme proposed by us, uses call duration to establish social network linkages and global reputations for callers, based on which call recipients can decide whether the caller is legitimate or not. CallRank has been implemented within a VoIP system simulation and our results show that we are able to achieve a false negative rate of 10% and a false positive rate of 3% even in the presence of a significant fraction of spammers.

With the advent to social media the number of reviews for any particular product is in millions, as there exist thousand of websites where that particular product exists. As the numbers of reviews are very high the user ends up spending a lot of time for searching best product based on the experiences shared by review writers. This paper presents a sentiment based rating approach for food recipes which sorts food recipes present on various website on the basis of sentiments of review writers. The results are shown with the help of a mobile application: Foodoholic. The output of the application is an ordered list of recipes with user input as their core ingredient.

People are highly dependent on online social networks (OSNs) which have attracted the interest of cyber criminals for carrying out a number of malicious activities. An entire industry of black-market services has emerged which offers fake accounts based services for sale. We, therefore, in our work, focus on detecting fake accounts on a very popular (and difficult for data collection) online social network, Facebook. Key contributions of our work are as follows. The first contribution has been collection of data related to real and fake accounts on Facebook. Due to strict privacy settings and ever evolving API of Facebook with each version adding more restrictions, collecting user accounts data became a major challenge. Our second contribution is the use of user-feed information on Facebook to understand user profile activity and identifying an extensive set of 17 features which play a key role in discriminating fake users on Facebook with real users. Third contribution is the use these features and identifying the key machine learning based classifiers who perform well in detection task out of a total of 12 classifiers employed. Fourth contribution is the identifying which type of activities (like, comment, tagging, sharing, etc) contribute the most in fake user detection. Results exhibit classification accuracy of 79% among the best performing classifiers. In terms of activities, likes and comments contribute well towards detection task. Although the accuracy is not very high, however, our work forms a baseline for further improvement. Our results indicate that many fake users are classified as real suggesting clearly that fake accounts are mimicking real user behavior to evade detection mechanisms. Our work concludes by enlisting a number of future course of actions that can be undertaken.

This paper proposes a hybrid solution of spam email classifier using context based email classification model as main algorithm complimented by information gain calculation to increase spam classification accuracy. Proposed solution consists of three stages email pre-processing, feature extraction and email classification. Research has found that LingerIG spam filter is highly effective at separating spam emails from cluster of homogenous work emails. Also experiment result proved the accuracy of spam filtering is 100% as recorded by the team of developers at University of Sydney. The study has shown that implementing the spam filter in the context-based email classification model is feasible. Experiment of the study has confirmed that spam filtering aspect of context-based classification model can be improved.

This article addresses the challenge of truth discovery from noisy social sensing data. The work is motivated by the emergence of social sensing as a data collection paradigm of growing interest, where humans perform sensory data collection tasks. Unlike the case with well-calibrated and well-tested infrastructure sensors, humans are less reliable, and the likelihood that participants' measurements are correct is often unknown a priori. Given a set of human participants of unknown trustworthiness together with their sensory measurements, we pose the question of whether one can use this information alone to determine, in an analytically founded manner, the probability that a given measurement is true. In our previous conference paper, we offered the first maximum likelihood solution to the aforesaid truth discovery problem for corroborating observations only. In contrast, this article extends the conference paper and provides the first maximum likelihood solution to handle the cases where measurements from different participants may be conflicting. The article focuses on binary measurements. The approach is shown to outperform our previous work used for corroborating observations, the state-of-the-art fact-finding baselines, as well as simple heuristics such as majority voting.

This work presents a new approach for analysing the ability of existing research metrics to identify research which has strongly influenced future developments. More specifically, we focus on the ability of citation counts and Mendeley reader counts to distinguish between publications regarded as seminal and publications regarded as literature reviews by field experts. The main motivation behind our research is to gain a better understanding of whether and how well the existing research metrics relate to research quality. For this experiment we have created a new dataset which we call TrueImpactDataset and which contains two types of publications, seminal papers and literature reviews. Using the dataset, we conduct a set of experiments to study how citation and reader counts perform in distinguishing these publication types, following the intuition that causing a change in a field signifies research quality. Our research shows that citation counts work better than a random baseline (by a margin of 10%) in distinguishing important seminal research papers from literature reviews while Mendeley reader counts do not work better than the baseline.

This paper tries to do the research of analyzing the relation of the customer trust, the customer satisfaction and purchase intention under the electronic commerce shopping environment. Drawing on the theory of customer trust, customer satisfaction and purchase intention, based on a survey of 102 respondents from the University in China, the United Kingdom, and the United States, the results showed that there is significant correlation between satisfaction and trust, satisfaction and purchase intention, trust and purchase intention, and more than 50% of respondents chose Taobao and Tmall, and the second is Amazon when they shopping onling. It could be expected that this kind of study is able to provide suggestion and guide for internet merchant.

Given a large online network of online auction users and their histories of transactions, how can we spot anomalies and auction fraud? This paper describes the design and implementation of NetProbe, a system that we propose for solving this problem. NetProbe models auction users and transactions as a Markov Random Field tuned to detect the suspicious patterns that fraudsters create, and employs a Belief Propagation mechanism to detect likely fraudsters. Our experiments show that NetProbe is both efficient and effective for fraud detection. We report experiments on synthetic graphs with as many as 7,000 nodes and 30,000 edges, where NetProbe was able to spot fraudulent nodes with over 90% precision and recall, within a matter of seconds. We also report experiments on a real dataset crawled from eBay, with nearly 700,000 transactions between more than 66,000users, where NetProbe was highly effective at unearthing hidden networks of fraudsters, within a realistic response time of about 6 minutes. For scenarios where the underlying data is dynamic in nature, we propose IncrementalNetProbe, which is an approximate, but fast, variant of NetProbe. Our experiments prove that Incremental NetProbe executes nearly doubly fast as compared to NetProbe, while retaining over 99% of its accuracy.

Low bug report quality and human conflicts pose challenges to keep bug tracking systems productive. This work proposes to address these issues by applying game mechanisms to bug tracking systems. We investigate the use of game mechanisms in Stack Overflow, an online community organized to resolve computer programming related problems, for which the improvements we seek for bug tracking systems also turn out to be relevant. The results of our Stack Overflow investigation show that its game mechanisms could be used to address these issues by motivating contributors to increase contribution frequency and quality, by filtering useful contributions, and by creating an agile and dependable moderation system. We proceed by mapping these mechanisms to open-source bug tracking systems, and find that most benefits are applicable. Additionally, our results motivate tailoring a reward and reputation system and summarizing bug reports as future directions for increasing the benefits of game mechanisms in bug tracking systems.


Question: A trust-based method for identifying and filtering out review spam.

This paper delves into the issue of fake review detection, a critical concern for both industries and researchers aiming to ensure the authenticity of user experiences and opinions reflected in online reviews. Supervised learning techniques have emerged as a leading strategy in tackling this challenge. However, a significant obstacle lies in the acquisition of accurately labeled fake reviews for training purposes, given the inherent difficulties in manually identifying such reviews with certainty. To circumvent this issue, researchers have turned to pseudo fake reviews as a training resource. Among the various types of pseudo reviews, those generated through Amazon Mechanical Turk (AMT), a popular crowdsourcing platform, have garnered particular interest. AMT-generated fake reviews have demonstrated remarkable accuracy in detection, with one study reporting an 89.6% success rate using solely word n-gram features. Despite this high accuracy, a critical distinction exists between AMT-generated reviews and authentic fake reviews found on commercial websites. The psychological mindset of AMT contributors, who write reviews without genuine business interests at stake, likely diverges from that of authors of real fake reviews, who may have vested interests in promoting or disparaging specific businesses. Our experimental findings support this hypothesis. To further explore this distinction, it becomes pertinent to compare the accuracy of fake review detection on AMT data against real-life data, to assess whether differing psychological states lead to variations in writing style and, subsequently, classification accuracy. For our real-life review dataset, we utilized filtered (fake) and unfiltered (non-fake) reviews from Yelp.com, considered the closest approximation to ground truth labels, to conduct a series of classification experiments utilizing only n-gram features. Our results revealed a detection accuracy of 67.8% on Yelp’s real-life data, underscoring the utility of n-gram features. To uncover the precise disparities between AMT and real-life review data, we introduced an innovative method employing the information theoretic measure of KL-divergence and its asymmetric property. This method unveiled intriguing psycholinguistic patterns between forced and natural fake reviewers. To enhance classification accuracy on real Yelp review data, we proposed an augmented set of behavioral features related to reviewers and their reviews for the learning process, resulting in a substantial improvement in the classification of real-life opinion spam data.

This work presents a new approach for analysing the ability of existing research metrics to identify research which has strongly influenced future developments. More specifically, we focus on the ability of citation counts and Mendeley reader counts to distinguish between publications regarded as seminal and publications regarded as literature reviews by field experts. The main motivation behind our research is to gain a better understanding of whether and how well the existing research metrics relate to research quality. For this experiment we have created a new dataset which we call TrueImpactDataset and which contains two types of publications, seminal papers and literature reviews. Using the dataset, we conduct a set of experiments to study how citation and reader counts perform in distinguishing these publication types, following the intuition that causing a change in a field signifies research quality. Our research shows that citation counts work better than a random baseline (by a margin of 10%) in distinguishing important seminal research papers from literature reviews while Mendeley reader counts do not work better than the baseline.

News professionals are facing the challenge of discovering news from more diverse and unreliable information in the age of social media. More and more news events break on social media first and are picked up by news media subsequently. The recent Brussels attack is such an example. At Reuters, a global news agency, we have observed the necessity of providing a more effective tool that can help our journalists to quickly discover news on social media, verify them and then inform the public.
 In this paper, we describe Reuters Tracer, a system for sifting through all noise to detect news events on Twitter and assessing their veracity. We disclose the architecture of our system and discuss the various design strategies that facilitate the implementation of machine learning models for noise filtering and event detection. These techniques have been implemented at large scale and successfully discovered breaking news faster than traditional journalism

Social network sites are becoming great through millions of users and information have been gathered from the users. This information have equal advantages to their friends and spammers. Twitter is one of the most popular social networks that, users can send short textual messages namely tweet. Researches have shown that this network is subject to spammer's invasion more than other social networks and more than six percent of its tweets are spam. So diagnose of the spam tweets is very important. Firstly, in this research, we determine various features for spam detection and then by using a clustering algorithm based on the data stream, we identify spam tweets. previous works in the field of spam tweets were done by classification algorithms. it is the first time that for spam tweets detection, an data stream clustering algorithm is used. Den stream Algorithm can cluster tweets and consider Outliers as spam. Results show, when this algorithm is set properly the amount of accuracy and precision of spam tweets detection will improve and false positive rate will reach to the minimum value in comparison with previous works.

Web spamming refers to actions intended to mislead search engines into ranking some pages higher than they deserve. Recently, the amount of web spam has increased dramatically, leading to a degradation of search results. This paper presents a comprehensive taxonomy of current spamming techniques, which we believe can help in developing appropriate countermeasures.

Filter lists are widely deployed by adblockers to block ads and other forms of undesirable content in web browsers. However, these filter lists are manually curated based on informal crowdsourced feedback, which brings with it a significant number of maintenance challenges. To address these challenges, we propose a machine learning approach for automatic and effective adblocking called AdGraph. Our approach relies on information obtained from multiple layers of the web stack (HTML, HTTP, and JavaScript) to train a machine learning classifier to block ads and trackers. Our evaluation on Alexa top-10K websites shows that AdGraph automatically and effectively blocks ads and trackers with 97.7% accuracy. Our manual analysis shows that AdGraph has better recall than filter lists, it blocks 16%more ads and trackers with 65% accuracy. We also show that AdGraph is fairly robust against adversarial obfuscation by publishers and advertisers that bypass filter lists.

Social media for news consumption is becoming popular nowadays. The low cost, easy access and rapid information dissemination of social media bring benefits for people to seek out news timely. However, it also causes the widespread of fake news, i.e., low-quality news pieces that are intentionally fabricated. The fake news brings about several negative effects on individual consumers, news ecosystem, and even society trust. Previous fake news detection methods mainly focus on news contents for deception classification or claim fact-checking. Recent Social and Psychology studies show potential importance to utilize social media data: 1) Confirmation bias effect reveals that consumers prefer to believe information that confirms their existing stances; 2) Echo chamber effect suggests that people tend to follow likeminded users and form segregated communities on social media. Even though users social engagements towards news on social media provide abundant auxiliary information for better detecting fake news, but existing work exploiting social engagements is rather limited. In this paper, we explore the correlations of publisher bias, news stance, and relevant user engagements simultaneously, and propose a Tri-Relationship Fake News detection framework (TriFN). We also provide two comprehensive real-world fake news datasets to facilitate fake news research. Experiments on these datasets demonstrate the effectiveness of the proposed approach. Introduction People nowadays tend to seek out and consume news from social media rather than traditional news organizations. For example, 62% of U.S. adults get news on social media in 2016, while in 2012, only 49 percent reported seeing news on social media. However, social media for news consumption is a double-edged sword. The quality of news on social media is much lower than traditional news organizations. Large volumes of “fake news”, i.e., those news articles with intentionally false information, are produced online for a variety of purposes, such as financial and political gain (Klein and Wueller 2017; Allcott and Gentzkow 2017). Fake news can have detrimental effects on individuals and the society. First, people may be misled by fake Copyright c © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. http://www.journalism.org/2016/05/26/news-use-acrosssocial-media-platforms-2016/ news and accept false beliefs (Nyhan and Reifler 2010; Paul and Matthews 2016). Second, fake news could change the way people respond to true news. Third, the widespread of fake news could break the trustworthiness of entire news ecosystem. Thus, it is important to detect fake news on social media. Fake news is intentionally written to mislead consumers, which makes it nontrivial to detect simply based on news content. Thus, it is necessary to explore auxiliary information to improve detection. For example, several style-based approaches try to capture the deceptive manipulators originated from the particular writing style of fake news (Rubin and Lukoianova 2015; Potthast et al. 2017). In addition, previous approaches try to aggregate users’ responses from relevant social engagements to infer the veracity of original news (Castillo, Mendoza, and Poblete 2011; Gupta, Zhao, and Han 2012). The news ecosystem on social media involves three basic entities, i.e., news publisher, news and social media users. Figure 1 gives an illustration of such ecosystem. In Figure 1, p1, p2 and p3 are news publishers who publish news a1, . . . , a4 and u1, . . . , u6 are users who have engaged in posting these news. In addition, users with similar interests can also form social links. The tri-relationship among publisher, news, and social engagements contains additional information to help detect fake news. First, sociallogical studies on journalism have theorized the correlation between the partisan bias of publisher and news contents veracity (Gentzkow, Shapiro, and Stone 2014; Entman 2007), where partisan means the perceived bias of the publisher in the selection of how news is reported and covered. For example, in Figure 1, for p1 with extreme left partisan bias and p2 with extreme right partisan bias, to support their own partisan, they have high degree to report fake news, such as a1 and a3; while for a mainstream publisher p3 that has least partisan bias, she has lower degree to manipulate original news events, and is more likely to write true news a4. Thus, exploiting publisher partisan information can bring additional benefits to predict fake news. Second, mining user engagements on social media towards the news also help fake news detection. Different users have different credibility levels on social media, and https://www.nytimes.com/2016/11/28/opinion/fake-newsand-the-internet-shell-game.html? Figure 1: Tri-relationship among publishers, news pieces, and social media users for news dissemination ecosystem. the credibility score which means “the quality of being trustworthy” (Abbasi and Liu 2013) has a strong indication of whether the user is more likely to engage fake news or not. Those less credible users, such as malicious accounts or normal users who are vulnerable to fake news, are more likely to spread fake news. For example, u2 and u4 are users with low credibility scores, and they tend to spread fake news more than other highly credible users. In addition, users tend to form relationships with like-minded people. For example, user u5 and u6 are friends on social media, so they tend to engage those news that confirm their own views, such as a4. Publisher partisan information can bridge the publishernews relationship, while social engagements can capture the news-user relationship. In other words, they provide complementary information that has potential to improve fake news prediction. Thus, it’s important to integrate these two components and model the tri-relationship simultaneously. In this paper, we study the novel problem of exploiting trirelationship for fake news detection. In essence, we need to address the following challenges (1) how to mathematically model the tri-relationship to extract news feature representations; and (2) how to take the advantage of tri-relationship learning for fake news detection. In an attempt to address these challenges, we propose a novel framework TriFN that captures the Tri-relationship for Fake News detection. The main contributions are: • We provide a principled way to model tri-relationship among publisher, news, and relevant user engagements simultaneously; • We propose a novel framework TriFN that exploits trirelationship for fake news prediction; and • We evaluate the effectiveness of the proposed framework for fake news detection through extensive experiments on newly collected real-world datasets. Problem Statement Even though fake news has been existed for long time, there is no agreed definition. In this paper, we follow the definition of fake news that is widely used in recent research (Shu et al. 2017; Zubiaga et al. 2017; Allcott and Gentzkow 2017), which has been shown to be able to 1) provide theoretical and practical values for fake news topic; and 2) eliminate the ambiguities between fake news and related concepts. DEFINITION 1 (FAKE NEWS) Fake news is a news article that is intentionally and verifiably false. Let A = {a1, a2, ..., an} be the set of n news articles and U = {u1, u2, ..., um} be the set of m users on social media engaging the news spreading process. We denote X ∈ R as the news feature matrix. Users can become friends with other users and we use A ∈ {0, 1} to denote the user-user adjacency matrix. On social media sites, users can easily share, comment and discuss about the news pieces. This kind of social engagements provide auxiliary information for fake news detection. We denote the social news engagement matrix as W ∈ {0, 1}, where Wij = 1 indicate that user ui has engaged in the spreading process of the news piece aj ; otherwise Wij = 0. It’s worth mentioning that we focus on those engagements that show that users agree with the news. For example, For example, we only utilize those users that directly post the news, or repost the news without adding comments. More details will introduced in Section . We also denote P = {p1, p2, ..., pl} as the set of l news publishers. In addition, we denote B ∈ R as the publisher-news relation matrix, and Bkj = 1 means news publisher pk publishes the news article aj ; otherwise Bkj = 0. We assume that the partisan labels of some publishers are given and available. We define o ∈ {−1, 0, 1} as the partisan label vectors, where -1, 0, 1 represents left-, neutral-, and right-partisan bias. Similar to previous research (Shu et al. 2017), we treat fake news detection problem as a binary classification problem. In other words, each news piece can be true or fake, and we use y = {y1;y2; ...;yn} ∈ R n×1 to represent the labels, and yj = 1 means news piece aj is fake news; yj = −1 means true news. With the notations given above, the problem is formally defined as, Given news article feature matrix X, user adjacency matrix A, user social engagement matrix W, publishernews publishing matrix B, publisher partisan label vector o, and partial labeled news vector yL, we aim to predict remaining unlabeled news label vector yU . A Tri-Relationship Embedding Framework In this section, we propose a semi-supervised detection framework by exploiting tri-relationship. The idea of modeling tri-relationship is demonstrated in Figure 2. Specifically, we first introduce the news latent feature embedding from news content, and then show how to model user social engagements and publisher partisan separately; At last, we integrate the components to model tri-relationship and provide a semi-supervised detection framework. A Basic Model for News Content Embedding The inherent manipulators of fake news can be reflected in the news content. Thus, it’s important to extract

Trust is a critical factor in cloud computing; in present practice it depends largely on perception of reputation, and self assessment by providers of cloud services. We begin this paper with a survey of existing mechanisms for establishing trust, and comment on their limitations. We then address those limitations by proposing more rigorous mechanisms based on evidence, attribute certification, and validation, and conclude by suggesting a framework for integrating various trust mechanisms together to reveal chains of trust in the cloud.

Recommender systems attempt to predict items in which a user might be interested, given some information about the user's and items' profiles. Most existing recommender systems use content-based or collaborative filtering methods or hybrid methods that combine both techniques (see the sidebar for more details). We created Informed Recommender to address the problem of using consumer opinion about products, expressed online in free-form text, to generate product recommendations. Informed recommender uses prioritized consumer product reviews to make recommendations. Using text-mining techniques, it maps each piece of each review comment automatically into an ontology.

Due to the increasing amount of opinions and reviews on the Internet, opinion mining has become a hot topic in data mining, in which extracting opinion features is a key step. Most of existing methods utilize a rule-based mechanism or statistics to extract opinion features, but they ignore the structure characteristics of reviews. The performance has hence not been promising. A new approach of OFESP (Opinion Feature Extraction based on Sentiment Patterns) is proposed in this paper, which takes into account the structure characteristics of reviews for higher values of precision and recall. With a self-constructed database of sentiment patterns, OFESP matches each review sentence to obtain its features, and then filters redundant features regarding relevance of the domain, statistics and semantic similarity. Experimental studies on real-world data demonstrate that as compared with the traditional method based on a window mechanism, OFESP outperforms it on precision, recall and F-score. Meanwhile, in comparison to the approach based on syntactic analysis, OFESP performs better on recall and F-score.

Mining opinion targets from online reviews is an important and challenging task in opinion mining. This paper proposes a novel approach to extract opinion targets by using partially-supervised word alignment model (PSWAM). At first, we apply PSWAM in a monolingual scenario to mine opinion relations in sentences and estimate the associations between words. Then, a graph-based algorithm is exploited to estimate the confidence of each candidate, and the candidates with higher confidence will be extracted as the opinion targets. Compared with existing syntax-based methods, PSWAM can effectively avoid parsing errors when dealing with informal sentences in online reviews. Compared with the methods using alignment model, PSWAM can capture opinion relations more precisely through partial supervision from partial alignment links. Moreover, when estimating candidate confidence, we make penalties on higherdegree vertices in our graph-based algorithm in order to decrease the probability of the random walk running into the unrelated regions in the graph. As a result, some errors can be avoided. The experimental results on three data sets with different sizes and languages show that our approach outperforms state-of-the-art methods.


Question: A trust-based method for identifying deceptive review spam.

This paper explores innovative methods to enhance the reliability of recommendations in an era dominated by information saturation, where finding pertinent content has become increasingly challenging. Recommender systems, which have gained widespread adoption in both academic research and e-commerce platforms, aim to mitigate this issue by filtering out irrelevant data. Here, we introduce novel strategies to bolster the precision of these systems through the application of rating variance as a measure of recommendation confidence. Through extensive experimentation, we illustrate the integration of these techniques with diverse recommendation methodologies, showcasing their effectiveness. Moreover, we highlight the ability of these approaches to produce more tailored recommendations, as quantified by the coverage metric. This enhancement empowers users with greater autonomy, allowing them to prioritize recommendations based on either coverage or accuracy, thus significantly improving the user experience.

The open nature of the World Wide Web makes evaluating webpage credibility challenging for users. In this paper, we aim to automatically assess web credibility by investigating various characteristics of webpages. Specifically, we first identify features from textual content, link structure, webpages design, as well as their social popularity learned from popular social media sites (e.g., Facebook, Twitter). A set of statistical analyses methods are applied to select the most informative features, which are then used to infer webpages credibility by employing supervised learning algorithms. Real dataset-based experiments under two application settings show that we attain an accuracy of 75% for classification, and an improvement of 53% for the mean absolute error (MAE), with respect to the random baseline approach, for regression.

The Internet has changed the way people look for information. Users now expect the answers to their questions to be available through a simple web search. Web search engines are increasingly efficient at identifying the best sources for any given keyword query, and are often able to identify the answer within the sources. Unfortunately, many web sources are not trustworthy, because of erroneous, misleading, biased, or outdated information. In many cases, users are not satisfied with —or do not trust— the results from any single source and prefer checking several sources for corroborating evidence. In this paper, we propose methods to aggregate query results from different sources in order to save users the hassle of individually checking query-related web sites to corroborate answers. To return the best aggregated answers to the users, our techniques consider the number, importance, and similarity of the web sources reporting each answer, as well as the importance of the answer within the source. We present an experimental evaluation of our technique on real web queries, comparing the corroborated answers returned with real user clicks.

In this paper we present a novel framework for extracting the ratable aspects of objects from online user reviews. Extracting such aspects is an important challenge in automatically mining product opinions from the web and in generating opinion-based summaries of user reviews [18, 19, 7, 12, 27, 36, 21]. Our models are based on extensions to standard topic modeling methods such as LDA and PLSA to induce multi-grain topics. We argue that multi-grain models are more appropriate for our task since standard models tend to produce topics that correspond to global properties of objects (e.g., the brand of a product type) rather than the aspects of an object that tend to be rated by a user. The models we present not only extract ratable aspects, but also cluster them into coherent topics, e.g., 'waitress' and 'bartender' are part of the same topic 'staff' for restaurants. This differentiates it from much of the previous work which extracts aspects through term frequency analysis with minimal clustering. We evaluate the multi-grain models both qualitatively and quantitatively to show that they improve significantly upon standard topic models.

In the past years we have witnessed Sentiment Analysis and Opinion Mining becoming increasingly popular topics in Information Retrieval and Web data analysis. With the rapid growth of the user-generated content represented in blogs, wikis and Web forums, such an analysis became a useful tool for mining the Web, since it allowed us to capture sentiments and opinions at a large scale. Opinion retrieval has established itself as an important part of search engines. Ratings, opinion trends and representative opinions enrich the search experience of users when combined with traditional document retrieval, by revealing more insights about a subject. Opinion aggregation over product reviews can be very useful for product marketing and positioning, exposing the customers’ attitude towards a product and its features along different dimensions, such as time, geographical location, and experience. Tracking how opinions or discussions evolve over time can help us identify interesting trends and patterns and better understand the ways that information is propagated in the Internet. In this study, we review the development of Sentiment Analysis and Opinion Mining during the last years, and also discuss the evolution of a relatively new research direction, namely, Contradiction Analysis. We give an overview of the proposed methods and recent advances in these areas, and we try to layout the future research directions in the field.

The increasing use of web 2.0 applications has generated numerous online user reviews. Prior studies have revealed the influence of user-generated reviews on the sales of products such as CDs, books, and movies. However, the influence of online user-generated reviews in the tourism industry is still largely unknown both to tourism researchers and practitioners. To bridge this knowledge gap in tourism management, we conducted an empirical study to identify the impact of online user-generated reviews on business performance using data extracted from a major online travel agency in China. The empirical findings show that traveler reviews have a significant impact on online sales, with a 10 percent increase in traveler review ratings boosting online bookings by more than five percent. Our results highlight the importance of online user-generated reviews to business performance in tourism. 2010 Elsevier Ltd. All rights reserved.

Recommender systems attempt to predict items in which a user might be interested, given some information about the user's and items' profiles. Most existing recommender systems use content-based or collaborative filtering methods or hybrid methods that combine both techniques (see the sidebar for more details). We created Informed Recommender to address the problem of using consumer opinion about products, expressed online in free-form text, to generate product recommendations. Informed recommender uses prioritized consumer product reviews to make recommendations. Using text-mining techniques, it maps each piece of each review comment automatically into an ontology.

Web 2.0 technologies have enabled more and more people to freely comment on different kinds of entities (e.g. sellers, products, services). The large scale of information poses the need and challenge of automatic summarization. In many cases, each of the user-generated short comments comes with an overall rating. In this paper, we study the problem of generating a ``rated aspect summary'' of short comments, which is a decomposed view of the overall ratings for the major aspects so that a user could gain different perspectives towards the target entity. We formally define the problem and decompose the solution into three steps. We demonstrate the effectiveness of our methods by using eBay sellers' feedback comments. We also quantitatively evaluate each step of our methods and study how well human agree on such a summarization task. The proposed methods are quite general and can be used to generate rated aspect summary automatically given any collection of short comments each associated with an overall rating.

Semantic Web endeavors have mainly focused on issues pertaining to knowledge representation and ontology design. However, besides understanding information metadata stated by subjects, knowing about their credibility becomes equally crucial. Hence, trust and trust metrics, conceived as computational means to evaluate trust relationships between individuals, come into play. Our major contribution to Semantic Web trust management through this work is twofold. First, we introduce a classification scheme for trust metrics along various axes and discuss advantages and drawbacks of existing approaches for Semantic Web scenarios. Hereby, we devise an advocacy for local group trust metrics, guiding us to the second part which presents Appleseed, our novel proposal for local group trust computation. Compelling in its simplicity, Appleseed borrows many ideas from spreading activation models in psychology and relates their concepts to trust evaluation in an intuitive fashion. Moreover, we provide extensions for the Appleseed nucleus that make our trust metric handle distrust statements.

Social media are nowadays one of the main news sources for millions of people around the globe due to their low cost, easy access, and rapid dissemination. This however comes at the cost of dubious trustworthiness and significant risk of exposure to ‘fake news’, intentionally written to mislead the readers. Automatically detecting fake news poses challenges that defy existing content-based analysis approaches. One of the main reasons is that often the interpretation of the news requires the knowledge of political or social context or ‘common sense’, which current natural language processing algorithms are still missing. Recent studies have empirically shown that fake and real news spread differently on social media, forming propagation patterns that could be harnessed for the automatic fake news detection. Propagation-based approaches have multiple advantages compared to their content-based counterparts, among which is language independence and better resilience to adversarial attacks. In this paper, we show a novel automatic fake news detection model based on geometric deep learning. The underlying core algorithms are a generalization of classical convolutional neural networks to graphs, allowing the fusion of heterogeneous data such as content, user profile and activity, social graph, and news propagation. Our model was trained and tested on news stories, verified by professional fact-checking organizations, that were spread on Twitter. Our experiments indicate that social network structure and propagation are important features allowing highly accurate (92.7% ROC AUC) fake news detection. Second, we observe that fake news can be reliably detected at an early stage, after just a few hours of propagation. Third, we test the aging of our model on training and testing data separated in time. Our results point to the promise of propagation-based approaches for fake news detection as an alternative or complementary strategy to content-based approaches. ar X iv :1 90 2. 06 67 3v 1 [ cs .S I] 1 0 Fe b 20 19

Online deception is disrupting our daily life, organizational process, and even national security. Existing deception detection approaches followed a traditional paradigm by using a set of cues as antecedents, and used a variety of data sets and common classification models to detect deception, which were demonstrated to be an accurate technique, but the previous results also showed the necessity to expand the deception feature set in order to improve the accuracy. In our study, we propose a novel feature selection method of the combination of CHI statistics and hypothesis testing, and achieve the accuracy level of 86% and F-measure of 0.84 by using the novel feature sets and SVM classification models, which exceeds the previous experiment results.