Question: Moving from Brexit to Trump: Examining social media's impact on democracy.

Characterizing the Dual Nature of Twitter: A Social and Information Network

This study delves into the intricate topology of the Twitter follow graph, offering a comprehensive analysis of its key features, including degree distributions, connected components, shortest path lengths, clustering coefficients, and degree assortativity. By juxtaposing these attributes against data from other social networks, the research uncovers a nuanced understanding of Twitter's network dynamics. Central to this inquiry is the recurring debate: Is Twitter predominantly a social network or an information network? Given that the "follow" action on Twitter primarily serves information consumption, yet is often rooted in social relationships, the platform emerges as a hybrid of both network types. Intriguingly, the study suggests that Twitter's evolution is marked by a transformation from an information-centric network to one that increasingly mirrors the characteristics of a social network. This evolution is evidenced through preliminary data, which could potentially inform a formal model delineating the growth of hybrid networks akin to Twitter.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems

Live-streaming mobile video is an emerging medium. Few have measured how this new form of production is contributing to civic engagement or broadening the public sphere by circulating visual footage of community interest. Here I explore the overall trends in production of live-streaming mobile video from producers around the world, and focus more narrowly on the motivations and practices surrounding the production of civic content. Informing my study are mobile videos on Qik.com, a popular web service for live-streaming mobile video. I offer a quantitative content analysis of 1,000 videos, summarizing general trends in content production as well as analyzing the motivations behind the production of civic content, based on qualitative interviews with frequent producers. Results indicate that production is higher among those who self-identify as activists, journalists, community leaders or educators, suggesting this new medium can be best appropriated by those who are already civically engaged.

As the use and influence of social networking continues to grow, researchers have begun to explore its consequences for psychological well-being. Some research suggests that Facebook use can have negative consequences for well-being. Instagram, a photo-sharing social network created in 2010, has particular characteristics that may make users susceptible to negative consequences. This study tested a theoretically grounded moderated meditation model of the association between Instagram use and depressive symptoms through the mechanism of negative social comparison, and moderation by amount of strangers one follows. One hundred and seventeen 18-29 year olds completed online questionnaires containing demographics, frequency of Instagram use, amount of strangers followed on Instagram, the Center for Epidemiological Resources Scale for Depression, and the Social Comparison Rating Scale. Instagram use was marginally positively associated with depressive symptoms, and positive social comparison was significantly negatively associated with depressive symptoms. Amount of strangers followed moderated the associations of Instagram use with social comparison (significantly) and depressive symptoms (marginally), and further significantly moderated the indirect association of Instagram use with depressive symptoms through social comparison. Findings generally suggest that more frequent Instagram use has negative associations for people who follow more strangers, but positive associations for people who follow fewer strangers, with social comparison and depressive symptoms. Implications of negative associations of social networking for people who follow strangers and the need for more research on Instagram use given its increasing popularity are explored.

The possibility of anonymity and lack of effective ways to identify inappropriate messages have resulted in a significant amount of online interaction data that attempt to harass, bully, or offend the recipient. In this work, we perform a preliminary linguistic study on messages exchanged using one such popular web/smartphone application—Sarahah, that allows friends to exchange messages anonymously. Since messages exchanged via Sarahah are private, we collect them when the recipient shares it on Twitter. We then perform an analysis of the different kinds of messages exchanged through this application. Our linguistic analysis reveals that a significant number of these messages (∼ 20%) include inappropriate, hurtful, or profane language intended to embarrass, offend, or bully the recipient. Our analysis helps in understanding the different ways in which anonymous message exchange platforms are used and the different types of bullying present in such exchanges.

Purpose – Digital libraries and social media are two sources of online information with different characteristics. This study integrates self-efficacy into the analysis of the relationship between information sources and decision-making, with the aim of exploring the effect of self-efficacy on decision-making, as well as the interacting effect of self-efficacy and information sources on decision-making. Design/methodology/approach –Survey data were collected and the partial least squares (PLS) structural equation modelling (SEM) was employed to verify the research model. Findings – The effect of digital library usage for acquiring information on perceived decision quality is larger than that of social media usage for acquiring information on perceived decision quality. Self-efficacy in acquiring information stands out as the key determinant for perceived decision quality. The effect of social media usage for acquiring information on perceived decision quality is positively moderated by self-efficacy in acquiring information. Practical implications – Decision-making is a fundamental activity for individuals, but human decision-making is often subject to biases. The findings of this study provide useful insights into decision quality improvement, highlighting the importance of self-efficacy in acquiring information in the face of information overload. Originality/value – This study integrates self-efficacy into the analysis of the relationship between information sources and decision-making, presenting a new perspective for decision-making research and practice alike.

This article explores the relationship between Internet use and the individual-level production of social capital. To do so, the authors adopt a motivational perspective to distinguish among types of Internet use when examining the factors predicting civic engagement, interpersonal trust, and life contentment. The predictive power of new media use is then analyzed relative to key demographic, contextual, and traditional media use variables using the 1999 DDB Life Style Study. Although the size of associations is generally small, the data suggest that informational uses of the Internet are positively related to individual differences in the production of social capital, whereas social-recreational uses are negatively related to these civic indicators. Analyses within subsamples defined by generational age breaks further suggest that social capital production is related to Internet use among Generation X, while it is tied to television use among Baby Boomers and newspaper use among members of the Civic Generation. The possibility of life cycle and cohort effects is discussed.

Geographically-grounded situational awareness (SA) is critical to crisis management and is essential in many other decision making domains that range from infectious disease monitoring, through regional planning, to political campaigning. Social media are becoming an important information input to support situational assessment (to produce awareness) in all domains. Here, we present a geovisual analytics approach to supporting SA for crisis events using one source of social media, Twitter. Specifically, we focus on leveraging explicit and implicit geographic information for tweets, on developing place-time-theme indexing schemes that support overview+detail methods and that scale analytical capabilities to relatively large tweet volumes, and on providing visual interface methods to enable understanding of place, time, and theme components of evolving situations. Our approach is user-centered, using scenario-based design methods that include formal scenarios to guide design and validate implementation as well as a systematic claims analysis to justify design choices and provide a framework for future testing. The work is informed by a structured survey of practitioners and the end product of Phase-I development is demonstrated / validated through implementation in SensePlace2, a map-based, web application initially focused on tweets but extensible to other media.

Weblogs are now a key part of online culture, and social scientists are interested in characterising the networks formed by bloggers and measuring their extent and impact in areas such as politics. However, researchers wishing to conduct quantitative social science analysis of the blogging phenomenon are faced with the challenge of using new methods of data collection and analysis largely derived from fields outside of the social sciences, such as the information sciences. This paper presents an overview of one new approach for collecting and analysing weblog data, and illustrates this approach in the context of a preliminary quantitative analysis of online networks formed by a sample of North-American “A-list” political bloggers. There are two aims to this paper. First is to assess (using different data and methods) the conclusion of Adamic and Glance (2005) that there are significant differences in the behaviour of liberal and conservative bloggers, with the latter forming more dense patterns of linkages. We find broad support for this conclusion, and empirically assess the implications of differences in conservative/liberal linking behaviour for the online visibility of different political messages or ideologies. The second aim is to highlight the role of web mining and data visualisation in the analysis of weblogs, and the opportunities and challenges inherent in this new field of research.

Fitness applications with social network services (SNS) have emerged for physical activity management. However, there is little understanding of the effects of these applications on users’ physical activity. Motivated thus, we develop a theoretical model based on social cognitive theory of self-regulation to explain the effects of goal setting, self-tracking, and SNS (through social comparison from browsing others’ tracking data and social support from sharing one’s own tracking data) on physical activity. The model was tested with objective data from 476 Runkeeper users. Our preliminary results show that goal setting, number of uses, the proportion of lower-performing friends, and number of likes positively influence users’ physical activities, while the proportion of higher-performing friends has a negative effect. Moreover, the effect of the proportion of higher-performing friends is moderated by the average difference between the friends and the individual. The initial contributions of the study and remaining research plan are described.

URLs: Self-disclosure on Facebook among female users and its relationship to feelings of loneliness Author(s): Al-Saggaf, Y.M. ; Nielsen, S.G. Computers in Human Behavior

Understanding human mobility is crucial for a broad range of applications from disease prediction to communication networks. Most efforts on studying human mobility have so far used private and low resolution data, such as call data records. Here, we propose Twitter as a proxy for human mobility, as it relies on publicly available data and provides high resolution positioning when users opt to geotag their tweets with their current location. We analyse a Twitter dataset with more than six million geotagged tweets posted in Australia, and we demonstrate that Twitter can be a reliable source for studying human mobility patterns. Our analysis shows that geotagged tweets can capture rich features of human mobility, such as the diversity of movement orbits among individuals and of movements within and between cities. We also find that short- and long-distance movers both spend most of their time in large metropolitan areas, in contrast with intermediate-distance movers' movements, reflecting the impact of different modes of travel. Our study provides solid evidence that Twitter can indeed be a useful proxy for tracking and predicting human movement.


Question: Moving from Brexit to Trump: Examining social media's impact on democracy.

This study explores the impact of Twitter on political engagement and opinion leadership. It investigates whether the use of Twitter as a platform for political discourse and information sharing can inspire individuals to become more politically active and assertive in their views. The research suggests that Twitter, as a form of social media, has the potential to transform passive observers into active participants in the political landscape. By facilitating the spread of information and enabling real-time discussions, Twitter empowers users to voice their opinions, engage with like-minded individuals, and potentially influence the political agenda. The findings indicate that Twitter users, particularly those who actively participate in political conversations, are more likely to assume the role of opinion leaders within their social networks. This enhanced political engagement can lead to increased participation in offline political activities, such as attending rallies, signing petitions, or voting. The study underscores the pivotal role of social media, particularly Twitter, in shaping contemporary democratic processes and fostering a more participatory political culture.

In this paper, we present a study on 4.4 million Twitter messages related to 24 systematically chosen real-world events. For each of the 4.4 million tweets, we first extracted sentiment scores based on the eight basic emotions according to Plutchik's wheel of emotions. Subsequently, we investigated the effects of shifts in the emotional valence on the spread of information. We found that in general OSN users tend to conform to the emotional valence of the respective real-world event. However, we also found empirical evidence that prospectively negative real-world events exhibit a significant amount of shifted emotions in the corresponding tweets (i.e. positive messages). To explain this finding, we use the theory of social connection and emotional contagion. To the best of our knowledge, this is the first study that provides empirical evidence for the undoing hypothesis in online social networks (OSNs). The undoing hypothesis postulates that positive emotions serve as an antidote during negative events.

This paper presents a study of the life cycle of news articles posted online. We describe the interplay between website visitation patterns and social media reactions to news content. We show that we can use this hybrid observation method to characterize distinct classes of articles. We also find that social media reactions can help predict future visitation patterns early and accurately. We validate our methods using qualitative analysis as well as quantitative analysis on data from a large international news network, for a set of articles generating more than 3,000,000 visits and 200,000 social media reactions. We show that it is possible to model accurately the overall traffic articles will ultimately receive by observing the first ten to twenty minutes of social media reactions. Achieving the same prediction accuracy with visits alone would require to wait for three hours of data. We also describe significant improvements on the accuracy of the early prediction of shelf-life for news stories.

Persuasion is a form of social influence that implies intentional but voluntary change of a person’s behaviours, feelings or thoughts about an issue, object or action. Successful persuasion by means of persuasive technology relies on a number of factors and motivators. Recently, the so called social acceptance or rejection motivator has formed the backbone of a new type of persuasion, called Mass Interpersonal Persuasion, or MIP for short. This type of persuasion uses the social influence that is generated on online social networks to persuade people. Though it has been established that normative social influence can be used effectively in persuasive technology, it is unknown if the application of more social pressure also makes it more effective. In order to test the hypothesis that the effectiveness of persuasion increases when the persuasion becomes more intense, a quantitative experiment was conducted on the online social network of Facebook. Although evidence to support the hypothesis was found, it cannot be concluded from this experiment that when utilizing normative social influence in persuasive technology more intense persuasion is also more effective.

This study examines the relationship between use of Facebook, a popular online social network site, and the formation and maintenance of social capital. In addition to assessing bonding and bridging social capital, we explore a dimension of social capital that assesses one's ability to stay connected with members of a previously inhabited community, which we call maintained social capital. Regression analyses conducted on results from a survey of undergraduate students (N=286) suggest a strong association between use of Facebook and the three types of social capital, with the strongest relationship being to bridging social capital. In addition, Facebook usage was found to interact with measures of psychological well-being, suggesting that it might provide greater benefits for users experiencing low self-esteem and low life satisfaction.

PURPOSE
Pro-eating disorder (ED) online movements support engagement with ED lifestyles and are associated with negative health consequences for adolescents with EDs. Twitter is a popular social media site among adolescents that provides a unique setting for Pro-ED content to be publicly exchanged. The purpose of this study was to investigate Pro-ED Twitter profiles' references to EDs and how their social connections (followers) reference EDs.

METHODS
A purposeful sample of 45 Pro-ED profiles was selected from Twitter. Profile information, all tweets, and a random sample of 100 of their followers' profile information were collected for content analysis using the Twitter Application Programming Interface. A codebook based on ED screening guidelines was applied to evaluate ED references. For each Pro-ED profile, proportion of tweets with ED references and proportion of followers with ED references in their own profile were evaluated.

RESULTS
In total, our 45 Pro-ED profiles generated 4,245 tweets for analysis. A median of 36.4% of profiles' tweets contained ED references. Pro-ED profiles had a median of 173 followers, and a median of 44.5% of followers had ED references. Pro-ED profiles with more tweets with ED references also tended to have more followers with ED references (β = .37, p < .01).

CONCLUSIONS
Findings suggest that profiles which self-identify as Pro-ED express disordered eating patterns through tweets and have an audience of followers, many of whom also reference ED in their own profiles. ED socialization on Twitter might provide social support, but in the Pro-ED context this activity might also reinforce an ED identity.

Motivated by the growing interconnection between online platforms, we examine the dynamic interplay between social buzz and contribution behavior in the crowdfunding context. Since the utility of crowdfunding projects is usually difficult to ascertain, prospective backers draw on quality signals, such as social buzz and prior-contribution behavior, to make their funding decisions. We employ the panel vector autoregression (PVAR) methodology to investigate both intraand cross-platform effects based on data collected from three platforms: Indiegogo, one of the largest crowdfunding platforms on the web, Twitter and Facebook. Our results show a positive influence of social buzz on project backing, but a negative relationship in the reverse direction. Furthermore, we observe strong positive feedback cycles within each platform. Our results are supplemented by split-sample analyses for project orientation (Social, Cause and Entrepreneurial) and project success (Winners vs. Losers), in which Facebook shares were identified as a critical success factor.

Paste the appropriate copyright statement here. ACM now supports three different copyright statements: • ACM copyright: ACM holds the copyright on the work. This is the historical approach. • License: The author(s) retain copyright, but ACM receives an exclusive publication license. • Open Access: The author(s) wish to pay for the work to be open access. The additional fee must be paid to ACM. This text field is large enough to hold the appropriate release statement assuming it is single spaced in a sans-serif 7 point font. Every submission will be assigned their own unique DOI string to be included here. Abstract Twitch.tv, a streaming platform known for video game content, has grown tremendously since its inception in 2011. We examine communication practices in Twitch chats for the popular game Hearthstone, comparing massive chats with at least 10,000 concurrent viewers and small chats with fewer than 2,000 concurrent viewers. Due to the large scale and fast pace of massive chats, communication patterns no longer follow models developed in previous studies of computer-mediated communication. Rather than what other studies have described as communication breakdowns and information overload, participants in massive chats communicate in what we call “crowdspeak.”

Platforms like Reddit have attracted large and vibrant communities, but the individuals in those communities are free to migrate to other platforms at any time. History has borne this out with the mass migration from Slashdot to Digg. The underlying motivations of individuals who migrate between platforms, and the conditions that favor migration online are not well-understood. We examine Reddit during a period of community unrest affecting millions of users in the summer of 2015, and analyze large-scale changes in user behavior and migration patterns to Reddit-like alternative platforms. Using self-reported statements from user comments, surveys, and a computational analysis of the activity of users with accounts on multiple platforms, we identify the primary motivations driving user migration. While a notable number of Reddit users left for other platforms, we found that an important pull factor that enabled Reddit to retain users was its long tail of niche content. Other platforms may reach critical mass to support popular or “mainstream” topics, but Reddit’s large userbase provides a key advantage in supporting niche top-

Social media play a decisive role in communicating and spreading information during global events. In particular, real-time microblogging platforms such as Twitter have become prevalent. Researchers have used microblogging for a number of tasks, including past events analysis, predictions, and information retrieval. Nevertheless, little attention has been given to quantitative data extraction. In this paper, we address two questions: can we develop a mechanism to extract quantitative data from a collection of tweets, and can we use the salient findings to describe an event? To answer the first question, we introduce Raimond, a virtual text curator, specialized in quantitative data extraction from Twitter. To address the second question, we use our system on three events and evaluate its output using a crowdsourcing strategy. We demonstrate the e↵ectiveness of our approach with a number of real world examples.

There is a widespread intuitive sense that different kinds of information spread differently on-line, but it has been difficult to evaluate this question quantitatively since it requires a setting where many different kinds of information spread in a shared environment. Here we study this issue on Twitter, analyzing the ways in which tokens known as hashtags spread on a network defined by the interactions among Twitter users. We find significant variation in the ways that widely-used hashtags on different topics spread.
 Our results show that this variation is not attributable simply to differences in "stickiness," the probability of adoption based on one or more exposures, but also to a quantity that could be viewed as a kind of "persistence" - the relative extent to which repeated exposures to a hashtag continue to have significant marginal effects. We find that hashtags on politically controversial topics are particularly persistent, with repeated exposures continuing to have unusually large marginal effects on adoption; this provides, to our knowledge, the first large-scale validation of the "complex contagion" principle from sociology, which posits that repeated exposures to an idea are particularly crucial when the idea is in some way controversial or contentious. Among other findings, we discover that hashtags representing the natural analogues of Twitter idioms and neologisms are particularly non-persistent, with the effect of multiple exposures decaying rapidly relative to the first exposure.
 We also study the subgraph structure of the initial adopters for different widely-adopted hashtags, again finding structural differences across topics. We develop simulation-based and generative models to analyze how the adoption dynamics interact with the network structure of the early adopters on which a hashtag spreads.


Question: An adaptive modification of the boost by majority algorithm.

This paper delves into a notable phenomenon observed in boosting experiments: the test error of the classifier tends not to rise as its size grows significantly large, and, intriguingly, it often continues to decline even after the training error has hit zero. We explore this phenomenon by examining the distribution of margins associated with the training examples, where the margin is defined as the difference between the number of correct votes and the highest number of votes for any incorrect label, in relation to the voting classification rule. By leveraging techniques utilized in the analysis of Vapnik's support vector classifiers and neural networks with small weights, we establish a connection between the margin distribution and the test error. Moreover, we demonstrate, both theoretically and through experimentation, that boosting is exceptionally proficient at augmenting the margins of the training examples. We also contrast our findings with explanations rooted in the bias-variance decomposition. This research contributes to a deeper understanding of the mechanisms underlying the effectiveness of boosting methods in machine learning.

Machine learning algorithms are increasingly being applied in security-related tasks such as spam and malware detection, although their security properties against deliberate attacks have not yet been widely understood. Intelligent and adaptive attackers may indeed exploit specific vulnerabilities exposed by machine learning techniques to violate system security. Being robust to adversarial data manipulation is thus an important, additional requirement for machine learning algorithms to successfully operate in adversarial settings. In this work, we evaluate the security of Support Vector Machines (SVMs) to well-crafted, adversarial label noise attacks. In particular, we consider an attacker that aims to maximize the SVM’s classification error by flipping a number of labels in the training data. We formalize a corresponding optimal attack strategy, and solve it by means of heuristic approaches to keep the computational complexity tractable. We report an extensive experimental analysis on the effectiveness of the considered attacks against linear and non-linear SVMs, both on synthetic and real-world datasets. We finally argue that our approach can also provide useful insights for developing more secure SVM learning algorithms, and also novel techniques in a number of related research areas, such as semi-supervised and active learning.

Evolutionary Learning proceeds by evolving a population of classifiers, from which it generally returns (with some notable exceptions) the single best-of-run classifier as final result. In the meanwhile, Ensemble Learning, one of the most efficient approaches in supervised Machine Learning for the last decade, proceeds by building a population of diverse classifiers. Ensemble Learning with Evolutionary Computation thus receives increasing attention. The Evolutionary Ensemble Learning (EEL) approach presented in this paper features two contributions. First, a new fitness function, inspired by co-evolution and enforcing the classifier diversity, is presented. Further, a new selection criterion based on the classification margin is proposed. This criterion is used to extract the classifier ensemble from the final population only (Off-EEL) or incrementally along evolution (On-EEL). Experiments on a set of benchmark problems show that Off-EEL outperforms single-hypothesis evolutionary learning and state-of-art Boosting and generates smaller classifier ensembles.

In this paper, we develop a novel Distance-weighted k -nearest Neighbor rule (DWKNN), using the dual distance-weighted function. The proposed DWKNN is motivated by the sensitivity problem of the selection of the neighborhood size k that exists in k -nearest Neighbor rule (KNN), with the aim of improving classification performance. The experiment results on twelve real data sets demonstrate that our proposed classifier is robust to different choices of k to some degree, and yields good performance with a larger optimal k, compared to the other state-of-art KNN-based methods.

Categorization of documents is challenging, as the number o f discriminating words can be very large. We present a nearest neighbor classification scheme for text categoriz ation in which the importance of discriminating words is learned using mutual information and weight adjustment tec hniques. The nearest neighbors for a particular document are then computed based on the matching words and their weigh ts. We evaluate our scheme on both synthetic and real world documents. Our experiments with synthetic data sets s how that this scheme is robust under different emulated conditions. Empirical results on real world documents demo nstrate that this scheme outperforms state of the art classification algorithms such as C4.5, RIPPER, Rainbow, an d PEBLS.

This article presents an algorithm for inducing multiclass decision trees with multivariate tests at internal decision nodes. Each test is constructed by training a linear machine and eliminating variables in a controlled manner. Empirical results demonstrate that the algorithm builds small accurate trees across a variety of tasks.

We consider algorithms for combining advice from a set of experts. In each trial, the algorithm receives the predictions of the experts and produces its own prediction. A loss function is applied to measure the discrepancy between the predictions and actual observations. The algorithm keeps a weight for each expert. At each trial the weights are first used to help produce the prediction and then updated according to the observed outcome. Our starting point is Vovk’s Aggregating Algorithm, in which the weights have a simple form: the weight of an expert decreases exponentially as a function of the loss incurred by the expert. The prediction of the Aggregating Algorithm is typically a nonlinear function of the weights and the experts’ predictions. We analyze here a simplified algorithm in which the weights are as in the original Aggregating Algorithm, but the prediction is simply the weighted average of the experts’ predictions. We show that for a large class of loss functions, even with the simplified prediction rule the additional loss of the algorithm over the loss of the best expert is at most c ln n, where n is the number of experts and c a constant that depends on the loss function. Thus, the bound is of the same form as the known bounds for the Aggregating Algorithm, although the constants here are not quite as good. We use relative entropy to rewrite the bounds in a stronger form and to motivate the update.

Training efficiency is one of the main problems for Neural Machine Translation (NMT). Deep networks need for very large data as well as many training iterations to achieve state-of-the-art performance. This results in very high computation cost, slowing down research and industrialisation. In this paper, we propose to alleviate this problem with several training methods based on data boosting and bootstrap with no modifications to the neural network. It imitates the learning process of humans, which typically spend more time when learning “difficult” concepts than easier ones. We experiment on an English-French translation task showing accuracy improvements of up to 1.63 BLEU while saving 20% of training time.

This paper presents the top 10 data mining algorithms identified by the IEEE International Conference on Data Mining (ICDM) in December 2006: C4.5, k-Means, SVM, Apriori, EM, PageRank, AdaBoost, kNN, Naive Bayes, and CART. These top 10 algorithms are among the most influential data mining algorithms in the research community. With each algorithm, we provide a description of the algorithm, discuss the impact of the algorithm, and review current and further research on the algorithm. These 10 algorithms cover classification, clustering, statistical learning, association analysis, and link mining, which are all among the most important topics in data mining research and development.

algorithms, utilize techniques from optimization, statistics, and functional analysis to achieve maximal generality, flexibility, and performance. These algorithms are different from earlier techniques used in machine learning in many respects: For example, they are explicitly based on a theoretical model of learning rather than on loose analogies with natural learning systems or other heuristics. They come with theoretical guarantees about their performance and have a modular design that makes it possible to separately implement and analyze their components. They are not affected by the problem of local minima because their training amounts to convex optimization. In the last decade, a sizable community of theoreticians and practitioners has formed around these methods, and a number of practical applications have been realized. Although the research is not concluded, already now kernel methods are considered the state of the art in several machine learning tasks. Their ease of use, theoretical appeal, and remarkable performance have made them the system of choice for many learning problems. Successful applications range from text categorization to handwriting recognition to classification of geneexpression data.

In an expert search task, the users' need is to identify people who have relevant expertise to a topic of interest. An expert search system predicts and ranks the expertise of a set of candidate persons with respect to the users' query. In this paper, we propose a novel approach for predicting and ranking candidate expertise with respect to a query. We see the problem of ranking experts as a voting problem, which we model by adapting eleven data fusion techniques.We investigate the effectiveness of the voting approach and the associated data fusion techniques across a range of document weighting models, in the context of the TREC 2005 Enterprise track. The evaluation results show that the voting paradigm is very effective, without using any collection specific heuristics. Moreover, we show that improving the quality of the underlying document representation can significantly improve the retrieval performance of the data fusion techniques on an expert search task. In particular, we demonstrate that applying field-based weighting models improves the ranking of candidates. Finally, we demonstrate that the relative performance of the adapted data fusion techniques for the proposed approach is stable regardless of the used weighting models.


Question: An adaptive modification of the boost by majority algorithm.

The paper introduces an adaptive version of the Boost by Majority algorithm, which has emerged as a significant advancement in the field of classification. Boosting techniques improve classification accuracy by iteratively applying a classification algorithm to weighted versions of the training dataset and then aggregating the resulting classifiers through a weighted majority vote. The effectiveness of this method can be explained through established statistical principles, specifically additive modeling and maximum likelihood estimation. In the context of binary classification, boosting can be seen as an approximation to additive modeling on the logistic scale, with the objective of maximizing Bernoulli likelihood. The authors present more direct approximations to this process and demonstrate that these alternatives yield results that are nearly identical to those obtained through traditional boosting methods. Moreover, the paper extends the boosting framework to multiclass classification by utilizing multinomial likelihood, which often outperforms other recent multiclass generalizations of boosting. A minor adjustment to the boosting algorithm is suggested, which can significantly reduce computational requirements, often by factors ranging from 10 to 50. The insights gained from this study are applied to develop an alternative formulation of boosting for decision trees. This approach, based on best-first truncated tree induction, frequently leads to improved performance compared to conventional boosting methods, while also providing more interpretable descriptions of the aggregate decision rule. Additionally, this variant is computationally more efficient, making it better suited for large-scale data mining applications.

This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the “Integral Image” which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers[6]. The third contribution is a method for combining increasingly more complex classifiers in a “cascade” which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.

We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting “kernel herding” algorithm is an infinite memory deterministic process that learns to approximate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert space at a rateO(1/T )which is much faster than the usual O(1/ √ T ) for iid random samples. We illustrate kernel herding by approximating Bayesian predictive distributions.

Recent work has shown that improving model learning for weak classifiers can yield significant gains in the overall accuracy of a boosted classifier. However, most published classifier boosting research relies only on rudimentary learning techniques for weak classifiers. So while it is known that improving the model learning can greatly improve the accuracy of the resulting strong classifier, it remains to be shown how much can yet be gained by further improving the model learning at the weak classifier level. This paper derives a very accurate model learning method for weak classifiers based on the popular Haar-like features and presents an investigation of its usefulness compared to the standard and recent approaches. The accuracy of the new method is shown by demonstrating the new models ability to predict ROC performance on validation data. A discussion of the problems in learning accurate weak hypotheses is given, along with example solutions. It is also shown that a previous simpler method can be further improved. Lastly, we show that improving model accuracy does not continue to yield improved overall classification beyond a certain point. At this point the learning technique, in this case RealBoost, is unable to make gains from the improved model data. The method has been tested on pedestrian detection tasks using classifiers boosted using the RealBoost boosting algorithm. A subset of our most interesting results is shown to demonstrate the value of method.

Classifier evasion consists in finding for a given instance x the “nearest” instance x′ such that the classifier predictions of x and x′ are different. We present two novel algorithms for systematically computing evasions for tree ensembles such as boosted trees and random forests. Our first algorithm uses a Mixed Integer Linear Program solver and finds the optimal evading instance under an expressive set of constraints. Our second algorithm trades off optimality for speed by using symbolic prediction, a novel algorithm for fast finite differences on tree ensembles. On a digit recognition task, we demonstrate that both gradient boosted trees and random forests are extremely susceptible to evasions. Finally, we harden a boosted tree model without loss of predictive accuracy by augmenting the training set of each boosting round with evading instances, a technique we call adversarial boosting.

In this paper, a new learning algorithm for adaptive network intrusion detection using naive Bayesian classifier and decision tree is presented, which performs balance detections and keeps false positives at acceptable level for different types of network attacks, and eliminates redundant attributes as well as contradictory examples from training data that make the detection model complex. The proposed algorithm also addresses some difficulties of data mining such as handling continuous attribute, dealing with missing attribute values, and reducing noise in training data. Due to the large volumes of security audit data as well as the complex and dynamic properties of intrusion behaviours, several data miningbased intrusion detection techniques have been applied to network-based traffic data and host-based data in the last decades. However, there remain various issues needed to be examined towards current intrusion detection systems (IDS). We tested the performance of our proposed algorithm with existing learning algorithms by employing on the KDD99 benchmark intrusion detection dataset. The experimental results prove that the proposed algorithm achieved high detection rates (DR) and significant reduce false positives (FP) for different types of network intrusions using limited computational resources.

This paper addresses the problem of dealing with a collection of labeled training documents, especially annotating negative training documents and presents a method of text classification from positive and unlabeled data. We applied an error detection and correction technique to the results of positive and negative documents classified by the Support Vector Machines (SVM). The results using Reuters documents showed that the method was comparable to the current state-of-the-art biasedSVM method as the F-score obtained by our method was 0.627 and biased-SVM was 0.614.

The support vector machine (SVM) is a popular classification technique. However, beginners who are not familiar with SVM often get unsatisfactory results since they miss some easy but significant steps. In this guide, we propose a simple procedure which usually gives reasonable results.

This paper compares the performance of several classi er algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also rejection, training time, recognition time, and memory requirements.

The objective of machine learning is to identify a model that yields good generalization performance. This involves repeatedly selecting a hypothesis class, searching the hypothesis class by minimizing a given objective function over the model’s parameter space, and evaluating the generalization performance of the resulting model. This search can be computationally intensive as training data continuously arrives, or as one needs to tune hyperparameters in the hypothesis class and the objective function. In this paper, we present a framework for exact incremental learning and adaptation of support vector machine (SVM) classifiers. The approach is general and allows one to learn and unlearn individual or multiple examples, adapt the current SVM to changes in regularization and kernel parameters, and evaluate generalization performance through exact leave-one-out error estimation. I. I NTRODUCTION SVM techniques for classification and regression provide powerful tools for learning models that generalize well even in sparse, high dimensional settings. Their success can be attributed to Vapnik’s seminal work in statistical learning theory [15] which provided key insights into the factors affecting generalization performance. SVM learning can be viewed as a practical implementation of Vapnik’s structural risk minimizationinduction principle which involves searching over hypothesis classes of varying capacity to find the model with the best generalization performance. SVM classifiers of the formf(x) = w ·Φ(x)+b are learned from the data{(xi, yi) ∈ R I m × {−1, 1} ∀ i ∈ {1, . . . , N}} by minimizing min w,b,ξ 1 2 ‖w‖ + C N ∑

In this paper, we propose a support vector machine with automatic confidence (SVMAC) for pattern classification. The main contributions of this work to learning machines are twofold. One is that we develop an algorithm for calculating the label confidence value of each training sample. Thus, the label confidence values of all of the training samples can be considered in training support vector machines. The other one is that we propose a method for incorporating the label confidence value of each training sample into learning and derive the corresponding quadratic programming problems. To demonstrate the e ff ctiveness of the proposed SVMACs, a series of experiments are performed on three benchmarking pattern classification problems and a challenging gender classification problem. Experimental results show that the generalization performance of our SVMACs is superior to that of traditional SVMs.


Question: An adaptive modification of the Boost by Majority algorithm.

This paper presents an innovative algorithm, coined as Matching Pursuit, designed to decompose a signal into a linear combination of waveforms selected from a dictionary of functions that is intentionally made redundant. The selection of these waveforms is strategically executed to optimally mirror the intricate structures within the signal. Matching pursuits are versatile methodologies that facilitate the computation of adaptive signal representations. When employing a dictionary consisting of Gabor functions, a matching pursuit effectively transforms into an adaptive time-frequency analysis tool. We introduce a novel signal energy distribution mapped onto the time-frequency plane, distinct from Wigner and Cohen class distributions, as it excludes interference terms. A matching pursuit adeptly segregates signal components that exhibit coherence relative to the selected dictionary. An instance of pattern retrieval from signals corrupted by noise is provided to illustrate the algorithm's practical utility. Moreover, a comparison is drawn between a matching pursuit decomposition and a signal expansion over a wavepacket orthonormal basis that has been optimized, as determined by the algorithm devised by Coifman and Wickerhauser.

Having access to massive amounts of data does not necessarily imply that induction algorithms must use them all. Samples often provide the same accuracy with far less computational cost. However, the correct sample size rarely is obvious. We analyze methods for progressive samplingusing progressively larger samples as long as model accuracy improves. We explore several notions of efficient progressive sampling. We analyze efficiency relative to induction with all instances; we show that a simple, geometric sampling schedule is asymptotically optimal, and we describe how best to take into account prior expectations of accuracy convergence. We then describe the issues involved in instantiating an efficient progressive sampler, including how to detect convergence. Finally, we provide empirical results comparing a variety of progressive sampling methods. We conclude that progressive sampling can be remarkably efficient .

Recently there has been a lot of interest in “ensemble learning” — methods that generate many classifiers and aggregate their results. Two well-known methods are boosting (see, e.g., Shapire et al., 1998) and bagging Breiman (1996) of classification trees. In boosting, successive trees give extra weight to points incorrectly predicted by earlier predictors. In the end, a weighted vote is taken for prediction. In bagging, successive trees do not depend on earlier trees — each is independently constructed using a bootstrap sample of the data set. In the end, a simple majority vote is taken for prediction. Breiman (2001) proposed random forests, which add an additional layer of randomness to bagging. In addition to constructing each tree using a different bootstrap sample of the data, random forests change how the classification or regression trees are constructed. In standard trees, each node is split using the best split among all variables. In a random forest, each node is split using the best among a subset of predictors randomly chosen at that node. This somewhat counterintuitive strategy turns out to perform very well compared to many other classifiers, including discriminant analysis, support vector machines and neural networks, and is robust against overfitting (Breiman, 2001). In addition, it is very user-friendly in the sense that it has only two parameters (the number of variables in the random subset at each node and the number of trees in the forest), and is usually not very sensitive to their values. The randomForest package provides an R interface to the Fortran programs by Breiman and Cutler (available at http://www.stat.berkeley.edu/ users/breiman/). This article provides a brief introduction to the usage and features of the R functions.

The problem of tracking a varying number of non-rigid objects has two major difficulties. First, the observation models and target distributions can be highly non-linear and non-Gaussian. Second, the presence of a large, varying number of objects creates complex interactions with overlap and ambiguities. To surmount these difficulties, we introduce a vision system that is capable of learning, detecting and tracking the objects of interest. The system is demonstrated in the context of tracking hockey players using video sequences. Our approach combines the strengths of two successful algorithms: mixture particle filters and Adaboost. The mixture particle filter [17] is ideally suited to multi-target tracking as it assigns a mixture component to each player. The crucial design issues in mixture particle filters are the choice of the proposal distribution and the treatment of objects leaving and entering the scene. Here, we construct the proposal distribution using a mixture model that incorporates information from the dynamic models of each player and the detection hypotheses generated by Adaboost. The learned Adaboost proposal distribution allows us to quickly detect players entering the scene, while the filtering process enables us to keep track of the individual players. The result of interleaving Adaboost with mixture particle filters is a simple, yet powerful and fully automatic multiple object tracking system.

Nearest neighbor classification expects the class conditional probabilities to be locally constant, and suffers from bias in high dimensions We propose a locally adaptive form of nearest neighbor classification to try to finesse this curse of dimensionality. We use a local linear discriminant analysis to estimate an effective metric for computing neighborhoods. We determine the local decision boundaries from centroid information, and then shrink neighborhoods in directions orthogonal to these local decision boundaries, and elongate them parallel to the boundaries. Thereafter, any neighborhood-based classifier can be employed, using the modified neighborhoods. The posterior probabilities tend to be more homogeneous in the modified neighborhoods. We also propose a method for global dimension reduction, that combines local dimension information. In a number of examples, the methods demonstrate the potential for substantial improvements over nearest neighbour classification. Introduction We consider a discrimination problem with d classes and N training observations. The training observations consist of predictor measurements x = (zl,z2,...zp) on p predictors and the known class memberships. Our goal is to predict the class membership of an observation with predictor vector x0 Nearest neighbor classification is a simple and appealing approach to this problem. We find the set of K nearest neighbors in the training set to x0 and then classify x0 as the most frequent class among the K neighbors. Nearest neighbors is an extremely flexible classification scheme, and does not involve any pre-processing (fitting) of the training data. This can offer both space and speed advantages in very large problems: see Cover (1968), Duda & Hart (1973), McLachlan (1992) for background material on nearest neighborhood classification. Cover & Hart (1967) show that the one nearest neighbour rule has asymptotic error rate at most twice the Bayes rate. However in finite samples the curse of dimensionality can severely hurt the nearest neighbor rule. The relative radius of the nearest-neighbor sphere grows like r 1/p where p is the dimension and r the radius for p = 1, resulting in severe bias at the target point x. Figure 1 illustrates the situation for a simple example. Figure 1: The vertical strip denotes the NN region using only the X coordinate to find the nearest neighbor for the target point (solid dot). The sphere shows the NN region using both coordinates, and we see in this case it has extended into the class 1 region (and found the wrong class in this instance). Our illustration here is based on a 1-NN rule, but the same phenomenon ccurs for k-NN rules as well. Nearest neighbor techniques are based on the assumption that locally the class posterior probabilities are constant. While that is clearly true in the vertical strip using only coordinate X, using X and Y this is no longer true. The techniques outlined in the abstract are designed to overcome these problems. Figure 2 shows an example. There are two classes in two dimensions, one of which almost completely surrounds the other. The left panel shows a nearest neighborhood of size 25 at the target point (shown as origin), which is chosen to near the class boundary. The right panel shows the same size neighborhood using our discriminant adap142 KDD--95 From: KDD-95 Proceedings. Copyright © 1995, AAAI (www.aaai.org). All rights reserved.

Extreme Learning Machine (ELM) is an elegant technique for training Single-hidden Layer Feedforward Networks (SLFNs) with extremely fast speed that attracts significant interest recently. One potential weakness of ELM is the random generation of the input weights and hidden biases, which may deteriorate the classification accuracy. In this paper, we propose a new Memetic Algorithm (MA) based Extreme Learning Machine (M-ELM) for classification problems. M-ELM uses Memetic Algorithm which is a combination of population-based global optimization technique and individual-based local heuristic search method to find optimal network parameters for ELM. The optimized network parameters will enhance the classification accuracy and generalization performance of ELM. Experiments and comparisons on 22 benchmark data sets demonstrate that M-ELM is able to provide highly competitive results compared with other state-of-the-art varieties of ELM algorithms.

Classifier evasion consists in finding for a given instance x the “nearest” instance x′ such that the classifier predictions of x and x′ are different. We present two novel algorithms for systematically computing evasions for tree ensembles such as boosted trees and random forests. Our first algorithm uses a Mixed Integer Linear Program solver and finds the optimal evading instance under an expressive set of constraints. Our second algorithm trades off optimality for speed by using symbolic prediction, a novel algorithm for fast finite differences on tree ensembles. On a digit recognition task, we demonstrate that both gradient boosted trees and random forests are extremely susceptible to evasions. Finally, we harden a boosted tree model without loss of predictive accuracy by augmenting the training set of each boosting round with evading instances, a technique we call adversarial boosting.

In this age, big data applications are increasingly becoming the main focus of attention because of the enormous increment of data generation and storage that has taken place in the last years. This situation becomes a challenge when huge amounts of data are processed to extract knowledge because the data mining techniques are not adapted to the new space and time requirements. Furthermore, real-world data applications usually present a class distribution where the samples that belong to one class, which is precisely the main interest, are hugely outnumbered by the samples of the other classes. This circumstance , known as the class imbalance problem, complicates the learning process as the standard learning techniques do not correctly address this situation. In this work, we analyse the performance of several techniques used to deal with imbal-anced datasets in the big data scenario using the Random Forest classifier. Specifically, oversampling, undersampling and cost-sensitive learning have been adapted to big data using MapReduce so that these techniques are able to manage datasets as large as needed providing the necessary support to correctly identify the underrepresented class. The Random Forest classifier provides a solid basis for the comparison because of its performance, robustness and versatility. An experimental study is carried out to evaluate the performance of the diverse algorithms considered. The results obtained show that there is not an approach to imbalanced big data classification that outperforms the others for all the data considered when using Random Forest. Moreover, even for the same type of problem, the best performing method is dependent on the number of mappers selected to run the experiments. In most of the cases, when the number of splits is increased, an improvement in the running times can be observed, however, this progress in times is obtained at the expense of a slight drop in the accuracy performance obtained. This decrement in the performance is related to the lack of density problem, which is evaluated in this work from the imbalanced data point of view, as this issue degrades the performance of classifiers in the imbalanced scenario more severely than in standard learning.

Previous studies propose that associative classification has high classification accuracy and strong flexibility at handling unstructured data. However, it still suffers from the huge set of mined rules and sometimes biased classification or overfitting since the classification is based on only single high-confidence rule. In this study, we propose a new associative classification method, CMAR, i.e., Classification based on Multiple Association Rules. The method extends an efficient frequent pattern mining method, FP-growth, constructs a class distribution-associated FP-tree, and mines large database efficiently. Moreover, it applies a CR-tree structure to store and retrieve mined association rules efficiently, and prunes rules effectively based on confidence, correlation and database coverage. The classification is performed based on a weighted analysis using multiple strong association rules. Our extensive experiments on databases from UCI machine learning database repository show that CMAR is consistent, highly effective at classification of various kinds of databases and has better average classification accuracy in comparison with CBA and C4.5. Moreover, our performance study shows that the method is highly efficient and scalable in comparison with other reported associative classification methods.

In this paper, a modification of v-support vector machines (v-SVM) for regression and classification is described, and the use of a parametric insensitive/margin model with an arbitrary shape is demonstrated. This can be useful in many cases, especially when the noise is heteroscedastic, that is, the noise strongly depends on the input value x. Like the previous v-SVM, the proposed support vector algorithms have the advantage of using the parameter 0 < or = v < or = 1 for controlling the number of support vectors. To be more precise, v is an upper bound on the fraction of training errors and a lower bound on the fraction of support vectors. The algorithms are analyzed theoretically and experimentally.

Support vector machines (SVMs) are regularly used for classification of unbalanced data by weighting more heavily the error contribution from the rare class. This heuristic technique is often used to learn classifiers with high F-measure, although this particular application of SVMs has not been rigorously examined. We provide significant and new theoretical results that support this popular heuristic. Specifically, we demonstrate that with the right parameter settings SVMs approximately optimize F-measure in the same way that SVMs have already been known to approximately optimize accuracy. This finding has a number of theoretical and practical implications for using SVMs in F-measure optimization.


Question: An adaptive modification of the boost by majority algorithm.

This paper delves into an enhanced adaptation of the Boost by Majority algorithm, a significant contribution to the field of ensemble learning methods. Unlike its traditional counterparts, this adaptive version introduces a dynamic mechanism that adjusts the algorithm's parameters based on the characteristics of the dataset and the performance of the weak learners during the boosting process. This adaptation aims to optimize the balance between bias and variance, enhancing the algorithm's robustness and effectiveness across various datasets. By continuously evaluating the performance of the ensemble and modifying the weights assigned to individual weak learners, the adaptive Boost by Majority algorithm can more effectively address complex classification and regression tasks. This innovation not only improves the generalization capability of the ensemble but also reduces overfitting, a common issue in machine learning models. The proposed adaptation is thoroughly evaluated through a series of experiments on diverse datasets, demonstrating its superiority over the standard Boost by Majority algorithm in terms of accuracy and stability. The results highlight the potential of adaptive mechanisms in boosting algorithms to significantly enhance their performance and applicability in real-world scenarios.

Many learning tasks such as spam filtering and credit card fraud detection face an active adversary that tries to avoid detection. For learning problems that deal with an active adversary, it is important to model the adversary's attack strategy and develop robust learning models to mitigate the attack. These are the two objectives of this paper. We consider two attack models: a free-range attack model that permits arbitrary data corruption and a restrained attack model that anticipates more realistic attacks that a reasonable adversary would devise under penalties. We then develop optimal SVM learning strategies against the two attack models. The learning algorithms minimize the hinge loss while assuming the adversary is modifying data to maximize the loss. Experiments are performed on both artificial and real data sets. We demonstrate that optimal solutions may be overly pessimistic when the actual attacks are much weaker than expected. More important, we demonstrate that it is possible to develop a much more resilient SVM learning model while making loose assumptions on the data corruption models. When derived under the restrained attack model, our optimal SVM learning strategy provides more robust overall performance under a wide range of attack parameters.

Peer prediction mechanisms allow the truthful elicitation of private signals (e.g., experiences, or opinions) in regard to a true world state when this ground truth is unobservable. The original peer prediction method is incentive compatible for any number of agents n ≥ 2, but relies on a common prior, shared by all agents and the mechanism. The Bayesian Truth Serum (BTS) relaxes this assumption. While BTS still assumes that agents share a common prior, this prior need not be known to the mechanism. However, BTS is only incentive compatible for a large enough number of agents, and the particular number of agents required is uncertain because it depends on this private prior. In this paper, we present a robust BTS for the elicitation of binary information which is incentive compatible for every n ≥ 3, taking advantage of a particularity of the quadratic scoring rule. The robust BTS is the first peer prediction mechanism to provide strict incentive compatibility for every n ≥ 3 without relying on knowledge of the common prior. Moreover, and in contrast to the original BTS, our mechanism is numerically robust and ex post individually rational. Introduction Web services that are built around user-generated content are ubiquitous. Examples include reputation systems, where users leave feedback about the quality of products or services, and crowdsourcing platforms, where users (workers) are paid small rewards to do human computation tasks, such as annotating an image. Whereas statistical estimation techniques (Raykar et al. 2010) can be used to resolve noisy inputs, for example in order to determine the image tags most likely to be correct, they are appropriate only when user inputs are informative in the first place. But what if providing accurate information is costly for users, or if users otherwise have an external incentive for submitting false inputs? The peer prediction method (Miller, Resnick, and Zeckhauser 2005) addresses the quality control problem by providing payments (in cash, points or otherwise) that align an agent’s own interest with providing inputs that are predictive of the inputs that will be provided by other agents. Formally, the peer prediction method provides strict incentives for providing truthful inputs (e.g., in regard to a user’s information about the quality of a product, or a user’s view on Copyright c © 2012, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. the correct label for a training example) for a system of two or more agents, and when there is a common prior amongst agents and, critically, known to the mechanism. The Bayesian Truth Serum (BTS) by Prelec (2004) still assumes that agents share a common prior, but does not require this to be known by the mechanism. In addition to an information report from an agent, BTS asks each agent for a prediction report, that reflects the agent’s belief about the distribution of information reports in the population. An agent’s payment depends on both reports, with an information component that rewards reports that are “surprisingly common,” i.e., more common than collectively predicted, and a prediction component that rewards accurate predictions of the reports made by others. A significant drawback of BTS is that it only aligns incentives for a large enough number of agents, where this number depends on the prior and is thus unknown to the mechanism. In addition, BTS may leave a participant with a negative payment, and is not numerically robust for all inputs. In this paper, we present the robust Bayesian Truth Serum (RBTS) mechanism, which, to the best of our knowledge, is the first peer prediction mechanism that does not rely on knowledge of the common prior to provide strict incentive compatibility for every number of agents n ≥ 3. RBTS is also ex post individually rational (so that no agent makes a negative payment in any outcome) and numerically robust, being well defined for all possible agent reports. Moreover, the mechanism seems conceptually simpler than BTS, and the incentive analysis is more straightforward. The main limitation of RBTS relative to earlier mechanisms, is that it applies only to the elicitation of binary information; e.g., good or bad experiences, or true or false classification labels.1 Extending RBTS to incorporate more than two signals is the most important direction for future research. RBTS takes the same reports as BTS, and an agent’s payment continues to consist of one component that depends on an agent’s information report and a second component that Many interesting applications involve binary information reports. This is supported by the fact that Prelec’s own experimental papers have adopted the binary signal case (Prelec and Seung 2006; John, Loewenstein, and Prelec 2011). Indeed, as the number of possible information reports increases, so does the difficulty imposed on users in providing the prediction report, which must include estimates for the additional possible information reports. depends on an agent’s prediction report. The main innovation is to induce a “shadow” posterior belief report for an agent i from her information report and the prediction report of another agent j, adjusting this prediction report in the direction suggested by agent i’s information report. We couple this with a particularity of the quadratic scoring rule, by which an agent prefers a shadow posterior belief that is as close as possible to her true posterior. In order to determine the agent’s payment, we then apply both the shadow posterior belief and the agent’s prediction report to the quadratic scoring rule, adopting the information report of a third agent k as the event to be predicted.

In this work we have compared different type of channel estimation algorithm for Orthogonal Frequency Division Multiplexing (OFDM) systems. The result of the Adaptive Boosting (AdaBoost) algorithm was compared with other algorithms such Least Square (LS), Best Linear Unbiased Estimator (BLUE) and Minimum Mean Square Error (MMSE).

In this paper we propose a novel method for human action recognition based on boosted key-frame selection and correlated pyramidal motion feature representations. Instead of using an unsupervised method to detect interest points, a Pyramidal Motion Feature (PMF), which combines optical flow with a biologically inspired feature, is extracted from each frame of a video sequence. The AdaBoost learning algorithm is then applied to select the most discriminative frames from a large feature pool. In this way, we obtain the top-ranked boosted frames of each video sequence as the key frames which carry the most representative motion information. Furthermore, we utilise the correlogram which focuses not only on probabilistic distributions within one frame but also on the temporal relationships of the action sequence. In the classification phase, a Support-Vector Machine (SVM) is adopted as the final classifier for human action recognition. To demonstrate generalizability, our method has been systematically tested on a variety of datasets and shown to be more effective and accurate for action recognition compared to the previous work. We obtain overall accuracies of: 95.5%, 93.7%, and 36.5% with our proposed method on the KTH, the multiview IXMAS and the challenging HMDB51 datasets, respectively. & 2012 Elsevier Ltd. All rights reserved.

In this paper, we develop a novel Distance-weighted k -nearest Neighbor rule (DWKNN), using the dual distance-weighted function. The proposed DWKNN is motivated by the sensitivity problem of the selection of the neighborhood size k that exists in k -nearest Neighbor rule (KNN), with the aim of improving classification performance. The experiment results on twelve real data sets demonstrate that our proposed classifier is robust to different choices of k to some degree, and yields good performance with a larger optimal k, compared to the other state-of-art KNN-based methods.

In multilabel learning, each instance in the training set is associated with a set of labels and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e., backpropagation for multilabel learning, is proposed. It is derived from the popular backpropagation algorithm through employing a novel error function capturing the characteristics of multilabel learning, i.e., the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two real-world multilabel learning problems, i.e., functional genomics and text categorization, show that the performance of BP-MLL is superior to that of some well-established multilabel learning algorithms

In this paper, an algorithm based on the concepts of genetic algorithms that uses an estimation of a probability distribution of promising solutions in order to generate new candidate solutions is proposed. To estimate the distribution, techniques for modeling multivariate data by Bayesian networks are used. The proposed algorithm identifies, reproduces and mixes building blocks up to a specified order. It is independent of the ordering of the variables in the strings representing the solutions. Moreover, prior information about the problem can be incorporated into the algorithm. However, prior information is not essential. Preliminary experiments show that the BOA outperforms the simple genetic algorithm even on decomposable functions with tight building blocks as a problem size grows.

Classifier evasion consists in finding for a given instance x the “nearest” instance x′ such that the classifier predictions of x and x′ are different. We present two novel algorithms for systematically computing evasions for tree ensembles such as boosted trees and random forests. Our first algorithm uses a Mixed Integer Linear Program solver and finds the optimal evading instance under an expressive set of constraints. Our second algorithm trades off optimality for speed by using symbolic prediction, a novel algorithm for fast finite differences on tree ensembles. On a digit recognition task, we demonstrate that both gradient boosted trees and random forests are extremely susceptible to evasions. Finally, we harden a boosted tree model without loss of predictive accuracy by augmenting the training set of each boosting round with evading instances, a technique we call adversarial boosting.

We present a new family of subgradient methods that dynamica lly incorporate knowledge of the geometry of the data observed in earlier iterations to perfo rm more informative gradient-based learning. Metaphorically, the adaptation allows us to find n eedles in haystacks in the form of very predictive but rarely seen features. Our paradigm stems fro m recent advances in stochastic optimization and online learning which employ proximal funct ions to control the gradient steps of the algorithm. We describe and analyze an apparatus for adap tively modifying the proximal function, which significantly simplifies setting a learning rate nd results in regret guarantees that are provably as good as the best proximal function that can be cho sen in hindsight. We give several efficient algorithms for empirical risk minimization probl ems with common and important regularization functions and domain constraints. We experimen tally study our theoretical analysis and show that adaptive subgradient methods outperform state-o f-the-art, yet non-adaptive, subgradient algorithms.

As the volumes of AI problems involving human knowledge are likely to soar, crowdsourcing has become essential in a wide range of world-wide-web applications. One of the biggest challenges of crowdsourcing is aggregating the answers collected from the crowd since the workers might have wide-ranging levels of expertise. In order to tackle this challenge, many aggregation techniques have been proposed. These techniques, however, have never been compared and analyzed under the same setting, rendering a ‘right’ choice for a particular application very difficult. Addressing this problem, this paper presents a benchmark that offers a comprehensive empirical study on the performance comparison of the aggregation techniques. Specifically, we integrated several stateof-the-art methods in a comparable manner, and measured various performance metrics with our benchmark, including computation time, accuracy, robustness to spammers, and adaptivity to multi-labeling. We then provide in-depth analysis of benchmarking results, obtained by simulating the crowdsourcing process with different types of workers. We believe that the findings from the benchmark will be able to serve as a practical guideline for crowdsourcing applications.


Question: An adaptive modification of the Boost by Majority algorithm.

This paper presents an innovative learning machine, the Support-Vector Network (SVN), specifically designed for two-group classification tasks. The SVN embodies a novel approach: input vectors are mapped non-linearly into a significantly higher-dimensional feature space. Within this space, a linear decision surface is constructed. The unique properties of this decision surface ensure the SVN possesses high generalization ability, a critical feature for effective machine learning. Initially, the concept of the SVN was applied to a restricted scenario where the training data could be perfectly separated. This paper extends the SVN's capabilities to handle non-separable training data, a more realistic scenario in many practical applications. The SVN's performance is bolstered by utilizing polynomial input transformations, which is demonstrated through empirical results. Additionally, a comparative analysis is conducted against a range of classical learning algorithms, all of which participated in a benchmark study on Optical Character Recognition (OCR). This comparison highlights the superior performance of the SVN in tackling classification problems, particularly in complex and real-world datasets.

In this paper, a new learning algorithm for adaptive network intrusion detection using naive Bayesian classifier and decision tree is presented, which performs balance detections and keeps false positives at acceptable level for different types of network attacks, and eliminates redundant attributes as well as contradictory examples from training data that make the detection model complex. The proposed algorithm also addresses some difficulties of data mining such as handling continuous attribute, dealing with missing attribute values, and reducing noise in training data. Due to the large volumes of security audit data as well as the complex and dynamic properties of intrusion behaviours, several data miningbased intrusion detection techniques have been applied to network-based traffic data and host-based data in the last decades. However, there remain various issues needed to be examined towards current intrusion detection systems (IDS). We tested the performance of our proposed algorithm with existing learning algorithms by employing on the KDD99 benchmark intrusion detection dataset. The experimental results prove that the proposed algorithm achieved high detection rates (DR) and significant reduce false positives (FP) for different types of network intrusions using limited computational resources.

The Random Forests algorithm belongs to the class of ensemble learning methods, which are common used in classification problem. In this paper, we studied the problem of adopting the Random Forests algorithm to learn raw data from real usage scenario. An improvement, which is stable, strict, high efficient, data-driven, problem independent and has no impact on algorithm performance, is proposed to investigate 2 actual issues of feature selection of the Random Forests algorithm. The first one is to eliminate noisy features, which are irrelevant to the classification. And the second one is to eliminate redundant features, which are highly relevant with other features, but useless. We implemented our improvement approach on Spark. Experiments are performed to evaluate our improvement and the results show that our approach has an ideal performance.

Many multimedia applications can benefit from techniques for adapting existing classifiers to data with different distributions. One example is cross-domain video concept detection which aims to adapt concept classifiers across various video domains. In this paper, we explore two key problems for classifier adaptation: (1) how to transform existing classifier(s) into an effective classifier for a new dataset that only has a limited number of labeled examples, and (2) how to select the best existing classifier(s) for adaptation. For the first problem, we propose Adaptive Support Vector Machines (A-SVMs) as a general method to adapt one or more existing classifiers of any type to the new dataset. It aims to learn the "delta function" between the original and adapted classifier using an objective function similar to SVMs. For the second problem, we estimate the performance of each existing classifier on the sparsely-labeled new dataset by analyzing its score distribution and other meta features, and select the classifiers with the best estimated performance. The proposed method outperforms several baseline and competing methods in terms of classification accuracy and efficiency in cross-domain concept detection in the TRECVID corpus.

In this paper, we address the problem of learning an adaptive appearance model for object tracking. In particular, a class of tracking techniques called “tracking by detection” have been shown to give promising results at real-time speeds. These methods train a discriminative classifier in an online manner to separate the object from the background. This classifier bootstraps itself by using the current tracker state to extract positive and negative examples from the current frame. Slight inaccuracies in the tracker can therefore lead to incorrectly labeled training examples, which degrades the classifier and can cause further drift. In this paper we show that using Multiple Instance Learning (MIL) instead of traditional supervised learning avoids these problems, and can therefore lead to a more robust tracker with fewer parameter tweaks. We present a novel online MIL algorithm for object tracking that achieves superior results with real-time performance.

In many practical learning scenarios, there is a small amount of labeled data along with a large pool of unlabeled data. Many supervised learning algorithms have been developed and extensively studied. We present a new \co-training" strategy for using un-labeled data to improve the performance of standard supervised learning algorithms. Unlike much of the prior work, such as the co-training procedure of Blum and Mitchell (1998), we do not assume there are two redundant views both of which are suucient for classiication. The only requirement our co-training strategy places on each supervised learning algorithm is that its hypothesis partitions the example space into a set of equivalence classes (e.g. for a decision tree each leaf deenes an equivalence class). We evaluate our co-training strategy via experiments using data from the UCI repository.

In this paper, we propose a method to classify movie review documents into positive or negative opinions. There are several approaches to classify documents. The previous studies, however, used only a single classifier for the classification task. We describe a multiple classifier for the review document classification task. The method consists of three classifiers based on SVMs, ME and score calculation. We apply two voting methods and SVMs to the integration process of single classifiers. The integrated methods improved the accuracy as compared with the three single classifiers. The experimental results show the effectiveness of our method.

Traditional machine learning algorithms for pattern recognition just output simple predictions, without any associated confidence values. Confidence values are an indication of how likely each prediction is of being correct. In the ideal case, a confidence of 99% or higher for all examples in a set, means that the percentage of erroneous predictions in that set will not exceed 1%. Knowing the likelihood of each prediction enables us to assess the extent to which we can rely on it. For this reason, predictions that are associated with some kind of confidence values are highly desirable in many risk-sensitive applications, such as those used for medical diagnosis or financial analysis. In fact, such information can benefit any application that requires human-computer interaction, as confidence values can be used to determine the way in which each prediction will be treated. For instance, a filtering mechanism can be employed so that only predictions which satisfy a certain level of confidence will be taken into account, while the rest can be discarded or passed on to a human for judgement. There are two main areas in mainstream machine learning that can be used in order to obtain some kind of confidence values; the Bayesian framework and the theory of Probably Approximately Correct learning (PAC theory). Quite often the Bayesian framework is used for producing algorithms that complement individual predictions with probabilistic measures of their quality. On the other hand, PAC theory can be used for producing upper bounds on the probability of error for a given algorithm with respect to some confidence level 1 − δ. Both of these approaches however, have their drawbacks. In order to apply the Bayesian framework one is required to have some prior knowledge about the distribution that generates the data. When the correct prior is known, Bayesian methods provide optimal decisions. For real world data sets though, as the required knowledge is not available, one has to assume the existence of an arbitrarily chosen prior. In this case, if the assumed prior is incorrect, the resulting confidence levels may also be “incorrect”; for example the predictive regions output for the 95% confidence level may contain the true label in much less than 95% of the cases. This signifies a major failure, as we would expect confidence levels to bound the percentage of expected errors. An experimental demonstration of how misleading Bayesian methods can become when their assumptions are violated can be found in (Melluish et al., 2001).

Gradient boosting of regression trees is a competitive procedure for learning predictive models of continuous data that fits the data with an additive non-parametric model. The classic version of gradient boosting assumes that the data is independent and identically distributed. However, relational data with interdependent, linked instances is now common and the dependencies in such data can be exploited to improve predictive performance. Collective inference is one approach to exploit relational correlation patterns and significantly reduce classification error. However, much of the work on collective learning and inference has focused on discrete prediction tasks rather than continuous. In this work, we investigate how to combine these two paradigms together to improve regression in relational domains. Specifically, we propose a boosting algorithm for learning a collective inference model that predicts a continuous target variable. In the algorithm, we learn a basic relational model, collectively infer the target values, and then iteratively learn relational models to predict the residuals. We evaluate our proposed algorithm on a real network dataset and show that it outperforms alternative boosting methods. However, our investigation also revealed that the relational features interact together to produce better predictions.

The continually increasing number of complex datasets each year necessitates ever improving machine learning methods for robust and accurate categorization of these data. This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble, deep learning approach for classification. Deep learning models have achieved state-of-the-art results across many domains. RMDL solves the problem of finding the best deep learning structure and architecture while simultaneously improving robustness and accuracy through ensembles of deep learning architectures. RDML can accept as input a variety data to include text, video, images, and symbolic. This paper describes RMDL and shows test results for image and text data including MNIST, CIFAR-10, WOS, Reuters, IMDB, and 20newsgroup. These test results show that RDML produces consistently better performance than standard methods over a broad range of data types and classification problems.

A large class of machine-learning problems in natural language require the characterization of linguistic context. Two characteristic properties of such problems are that their feature space is of very high dimensionality, and their target concepts depend on only a small subset of the features in the space. Under such conditions, multiplicative weight-update algorithms such as Winnow have been shown to have exceptionally good theoretical properties. In the work reported here, we present an algorithm combining variants of Winnow and weighted-majority voting, and apply it to a problem in the aforementioned class: context-sensitive spelling correction. This is the task of fixing spelling errors that happen to result in valid words, such as substituting to for too, casual for causal, and so on. We evaluate our algorithm, WinSpell, by comparing it against BaySpell, a statistics-based method representing the state of the art for this task. We find: (1) When run with a full (unpruned) set of features, WinSpell achieves accuracies significantly higher than BaySpell was able to achieve in either the pruned or unpruned condition; (2) When compared with other systems in the literature, WinSpell exhibits the highest performance; (3) While several aspects of WinSpell's architecture contribute to its superiority over BaySpell, the primary factor is that it is able to learn a better linear separator than BaySpell learns; (4) When run on a test set drawn from a different corpus than the training set was drawn from, WinSpell is better able than BaySpell to adapt, using a strategy we will present that combines supervised learning on the training set with unsupervised learning on the (noisy) test set.