Question: Multi-level preference regression for addressing the cold-start recommendation challenge.

This study explores the cold-start recommendation challenge within the context of online retail environments for users who have not yet engaged in any transactions or meaningful interactions with available products but have consented to share limited auxiliary information, such as basic demographics (gender, age, location) or social network details (Facebook connections or page preferences). We propose a novel framework for neighborhood-based collaborative filtering, tailored to cold-start scenarios, which operates within a generalized matrix algebra context. This approach does not necessitate purchase history for the target users when their auxiliary information is accessible. Through empirical evaluations using a dataset comprising 30,000 users who have made over 80,000 book purchases and have over 9,000,000 Facebook friends and 6,000,000 page likes, we illustrate that leveraging Facebook page preferences for cold-start recommendations can lead to a significant enhancement in recommendation accuracy. Specifically, we observe up to a threefold increase in mean average precision (mAP) and up to sixfold improvements in Precision@k and Recall@k metrics when compared to recommendations based on the most popular items, demographics, or Facebook friendship networks. These findings underscore the substantial predictive value of social network data and its crucial role in addressing the complex issue of recommending items to cold-start users.

The primary premise upon which top-N recommender systems operate is that similar users are likely to have similar tastes with regard to their product choices. For this reason, recommender algorithms depend deeply on similarity metrics to build the recommendation lists for end-users.
 However, it has been noted that the products offered on recommendation lists are often too similar to each other and attention has been paid towards the goal of improving diversity to avoid monotonous recommendations.
 Noting that the retrieval of a set of items matching a user query is a common problem across many applications of information retrieval, we model the competing goals of maximizing the diversity of the retrieved list while maintaining adequate similarity to the user query as a binary optimization problem. We explore a solution strategy to this optimization problem by relaxing it to a trust-region problem.This leads to a parameterized eigenvalue problem whose solution is finally quantized to the required binary solution. We apply this approach to the top-N prediction problem, evaluate the system performance on the Movielens dataset and compare it with a standard item-based top-N algorithm. A new evaluation metric ItemNovelty is proposed in this work. Improvements on both diversity and accuracy are obtained compared to the benchmark algorithm.

Personalized recommendations have become a common feature of many modern online services. In particular on ecommerce sites, one value of such recommendations is that they help consumers find items of interest in large product assortments more quickly. Many of today’s sites take advantage of modern recommendation technologies to create personalized item suggestions for consumers navigating the site. However, limited research exists on the use of personalization and recommendation technology when consumers rely on the site’s catalog search functionality to discover relevant items. In this work we explore the value of personalizing search results on e-commerce sites using recommendation technology. We design and evaluate different personalization strategies using log data of an online retail site. Our results show that considering several item relevance signals within the recommendation process in parallel leads to the best ranking of the search results. Specifically, the factors taken into account include the users’ general interests, their most recent browsing behavior, as well as the consideration of current sales trends.

We implement a personal recommendation system on the Yelp academic dataset using a novel algorithm proposed by [Li and Chen., 2012]. By constructing a bipartite userbusiness graph where positive reviews are edges between nodes, we pose the recommendation problem as link prediction. More specifically, we construct a kernel which combines the information from both the node features and local graph structure. This kernel is then put into a one-class support vector machine, which generates an ordering of likely user-business links. We downsample the graph and tune the parameters of our algorithm (via cross validation). In what follows we provide a summary of the Yelp data, describe the recommendation algorithm in detail, and present the results of our implementation.

This paper suggests a number of research directions in which the recommender systems can improve their quality, by moving beyond the assumptions of linearity and independence that are traditionally made. These assumptions, while producing effective and meaningful results, can be suboptimal, as in lots of cases they do not represent the real datasets. In this paper, we discuss three different ways to address some of the previous constraints. More specifically, we focus on the development of methods capturing higher-order relations between the items, cross-feature interactions and intra-set dependencies which can potentially lead to a considerable enhancement of the recommendation accuracy.

Recommender systems have become an important research field since the emergence of the first paper on collaborative filtering in the mid‐1990s. In general, recommender systems are defined as the supporting systems which help users to find information, products, or services (such as books, movies, music, digital products, web sites, and TV programs) by aggregating and analyzing suggestions from other users, which mean reviews from various authorities, and user attributes. However, as academic researches on recommender systems have increased significantly over the last ten years, more researches are required to be applicable in the real world situation. Because research field on recommender systems is still wide and less mature than other research fields. Accordingly, the existing articles on recommender systems need to be reviewed toward the next generation of recommender systems. However, it would be not easy to confine the recommender system researches to specific disciplines, considering the nature of the recommender system researches. So, we reviewed all articles on recommender systems from 37 journals which were published from 2001 to 2010. The 37 journals are selected from top 125 journals of the MIS Journal Rankings. Also, the literature search was based on the descriptors “Recommender system”, “Recommendation system”, “Personalization system”, “Collaborative filtering” and “

Evaluating user preferences of web search results is crucial for search engine development, deployment, and maintenance. We present a real-world study of modeling the behavior of web search users to predict web search result preferences. Accurate modeling and interpretation of user behavior has important applications to ranking, click spam detection, web search personalization, and other tasks. Our key insight to improving robustness of interpreting implicit feedback is to model query-dependent deviations from the expected "noisy" user behavior. We show that our model of clickthrough interpretation improves prediction accuracy over state-of-the-art clickthrough methods. We generalize our approach to model user behavior beyond clickthrough, which results in higher preference prediction accuracy than models based on clickthrough information alone. We report results of a large-scale experimental evaluation that show substantial improvements over published implicit feedback interpretation methods.

Recommendation technologies and trust metrics constitute the two pillars of trust-enhanced recommender systems. We discuss and illustrate the basic trust concepts such as trust and distrust modeling, propagation and aggregation. These concepts are needed to fully grasp the rationale behind the trust-enhanced recommender techniques that are discussed in the central part of the chapter, which focuses on the application of trust metrics and their operators in recommender systems. We explain the benefits of using trust in recommender algorithms and give an overview of state-of-the-art approaches for trust-enhanced recommender systems. Furthermore, we explain the details of three well-known trust-based systems and provide a comparative analysis of their performance. We conclude with a discussion of some recent developments and open challenges, such as visualizing trust relationships in a recommender system, alleviating the cold start problem in a trust network of a recommender system, studying the effect of involving distrust in the recommendation process, and investigating the potential of other types of social relationships.

Matrix factorization is one of the most powerful techniques in collaborative filtering, which models the (user, item) interactions behind historical explicit or implicit feedbacks. However, plain matrix factorization may not be able to uncover the structure correlations among users and items well such as communities and taxonomies. As a response, we design a novel algorithm, i.e., hierarchical group matrix factorization (HGMF), in order to explore and model the structure correlations among users and items in a principled way. Specifically, we first define four types of correlations, including (user, item), (user, item group), (user group, item) and (user group, item group); we then extend plain matrix factorization with a hierarchical group structure; finally, we design a novel clustering algorithm to mine the hidden structure correlations. In the experiments, we study the effectiveness of our HGMF for both rating prediction and item recommendation, and find that it is better than some state-of-the-art methods on several real-world data sets.

The popularity of online recommender systems has soared; they are deployed in numerous websites and gather tremendous amounts of user data that are necessary for the recommendation purposes. This data, however, may pose a severe threat to user privacy, if accessed by untrusted parties or used inappropriately. Hence, it is of paramount importance for recommender system designers and service providers to find a sweet spot, which allows them to generate accurate recommendations and guarantee the privacy of their users. In this chapter we overview the state of the art in privacy enhanced recommendations. We analyze the risks to user privacy imposed by recommender systems, survey the existing solutions, and discuss the privacy implications for the users of the recommenders. We conclude that a considerable effort is still required to develop practical recommendation solutions that provide adequate privacy guarantees, while at the same time facilitating the delivery of high-quality recommendations to their users. Arik Friedman NICTA, Australia, e-mail: arik.friedman@nicta.com.au Bart Knijnenburg UC Irvine, USA, e-mail: bart.k@uci.edu Kris Vanhecke iMinds Ghent University, Belgium, e-mail: kris.vanhecke@intec.ugent.be Luc Martens iMinds Ghent University, Belgium, e-mail: luc.martens@intec.ugent.be Shlomo Berkovsky CSIRO, Australia, e-mail: shlomo.berkovsky@csiro.au 1 Accepted to be included in the 2nd edition of the Recommender Systems handbook. Please do not share publicly, and consult the authors before citing this chapter.

Preface It was January 2007 when Dolf Trieschnigg, my supervisor for the course Information Retrieval , first told me about the availability of graduation projects at TNO Information and Communication Technology. This was a bull's eye since I just started to orientate myself on a subject to investigate. I was also looking for a company where I could perform my research since I wanted to get familiar with working in a professional environment. TNO Information and Communication Technology was one of the companies that seemed interesting to me, so I contacted Stephan Raaijmakers for more information. The subject of the proposed research project, evaluating recommender systems, was completely new to me, but seemed fascinating. And it is. In September 2008 I started my research by crafting the goals and research questions for the project. Now, almost nine months later, the research has resulted in the report that you just started reading. It marks the end of my life as a student (at least for now) and the start of my professional career at TNO. At TNO I can continue to work on personalisation and recommender systems. Although I wrote this thesis the research would never have been completed without the support of other people. In the first place, I would like to thank my supervisors, Erik, Djoerd, Stephan and Dolf for supervising me throughout the research project. They were always there for me when I needed their help and/or expertise and provided loads of valuable comments and suggestions. I also like to thank Jan Telman for his help and advice with respect to statistics (and hiking through the Alps). Furthermore I would like to thank all people that participated in the user study. Without their devotion to the demanding study, the research would not have been completed. I would also like to thank my lovely girlfriend Linda, my family and my colleagues at TNO Information and Communication Technology for being all ears when I tried to explain my approach and ideas to them. They helped me to abstract from all the details and forced me to explain things clearly so I better understood it myself. Finally, I owe special thanks to Mark Prins, who provided me with an inexhaustible source of Tiësto's Club Life, A State of Trance and other dance and trance music.


Question: Multi-level preference prediction for cold-start recommendation systems.

This paper explores the potential of social media profiles in forecasting consumer buying patterns on e-commerce platforms. In the context of social commerce, where users frequently navigate between e-commerce sites and social networks like Facebook and Twitter, little has been done to explore the relationship between social media profiles and e-commerce actions. Our study introduces a methodology for predicting a user's purchasing habits on e-commerce sites based on their social media profile. Specifically, we investigate whether information from a user's social network profile (such as Facebook) can be utilized to forecast the categories of products they are likely to purchase (for example, from eBay Electronics). The paper offers a comprehensive analysis of the correlation between users' Facebook profile data and their purchasing behavior on eBay. It also evaluates the effectiveness of various feature sets and learning algorithms in predicting purchase behavior, providing insights into the predictive power of social media profiles in e-commerce contexts.

Recommender systems perform much better on users for which they have more information. This gives rise to a problem of satisfying users new to a system. The problem is even more acute considering that some of these hard to profile new users judge the unfamiliar system by its ability to immediately provide them with satisfying recommendations, and may quickly abandon the system when disappointed. Rapid profiling of new users by a recommender system is often achieved through a bootstrapping process - a kind of an initial interview - that elicits users to provide their opinions on certain carefully chosen items or categories. The elicitation process becomes particularly effective when adapted to users' responses, making best use of users' time by dynamically modifying the questions to improve the evolving profile. In particular, we advocate a specialized version of decision trees as the most appropriate tool for this task. We detail an efficient tree learning algorithm, specifically tailored to the unique properties of the problem. Several extensions to the tree construction are also introduced, which enhance the efficiency and utility of the method. We implemented our methods within a movie recommendation service. The experimental study delivered encouraging results, with the tree-based bootstrapping process significantly outperforming previous approaches.

Recommender Systems (RSs) are software tools and techniques providing suggestions for items to be of use to a user. [1] For a recommender system to be effective, it has to have a comprehensive and appropriately catalogued repository of resources. Also, it has to be accurate in understanding the need of the user, and basically has to correctly profile the user. Ideally, one needs to take into consideration the fact that a user’s preferences are constantly changing and that is often not done by recommender systems. To meet this need, we propose an interactive research paper recommender system that observes the various themes that occur in a researcher’s work over time and keeps learning the user’s preferences constantly based on the user’s feedback of the papers recommended to them. We use a topic model to take into account the themes occurring in the document. The query is modeled as a bag of topics, where topics represent latent generative clusters of relative words. The entire collection or research papers is also modeled this way. We then estimate the similarity between the query and each research paper using a bag-of-topics based similarity measure and find the best ones. To take into account the users preferences, we keep track of the papers which the user likes and augment the query with the topics which recur in a lot of the users preferred papers the next time recommendations are required. We also truncate the topics which the user seems to ignore.

Given the abundance of online information available to mobile users, particularly tourists and weekend travelers, recommender systems that effectively filter this information and suggest interesting participatory opportunities will become increasingly important. Previous work has explored recommending interesting locations; however, users would also benefit from recommendations for activities in which to participate at those locations along with suitable times and days. Thus, systems that provide collaborative recommendations involving multiple dimensions such as location, activities and time would enhance the overall experience of users.The relationship among these dimensions can be modeled by higher-order matrices called tensors which are then solved by tensor factorization. However, these tensors can be extremely sparse. In this paper, we present a system and an approach for performing multi-dimensional collaborative recommendations for Who (User), What (Activity), When (Time) and Where (Location), using tensor factorization on sparse user-generated data. We formulate an objective function which simultaneously factorizes coupled tensors and matrices constructed from heterogeneous data sources. We evaluate our system and approach on large-scale real world data sets consisting of 588,000 Flickr photos collected from three major metro regions in USA. We compare our approach with several state-of-the-art baselines and demonstrate that it outperforms all of them.

Most previous research on location recommendation services in location-based social networks (LBSNs) makes recommendations without considering where the targeted user is currently located. Such services may recommend a place near her hometown even if the user is traveling out of town. In this paper, we study the issues in making location recommendations for out-of-town users by taking into account user preference, social influence and geographical proximity. Accordingly, we propose a collaborative recommendation framework, called User Preference, Proximity and Social-Based Collaborative Filtering} (UPS-CF), to make location recommendation for mobile users in LBSNs. We validate our ideas by comprehensive experiments using real datasets collected from Foursquare and Gowalla. By comparing baseline algorithms and conventional collaborative filtering approach (and its variants), we show that UPS-CF exhibits the best performance. Additionally, we find that preference derived from similar users is important for in-town users while social influence becomes more important for out-of-town users.

Contextual behavior modeling uses data from multiple contexts to discover patterns for predictive analysis. However, existing behavior prediction models often face difficulties when scaling for massive datasets. In this work, we formulate a behavior as a set of context items of different types (such as decision makers, operators, goals and resources), consider an observable itemset as a behavior success, and propose a novel scalable method, "multi-type itemset embedding", to learn the context items' representations preserving the success structures. Unlike most of existing embedding methods that learn pair-wise proximity from connection between a behavior and one of its items, our method learns item embeddings collectively from interaction among all multi-type items of a behavior, based on which we develop a novel framework, LearnSuc, for (1) predicting the success rate of any set of items and (2) finding complementary items which maximize the probability of success when incorporated into an itemset. Extensive experiments demonstrate both effectiveness and efficency of the proposed framework.

We study the problem of cross-domain ranking, which addresses learning to rank objects from multiple interrelated domains. In many applications, we may have multiple interrelated domains, some of them with a large amount of training data and others with very little. We often wish to utilize the training data from all these related domains to help improve ranking performance. In this paper, we present a unified model: BayCDR for cross-domain ranking. BayCDR uses a latent space to measure the correlation between different domains, and learns the ranking functions from the interrelated domains via the latent space by a Bayesian model, where each ranking function is based on a weighted average model. An efficient learning algorithm based on variational inference and a generalization bound has been developed. To scale up to handle real large data, we also present a learning algorithm under the Map-Reduce programming model. Finally, we demonstrate the effectiveness and efficiency of BayCDR on large datasets.

Collaborative recommender systems allow personalization for e-commerce by exploiting similarities and dissimilarities among users' preferences. We investigate the use of association rule mining as an underlying technology for collaborative recommender systems. Association rules have been used with success in other domains. However, most currently existing association rule mining algorithms were designed with market basket analysis in mind. Such algorithms are ine cient for collaborative recommendation because they mine many rules that are not relevant to a given user. Also, it is necessary to specify the minimum support of the mined rules in advance, often leading to either too many or too few rules; this negatively impacts the performance of the overall system. We describe a collaborative recommendation technique based on a new algorithm speci cally designed to mine association rules for this purpose. Our algorithm does not require the minimum support to be speci ed in advance. Rather, a target range is given for the number of rules, and the algorithm adjusts the minimum support for each user in order to obtain a ruleset whose size is in the desired range. Rules are mined for a speci c target user, reducing the time required for the mining process. We employ associations between users as well as associations between items in making recommendations. Experimental evaluation of a system based on our algorithm reveals performance that is signi cantly better than that of traditional correlationbased approaches. Corresponding author. Present a liation: Department of Computer Science, Wellesley College, 106 Central Street, Wellesley, MA 02481 USA, e-mail: salvarez@wellesley.edu

Traditional recommender systems assume that all the users are independent, and they usually face the cold start and data sparse problems. To alleviate these problems, social recommender systems use social relations as an additional input to improve recommendation accuracy. Social recommendation follows the intuition that people with social relationships share some kinds of preference towards items. Current social recommendation methods commonly apply the Matrix Factorization (MF) model to incorporate social information into the recommendation process. As an alternative model to MF, we propose a novel social recommendation approach based on Euclidean Embedding (SREE) in this paper. The idea is to embed users and items in a unified Euclidean space, where users are close to both their desired items and social friends. Experimental results conducted on two real-world data sets illustrate that our proposed approach outperforms the state-of-the-art methods in terms of recommendation accuracy.

Abstract: This research proposes a two-stage user-based collaborative filtering process using an artificial immune system for the prediction of student grades, along with a filter for professor ratings in the course recommendation for college students. We test for cosine similarity and Karl Pearson (KP) correlation in affinity calculations for clustering and prediction. This research uses student information and professor information datasets of Yuan Ze University from the years 2005–2009 for the purpose of testing and training. The mean average error and confusion matrix analysis form the testing parameters. A minimum professor rating was tested to check the results, and observed that the recommendation systems herein provide highly accurate results for students with higher mean grades.

In this paper, we propose a machine-learning solution to problems consisting of many similar prediction tasks. Each of the individual tasks has a high risk of overrtting. We combine two types of knowledge transfer between tasks to reduce this risk: multi-task learning and hierarchical Bayesian modeling. Multi-task learning is based on the assumption that there exist features typical to the task at hand. To nd these features, we train a huge two-layered neural network. Each task has its own output, but shares the weights from the input to the hidden units with all other tasks. In this way a relatively large set of possible explanatory variables (the network inputs) is reduced to a smaller and easier to handle set of features (the hidden units). Given this set of features and after an appropriate scale transformation, we assume that the tasks are exchangeable. This assumption allows for a hierarchical Bayesian analysis in which the hyperparameters can be estimated from the data. EEectively, these hyperpa-rameters act as regularizers and prevent over-tting. We describe how to make the system robust against nonstationarities in the time series and give directions for further improvement. We illustrate our ideas on a database regarding the prediction of newspaper sales.


Question: Multi-level preference prediction for cold-start recommendation systems.

This paper presents a pioneering algorithm designed to approximate the matrix with the least nuclear norm from a set of matrices that adhere to a series of convex constraints. This problem can be seen as the convex relaxation of a rank minimization issue, which is a critical component in various applications, notably in recovering a vast matrix from a minuscule subset of its entries, akin to the Netflix challenge. Traditional algorithms, such as interior point methods, are not well-suited for large-scale problems of this nature with over a million unknown entries. To address this, the paper proposes a straightforward, first-order algorithm that is highly efficient in handling problems where the optimal solution exhibits low rank. The algorithm is iterative, generating a sequence of matrices {Xk, Yk}, and at each iteration, primarily executes a soft-thresholding operation on the singular values of the matrix Yk. Two significant features make this algorithm particularly suitable for low-rank matrix completion problems. Firstly, the soft-thresholding operation is applied to a sparse matrix, and secondly, the rank of the iterates {Xk} empirically increases as the algorithm progresses. These attributes allow the algorithm to operate with minimal storage requirements and maintain low computational costs per iteration. From a theoretical standpoint, we offer a convergence analysis demonstrating that the sequence of iterates converges. Practically, we provide empirical evidence where 1,000 × 1,000 matrices are reconstructed in under a minute on a standard desktop computer. We also illustrate the algorithm's scalability to enormous problems by recovering matrices of rank approximately 10 with nearly a billion unknowns from merely 0.4% of their sampled entries. Our methodology is linked to the current literature on linearized Bregman iterations for l1 minimization, and we develop a conceptual framework to interpret these algorithms within the context of established Lagrange multiplier methods.

This paper introduces a new dataset and compares several methods for the recommendation of non-fiction audio visual material, namely lectures from the TED website. The TED dataset contains 1,149 talks and 69,023 profiles of users, who have made more than 100,000 ratings and 200,000 comments. The corresponding metadata, which we make available, can be used for training and testing generic or personalized recommender systems. We define content-based, collaborative, and combined recommendation methods for TED lectures and use cross-validation to select the best parameters of keyword-based (TFIDF) and semantic vector space-based methods (LSI, LDA, RP, and ESA). We compare these methods on a personalized recommendation task in two settings, a cold-start and a non-cold-start one. In the cold-start setting, semantic vector spaces perform better than keywords. In the non-cold-start setting, where collaborative information can be exploited, content-based methods are outperformed by collaborative filtering ones, but the proposed combined method shows acceptable performances, and can be used in both settings. For the generic recommendation task, LSI and RP again outperform TF-IDF.

Collaborative recommender systems allow personalization for e-commerce by exploiting similarities and dissimilarities among customers' preferences. We investigate the use of association rule mining as an underlying technology for collaborative recommender systems. Association rules have been used with success in other domains. However, most currently existing association rule mining algorithms were designed with market basket analysis in mind. Such algorithms are inefficient for collaborative recommendation because they mine many rules that are not relevant to a given user. Also, it is necessary to specify the minimum support of the mined rules in advance, often leading to either too many or too few rules; this negatively impacts the performance of the overall system. We describe a collaborative recommendation technique based on a new algorithm specifically designed to mine association rules for this purpose. Our algorithm does not require the minimum support to be specified in advance. Rather, a target range is given for the number of rules, and the algorithm adjusts the minimum support for each user in order to obtain a ruleset whose size is in the desired range. Rules are mined for a specific target user, reducing the time required for the mining process. We employ associations between users as well as associations between items in making recommendations. Experimental evaluation of a system based on our algorithm reveals performance that is significantly better than that of traditional correlation-based approaches.

Critiquing-based recommender systems elicit users’ feedback, called critiques, which they made on the recommended items. This conversational style of interaction is in contract to the standard model where users receive recommendations in a single interaction. Through the use of the critiquing feedback, the recommender systems are able to more accurately learn the users’ profiles, and therefore suggest better recommendations in the subsequent rounds. Critiquing-based recommenders have been widely studied in knowledge-, content-, and preference-based recommenders and are beginning to be tried in several online websites, such as MovieLens. This article examines the motivation and development of the subject area, and offers a detailed survey of the state of the art concerning the design of critiquing interfaces and development of algorithms for critiquing generation. With the help of categorization analysis, the survey reveals three principal branches of critiquing based recommender systems, using respectively natural language based, system-suggested, and user-initiated critiques. Representative example systems will be presented and analyzed for each branch, and their respective pros and cons will be discussed. Subsequently, a hybrid framework is developed to unify the advantages of different methods and overcome their respective limitations. Empirical findings from user studies are further presented, indicating how hybrid critiquing supports could effectively enable end-users to achieve more confident decisions. Finally, the article will point out several future trends to boost the advance of critiquing-based recommenders.

Representing relationships as translations in vector space lives at the heart of many neural embedding models such as word embeddings and knowledge graph embeddings. In this work, we study the connections of this translational principle with collaborative ltering algorithms. We propose Translational Recommender Networks (TransRec), a new aentive neural architecture that utilizes the translational principle to model the relationships between user and item pairs. Our model employs a neural aention mechanism over a Latent Relational Aentive Memory (LRAM) module to learn the latent relations between user-item pairs that best explains the interaction. By exploiting adaptive user-item specic translations in vector space, our model also alleviates the geometric inexibility problem of other metric learning algorithms while enabling greater modeling capability and ne-grained ing of users and items in vector space. e proposed architecture not only demonstrates the state-of-the-art performance across multiple recommendation benchmarks but also boasts of improved interpretability. alitative studies over the LRAM module shows evidence that our proposed model is able to infer and encode explicit sentiment, temporal and aribute information despite being only trained on implicit feedback. As such, this ascertains the ability of TransRec to uncover hidden relational structure within implicit datasets.

A mixture model for preferences data, which adequately represents the composite nature of the elicitation mechanism in ranking processes, is proposed. Both probabilistic features of the mixture distribution and inferential and computational issues arising from the maximum likelihood parameters estimation are addressed. Moreover, empirical evidence from different data sets confirming the goodness of fit of the proposed model to many real preferences data is shown. © 2004 Elsevier B.V. All rights reserved.

X iv :1 40 3. 57 53 v1 [ cs .A I] 2 3 M ar 2 01 4 D-CFPR: D numbers extended consistent fuzzy preference relations Xinyang Deng, Felix T.S. Chan, Rehan Sadiq, Sankaran Mahadevan, Yong Deng School of Computer and Information Science, Southwest University, Chongqing, 400715, China Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, China School of Engineering, University of British Columbia, 1137 Alumni Ave, Kelowna, BC V1V 1V7, Canada School of Engineering, Vanderbilt University, Nashville, TN, 37235, USA

Recommendation systems are widely used to recommend products to the end users that are most appropriate. Online book selling websites now-a-days are competing with each other by many means. Recommendation system is one of the stronger tools to increase profit and retaining buyer. The book recommendation system must recommend books that are of buyer's interest. This paper presents book recommendation system based on combined features of content filtering, collaborative filtering and association rule mining.

Point-of-interest (POI) recommendation that suggests new places for users to visit arises with the popularity of location-based social networks (LBSNs). Due to the importance of POI recommendation in LBSNs, it has attracted much academic and industrial interest. In this paper, we offer a systematic review of this field, summarizing the contributions of individual efforts and exploring their relations. We discuss the new properties and challenges in POI recommendation, compared with traditional recommendation problems, e.g., movie recommendation. Then, we present a comprehensive review in three aspects: influential factors for POI recommendation, methodologies employed for POI recommendation, and different tasks in POI recommendation. Specifically, we propose three taxonomies to classify POI recommendation systems. First, we categorize the systems by the influential factors check-in characteristics, including the geographical information, social relationship, temporal influence, and content indications. Second, we categorize the systems by the methodology, including systems modeled by fused methods and joint methods. Third, we categorize the systems as general POI recommendation and successive POI recommendation by subtle differences in the recommendation task whether to be bias to the recent check-in. For each category, we summarize the contributions and system features, and highlight the representative work. Moreover, we discuss the available data sets and the popular metrics. Finally, we point out the possible future directions in this area and conclude this survey.

Providing model-generated explanations in recommender systems is important to user experience. State-of-the-art recommendation algorithms — especially the collaborative filtering (CF)-based approaches with shallow or deep models — usually work with various unstructured information sources for recommendation, such as textual reviews, visual images, and various implicit or explicit feedbacks. Though structured knowledge bases were considered in content-based approaches, they have been largely ignored recently due to the availability of vast amounts of data and the learning power of many complex models. However, structured knowledge bases exhibit unique advantages in personalized recommendation systems. When the explicit knowledge about users and items is considered for recommendation, the system could provide highly customized recommendations based on users’ historical behaviors and the knowledge is helpful for providing informed explanations regarding the recommended items. A great challenge for using knowledge bases for recommendation is how to integrate large-scale structured and unstructured data, while taking advantage of collaborative filtering for highly accurate performance. Recent achievements in knowledge-base embedding (KBE) sheds light on this problem, which makes it possible to learn user and item representations while preserving the structure of their relationship with external knowledge for explanation. In this work, we propose to explain knowledge-base embeddings for explainable recommendation. Specifically, we propose a knowledge-base representation learning framework to embed heterogeneous entities for recommendation, and based on the embedded knowledge base, a soft matching algorithm is proposed to generate personalized explanations for the recommended items. Experimental results on real-world e-commerce datasets verified the superior recommendation performance and the explainability power of our approach compared with state-of-the-art baselines.

Many businesses are using recommender systems for marketing outreach. Recommendation algorithms can be either based on content or driven by collaborative filtering. We study different ways to incorporate content information directly into the matrix factorization approach of collaborative filtering. These content-boosted matrix factorization algorithms not only improve recommendation accuracy, but also provide useful insights about the contents, as well as make recommendations more easily interpretable.


Question: Applying machine learning techniques to enhance parallel processing in large-scale data applications.

This paper introduces MapReduce, a pioneering programming model designed for the efficient processing and generation of extensive datasets. Its versatility enables it to handle a wide range of practical tasks. Users define computations using map and reduce functions, while the underlying runtime system automates the parallelization process across large-scale clusters, manages hardware failures, and optimizes inter-machine communication to maximize network and disk efficiency. The simplicity of the system for programmers is a significant advantage; at Google, over ten thousand unique MapReduce programs have been developed over the past four years. Currently, an average of one hundred thousand MapReduce jobs are executed daily on Google's clusters, processing more than twenty petabytes of data per day. This showcases the model's effectiveness in optimizing parallelism in big data applications, aligning well with contemporary machine learning techniques for data processing.

In recent years, deep neural networks (DNN) have demonstrated significant business impact in large scale analysis and classification tasks such as speech recognition, visual object detection, pattern extraction, etc. Training of large DNNs, however, is universally considered as time consuming and computationally intensive task that demands datacenter-scale computational resources recruited for many days. Here we propose a concept of resistive processing unit (RPU) devices that can potentially accelerate DNN training by orders of magnitude while using much less power. The proposed RPU device can store and update the weight values locally thus minimizing data movement during training and allowing to fully exploit the locality and the parallelism of the training algorithm. We evaluate the effect of various RPU device features/non-idealities and system parameters on performance in order to derive the device and system level specifications for implementation of an accelerator chip for DNN training in a realistic CMOS-compatible technology. For large DNNs with about 1 billion weights this massively parallel RPU architecture can achieve acceleration factors of 30, 000 × compared to state-of-the-art microprocessors while providing power efficiency of 84, 000 GigaOps∕s∕W. Problems that currently require days of training on a datacenter-size cluster with thousands of machines can be addressed within hours on a single RPU accelerator. A system consisting of a cluster of RPU accelerators will be able to tackle Big Data problems with trillions of parameters that is impossible to address today like, for example, natural speech recognition and translation between all world languages, real-time analytics on large streams of business and scientific data, integration, and analysis of multimodal sensory data flows from a massive number of IoT (Internet of Things) sensors.

Current trends in parallel processors call for the design of efficient massively parallel algorithms for scientific computi ng. Parallel algorithms for Monte Carlo simulations of thermodynamic en s mbles of particles have received little attention becaus e of the inherent serial nature of the statistical sampling. In this paper, we present a massively parallel method that obeys det ailed balance and implement it for a system of hard disks on the GPU. We repro duce results of serial high-precision Monte Carlo runs to ve rify the method. This is a good test case because the hard disk equa tion of state over the range where the liquid transforms into the solid is particularly sensitive to small deviations away from the balance conditions. On a Tesla K20, our GPU implementation e xecutes over one billion trial moves per second, which is 148 times fa ster than on a single Intel Xeon E5540 CPU core, enables 27 tim es better performance per dollar, and cuts energy usage by a fac tor of 13. With this improved performance we are able to calcu l te the equation of state for systems of up to one million hard dis ks. These large system sizes are required in order to probe th e nature of the melting transition, which has been debated for the las t forty years. In this paper we present the details of our comp utational method, and discuss the thermodynamics of hard disks separa t ly in a companion paper.

algorithms, utilize techniques from optimization, statistics, and functional analysis to achieve maximal generality, flexibility, and performance. These algorithms are different from earlier techniques used in machine learning in many respects: For example, they are explicitly based on a theoretical model of learning rather than on loose analogies with natural learning systems or other heuristics. They come with theoretical guarantees about their performance and have a modular design that makes it possible to separately implement and analyze their components. They are not affected by the problem of local minima because their training amounts to convex optimization. In the last decade, a sizable community of theoreticians and practitioners has formed around these methods, and a number of practical applications have been realized. Although the research is not concluded, already now kernel methods are considered the state of the art in several machine learning tasks. Their ease of use, theoretical appeal, and remarkable performance have made them the system of choice for many learning problems. Successful applications range from text categorization to handwriting recognition to classification of geneexpression data.

One of the main sources of sparse matrices is the discretization of partial differential equations that govern continuumphysics phenomena such as fluid flow and transport, phase separation, mechanical deformation, electromagnetic wave propagation, and others. Recent advances in high-performance computing area have been enabling researchers to tackle increasingly larger problems leading to sparse linear systems with hundreds of millions to a few tens of billions of unknowns, e.g., [5, 6]. Iterative linear solvers are popular in large-scale computing as they consume less memory than direct solvers. Contrary to direct linear solvers, iterative solvers approach the solution gradually requiring the computation of sparse matrixvector (SpMV) products. The evaluation of SpMV products can emerge as a bottleneck for computational performance within the context of the simulation of large problems. In this work, we focus on a linear system arising from the discretization of the Cahn–Hilliard equation, which is a fourth order nonlinear parabolic partial differential equation that governs the separation of a two-component mixture into phases [3]. The underlying spatial discretization is performed using the discontinuous Galerkin method (e.g. [10]) and Newton’s method. A number of parallel algorithms and strategies have been evaluated in this work to accelerate the evaluation of SpMV products.

Data-driven applications are required to adapt to the everincreasing volume, velocity and veracity of data generated by a variety of sources including the Web and Internet of Things devices. At the same time, an event-driven computational paradigm is emerging as the core of modern systems designed for both database queries, data analytics and on-demand applications. MapReduce has been generalized to Map Collective and shown to be very effective in machine learning. However one often uses a dataflow computing model, which has been adopted by most major big data processing runtimes. The HPC community has also developed several asynchronous many tasks (AMT) systems according to the dataflow model. From a different point of view, the services community is moving to an increasingly event-driven model where (micro)services are composed of small functions driven by events in the form of Function as a Service(Faas) and serverless computing. Such designs allow the applications to scale quickly as well as be cost effective in cloud environments. An event-driven runtime designed for data processing consists of well-understood components such as communication, scheduling, and fault tolerance. One can make different design decisions for these components that will determine the type of applications a system can support efficiently. We find that modern systems are designed in a monolithic approach with a fixed set of choices that cannot be changed easily afterwards. Because of these design choices their functionality is limited to specific sets of applications. In this paper we study existing systems (candidate event-driven runtimes), the design choices they have made for each component, and how this affects the type of applications they can support. Further we propose a loosely coupled component-based approach for designing a big data toolkit where each component can have different implementations to support various applications. We believe such a polymorphic design would allow services and data analytics to be integrated seamlessly and expand from edge to cloud to high performance computing environments.

As datasets grow and analytic algorithms become more complex, the typical workflow of analysts launching an analytic, waiting for it to complete, inspecting the results, and then re-Iaunching the computation with adjusted parameters is not realistic for many real-world tasks. This paper presents an alternative workflow, progressive visual analytics, which enables an analyst to inspect partial results of an algorithm as they become available and interact with the algorithm to prioritize subspaces of interest. Progressive visual analytics depends on adapting analytical algorithms to produce meaningful partial results and enable analyst intervention without sacrificing computational speed. The paradigm also depends on adapting information visualization techniques to incorporate the constantly refining results without overwhelming analysts and provide interactions to support an analyst directing the analytic. The contributions of this paper include: a description of the progressive visual analytics paradigm; design goals for both the algorithms and visualizations in progressive visual analytics systems; an example progressive visual analytics system (Progressive Insights) for analyzing common patterns in a collection of event sequences; and an evaluation of Progressive Insights and the progressive visual analytics paradigm by clinical researchers analyzing electronic medical records.

What’s more, Big Data Analytics with Spark provides an introduction to other big data technologies that are commonly used along with Spark, such as HDFS, Avro, Parquet, Ka a, Cassandra, HBase, Mesos, and so on. It also provides an introduction to machine learning and graph concepts. So the book is self-suffi cient; all the technologies that you need to know to use Spark are covered. The only thing that you are expected to have is some programming knowledge in any language.

Parallel coordinate plots (PCPs) are commonly used in information visualization to provide insight into multi-variate data. These plots help to spot correlations between variables. PCPs have been successfully applied to unstructured datasets up to a few millions of points. In this paper, we present techniques to enhance the usability of PCPs for the exploration of large, multi-timepoint volumetric data sets, containing tens of millions of points per timestep. The main difficulties that arise when applying PCPs to large numbers of data points are visual clutter and slow performance, making interactive exploration infeasible. Moreover, the spatial context of the volumetric data is usually lost. We describe techniques for preprocessing using data quantization and compression, and for fast GPU-based rendering of PCPs using joint density distributions for each pair of consecutive variables, resulting in a smooth, continuous visualization. Also, fast brushing techniques are proposed for interactive data selection in multiple linked views, including a 3D spatial volume view. These techniques have been successfully applied to three large data sets: Hurricane Isabel (Vis'04 contest), the ionization front instability data set (Vis'08 design contest), and data from a large-eddy simulation of cumulus clouds. With these data, we show how PCPs can be extended to successfully visualize and interactively explore multi-timepoint volumetric datasets with an order of magnitude more data points.

Physical problems offer scope for macro level parallelization of solution by their essential structure. For parallelization of electrical network simulation, the most natural structure based method is that of multiport decomposition. In this paper this method is used for the simulation of electrical networks consisting of resistances, voltage and current sources using a distributed cluster of weakly coupled processors. At the two levels in which equations are solved in this method the authors have used sparse LU for both levels in the first scheme and sparse LU in the inner level and conjugate gradient in the outer level in the second scheme. Results are presented for planar networks, for the cases where the numbers of slave processors are 1 and 2, and for circuit sizes up to 8.2 million nodes and 16.4 million edges using 8 slave processors. The authors use a cluster of Pentium IV processors linked through a 10/100MBPS Ethernet switch

Exploring data requires a fast feedback loop from the analyst to the system, with a latency below about 10 seconds because of human cognitive limitations. When data becomes large or analysis becomes complex, sequential computations can no longer be completed in a few seconds and data exploration is severely hampered. This article describes a novel computation paradigm called Progressive Computation for Data Analysis or more concisely Progressive Analytics, that brings at the programming language level a low-latency guarantee by performing computations in a progressive fashion. Moving this progressive computation at the language level relieves the programmer of exploratory data analysis systems from implementing the whole analytics pipeline in a progressive way from scratch, streamlining the implementation of scalable exploratory data analysis systems. This article describes the new paradigm through a prototype implementation called ProgressiVis, and explains the requirements it implies through examples.


Question: Applying machine learning techniques to enhance parallelism in large-scale data applications.

This paper introduces Resilient Distributed Datasets (RDDs), a novel fault-tolerant abstraction designed for in-memory cluster computing, aimed at enhancing the efficiency of large-scale data processing while maintaining the robustness of data flow models akin to MapReduce. RDDs are particularly beneficial for two categories of applications that traditional data flow systems struggle to handle effectively: iterative algorithms, which are prevalent in graph analytics and machine learning tasks, and interactive data mining tools. For these scenarios, in-memory computation can significantly boost performance, often by a factor of ten. To ensure fault tolerance without compromising efficiency, RDDs enforce a strict shared memory model: they are immutable datasets that can only be created through bulk operations on other RDDs. Despite these constraints, RDDs prove to be versatile, capable of encompassing a broad range of computations, including MapReduce and specialized frameworks for iterative processing like Pregel. Our implementation demonstrates superior performance compared to Hadoop, achieving up to 20 times faster execution for iterative tasks and enabling interactive queries on terabyte-scale datasets with response times as low as 5-7 seconds. This showcases the potential of RDDs in optimizing parallelism in big data applications, particularly in the realm of machine learning and interactive data analysis.

The performance of parallel scientific applications depends on many factors which are determined by the execution environment and the parallel application. Especially on large parallel systems, it is too expensive to explore the solution space with series of experiments. Deriving analytical models for applications and platforms allow estimating and extrapolating their execution performance, bottlenecks, and the potential impact of optimization options. We propose to use such "performance modeling" techniques beginning from the application design process throughout the whole software development cycle and also during the lifetime of supercomputer systems. Such models help to guide supercomputer system design and re-engineering efforts to adopt applications to changing platforms and allow users to estimate costs to solve a particular problem. Models can often be built with the help of well-known performance profiling tools. We discuss how we successfully used modeling throughout the proposal, initial testing, and beginning deployment phase of the Blue Waters supercomputer system.

Massive data analysis on large clusters presents new opportunities and challenges for query optimization. Data partitioning is crucial to performance in this environment. However, data repartitioning is a very expensive operation so minimizing the number of such operations can yield very significant performance improvements. A query optimizer for this environment must therefore be able to reason about data partitioning including its interaction with sorting and grouping. SCOPE is a SQL-like scripting language used at Microsoft for massive data analysis. A transformation-based optimizer is responsible for converting scripts into efficient execution plans for the Cosmos distributed computing platform. In this paper, we describe how reasoning about data partitioning is incorporated into the SCOPE optimizer. We show how relational operators affect partitioning, sorting and grouping properties and describe how the optimizer reasons about and exploits such properties to avoid unnecessary operations. In most optimizers, consideration of parallel plans is an afterthought done in a postprocessing step. Reasoning about partitioning enables the SCOPE optimizer to fully integrate consideration of parallel, serial and mixed plans into the cost-based optimization. The benefits are illustrated by showing the variety of plans enabled by our approach.

Hyracks is a new partitioned-parallel software platform designed to run data-intensive computations on large shared-nothing clusters of computers. Hyracks allows users to express a computation as a DAG of data operators and connectors. Operators operate on partitions of input data and produce partitions of output data, while connectors repartition operators' outputs to make the newly produced partitions available at the consuming operators. We describe the Hyracks end user model, for authors of dataflow jobs, and the extension model for users who wish to augment Hyracks' built-in library with new operator and/or connector types. We also describe our initial Hyracks implementation. Since Hyracks is in roughly the same space as the open source Hadoop platform, we compare Hyracks with Hadoop experimentally for several different kinds of use cases. The initial results demonstrate that Hyracks has significant promise as a next-generation platform for data-intensive applications.

5 Nowadays computers are in the middle of most economic transactions. 6 These “computer-mediated transactions” generate huge amounts of 7 data, and new tools can be used to manipulate and analyze this data. 8 This essay offers a brief introduction to some of these tools and meth9 ods. 10 Computers are now involved in many economic transactions and can cap11 ture data associated with these transactions, which can then be manipulated 12 and analyzed. Conventional statistical and econometric techniques such as 13 regression often work well but there are issues unique to big data sets that 14 may require different tools. 15 First, the sheer size of the data involved may require more powerful data 16 manipulation tools. Second, we may have more potential predictors than 17 appropriate for estimation, so we need to do some kind of variable selection. 18 Third, large data sets may allow for more flexible relationships than simple 19 ∗Hal Varian is Chief Economist, Google Inc., Mountain View, California, and Emeritus Professor of Economics, University of California, Berkeley, California. Thanks to Jeffrey Oldham, Tom Zhang, Rob On, Pierre Grinspan, Jerry Friedman, Art Owen, Steve Scott, Bo Cowgill, Brock Noland, Daniel Stonehill, Robert Snedegar, Gary King, the editors of this journal for comments on earlier versions of this paper. 1 linear models. Machine learning techniques such as decision trees, support 20 vector machines, neural nets, deep learning and so on may allow for more 21 effective ways to model complex relationships. 22 In this essay I will describe a few of these tools for manipulating and an23 alyzing big data. I believe that these methods have a lot to offer and should 24 be more widely known and used by economists. In fact, my standard advice 25 to graduate students these days is “go to the computer science department 26 and take a class in machine learning.” There have been very fruitful collabo27 rations between computer scientists and statisticians in the last decade or so, 28 and I expect collaborations between computer scientists and econometricians 29 will also be productive in the future. 30 1 Tools to manipulate big data 31 Economists have historically dealt with data that fits in a spreadsheet, but 32 that is changing as new more detailed data becomes available; see Einav 33 and Levin [2013] for several examples and discussion. If you have more 34 than a million or so rows in a spreadsheet, you probably want to store it in a 35 relational database, such as MySQL. Relational databases offer a flexible way 36 to store, manipulate and retrieve data using a Structured Query Language 37 (SQL) which is easy to learn and very useful for dealing with medium-sized 38 data sets. 39 However, if you have several gigabytes of data or several million observa40 tions, standard relational databases become unwieldy. Databases to manage 41 data of this size are generically known as “NoSQL” databases. The term is 42 used rather loosely, but is sometimes interpreted as meaning “not only SQL.” 43 NoSQL databases are more primitive than SQL databases in terms of data 44 manipulation capabilities but can handle larger amounts of data. 45 Due to the rise of computer mediated transactions, many companies have 46 found it necessary to develop systems to process billions of transactions per 47

In the process of knowledge discovery and representation in large datasets using formal concept analysis, complexity plays a major role in identifying all the formal concepts and constructing the concept lattice(digraph of the concepts). For identifying the formal concepts and constructing the digraph from the identified concepts in very large datasets, various distributed algorithms are available in the literature. However, the existing distributed algorithms are not very well suitable for concept generation because it is an iterative process. The existing algorithms are implemented using distributed frameworks like MapReduce and Open MP, these frameworks are not appropriate for iterative applications. Hence, in this paper we proposed efficient distributed algorithms for both formal concept generation and concept lattice digraph construction in large formal contexts using Apache Spark. Various performance metrics are considered for the evaluation of the proposed work, the results of the evaluation proves that the proposed algorithms are efficient for concept generation and lattice graph construction in comparison with the existing algorithms.

The development of data-mining applications such as classification and clustering has shown the need for machine learning algorithms to be applied to large scale data. In this paper we present the comparison of different classification techniques using Waikato Environment for Knowledge Analysis or in short, WEKA. WEKA is an open source software which consists of a collection of machine learning algorithms for data mining tasks. The aim of this paper is to investigate the performance of different classification or clustering methods for a set of large data. The algorithm or methods tested are Bayes Network, Radial Basis Function, Pruned Tree, Single Conjunctive Rule Learner and Nearest Neighbors Algorithm. A fundamental review on the selected technique is presented for introduction purposes. The data breast cancer data with a total data of 6291 and a dimension of 699 rows and 9 columns will be used to test and justify the differences between the classification methods or algorithms. Subsequently, the classification technique that has the potential to significantly improve the common or conventional methods will be suggested for use in large scale data, bioinformatics or other general applications. Keywords— Machine Learning, Data Mining, WEKA, Classification, Bioinformatics.

As applications such as Apple Siri, Google Now, Microsoft Cortana, and Amazon Echo continue to gain traction, web-service companies are adopting large deep neural networks (DNN) for machine learning challenges such as image processing, speech recognition, natural language processing, among others. A number of open questions arise as to the design of a server platform specialized for DNN and how modern warehouse scale computers (WSCs) should be outfitted to provide DNN as a service for these applications.
 In this paper, we present DjiNN, an open infrastructure for DNN as a service in WSCs, and Tonic Suite, a suite of 7 end-to-end applications that span image, speech, and language processing. We use DjiNN to design a high throughput DNN system based on massive GPU server designs and provide insights as to the varying characteristics across applications. After studying the throughput, bandwidth, and power properties of DjiNN and Tonic Suite, we investigate several design points for future WSC architectures. We investigate the total cost of ownership implications of having a WSC with a disaggregated GPU pool versus a WSC composed of homogeneous integrated GPU servers. We improve DNN throughput by over 120x for all but one application (40x for Facial Recognition) on an NVIDIA K40 GPU. On a GPU server composed of 8 NVIDIA K40s, we achieve near-linear scaling (around 1000x throughput improvement) for 3 of the 7 applications. Through our analysis, we also find that GPU-enabled WSCs improve total cost of ownership over CPU-only designs by 4-20x, depending on the composition of the workload

We present Resilient Distributed Datasets (RDDs), a distributed memory abstraction that allows programmers to perform in-memory computations on large clusters while retaining the fault tolerance of data flow models like MapReduce. RDDs are motivated by two types of applications that current data flow systems handle inefficiently: iterative algorithms, which are common in graph applications and machine learning, and interactive data mining tools. In both cases, keeping data in memory can improve performance by an order of magnitude. To achieve fault tolerance efficiently, RDDs provide a highly restricted form of shared memory: they are read-only datasets that can only be constructed through bulk operations on other RDDs. However, we show that RDDs are expressive enough to capture a wide class of computations, including MapReduce and specialized programming models for iterative jobs such as Pregel. Our implementation of RDDs can outperform Hadoop by 20× for iterative jobs and can be used interactively to search a 1 TB dataset with latencies of 5–7 seconds.

The management and analysis of big data has been identified as one of the most important emerging needs in recent years. This is because of the sheer volume and increasing complexity of data being created or collected. Current clustering algorithms can not handle big data, and therefore, scalable solutions are necessary. Since fuzzy clustering algorithms have shown to outperform hard clustering approaches in terms of accuracy, this paper investigates the parallelization and scalability of a common and effective fuzzy clustering algorithm named Fuzzy C-Means (FCM) algorithm. The algorithm is parallelized using the MapReduce paradigm outlining how the Map and Reduce primitives are implemented. A validity analysis is conducted in order to show that the implementation works correctly achieving competitive purity results compared to state-of-the art clustering algorithms. Furthermore, a scalability analysis is conducted to demonstrate the performance of the parallel FCM implementation with increasing number of computing nodes used.

This paper documents the development of ml.lib: a set of opensource tools designed for employing a wide range of machine learning techniques within two popular real-time programming environments, namely Max and Pure Data. ml.lib is a crossplatform, lightweight wrapper around Nick Gillian’s Gesture Recognition Toolkit, a C++ library that includes a wide range of data processing and machine learning techniques. ml.lib adapts these techniques for real-time use within popular dataflow IDEs, allowing instrument designers and performers to integrate robust learning, classification and mapping approaches within their existing workflows. ml.lib has been carefully designed to allow users to experiment with and incorporate machine learning techniques within an interactive arts context with minimal prior knowledge. A simple, logical and consistent, scalable interface has been provided across over sixteen externals in order to maximize learnability and discoverability. A focus on portability and maintainability has enabled ml.lib to support a range of computing architectures—including ARM— and operating systems such as Mac OS, GNU/Linux and Windows, making it the most comprehensive machine learning implementation available for Max and Pure Data.


Question: Applying machine learning techniques to enhance parallelism in large-scale data processing applications.

This study delves into the comparative analysis of cross-validation and bootstrap techniques for accuracy estimation and model selection, with a focus on their application in real-world datasets. Recent experiments on artificial data and theoretical findings in controlled environments have suggested that for the purpose of selecting an optimal classifier from a pool of options, model selection using ten-fold cross-validation might outperform the computationally more intensive leave-one-out cross-validation method. To further investigate the impact of different parameters on these algorithms, we conducted a comprehensive experiment involving over half a million runs of the C algorithm and Naive Bayes. In our examination of cross-validation, we explored the influence of varying the number of folds and whether stratification was applied, while for bootstrap, we altered the number of bootstrap samples. Our findings revealed that, for datasets akin to those used in our study, the most effective method for model selection is ten-fold stratified cross-validation, regardless of the availability of computational resources that could accommodate additional folds. This insight underscores the practical utility of ten-fold stratified cross-validation in achieving accurate model selection in real-world applications.

Deep learning thrives with large neural networks and large datasets. However, larger networks and larger datasets result in longer training times that impede research and development progress. Distributed synchronous SGD offers a potential solution to this problem by dividing SGD minibatches over a pool of parallel workers. Yet to make this scheme efficient, the per-worker workload must be large, which implies nontrivial growth in the SGD minibatch size. In this paper, we empirically show that on the ImageNet dataset large minibatches cause optimization difficulties, but when these are addressed the trained networks exhibit good generalization. Specifically, we show no loss of accuracy when training with large minibatch sizes up to 8192 images. To achieve this result, we adopt a linear scaling rule for adjusting learning rates as a function of minibatch size and develop a new warmup scheme that overcomes optimization challenges early in training. With these simple techniques, our Caffe2-based system trains ResNet50 with a minibatch size of 8192 on 256 GPUs in one hour, while matching small minibatch accuracy. Using commodity hardware, our implementation achieves ∼90% scaling efficiency when moving from 8 to 256 GPUs. This system enables us to train visual recognition models on internetscale data with high efficiency.

For the past 5 years, the ILSVRC competition and the ImageNet dataset have attracted a lot of interest from the Computer Vision community, allowing for state-of-the-art accuracy to grow tremendously. This should be credited to the use of deep artificial neural network designs. As these became more complex, the storage, bandwidth, and compute requirements increased. This means that with a non-distributed approach, even when using the most high-density server available, the training process may take weeks, making it prohibitive. Furthermore, as datasets grow, the representation learning potential of deep networks grows as well by using more complex models. This synchronicity triggers a sharp increase in the computational requirements and motivates us to explore the scaling behaviour on petaflop scale supercomputers. In this paper we will describe the challenges and novel solutions needed in order to train ResNet50 in this large scale environment. We demonstrate above 90% scaling efficiency and a training time of 28 minutes using up to 104K x86 cores. This is supported by software tools from Intel’s ecosystem. Moreover, we show that with regular 90 120 epoch train runs we can achieve a top-1 accuracy as high as 77% for the unmodified ResNet-50 topology. We also introduce the novel Collapsed Ensemble (CE) technique that allows us to obtain a 77.5% top-1 accuracy, similar to that of a ResNet-152, while training a unmodified ResNet-50 topology for the same fixed training budget. All ResNet-50 models as well as the scripts needed to replicate them will be posted shortly. Keywords—deep learning, scaling, convergence, large minibatch, ensembles.

The performance of parallel scientific applications depends on many factors which are determined by the execution environment and the parallel application. Especially on large parallel systems, it is too expensive to explore the solution space with series of experiments. Deriving analytical models for applications and platforms allow estimating and extrapolating their execution performance, bottlenecks, and the potential impact of optimization options. We propose to use such "performance modeling" techniques beginning from the application design process throughout the whole software development cycle and also during the lifetime of supercomputer systems. Such models help to guide supercomputer system design and re-engineering efforts to adopt applications to changing platforms and allow users to estimate costs to solve a particular problem. Models can often be built with the help of well-known performance profiling tools. We discuss how we successfully used modeling throughout the proposal, initial testing, and beginning deployment phase of the Blue Waters supercomputer system.

The emerging discipline of algorithm engineering has primarily focussed on transforming pencil-and-paper sequential algorithms into robust, efficient, well tested, and easily used implementations. As parallel computing becomes ubiquitous, we need to extend algorithm engineering techniques to parallel computation. Such an extension adds significant complications. After a short review of algorithm engineering achievements for sequential computing, we review the various complications caused by parallel computing, present some examples of successful efforts, and give a personal view of possible future research.

A study was made in those region that expose major area of machine learning techniques which available in data mining. There are lots of Research work is analysized in agriculture field. It is a smart techniques is needed to find the resolution over the modern method. Cotton is an significant beneficial harvest and extensively traded commodity across the world. Its yield is insightful to weather, soil as well as organization practices. This study presents a concise idea of several of the broadly capitalized machine learning technique over agricultural domain. Among the algorithms like Navie bayes, j48 ,MLP,Randomforest,Random tree, I have founded J48 gives better result over the other algorithms using cotton seed quality accuracy is best.

With the advent of the multicore era, it is clear that future growth in application performance will primarily come from increased parallelism. We believe parallelism should be introduced early into the Computer Science curriculum to educate students on the fundamentals of parallel computation. In this paper, we introduce the newly-created Habanero-Java library (HJlib), a pure Java 8 library implementation of the pedagogic parallel programming model [12]. HJlib has been used in teaching a sophomore-level course titled "Fundamentals of Parallel Programming" at Rice University.
 HJlib adds to the Java ecosystem a powerful and portable task parallel programming model that can be used to parallelize both regular and irregular applications. By relying on simple orthogonal parallel constructs with important safety properties, HJlib allows programmers with a basic knowledge of Java to get started with parallel programming concepts by writing or refactoring applications to harness the power of multicore architecture. The HJlib APIs make extensive use of lambda expressions and can run on any Java 8 JVM. HJlib runtime feedback capabilities, such as the abstract execution metrics and the deadlock detector, help the programmer to obtain feedback on theoretical performance as well as the presence of potential bugs in their program.
 Being an implementation of a pedagogic programming model, HJlib is also an attractive tool for both educators and researchers. HJlib is actively being used in multiple research projects at Rice and also by external independent collaborators. These projects include exploitation of homogeneous and heterogeneous multicore parallelism in big data applications written for the Hadoop platform [20, 43].

The fundamental feature of an OpenFlow network is that the controller is responsible for the initial establishment of every flow by contacting related switches. Thus the performance of the controller could be a bottleneck. This paper shows how this fundamental problem is addressed by parallelism. The state of the art OpenFlow controller, called NOX, achieves a simple programming model for control function development by having a single-threaded event-loop. Yet NOX has not considered exploiting parallelism. We propose Maestro which keeps the simple programming model for programmers, and exploits parallelism in every corner together with additional throughput optimization techniques. We experimentally show that the throughput of Maestro can achieve near linear scalability on an eight core server machine. Keywords-OpenFlow, network management, multithreading, performance optimization

Increasingly, organizations capture, transform and analyze enormous data sets. Prominent examples include internet companies and e-science. The Map-Reduce scalable dataflow paradigm has become popular for these applications. Its simple, explicit dataflow programming model is favored by some over the traditional high-level declarative approach: SQL. On the other hand, the extreme simplicity of MapReduce leads to much low-level hacking to deal with the many-step, branching dataflows that arise in practice. Moreover, users must repeatedly code standard operations such as join by hand. These practices waste time, introduce bugs, harm readability, and impede optimizations. Pig is a high-level dataflow system that aims at a sweet spot between SQL and Map-Reduce. Pig offers SQL-style high-level data manipulation constructs, which can be assembled in an explicit dataflow and interleaved with custom Mapand Reduce-style functions or executables. Pig programs are compiled into sequences of Map-Reduce jobs, and executed in the Hadoop Map-Reduce environment. Both Pig and Hadoop are open-source projects administered by the Apache Software Foundation. This paper describes the challenges we faced in developing Pig, and reports performance comparisons between Pig execution and raw Map-Reduce execution.

We present a new programming model for deterministic parallel computation in a pure functional language. The model is monadic and has explicit granularity, but allows dynamic construction of dataflow networks that are scheduled at runtime, while remaining deterministic and pure. The implementation is based on monadic concurrency, which has until now only been used to simulate concurrency in functional languages, rather than to provide parallelism. We present the API with its semantics, and argue that parallel execution is deterministic. Furthermore, we present a complete work-stealing scheduler implemented as a Haskell library, and we show that it performs at least as well as the existing parallel programming models in Haskell.

This paper describes the Pegasus framework that can be used to map complex scientific workflows onto distributed resources. Pegasus enables users to represent the workflows at an abstract level without needing to worry about the particulars of the target execution systems. The paper describes general issues in mapping applications and the functionality of Pegasus. We present the results of improving application performance through workflow restructuring which clusters multiple tasks in a workflow into single entities. A real-life astronomy application is used as the basis for the study.


Question: Applying machine learning techniques to enhance parallelism in large-scale data applications.

Introducing Paragon: A Scalable and QoS-aware Scheduler for Heterogeneous Datacenters
In the realm of large-scale datacenters (DCs), managing the coexistence of diverse applications on a daily basis poses significant challenges. Performance degradation due to workload interference and the complexity of aligning applications with suitable hardware platforms can compromise the quality of service (QoS) essential for many cloud-based workloads. Although prior research has acknowledged the implications of heterogeneity and interference, existing solutions are often computationally demanding, unsuitable for real-time application, and lack scalability for handling a multitude of applications.
We propose Paragon, a novel, scalable, and online scheduler for datacenters that is adept at handling heterogeneity and interference. Paragon's design is rooted in robust analytical methods, distinguishing itself by utilizing existing system information about previously encountered applications, rather than requiring detailed profiling of each application. It employs collaborative filtering techniques to swiftly and accurately categorize incoming, uncharted workloads in terms of their heterogeneity and interference across various shared resources. This categorization is achieved by identifying patterns that mirror previously scheduled applications. Armed with this classification, Paragon can then schedule applications in a way that curtails interference while optimizing server utilization. Remarkably, Paragon achieves this while maintaining minimal overheads in terms of time and state, even when scaling to manage tens of thousands of servers.
To gauge Paragon's efficacy, we subject it to a comprehensive evaluation encompassing a broad spectrum of workload scenarios on both small-scale and large-scale systems, including a deployment on 1,000 servers on Amazon's EC2 platform. In a scenario involving 2,500 workloads, Paragon successfully upholds performance guarantees for an impressive 91% of applications, concurrently boosting utilization. This stands in stark contrast to heterogeneity-oblivious, interference-oblivious, and least-loaded schedulers, which can only offer similar guarantees to 14%, 11%, and 3% of workloads, respectively. These disparities become even more pronounced in environments where resource efficiency is paramount, such as in oversubscribed scenarios.

Context: Big data has become the new buzzword in the information and communication technology industry. Researchers and major corporations are looking into big data applications to extract the maximum value from the data available to them. However, developing and maintaining stable and scalable big data applications is still a distant milestone. Objective: To look at existing research on how software engineering concepts, namely the phases of the software development project life cycle (SDPLC), can help build better big data application projects. Method: A literature survey was performed. A manual search covered papers returned by search engines resulting in approximately 2,000 papers being searched and 170 papers selected for review. Results: The search results helped in identifying data rich application projects that have the potential to utilize big data successfully. The review helped in exploring SDPLC phases in the context of big data applications and performing a gap analysis of the phases that have yet to see detailed research efforts but deserve attention.

Recent advances in computing have led to an explosion in the amount of data being generated. Processing the ever-growing data in a timely manner has made throughput computing an important aspect for emerging applications. Our analysis of a set of important throughput computing kernels shows that there is an ample amount of parallelism in these kernels which makes them suitable for today's multi-core CPUs and GPUs. In the past few years there have been many studies claiming GPUs deliver substantial speedups (between 10X and 1000X) over multi-core CPUs on these kernels. To understand where such large performance difference comes from, we perform a rigorous performance analysis and find that after applying optimizations appropriate for both CPUs and GPUs the performance gap between an Nvidia GTX280 processor and the Intel Core i7-960 processor narrows to only 2.5x on average. In this paper, we discuss optimization techniques for both CPU and GPU, analyze what architecture features contributed to performance differences between the two architectures, and recommend a set of architectural features which provide significant improvement in architectural efficiency for throughput kernels.

Mainstream parallel algorithms for mining frequent itemsets (patterns) were designed by implementing FP-Growth or Apriori algorithms on MapReduce (MR) framework. Existing MR FP-Growth algorithms can not distribute data equally among nodes, and MR Apriori algorithms utilize multiple map/reduce procedures and generate too many key-value pairs with value of 1; these disadvantages hinder their performance. This paper proposes an algorithm FIMMR: it firstly mines local frequent itemsets for each data chunk as candidates, applies prune strategies to the candidates, and then identifies global frequent itemsets from candidates. Experimental results show that the time efficiency of FIMMR outperforms PFP and SPC significantly; and under small minimum support threshold, FIMMR can achieve one order of magnitude improvement than the other two algorithms; meanwhile, the speedup of FIMMR is also satisfactory.

The use of large-scale data mining and machine learning has proliferated through the adoption of technologies such as Hadoop, with its simple programming semantics and rich and active ecosystem. This paper presents LinkedIn's Hadoop-based analytics stack, which allows data scientists and machine learning researchers to extract insights and build product features from massive amounts of data. In particular, we present our solutions to the ``last mile'' issues in providing a rich developer ecosystem. This includes easy ingress from and egress to online systems, and managing workflows as production processes. A key characteristic of our solution is that these distributed system concerns are completely abstracted away from researchers. For example, deploying data back into the online system is simply a 1-line Pig command that a data scientist can add to the end of their script. We also present case studies on how this ecosystem is used to solve problems ranging from recommendations to news feed updates to email digesting to descriptive analytical dashboards for our members.

Next generation analytics will be all about graphs, though performance has been a fundamental challenge for large scale graph processing. In this paper, we present an industrial graph processing engine for exploring various large scale linked data, which exhibits superior performance due to the several innovations. This engine organizes a graph as a set of edge-sets, compatible with the traditional edge-centric sharding for graphs, but becomes more amenable for large scale processing. Each time only a portion of the sets are needed for computation and the data access patterns can be highly predictable for prefetch for many graph computing algorithms. Due to the sparsity of large scale graph structure, this engine differentiates logical edge-sets from the edge-sets physically stored on the disk, where multiple logical edge-sets can be organized into a same physical edge-set to increase the data locality. Besides, in contrast to existing solution, the data structures utilized for the physical edge-sets can vary from one to another. Such heterogeneous edge-set representation explores the best graph processing performance according to local data access patterns. We conduct experiments on a representative set of property graphs on multiple platforms, where the proposed system outperform the baseline systems consistently.

Big data, which is maturing over time, has a high impact in analyzing scenarios and coming up with decision making strategies pertaining to any sector. Consequently the number of applications are also increasing where in the past, they were limited to the confines of company sales departments. In lieu, this paper is primarily focused on applying Data Sciences within the education sector. We will be discussing the overall application to address higher-education issues. We will also present a framework for capturing, analyzing it and concluding on data in order to benefit decision-makers, thus ultimately manifesting in informing design and finalizing policy.

Big Data, the analysis of large quantities of data to gain new insight has become a ubiquitous phrase in recent years. Day by day the data is growing at a staggering rate. One of the efficient technologies that deal with the Big Data is Hadoop, which will be discussed in this paper. Hadoop, for processing large data volume jobs uses MapReduce programming model. Hadoop makes use of different schedulers for executing the jobs in parallel. The default scheduler is FIFO (First In First Out) Scheduler. Other schedulers with priority, pre-emption and non-pre-emption options have also been developed. As the time has passed the MapReduce has reached few of its limitations. So in order to overcome the limitations of MapReduce, the next generation of MapReduce has been developed called as YARN (Yet Another Resource Negotiator). So, this paper provides a survey on Hadoop, few scheduling methods it uses and a brief introduction to YARN. Keywords—Hadoop, HDFS, MapReduce, Schedulers, YARN.

Many widely used, latency sensitive, data-parallel distributed systems, such as HDFS, Hive, and Spark choose to use the Java Virtual Machine (JVM), despite debate on the overhead of doing so. This paper analyzes the extent and causes of the JVM performance overhead in the above mentioned systems. Surprisingly, we find that the warm-up overhead, i.e., class loading and interpretation of bytecode, is frequently the bottleneck. For example, even an I/O intensive, 1GB read on HDFS spends 33% of its execution time in JVM warm-up, and Spark queries spend an average of 21 seconds in warm-up. The findings on JVM warm-up overhead reveal a contradiction between the principle of parallelization, i.e., speeding up long running jobs by parallelizing them into short tasks, and amortizing JVM warm-up overhead through long tasks. We solve this problem by designing HotTub, a new JVM that amortizes the warm-up overhead over the lifetime of a cluster node instead of over a single job by reusing a pool of already warm JVMs across multiple applications. The speed-up is significant. For example, using HotTub results in up to 1.8X speedups for Spark queries, despite not adhering to the JVM specification in edge cases.

We present Resilient Distributed Datasets (RDDs), a distributed memory abstraction that allows programmers to perform in-memory computations on large clusters while retaining the fault tolerance of data flow models like MapReduce. RDDs are motivated by two types of applications that current data flow systems handle inefficiently: iterative algorithms, which are common in graph applications and machine learning, and interactive data mining tools. In both cases, keeping data in memory can improve performance by an order of magnitude. To achieve fault tolerance efficiently, RDDs provide a highly restricted form of shared memory: they are read-only datasets that can only be constructed through bulk operations on other RDDs. However, we show that RDDs are expressive enough to capture a wide class of computations, including MapReduce and specialized programming models for iterative jobs such as Pregel. Our implementation of RDDs can outperform Hadoop by 20× for iterative jobs and can be used interactively to search a 1 TB dataset with latencies of 5–7 seconds.

In this paper, we characterize Google applications, based on a one-month Google trace with over 650k jobs running across over 12000 heterogeneous hosts from a Google data center. On one hand, we carefully compute the valuable statistics about task events and resource utilization for Google applications, based on various types of resources (such as CPU, memory) and execution types (e.g., whether they can run batch tasks or not). Resource utilization per application is observed with an extremely typical Pareto principle. On the other hand, we classify applications via a K-means clustering algorithm with optimized number of sets, based on task events and resource usage. The number of applications in the K-means clustering sets follows a Pareto-similar distribution. We believe our work is very interesting and valuable for the further investigation of Cloud environment.