Question: The argument for a visual discovery aide: a comprehensive resolution to expedite visual data exploration.

Voice-Activated Engagement: Unraveling the Human-Computer Bond through Dialogue
In the realm of human-computer interaction, spoken dialogue systems are emerging as a more intuitive and potent alternative to traditional graphic interfaces. "Voice-Activated Engagement" delves into the psychological and sociological underpinnings of voice synthesis and recognition, revealing that humans inherently perceive spoken dialogue systems as possessing human-like traits—a phenomenon rooted in our evolutionary history. While the book offers a wealth of fundamental research, its primary focus is on providing practical guidance for technologists and marketers looking to incorporate voice interfaces in commercial applications.

The narrative is structured around a series of straightforward experiments that highlight the profound impact of psychological and social factors on individuals' perceptions and behaviors when interacting with voice interfaces. Each chapter explores a specific hypothesis, presents an experiment to test it, and discusses the implications for the design of voice interfaces, covering topics such as gender, personality, accent, ethnicity, emotion, the use of multiple voices, system use of "I," voices synchronized with visual cues, mixed synthetic and recorded voices, context, and error handling in human-computer collaboration.

Despite its accessibility, particularly to readers without a scientific background, the book adopts an unconventional academic style, with all references and detailed information relegated to a notes section that comprises a third of the book's content. This approach may not fully satisfy either the scientific or practical audience. Scientists may find the arguments lacking in precision, the details sparse, and the frequent need to consult the notes section disruptive. For those seeking practical guidance on implementing voice interfaces, the inclusion of academic references like Chomsky, Grice, and Zemlin may seem misplaced.

From a scientific standpoint, the book's tendency to interpret every experimental outcome as actionable advice for businesses and marketers, often using e-commerce scenarios like online book purchasing, is noteworthy. The small sample sizes of the experiments, typically between 40 and 50 participants, are treated as sufficient grounds for advising large corporations on system design, which could be seen as overly presumptuous.

In essence, "Voice-Activated Engagement" struggles to clearly define its target audience, echoing the very lesson it attempts to impart: the importance of understanding the user in the context of voice interface design. This oversight, combined with the book's focus on immediate business applications of its findings, raises questions about its broader applicability and the depth of its scientific inquiry.

In recent years a new category of data analysis applications have evolved, known as data pipelining tools, which enable even nonexperts to perform complex analysis tasks on potentially huge amounts of data. Due to the complex and computing intensive analysis processes and methods used, it is often neither sufficient nor possible to simply rely on the increase of performance of single processors. Promising solutions to this problem are parallel and distributed approaches that can accelerate the analysis process. In this paper we discuss the parallel and distribution potential of pipelining tools by demonstrating several parallel and distributed implementations in the open source pipelining platform KNIME. We verify the practical applicability in a number of real world experiments.

We present a web-based software, named TrajAnalytics, for the visual analytics of urban trajectory datasets. It allows users to interactively visualize and analyze the massive taxi trajectories over urban spaces. The software offers data management capability and enable various visual queries through a web interface. A set of visualization widgets and interaction functions are developed to promote easy user engagement. We implemented two prototypes with realworld datasets: (1) the origin/destination (OD) data of taxi trips collected in New York City; and (2) the taxi trajectory data collected in Porto City in Portugal. This open source software supports practitioners, researchers, and decision-makers in advancing transportation and urban studies in the new era of smart city.

In this paper we examine how the Minimum Description Length (MDL) principle can be used to efficiently select aggregated views of hierarchical datasets that feature a good balance between clutter and information. We present MDL formulae for generating uneven tree cuts tailored to treemap and sunburst diagrams, taking into account the available display space and information content of the data. We present the results of a proof-of-concept implementation. In addition, we demonstrate how such tree cuts can be used to enhance drill-down interaction in hierarchical visualizations by implementing our approach in an existing visualization tool. Validation is done with the feature congestion measure of clutter in views of a subset of the current DMOZ web directory, which contains nearly half million categories. The results show that MDL views achieve near constant clutter level across display resolutions. We also present the results of a crowdsourced user study where participants were asked to find targets in views of DMOZ generated by our approach and a set of baseline aggregation methods. The results suggest that, in some conditions, participants are able to locate targets (in particular, outliers) faster using the proposed approach.

We review some applications of human-computer interaction that alleviate the complexity of visual recognition by partitioning it into human and machine tasks to exploit the differences between human and machine capabilities. Human involvement offers advantages, both in the design of automated pattern classification systems, and at the operational level of some image retrieval and classification tasks. Recent development of interactive systems has benefited from the convergence of computer vision and psychophysics in formulating visual tasks as computational processes. Computer-aided classifier design and exploratory data analysis are already well established in pattern recognition and machine learning, but interfaces and functionality are improving. On the operational side, earlier recognition systems made use of human talent only in preprocessing and in coping with rejects. Most current content-based image retrieval systems make use of relevance feedback without direct image interaction. In contrast, some visual object classification systems can exploit such interaction. They require, however, a domain-specific visible model that makes sense to both human and computer.

As heterogeneous data from different sources are being increasingly linked, it becomes difficult for users to understand how the data are connected, to identify what means are suitable to analyze a given data set, or to find out how to proceed for a given analysis task. We target this challenge with a new model-driven design process that effectively codesigns aspects of data, view, analytics, and tasks. We achieve this by using the workflow of the analysis task as a trajectory through data, interactive views, and analytical processes. The benefits for the analysis session go well beyond the pure selection of appropriate data sets and range from providing orientation or even guidance along a preferred analysis path to a potential overall speedup, allowing data to be fetched ahead of time. We illustrate the design process for a biomedical use case that aims at determining a treatment plan for cancer patients from the visual analysis of a large, heterogeneous clinical data pool. As an example for how to apply the comprehensive design approach, we present Stack'n'flip, a sample implementation which tightly integrates visualizations of the actual data with a map of available data sets, views, and tasks, thus capturing and communicating the analytical workflow through the required data sets.

Examining large-scale, long-term application use is critical to understanding how an application meets the needs of its user community. However, there have been few published analyses of long-term use of desktop applications, and none that have examined applications that support creating and modifying content using direct manipulation. In this paper, we present an analysis of 2 years of usage data from an instrumented version of the GNU Image Manipulation Program, including data from over 200 users. In the course of our analysis, we show that previous findings concerning the sparseness of command use and idiosyncrasy of users’ command vocabularies extend to a new domain and interaction style. These findings motivate continued research in adaptive and mixed-initiative interfaces. We also describe the novel application of a clustering technique to characterize a user community’s higher-level tasks from low-level logging data.

Going beyond the desktop to leverage novel devices—such as smartphones, tablets, or large displays—for visual sensemaking typically requires supporting extraneous operations for device discovery, interaction sharing, and view management. Such operations can be time-consuming and tedious, and distract the user from the actual analysis. Embodied interaction models in these multi-device environments can take advantage of the natural interaction and physicality afforded by multimodal devices and help effectively carry out these operations in visual sensemaking. In this paper, we present cross-device interaction models for visualization spaces, that are embodied in nature, by conducting a user study to elicit actions from participants that could trigger a portrayed effect of sharing visualizations (and therefore information) across devices. We then explore one common interaction style from this design elicitation called Visfer, a technique for effortlessly sharing visualizations across devices using the visual medium. More specifically, this technique involves taking pictures of visualizations, or rather the QR codes augmenting them, on a display using the built-in camera on a handheld device. Our contributions include a conceptual framework for cross-device interaction and the Visfer technique itself, as well as transformation guidelines to exploit the capabilities of each specific device and a web framework for encoding visualization components into animated QR codes, which capture multiple frames of QR codes to embed more information. Beyond this, we also present the results from a performance evaluation for the visual data transfer enabled by Visfer. We end the paper by presenting the application examples of our Visfer framework.

In many research domains, such as mechanical engineering, game development and virtual reality, a typical output model is usually produced in a multi-object manner for efficient data management. To have a complete description of the whole model, tens of thousands of, or even millions of, such objects are created, which makes the entire dataset exceptionally complex. Consequently, visualizing the model becomes a computationally intensive process that impedes a real-time rendering and interaction.

Visual analysis of high-volume time series data is ubiquitous in many industries, including finance, banking, and discrete manufacturing. Contemporary, RDBMS-based systems for visualization of high-volume time series data have difficulty to cope with the hard latency requirements and high ingestion rates of interactive visualizations. Existing solutions for lowering the volume of time series data disregard the semantics of visualizations and result in visualization errors. In this work, we introduce M4, an aggregation-based time series dimensionality reduction technique that provides errorfree visualizations at high data reduction rates. Focusing on line charts, as the predominant form of time series visualization, we explain in detail the drawbacks of existing data reduction techniques and how our approach outperforms state of the art, by respecting the process of line rasterization. We describe how to incorporate aggregation-based dimensionality reduction at the query level in a visualizationdriven query rewriting system. Our approach is generic and applicable to any visualization system that uses an RDBMS as data source. Using real world data sets from high tech manufacturing, stock markets, and sports analytics domains we demonstrate that our visualization-oriented data aggregation can reduce data volumes by up to two orders of magnitude, while preserving perfect visualizations.

The goal of the research described in this paper is to develop an application-independent presentation tool that automatically designs effective graphical presentations (such as bar charts, scatter plots, and connected graphs) of relational information. Two problems are raised by this goal: The codification of graphic design criteria in a form that can be used by the presentation tool, and the generation of a wide variety of designs so that the presentation tool can accommodate a wide variety of information. The approach described in this paper is based on the view that graphical presentations are sentences of graphical languages. The graphic design issues are codified as expressiveness and effectiveness criteria for graphical languages. Expressiveness criteria determine whether a graphical language can express the desired information. Effectiveness criteria determine whether a graphical language exploits the capabilities of the output medium and the human visual system. A wide variety of designs can be systematically generated by using a composition algebra that composes a small set of primitive graphical languages. Artificial intelligence techniques are used to implement a prototype presentation tool called APT (A Presentation Tool), which is based on the composition algebra and the graphic design criteria.


Question: The rationale for a visual discovery aide: a comprehensive strategy to expedite the exploration of visual data.

This paper explores the utilization of search query logs to enhance the understanding of user needs and tasks in interactive applications, proposing a method termed Characterizing Usability through Search (CUTS). Users frequently turn to internet search engines to guide their interaction with various systems, seeking information on task completion, problem resolution, and product education. Recognizing this pattern, CUTS leverages search query logs to complement traditional usability evaluation techniques by uncovering the primary concerns and activities of the user base. The paper outlines an automated procedure for collecting, categorizing, annotating, filtering, and clustering search queries relevant to a specific product. Key advantages of this approach include rapid data assembly, up-to-date insights, ecological validity, and reduced susceptibility to self-selection bias compared to conventional usability assessment methods. The effectiveness of CUTS is illustrated through its application to prominent software and hardware systems, demonstrating its potential to provide a more comprehensive view of user behavior and needs.

The increasing availability of affordable color raster graphics displays has made it important to develop a better understanding of how color can be used effectively in an interactive environment. Most contemporary graphics displays offer a choice of some 16 million colors; the user's problem is to find the right color.
Folklore has it that the RGB color space arising naturally from color display hardware is user-hostile and that other color models such as the HSV scheme are preferable. Until now there has been virtually no experimental evidence addressing this point.
We describe a color matching experiment in which subjects used one of two tablet-based input techniques, interfaced through one of five color models, to interactively match target colors displayed on a CRT.
The data collected show small but significant differences between models in the ability of subjects to match the five target colors used in this experiment. Subjects using the RGB color model matched quickly but inaccurately compared with those using the other models. The largest speed difference occurred during the early convergence phase of matching. Users of the HSV color model were the slowest in this experiment, both during the convergence phase and in total time to match, but were relatively accurate. There was less variation in performance during the second refinement phase of a match than during the convergence phase.
Two-dimensional use of the tablet resulted in faster but less accurate performance than did strictly one-dimensional usage.
Significant learning occurred for users of the Opponent, YIQ, LAB, and HSV color models, and not for users of the RGB color model.

"How common is interactive visualization on the web?" "What is the most popular visualization design?" "How prevalent are pie charts really?" These questions intimate the role of interactive visualization in the real (online) world. In this paper, we present our approach (and findings) to answering these questions. First, we introduce Beagle, which mines the web for SVG-based visualizations and automatically classifies them by type (i.e., bar, pie, etc.). With Beagle, we extract over 41,000 visualizations across five different tools and repositories, and classify them with 85% accuracy, across 24 visualization types. Given this visualization collection, we study usage across tools. We find that most visualizations fall under four types: bar charts, line charts, scatter charts, and geographic maps. Though controversial, pie charts are relatively rare for the visualization tools that were studied. Our findings also suggest that the total visualization types supported by a given tool could factor into its ease of use. However this effect appears to be mitigated by providing a variety of diverse expert visualization examples to users.

The cognitive dimensions framework is a broad-brush evaluation technique for interactive devices and for non-interactive notations. It sets out a small vocabulary of terms designed to capture the cognitively-relevant aspects of structure, and shows how they can be traded off against each other. The purpose of this paper is to propose the framework as an evaluation technique for visual programming environments. We apply it to two commercially-available dataflow languages (with further examples from other systems) and conclude that it is effective and insightful; other HCI-based evaluation techniques focus on different aspects and would make good complements. Insofar as the examples we used are representative, current VPLs are successful in achieving a good ‘closeness of match’, but designers need to consider the ‘viscosity’ (resistance to local change) and the ‘secondary notation’ (possibility of conveying extra meaning by choice of layout, colour, etc.).

Big Data Analysis (BDA) has attracted considerable interest and curiosity from scientists of various fields recently. As big size and complexity of big data, it is pivotal to uncover hidden patterns, bursts of activity, correlations and laws of it. Complex network analysis could be effective method for this purpose, because of its powerful data organization and visualization ability. Besides the general complex software (Gephi), some bibliometrics software (Citespace and SCI2) could also be used for this analysis, due to their powerful data process and visualization functions. This paper presents a comparison of Citespace, SCI2 and Gephi from the following three aspects: (1) Data preprocessing. Citespace is time-consuming and laborious on merging synonyms. SCI2 is efficient and suitable for massive data. Both of them could remove duplicate records. However, Gephi lacks these. (2) Network extraction. Citespace pays more attention on temporal analysis, but SCI2 and Gephi pay less attention on it. Besides, Citespace and SCI2 could provide pruning algorithms highlight the main structure of social network, but Gephi lacks this. (3) Visualization. In Citespace, co-occurrence network could present time, frequency and betweenness centrality simultaneously; cluster view could label the clusters with phrases. However, SCI2 and Gephi provide more various layout algorithms to present network. Besides, they have a better edit-ability than Citespace on generated network.

Evaluation of clustering results (or cluster validation) is an important and necessary step in cluster analysis, but it is often time-consuming and complicated work. We present a visual cluster validation tool, the Cluster Validity Analysis Platform (CVAP), to facilitate cluster validation. The CVAP provides necessary methods (e.g., many validity indices, several clustering algorithms and procedures) and an analysis environment for clustering, evaluation of clustering results, estimation of the number of clusters, and performance comparison among different clustering algorithms. It can help users accomplish their clustering tasks faster and easier and help achieve good clustering quality when there is little prior knowledge about the cluster structure of a data set.

In this paper, we describe a novel analytics system that enables query processing and predictive analytics over streams of aviation data. As part of an Internal Research and Development project, Boeing Research and Technology (BR&T) Advanced Air Traffic Management (AATM) built a system that makes predictions based upon descriptive patterns of archived aviation data. Boeing AATM has been receiving live Aircraft Situation Display to Industry (ASDI) data and archiving it for over two years. At the present time, there is not an easy mechanism to perform analytics on the data. The incoming ASDI data is large, compressed, and requires correlation with other flight data before it can be analyzed.
 The service exposes this data once it has been uncompressed, correlated, and stored in a data warehouse for further analysis using a variety of descriptive, predictive, and possibly prescriptive analytics tools. The service is being built partially in response to requests from Boeing Commercial Aviation (BCA) for analysis of capacity and flow in the US National Airspace System (NAS). The service utilizes a custom tool for correlating the raw ASDI feed, IBM Warehouse with DB2 for data management, WebSphere Message Broker for real-time message brokering, SPSS Modeler for statistical analysis, and Cognos BI for front-end business intelligence (BI) visualization. This paper describes a scalable service architecture, implementation and the value it adds to the aviation domain.

Virtual Reality (VR) has generated much excitement but little formal proof that it is useful. Because VR interfaces are difficult and expensive to build, the computer graphics community needs to be able to predict which applications will benefit from VR. In this paper, we show that users with a VR interface complete a search task faster than users with a stationary monitor and a hand-based input device. We placed users in the center of the virtual room shown in Figure 1 and told them to look for camouflaged targets. VR users did not do significantly better than desktop users. However, when asked to search the room and conclude if a target existed, VR users were substantially better at determining when they had searched the entire room. Desktop users took 41% more time, re-examining areas they had already searched. We also found a positive transfer of training from VR to stationary displays and a negative transfer of training from stationary displays to VR.

The Large Hadron Collider (LHC) is one of the most complex machines ever build. It is composed of many components which constitute a large system. The tunnel and the accelerator is just one of a very critical fraction of the whole LHC infrastructure. Hardware comissioning as one of the critical processes before running the LHC is implemented during the Long Shutdown (LS) states of the macine, where Electrical Quality Assurance (ELQA) is one of its key components. Here a huge data is collected when implementing various ELQA electrical tests. In this paper we present a conceptual framework for supporting a rapid design of web applications for ELQA data analysis. We show a framework’s main components, their possible integration with other systems and machine learning algorithms and a simple use case of prototyping an application for Electrical Quality Assurance of the LHC.

Behavioral researchers spend considerable amount of time coding video data to systematically extract meaning from subtle human actions and emotions. In this paper, we present Glance, a tool that allows researchers to rapidly query, sample, and analyze large video datasets for behavioral events that are hard to detect automatically. Glance takes advantage of the parallelism available in paid online crowds to interpret natural language queries and then aggregates responses in a summary view of the video data. Glance provides analysts with rapid responses when initially exploring a dataset, and reliable codings when refining an analysis. Our experiments show that Glance can code nearly 50 minutes of video in 5 minutes by recruiting over 60 workers simultaneously, and can get initial feedback to analysts in under 10 seconds for most clips. We present and compare new methods for accurately aggregating the input of multiple workers marking the spans of events in video data, and for measuring the quality of their coding in real-time before a baseline is established by measuring the variance between workers. Glance's rapid responses to natural language queries, feedback regarding question ambiguity and anomalies in the data, and ability to build on prior context in followup queries allow users to have a conversation-like interaction with their data - opening up new possibilities for naturally exploring video data.

The problem of visual presentation of multidimensional data is discussed. The projection methods for dimension reduction are reviewed. The chapter deals with the artificial neural networks that may be used for reducing dimension and data visualization, too. The stress is put on combining the selforganizing map (SOM) and Sammon mapping and on the neural network for Sammon’s mapping SAMANN. Large scale applications are discussed: environmental data analysis, statistical analysis of curricula, comparison of schools, analysis of the economic and social conditions of countries, analysis of data on the fundus of eyes and analysis of physiological data on men’s health.


Question: Forecasting movie success using machine learning methods: strategies to enhance precision

This paper presents an innovative approach to enhancing the accuracy of movie success predictions through machine learning techniques, specifically focusing on multi-class classification with AdaBoost. AdaBoost, a popular boosting method for binary classification, is extended to handle multi-class problems directly without resorting to the reduction into multiple two-class tasks. The proposed multi-class AdaBoost algorithm is shown to be equivalent to a forward stagewise additive modeling technique that minimizes a unique exponential loss function tailored for multi-class classification. It is further demonstrated that this exponential loss function belongs to a family of Fisher-consistent loss functions suitable for multi-class classification tasks. The algorithm's implementation is straightforward, and its effectiveness is evidenced by its competitive performance in terms of misclassification error rates, making it a promising method for improving the accuracy of movie success predictions.

C. L. Wu and K. W. Chau* 2 Dept. of Civil and Structural Engineering, Hong Kong Polytechnic University, 3 Hung Hom, Kowloon, Hong Kong, People’s Republic of China 4 5 *Email: cekwchau@polyu.edu.hk 6 ABSTRACT 7 Data-driven techniques such as Auto-Regressive Moving Average (ARMA), K-Nearest-Neighbors (KNN), and 8 Artificial Neural Networks (ANN), are widely applied to hydrologic time series prediction. This paper investigates 9 different data-driven models to determine the optimal approach of predicting monthly streamflow time series. Four 10 sets of data from different locations of People’s Republic of China (Xiangjiaba, Cuntan, Manwan, and 11 Danjiangkou) are applied for the investigation process. Correlation integral and False Nearest Neighbors (FNN) 12 are first employed for Phase Space Reconstruction (PSR). Four models, ARMA, ANN, KNN, and Phase Space 13 Reconstruction-based Artificial Neural Networks (ANN-PSR) are then compared by one-month-ahead forecast 14 using Cuntan and Danjiangkou data. The KNN model performs the best among the four models, but only exhibits 15 weak superiority to ARMA. Further analysis demonstrates that a low correlation between model inputs and 16 outputs could be the main reason to restrict the power of ANN. A Moving Average Artificial Neural Networks 17 (MA-ANN), using the moving average of streamflow series as inputs, is also proposed in this study. The results 18 show that the MA-ANN has a significant improvement on the forecast accuracy compared with the original four 19 models. This is mainly due to the improvement of correlation between inputs and outputs depending on the 20 moving average operation. The optimal memory lengths of the moving average were three and six for Cuntan and 21 Danjiangkou respectively when the optimal model inputs are recognized as the previous twelve months. 22 23

The machine learning approach is represented with two methods: the maximum entropy method and support vector machine. Text representation for the maximum entropy method includes the information about the proportion of positive and negative words and collocations, the quantity of interrogation and exclamation marks, emoticons, obscene language. For the support vector machine binary vectors with cosine normalization are built on texts.

Starting with the Netflix Prize, which fueled much recent progress in the field of collaborative filtering, recent years have witnessed rapid development of new recommendation algorithms and increasingly more complex systems, which greatly differ from their early content-based and collaborative filtering systems. Modern recommender systems leverage several novel algorithmic approaches: from matrix factorization methods and multi-armed bandits to deep neural networks. In this tutorial, we will cover recent algorithmic advances in recommender systems, highlight their capabilities, and their impact. We will give many examples of industrial-scale recommender systems that define the future of the recommender systems area. We will discuss related evaluation issues, and outline future research directions. The ultimate goal of the tutorial is to encourage the application of novel recommendation approaches to solve problems that go beyond user consumption and to further promote research in the intersection of recommender systems and databases.

Crowdfunding has gained widespread attention in recent years. Despite the huge success of crowdfunding platforms, the percentage of projects that succeed in achieving their desired goal amount is only around 40%. Moreover, many of these crowdfunding platforms follow "all-or-nothing" policy which means the pledged amount is collected only if the goal is reached within a certain predefined time duration. Hence, estimating the probability of success for a project is one of the most important research challenges in the crowdfunding domain. To predict the project success, there is a need for new prediction models that can potentially combine the power of both classification (which incorporate both successful and failed projects) and regression (for estimating the time for success). In this paper, we formulate the project success prediction as a survival analysis problem and apply the censored regression approach where one can perform regression in the presence of partial information. We rigorously study the project success time distribution of crowdfunding data and show that the logistic and log-logistic distributions are a natural choice for learning from such data. We investigate various censored regression models using comprehensive data of 18K Kickstarter (a popular crowdfunding platform) projects and 116K corresponding tweets collected from Twitter. We show that the models that take complete advantage of both the successful and failed projects during the training phase will perform significantly better at predicting the success of future projects compared to the ones that only use the successful projects. We provide a rigorous evaluation on many sets of relevant features and show that adding few temporal features that are obtained at the project's early stages can dramatically improve the performance.

We introduce supervised latent Dirichlet allocation (sLDA), a statistical model of labelled documents. The model accommodates a variety of response types. We derive a maximum-likelihood procedure for parameter estimation, which relies on variational approximations to handle intractable posterior expectations. Prediction problems motivate this research: we use the fitted model to predict response values for new documents. We test sLDA on two real-world problems: movie ratings predicted from reviews, and web page popularity predicted from text descriptions. We illustrate the benefits of sLDA versus modern regularized regression, as well as versus an unsupervised LDA analysis followed by a separate regression.

In recent years the amount of data stored in educational database is growing rapidly. The stored database contains hidden information which if used aids improvement of student's performance and behaviour. In this paper predictive modelling approach is used for extracting this hidden information. Data is collected, a predictive model is formulated, predictions are made, and the model is validated as additional data becomes available. The predictive models will help the instructor to understand how well or how poorly the students in his/her class will perform, and hence the instructor can choose proper pedagogical and instructional interventions to enhance student learning outcomes. The implementation is done in Hadoop framework with MapReduce and Revolutionary R Enterprise RRE.

This study investigated the accuracy of gender-specific stereotypes about movie-genre preferences for 17 genres. In Study 1, female and male participants rated the extent to which 17 movie genres are preferred by women or men. In Study 2, another sample of female and male participants rated their own preference for each genre. There were three notable results. First, Study 1 revealed the existence of gender stereotypes for the majority of genres (i.e., for 15 of 17 genres). Second, Study 2 revealed the existence of actual gender differences in preferences for the majority of genres (i.e., for 11 of 17 genres). Third, in order to assess the accuracy of gender stereotypes on movie preferences, we compared the results of both studies and found that the majority of gender stereotypes were accurate in direction, but inaccurate in size. In particular, the stereotypes overestimated actual gender differences for the majority of movie genres (i.e., 10 of 17). Practical and theoretical implications of these findings are discussed.

This paper proposes AutoRec, a novel autoencoder framework for collaborative filtering (CF). Empirically, AutoRec's compact and efficiently trainable model outperforms state-of-the-art CF techniques (biased matrix factorization, RBM-CF and LLORMA) on the Movielens and Netflix datasets.

This paper investigates the method of forecasting stock price difference on artificially generated price series data using neuro-fuzzy systems and neural networks. As trading profits is more important to an investor than statistical performance, this paper proposes a novel rough set-based neuro-fuzzy stock trading decision model called stock trading using rough set-based pseudo outer-product (RSPOP) which synergizes the price difference forecast method with a forecast bottleneck free trading decision model. The proposed stock trading with forecast model uses the pseudo outer-product based fuzzy neural network using the compositional rule of inference [POPFNN-CRI(S)] with fuzzy rules identified using the RSPOP algorithm as the underlying predictor model and simple moving average trading rules in the stock trading decision model. Experimental results using the proposed stock trading with RSPOP forecast model on real world stock market data are presented. Trading profits in terms of portfolio end values obtained are benchmarked against stock trading with dynamic evolving neural-fuzzy inference system (DENFIS) forecast model, the stock trading without forecast model and the stock trading with ideal forecast model. Experimental results showed that the proposed model identified rules with greater interpretability and yielded significantly higher profits than the stock trading with DENFIS forecast model and the stock trading without forecast model

With the exponential growth of social media i.e. blogs and social networks, organizations and individual persons are increasingly using the number of reviews of these media for decision making about a product or service. Opinion mining detects whether the emotions of an opinion expressed by a user on Web platforms in natural language, is positive or negative. This paper presents extensive experiments to study the effectiveness of the classification of Spanish opinions in five categories: highly positive, highly negative, positive, negative and neutral, using the combination of the psychological and linguistic features of LIWC. LIWC is a text analysis software that enables the extraction of different psychological and linguistic features from natural language text. For this study, two corpora have been used, one about movies and one about technological products. Furthermore, we have conducted a comparative assessment of the performance of various classification techniques: J48, SMO and BayesNet, using precision, recall and F-measure metrics. All in all, findings have revealed that the positive and negative categories provide better results than the other categories. Finally, experiments on both corpora indicated that SMO produces better results than BayesNet and J48 algorithms, obtaining an F-measure of 90.4% and 87.2% in each domain.


Question: Forecasting movie success using machine learning methods: strategies to enhance precision

Forecasting Movie Box Office Performance through Sentiment Analysis of Weblogs
This research explores the predictive power of sentiment analysis applied to online discussions in weblogs regarding its correlation with a movie's financial success. Previous studies have demonstrated that the volume of online conversations about a product can be indicative of its commercial performance. Building on this premise, our investigation specifically examines whether integrating sentiment analysis into the analysis of weblog data enhances predictive accuracy in the movie industry. Our key finding reveals that positive sentiment, when analyzed within a narrowed context surrounding explicit mentions of the movie in weblogs before its release, serves as a more reliable indicator of box office success. This insight underscores the significance of not only measuring the volume of online discourse but also assessing the sentiment within such conversations for a more accurate forecast of a movie's financial outcomes.

**Question**: Understanding the impact of social media on political polarization

**Documents**: Social Media and Political Polarization: Is Facebook Making Us More Divided?
Social media platforms, particularly Facebook, have become central to how people consume and share news. This paper examines whether social media is contributing to political polarization. The findings suggest that while social media can amplify political echo chambers, it is not the sole factor driving political polarization. Factors such as partisanship, media consumption habits, and pre-existing beliefs play a significant role.

**Similar Documents**:
The Role of Social Media in Exacerbating Political Divisions: A Comprehensive Analysis
In recent years, social media platforms, with Facebook being a prominent example, have transformed the landscape of news consumption and dissemination. This study delves into the question of whether social media platforms are exacerbating political polarization. Our analysis reveals that while social media can indeed intensify echo chambers, amplifying individuals' exposure to like-minded opinions, it is not the only culprit behind increasing political divisions. Other influential factors, including strong partisanship, selective media consumption patterns, and deeply ingrained personal beliefs, significantly contribute to the polarization phenomenon. These findings highlight the complex interplay between social media use and political polarization, suggesting that addressing polarization requires a multifaceted approach that considers various societal and individual factors.

The classification of file fragments is an important problem in digital forensics. The literature does not include comprehensive work on applying machine learning techniques to this problem. In this work, we explore the use of techniques from natural language processing to classify file fragments. We take a supervised learning approach, based on the use of support vector machines combined with the bag-of-words model, where text documents are represented as unordered bags of words. This technique has been repeatedly shown to be effective and robust in classifying text documents (e.g., in distinguishing positive movie reviews from negative ones). In our approach, we represent file fragments as “bags of bytes” with feature vectors consisting of unigram and bigram counts, as well as other statistical measurements (including entropy and others). We made use of the publicly available Garfinkel data corpus to generate file fragments for training and testing. We ran a series of experiments, and found that this approach is effective in this domain as well. a 2012 O. Zhulyn, S. Fitzgerald & G. Mathews. Published by Elsevier Ltd. All rights reserved.

While accurately predicting mood and wellbeing could have a number of important clinical benefits, traditional machine learning (ML) methods frequently yield low performance in this domain. We posit that this is because a one-size-fits-all machine learning model is inherently ill-suited to predicting outcomes like mood and stress, which vary greatly due to individual differences. Therefore, we employ Multitask Learning (MTL) techniques to train personalized ML models which are customized to the needs of each individual, but still leverage data from across the population. Three formulations of MTL are compared: i) MTL deep neural networks, which share several hidden layers but have final layers unique to each task; ii) Multi-task Multi-Kernel learning, which feeds information across tasks through kernel weights on feature types; and iii) a Hierarchical Bayesian model in which tasks share a common Dirichlet Process prior. We offer the code for this work in open source. These techniques are investigated in the context of predicting future mood, stress, and health using data collected from surveys, wearable sensors, smartphone logs, and the weather. Empirical results demonstrate that using MTL to account for individual differences provides large performance improvements over traditional machine learning methods and provides personalized, actionable insights.

This paper introduces an approach to sentiment analysis which uses support vector machines (SVMs) to bring together diverse sources of potentially pertinent information, including several favorability measures for phrases and adjectives and, where available, knowledge of the topic of the text. Models using the features introduced are further combined with unigram models which have been shown to be effective in the past (Pang et al., 2002) and lemmatized versions of the unigram models. Experiments on movie review data from Epinions.com demonstrate that hybrid SVMs which combine unigram-style feature-based SVMs with those based on real-valued favorability measures obtain superior performance, producing the best results yet published using this data. Further experiments using a feature set enriched with topic information on a smaller dataset of music reviews handannotated for topic are also reported, the results of which suggest that incorporating topic information into such models may also yield improvement.

Multiplicative weight-updating algorithms such as Winnow have been studied extensively in the COLT literature, but only recently have people started to use them in applications. In this paper, we apply a Winnow-based algorithm to a task in natural language: contextsensitive spelling correction. This is the task of xing spelling errors that happen to result in valid words, such as substituting to for too, casual for causal, and so on. Previous approaches to this problem have been statistics-based; we compare Winnow to one of the more successful such approaches, which uses Bayesian classi ers. We nd that: (1) When the standard (heavily-pruned) set of features is used to describe problem instances, Winnow performs comparably to the Bayesian method; (2) When the full (unpruned) set of features is used, Winnow is able to exploit the new features and convincingly outperform Bayes; and (3) When a test set is encountered that is dissimilar to the training set, Winnow is better than Bayes at adapting to the unfamiliar test set, using a strategy we will present for combining learning on the training set with unsupervised learning on the (noisy) test set. In Machine Learning: Proceedings of the 13th International Conference, Lorenza Saitta, ed., Morgan Kaufmann, San Francisco, CA, 1996, pages 182{190. This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonpro t educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Information Technology Center America; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Information Technology Center America. All rights reserved. Copyright c Mitsubishi Electric Information Technology Center America, 1996 201 Broadway, Cambridge, Massachusetts 02139 Dept. of Appl. Math. & CS, Weizmann Institute of Science, Rehovot 76100, Israel. 1. First printing, TR96-07, April 1996

The Recommender System (RS) is an efficient tool for decision makers that assists in the selection of appropriate items according to their preferences and interests. This system has been applied to various domains to personalize applications by recommending items such as books, movies, songs, restaurants, news articles and jokes, among others. An important issue problem, which occurs when there is a new user that has been registered to the system and no prior rating of this user is found in the rating table. In this paper, we first present a classification that divides the relevant studies addressing the new user cold-start problem into three major groups and summarize their advantages and disadvantages in a tabular format. Next, some typical algorithms of these groups, such as MIPFGWC-CS, NHSM, FARAMS and HU–FCF, are described. Finally, these algorithms are implemented and validated on some benchmark RS datasets under various settings of the new user cold start. The experimental results indicate that NHSM achieves better accuracy and computational time than the relevant methods. & 2014 Elsevier Ltd. All rights reserved.

An enduring issue in higher education is student retention to successful graduation. National statistics indicate that most higher education institutions have four-year degree completion rates around 50%, or just half of their student populations. While there are prediction models which illuminate what factors assist with college student success, interventions that support course selections on a semester-to-semester basis have yet to be deeply understood. To further this goal, we develop a system to predict students’ grades in the courses they will enroll in during the next enrollment term by learning patterns from historical transcript data coupled with additional information about students, courses and the instructors teaching them. We explore a variety of classic and state-of-the-art techniques which have proven effective for recommendation tasks in the e-commerce domain. In our experiments, Factorization Machines (FM), Random Forests (RF), and the Personalized Linear Multiple Regression model achieve the lowest prediction error. Application of a novel feature selection technique is key to the predictive success and interpretability of the FM. By comparing feature importance across populations and across models, we uncover strong connections between instructor characteristics and student performance. We also discover key differences between transfer and non-transfer students. Ultimately we find that a hybrid FM-RF method can be used to accurately predict grades for both new and returning students taking both new and existing courses. Application of these techniques holds promise for student degree planning, instructor interventions, and personalized advising, all of which could improve retention and academic performance.

The continually increasing number of complex datasets each year necessitates ever improving machine learning methods for robust and accurate categorization of these data. This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble, deep learning approach for classification. Deep learning models have achieved state-of-the-art results across many domains. RMDL solves the problem of finding the best deep learning structure and architecture while simultaneously improving robustness and accuracy through ensembles of deep learning architectures. RDML can accept as input a variety data to include text, video, images, and symbolic. This paper describes RMDL and shows test results for image and text data including MNIST, CIFAR-10, WOS, Reuters, IMDB, and 20newsgroup. These test results show that RDML produces consistently better performance than standard methods over a broad range of data types and classification problems.

Time series forecasting plays an important role in many fields such as economics, finance, business intelligence, natural sciences, and the social sciences. This forecasting task can be achieved by using different techniques such as statistical methods or Artificial Neural Networks (ANN). In this paper, we present two different approaches to time series forecasting: evolving Takagi-Sugeno (eTS) fuzzy model and ANN. These two different methods will be compared taking into account the different characteristic of each approach.

Sentiment classification has adopted machine learning techniques to improve its precision and efficiency. However, the features are always produced by basic words-bag methods without much consideration for words' syntactic properties, which could play an important role in the judgment of sentiment meanings. To remedy this, we firstly generate syntax trees of the sentences, with the analysis of syntactic features of the sentences. Then we introduce multiple sentiment features into the basic words-bag features. Such features were trained on movie reviews as data, with machine learning methods (Naive Bayes and support vector machines). The features and factors introduced by syntax tree were examined to generate a more accurate solution for sentiment classification.

Detection of malicious software (malware) using ma chine learning methods has been explored extensively to enable fas t detection of new released malware. The performance of these classifiers depen ds on the induction algorithms being used. In order to benefit from mul tiple different classifiers, and exploit their strengths we suggest using an ens emble method that will combine the results of the individual classifiers i nto one final result to achieve overall higher detection accuracy. In this paper we evaluate several combining methods using five different base inducers (C4.5 Dec ision Tree, Naïve Bayes, KNN, VFI and OneR) on five malware datasets. The mai n goal is to find the best combining method for the task of detecting mal icious files in terms of accuracy, AUC and execution time.


Question: Forecasting movie success through machine learning methods: strategies to enhance precision

Decision Tree Models for Movie Success Prediction

This paper explores the application of decision tree models, specifically classification and regression trees, to predict the success of movies. Decision trees are versatile machine learning techniques that can handle both categorical and continuous data, making them particularly suitable for the multifaceted nature of movie data, which includes factors such as genre, budget, release date, and critical reception. The methodology involves constructing a decision tree model using historical movie data, where each node represents a feature or decision point, and each branch represents an outcome. The model aims to find the optimal path through these decisions to accurately predict whether a movie will be a box office hit or a flop. By analyzing various attributes that contribute to a movie's success, the decision tree model can identify the most significant predictors and make data-driven decisions. The accuracy of the model can be improved by employing techniques such as pruning to avoid overfitting, using ensemble methods like random forests for better generalization, and incorporating feature engineering to enhance the predictive power of the model. Through rigorous testing and validation, this approach demonstrates a promising avenue for improving the accuracy of movie success predictions using machine learning techniques.

Despite the recognized value of Machine Learning (ML) techniques and high expectation of applying ML techniques within various applications, significant barriers to the widespread adoption and local implementation of ML approaches still exist in the areas of trust (of ML results), comprehension (of ML processes), as well as confidence (in decision making) by users. This paper investigates the effects of correlation between features and target values on user confidence in data analytics-driven decision making. Our user study found that revealing the correlation between features and target variables affected user confidence in decision making significantly. Moreover, users felt more confident in decision making when correlation shared the same trend with the prediction model performance. These findings would help design intelligent user interfaces and evaluate the effectiveness of machine learning models in applications.

This paper covers the two approaches for sentiment analysis: i) lexicon based method; ii) machine learning method. We describe several techniques to implement these approaches and discuss how they can be adopted for sentiment classification of Twitter messages. We present a comparative study of different lexicon combinations and show that enhancing sentiment lexicons with emoticons, abbreviations and social-media slang expressions increases the accuracy of lexicon-based classification for Twitter. We discuss the importance of feature generation and feature selection processes for machine learning sentiment classification. To quantify the performance of the main sentiment analysis methods over Twitter we run these algorithms on a benchmark Twitter dataset from the SemEval-2013 competition, task 2-B. The results show that machine learning method based on SVM and Naive Bayes classifiers outperforms the lexicon method. We present a new ensemble method that uses a lexicon based sentiment score as input feature for the machine learning approach. The combined method proved to produce more precise classifications. We also show that employing a cost-sensitive classifier for highly unbalanced datasets yields an improvement of sentiment classification performance up to 7%.

Unlike financial forecasting, a type of mechanical trading technique called Trend Following (TF) doesn’t predict any market movement; instead it identifies a trend at early time of the day, and trades automatically afterwards by a pre-defined strategy regardless of the moving market directions during run time. Trend following trading has a long and successful history among speculators. The traditional TF trading method is by human judgment in setting the rules (aka the strategy). Subsequently the TF strategy is executed in pure objective operational manner. Finding the correct strategy at the beginning is crucial in TF. This usually involves human intervention in first identifying a trend, and configuring when to place an order and close it out, when certain conditions are met. In this paper, we presented a new type of TF, namely Trend Recalling algorithm that operates in a totally automated manner. It works by partially matching the current trend with one of the proven successful patterns from the past. Our experiments based on real stock market data show that this algorithm has an edge over the other trend following methods in profitability. The new algorithm is also compared to time-series forecasting type of stock trading, and it can even outperform the best forecasting type in a

Detection of malicious software (malware) using ma chine learning methods has been explored extensively to enable fas t detection of new released malware. The performance of these classifiers depen ds on the induction algorithms being used. In order to benefit from mul tiple different classifiers, and exploit their strengths we suggest using an ens emble method that will combine the results of the individual classifiers i nto one final result to achieve overall higher detection accuracy. In this paper we evaluate several combining methods using five different base inducers (C4.5 Dec ision Tree, Naïve Bayes, KNN, VFI and OneR) on five malware datasets. The mai n goal is to find the best combining method for the task of detecting mal icious files in terms of accuracy, AUC and execution time.

Machine Learning is the technology, in which algorithms which are capable of learning from previous cases and past experiences are designed. It is implemented using various algorithms which reiterate over the same data repeatedly to analyze the pattern of data. The techniques of data mining are no far behind and are widely used to extract data from large databases to discover some patterns making decisions. This paper presents the Naïve Bayes improved K-Nearest Neighbor method (NBKNN) for Fraud Detection of Credit Card. Experimental results illustrate that both classifiers work differently for the same dataset. The purpose is to enhance the accuracy and enhance the flexibility of the algorithm.

Model selection strategies for machine learning algorithm s typically involve the numerical optimisation of an appropriate model selection criterion, ofte n based on an estimator of generalisation performance, such as k-fold cross-validation. The error of such an estimator can b e broken down into bias and variance components. While unbiasedness is oft en cited as a beneficial quality of a model selection criterion, we demonstrate that a low varian ce is at least as important, as a nonnegligible variance introduces the potential for over-fitt ing in model selection as well as in training the model. While this observation is in hindsight perhaps rat her obvious, the degradation in performance due to over-fitting the model selection criterion can b e surprisingly large, an observation that appears to have received little attention in the machine lea rning literature to date. In this paper, we show that the effects of this form of over-fitting are often of c mparable magnitude to differences in performance between learning algorithms, and thus canno t be ignored in empirical evaluation. Furthermore, we show that some common performance evaluati on practices are susceptible to a form of selection bias as a result of this form of over-fitting and hence are unreliable. We discuss methods to avoid over-fitting in model selection and sub sequent selection bias in performance evaluation, which we hope will be incorporated into best pra ctice. While this study concentrates on cross-validation based model selection, the findings are quit general and apply to any model selection practice involving the optimisation of a model se lection criterion evaluated over a finite sample of data, including maximisation of the Bayesian evid nce and optimisation of performance bounds.

Amazon sales-rank gives a relative estimate of a product item’s popularity among other items in the same category. An early prediction of the Amazon sales-rank of a product would imply an early guess of its sales-popularity relative to the other products on Amazon, which is one of the largest e-commerce hub across the globe. Traditional methods suggest use of product review related features, e.g., volume of reviews, text content of the reviews etc. for the purpose of prediction. In contrast, we propose in this paper for the first time a network-assisted approach to construct suitable features for prediction. In particular, we build a co-purchase network treating the individual products as nodes, with edges in between if two products are bought with one another. The way a product is positioned in this network (e.g., its centrality, clustering coefficient etc.) turns out to be a strong indicator of its sales-rank. This network-assisted approach has two distinct advantages over the traditional baseline method based on review analysis – (i) it works even if the product has no reviews (relevant especially in the early stages of the product launch) and (ii) it is notably more discriminative in classifying a popular (i.e., low sales-rank) product from an unpopular (i.e., high sales-rank) one. Based on this observation, we build a supervised model to early classify a popular product from an unpopular one. We report our results on two different product categories (CDs and cell phones) and obtain remarkably better classification accuracy compared to the baseline scheme. When the top 100 (700) products based on sales-rank are labelled as popular and the bottom 100 (700) are labelled as unpopular, the classification accuracy of our method is 89.85% (82.1%) for CDs and 84.11% (84.8%) for cell phones compared to 46.37% (68.75%) and 83.17% (71.95%) respectively from the baseline method.

In recent years, the shared bicycle project has developed rapidly. In use of shared bicycles, a great deal of user riding information is recorded. How to extract effective knowledge from these vast amounts of information, how to use this knowledge to improve the shared bicycle system, and how to improve the user experience, are problems to solve. Citi Bike is selected as the research target. Data on Citi Bike’s user historical behavior, weather information, and holiday information are collected from three different sources, and converted into appropriate formats for model training. Spark MLlib is used to construct three different predictive models, advantages and disadvantages of different forecasting models are compared. Some techniques are used to enhance the accuracy of random forests model. The experimental results show that the root mean square error RMSE of the final model is reduced from 305.458 to 243.346.

This paper introduces a new dataset and compares several methods for the recommendation of non-fiction audio visual material, namely lectures from the TED website. The TED dataset contains 1,149 talks and 69,023 profiles of users, who have made more than 100,000 ratings and 200,000 comments. The corresponding metadata, which we make available, can be used for training and testing generic or personalized recommender systems. We define content-based, collaborative, and combined recommendation methods for TED lectures and use cross-validation to select the best parameters of keyword-based (TFIDF) and semantic vector space-based methods (LSI, LDA, RP, and ESA). We compare these methods on a personalized recommendation task in two settings, a cold-start and a non-cold-start one. In the cold-start setting, semantic vector spaces perform better than keywords. In the non-cold-start setting, where collaborative information can be exploited, content-based methods are outperformed by collaborative filtering ones, but the proposed combined method shows acceptable performances, and can be used in both settings. For the generic recommendation task, LSI and RP again outperform TF-IDF.

Abstract: This research proposes a two-stage user-based collaborative filtering process using an artificial immune system for the prediction of student grades, along with a filter for professor ratings in the course recommendation for college students. We test for cosine similarity and Karl Pearson (KP) correlation in affinity calculations for clustering and prediction. This research uses student information and professor information datasets of Yuan Ze University from the years 2005–2009 for the purpose of testing and training. The mean average error and confusion matrix analysis form the testing parameters. A minimum professor rating was tested to check the results, and observed that the recommendation systems herein provide highly accurate results for students with higher mean grades.


Question: Forecasting movie success utilizing machine learning methods: strategies to enhance precision

This paper introduces Opinion Observer, a novel framework designed to analyze and compare consumer opinions on competing products, specifically focusing on online customer reviews. It presents two key contributions. Firstly, it offers a unique approach for visualizing consumer sentiments regarding different products, highlighting their strengths and weaknesses across various features. This visualization provides a quick and comprehensive comparison that is beneficial to both potential buyers, aiding in their purchasing decisions, and manufacturers, enabling them to gather valuable marketing intelligence and product benchmarking insights. Secondly, the paper introduces a cutting-edge technique that employs language pattern mining to extract product features from the pros and cons mentioned in specific types of reviews. These features serve as the foundation for the comparative analysis. The effectiveness of this technique is substantiated through experimental results, demonstrating its superior performance compared to existing methods.

**Question**: Evaluating the impact of renewable energy on electricity markets

**Documents**: The role of wind power in the European electricity market
The European electricity market is undergoing a significant transformation due to the integration of renewable energy sources, particularly wind power. This paper examines the impact of wind power on the European electricity market, focusing on its role in changing market dynamics, pricing mechanisms, and the behavior of market participants. The analysis reveals that wind power has led to a decrease in wholesale electricity prices, as it is often the marginal source of electricity generation. However, this has also created challenges for conventional power plants, which have to operate at lower capacities and face increased operational costs. Moreover, the intermittency of wind power has introduced new risks and uncertainties into the market, requiring the development of new market mechanisms and products to manage these risks. The paper also discusses the implications of these changes for the design of future electricity markets and the need for a more flexible and adaptive market structure to accommodate the increasing share of renewable energy.

**Similar Documents**: This study explores the profound effects of wind power integration on the European electricity market, illustrating how renewable energy sources are reshaping market dynamics, pricing strategies, and participant behavior. Wind power's role in reducing wholesale electricity prices is highlighted, as it often becomes the marginal cost of generation, impacting conventional power plants by forcing them to operate at reduced capacities and incurring higher operational expenses. The paper delves into the complexities introduced by wind power's variability, including heightened risks and uncertainties that necessitate the evolution of market mechanisms and the creation of innovative products to mitigate these challenges. Additionally, it discusses the broader implications for the future design of electricity markets, emphasizing the necessity for a more adaptable and resilient market framework capable of handling the growing proportion of renewable

The classification of file fragments is an important problem in digital forensics. The literature does not include comprehensive work on applying machine learning techniques to this problem. In this work, we explore the use of techniques from natural language processing to classify file fragments. We take a supervised learning approach, based on the use of support vector machines combined with the bag-of-words model, where text documents are represented as unordered bags of words. This technique has been repeatedly shown to be effective and robust in classifying text documents (e.g., in distinguishing positive movie reviews from negative ones). In our approach, we represent file fragments as “bags of bytes” with feature vectors consisting of unigram and bigram counts, as well as other statistical measurements (including entropy and others). We made use of the publicly available Garfinkel data corpus to generate file fragments for training and testing. We ran a series of experiments, and found that this approach is effective in this domain as well. a 2012 O. Zhulyn, S. Fitzgerald & G. Mathews. Published by Elsevier Ltd. All rights reserved.

To help generate relevant suggestions for researchers, recommendation systems have started to leverage the latent interests in the publication profiles of the researchers themselves. While using such a publication citation network has been shown to enhance performance, the network is often sparse, making recommendation difficult. To alleviate this sparsity, we identify "potential citation papers" through the use of collaborative filtering. Also, as different logical sections of a paper have different significance, as a secondary contribution, we investigate which sections of papers can be leveraged to represent papers effectively.
 On a scholarly paper recommendation dataset, we show that recommendation accuracy significantly outperforms state-of-the-art recommendation baselines as measured by nDCG and MRR, when we discover potential citation papers using imputed similarities via collaborative filtering and represent candidate papers using both the full text and assigning more weight to the conclusion sections.

Many businesses are using recommender systems for marketing outreach. Recommendation algorithms can be either based on content or driven by collaborative filtering. We study different ways to incorporate content information directly into the matrix factorization approach of collaborative filtering. These content-boosted matrix factorization algorithms not only improve recommendation accuracy, but also provide useful insights about the contents, as well as make recommendations more easily interpretable.

The support vector machine (SVM) is a popular classification technique. However, beginners who are not familiar with SVM often get unsatisfactory results since they miss some easy but significant steps. In this guide, we propose a simple procedure which usually gives reasonable results.

The popularity of using Internet contains some risks of network attacks. Intrusion detection is one major research problem in network security, whose aim is to identify unusual access or attacks to secure internal networks. In literature, intrusion detection systems have been approached by various machine learning techniques. However, there is no a review paper to examine and understand the current status of using machine learning techniques to solve the intrusion detection problems. This chapter reviews 55 related studies in the period between 2000 and 2007 focusing on developing single, hybrid, and ensemble classifiers. Related studies are compared by their classifier design, datasets used, and other experimental setups. Current achievements and limitations in developing intrusion detection systems by machine learning are present and discussed. A number of future research directions are also provided. 2009 Elsevier Ltd. All rights reserved.

We describe our first-place solution to the ECML/PKDD discovery challenge on taxi destination prediction. The task consisted in predicting the destination of a taxi based on the beginning of its trajectory, represented as a variable-length sequence of GPS points, and diverse associated meta-information, such as the departure time, the driver id and client information. Contrary to most published competitor approaches, we used an almost fully automated approach based on neural networks and we ranked first out of 381 teams. The architectures we tried use multi-layer perceptrons, bidirectional recurrent neural networks and models inspired from recently introduced memory networks. Our approach could easily be adapted to other applications in which the goal is to predict a fixed-length output from a variable-length sequence. Random order, this does not reflect the weights of contributions. ar X iv :1 50 8. 00 02 1v 2 [ cs .L G ] 2 1 Se p 20 15 2 Artificial Neural Networks Applied to Taxi Destination Prediction

Due to its high popularity, Weblogs (or blogs in short) present a wealth of information that can be very helpful in assessing the general public's sentiments and opinions. In this paper, we study the problem of mining sentiment information from blogs and investigate ways to use such information for predicting product sales performance. Based on an analysis of the complex nature of sentiments, we propose Sentiment PLSA (S-PLSA), in which a blog entry is viewed as a document generated by a number of hidden sentiment factors. Training an S-PLSA model on the blog data enables us to obtain a succinct summary of the sentiment information embedded in the blogs. We then present ARSA, an autoregressive sentiment-aware model, to utilize the sentiment information captured by S-PLSA for predicting product sales performance. Extensive experiments were conducted on a movie data set. We compare ARSA with alternative models that do not take into account the sentiment information, as well as a model with a different feature selection method. Experiments confirm the effectiveness and superiority of the proposed approach.

A decent movie summary is helpful for movie producer to promote the movie as well as audience to capture the theme of the movie before watching the whole movie. Most exiting automatic movie summarization approaches heavily rely on video content only, which may not deliver ideal result due to the semantic gap between computer calculated low-level features and human used high-level understanding. In this paper, we incorporate script into movie analysis and propose a novel character-based movie summarization approach, which is validated by modern film theory that what actually catches audiences' attention is the character. We first segment scenes in the movie by analysis and alignment of script and movie. Then we conduct substory discovery and content attention analysis based on the scent analysis and character interaction features. Given obtained movie structure and content attention value, we calculate movie attraction scores at both shot and scene levels and adopt this as criterion to generate movie summary. The promising experimental results demonstrate that character analysis is effective for movie summarization and movie content understanding.

In the present age of information overload, it is becoming increasingly harder to find relevant content. Recommender systems have been introduced to help people deal with these vast amounts of information and have been widely used in research as well as e-commerce applications. In this paper, we propose several new approaches to improve the accuracy of recommender systems by using rating variance to gauge the confidence of recommendations. We then empirically demonstrate how these approaches work with various recommendation techniques. We also show how these approaches can generate more personalized recommendations, as measured by the coverage metric. As a result, users can be given a better control to choose whether to receive recommendations with higher coverage or higher accuracy.

Evolutionary computation (EC) is a kind of optimization methodology inspired by the mechanisms of biological evolution and behaviors of living organisms. In the literature, the terminology evolutionary algorithms is frequently treated the same as EC. This article focuses on making a survey of researches based on using ML techniques to enhance EC algorithms. In the framework of an ML-technique enhanced-EC algorithm (MLEC), the main idea is that the EC algorithm has stored ample data about the search space, problem features, and population information during the iterative search process, thus the ML technique is helpful in analyzing these data for enhancing the search performance. The paper presents a survey of five categories: ML for population initialization, ML for fitness evaluation and selection, ML for population reproduction and variation, ML for algorithm adaptation, and ML for local search.


Question: Utilizing machine learning methods for forecasting film success: strategies to enhance precision

This study explores the innovative use of socially generated big data to forecast the early success of movies in terms of box office performance. The advent of computational social science has introduced a new methodology for gauging collective human behavior by analyzing vast datasets produced by society. A practical application of this approach is predicting public response and popularity of new products, including movies. Despite the potential of real-time monitoring through big data, accurately converting this information into predictive insights poses significant challenges. In this paper, we present a streamlined predictive model designed to anticipate the financial success of movies well before their release. Our model leverages the collective online activity of editors and viewers interacting with Wikipedia pages dedicated to upcoming films. We demonstrate that by quantifying and examining this activity data, it is possible to forecast a movie's popularity significantly in advance, offering a new dimension to the prediction of box office success.

Sentiment analysis seeks to identify the viewpoint(s) underlying a text span; an example application is classifying a movie review as “thumbs up” or “thumbs down”. To determine this sentiment polarity, we propose a novel machine-learning method that applies text-categorization techniques to just the subjective portions of the document. Extracting these portions can be implemented using efficient techniques for finding minimum cuts in graphs; this greatly facilitates incorporation of cross-sentence contextual constraints. Publication info: Proceedings of the ACL, 2004.

0957-4174/$ see front matter 2008 Elsevier Ltd. A doi:10.1016/j.eswa.2008.09.042 q Part of the content of this paper has been publi International Conference on Multimedia, 2005. * Corresponding author. Tel.: +886 2 29393091x67 E-mail address: mkshan@cs.nccu.edu.tw (M.-K. Sh With the growth of digital music, the development of music recommendation is helpful for users to pick desirable music pieces from a huge repository of music. The existing music recommendation approaches are based on a user’s preference on music. However, sometimes, it might better meet users’ requirement to recommend music pieces according to emotions. In this paper, we propose a novel framework for emotion-based music recommendation. The core of the recommendation framework is the construction of the music emotion model by affinity discovery from film music, which plays an important role in conveying emotions in film. We investigate the music feature extraction and propose the Music Affinity Graph and Music Affinity Graph-Plus algorithms for the construction of music emotion model. Experimental result shows the proposed emotion-based music recommendation achieves 85% accuracy in average. 2008 Elsevier Ltd. All rights reserved.

Evolutionary computation (EC) is a kind of optimization methodology inspired by the mechanisms of biological evolution and behaviors of living organisms. In the literature, the terminology evolutionary algorithms is frequently treated the same as EC. This article focuses on making a survey of researches based on using ML techniques to enhance EC algorithms. In the framework of an ML-technique enhanced-EC algorithm (MLEC), the main idea is that the EC algorithm has stored ample data about the search space, problem features, and population information during the iterative search process, thus the ML technique is helpful in analyzing these data for enhancing the search performance. The paper presents a survey of five categories: ML for population initialization, ML for fitness evaluation and selection, ML for population reproduction and variation, ML for algorithm adaptation, and ML for local search.

C. L. Wu and K. W. Chau* 2 Dept. of Civil and Structural Engineering, Hong Kong Polytechnic University, 3 Hung Hom, Kowloon, Hong Kong, People’s Republic of China 4 5 *Email: cekwchau@polyu.edu.hk 6 ABSTRACT 7 Data-driven techniques such as Auto-Regressive Moving Average (ARMA), K-Nearest-Neighbors (KNN), and 8 Artificial Neural Networks (ANN), are widely applied to hydrologic time series prediction. This paper investigates 9 different data-driven models to determine the optimal approach of predicting monthly streamflow time series. Four 10 sets of data from different locations of People’s Republic of China (Xiangjiaba, Cuntan, Manwan, and 11 Danjiangkou) are applied for the investigation process. Correlation integral and False Nearest Neighbors (FNN) 12 are first employed for Phase Space Reconstruction (PSR). Four models, ARMA, ANN, KNN, and Phase Space 13 Reconstruction-based Artificial Neural Networks (ANN-PSR) are then compared by one-month-ahead forecast 14 using Cuntan and Danjiangkou data. The KNN model performs the best among the four models, but only exhibits 15 weak superiority to ARMA. Further analysis demonstrates that a low correlation between model inputs and 16 outputs could be the main reason to restrict the power of ANN. A Moving Average Artificial Neural Networks 17 (MA-ANN), using the moving average of streamflow series as inputs, is also proposed in this study. The results 18 show that the MA-ANN has a significant improvement on the forecast accuracy compared with the original four 19 models. This is mainly due to the improvement of correlation between inputs and outputs depending on the 20 moving average operation. The optimal memory lengths of the moving average were three and six for Cuntan and 21 Danjiangkou respectively when the optimal model inputs are recognized as the previous twelve months. 22 23

This paper proposes a stock market prediction method exploiting sentiment analysis using financial microblogs (Sina Weibo). We analyze the microblog texts to find the financial sentiments, then combine the sentiments and the historical data of the Shanghai Composite Index (SH000001) to predict the stock market movements. Our framework includes three modules: Microblog Filter (MF), Sentiment Analysis (SA), and Stock Prediction (SP). The MF module is based on LDA to get the financial microblogs. The SA module first sets up a financial lexicon, then gets the sentiments of the microblogs obtained from the MF module. The SP module proposes a user-group model which adjusts the importance of different people, and combines it with stock historical data to predict the movement of the SH000001. We use about 6.1 million microblogs to test our method, and the result demonstrates our method is effective.

Management and analysis of big data are systematically associated with a data distributed architecture in the Hadoop and now Spark frameworks. This article offers an introduction for statisticians to these technologies by comparing the performance obtained by the direct use of three reference environments : R, Python Scikit-learn, ∗Université de Toulouse – INSA, Institut de Mathématiques, UMR CNRS 5219 †Institut de Mathématiques, UMR CNRS 5219 ‡Université de Toulouse – UT3, Institut de Mathématiques, UMR CNRS 5219 1 ar X iv :1 60 9. 09 61 9v 1 [ st at .A P] 3 0 Se p 20 16 Spark MLlib on three public use cases : character recognition, recommending films, categorizing products. As main result, it appears that, if Spark is very efficient for data munging and recommendation by collaborative filtering (non-negative factorization), current implementations of conventional learning methods (logistic regression, random forests) in MLlib or SparkML do not ou poorly compete habitual use of these methods (R, Python Scikit-learn) in an integrated or undistributed architecture.

The increasing availability of digital health records should ideally improve accountability in healthcare. In this context, the study of predictive modeling of healthcare costs forms a foundation for accountable care, at both population and individual patient-level care. In this research we use machine learning algorithms for accurate predictions of healthcare costs on publicly available claims and survey data. Specifically, we investigate the use of the regression trees, M5 model trees and random forest, to predict healthcare costs of individual patients given their prior medical (and cost) history.
 Overall, three observations showcase the utility of our research: (a) prior healthcare cost alone can be a good indicator for future healthcare cost, (b) M5 model tree technique led to very accurate future healthcare cost prediction, and (c) although state-of-the-art machine learning algorithms are also limited by skewed cost distributions in healthcare, for a large fraction (75%) of population, we were able to predict with higher accuracy using these algorithms. In particular, using M5 model trees we were able to accurately predict costs within less than $125 for 75% of the population when compared to prior techniques. Since models for predicting healthcare costs are often used to ascertain overall population health, our work is useful to evaluate future costs for large segments of disease populations with reasonably low error as demonstrated in our results on real-world publicly available datasets.

In recent years the amount of data stored in educational database is growing rapidly. The stored database contains hidden information which if used aids improvement of student's performance and behaviour. In this paper predictive modelling approach is used for extracting this hidden information. Data is collected, a predictive model is formulated, predictions are made, and the model is validated as additional data becomes available. The predictive models will help the instructor to understand how well or how poorly the students in his/her class will perform, and hence the instructor can choose proper pedagogical and instructional interventions to enhance student learning outcomes. The implementation is done in Hadoop framework with MapReduce and Revolutionary R Enterprise RRE.

This paper suggests a number of research directions in which the recommender systems can improve their quality, by moving beyond the assumptions of linearity and independence that are traditionally made. These assumptions, while producing effective and meaningful results, can be suboptimal, as in lots of cases they do not represent the real datasets. In this paper, we discuss three different ways to address some of the previous constraints. More specifically, we focus on the development of methods capturing higher-order relations between the items, cross-feature interactions and intra-set dependencies which can potentially lead to a considerable enhancement of the recommendation accuracy.

Recommender systems have become an important research field since the emergence of the first paper on collaborative filtering in the mid‐1990s. In general, recommender systems are defined as the supporting systems which help users to find information, products, or services (such as books, movies, music, digital products, web sites, and TV programs) by aggregating and analyzing suggestions from other users, which mean reviews from various authorities, and user attributes. However, as academic researches on recommender systems have increased significantly over the last ten years, more researches are required to be applicable in the real world situation. Because research field on recommender systems is still wide and less mature than other research fields. Accordingly, the existing articles on recommender systems need to be reviewed toward the next generation of recommender systems. However, it would be not easy to confine the recommender system researches to specific disciplines, considering the nature of the recommender system researches. So, we reviewed all articles on recommender systems from 37 journals which were published from 2001 to 2010. The 37 journals are selected from top 125 journals of the MIS Journal Rankings. Also, the literature search was based on the descriptors “Recommender system”, “Recommendation system”, “Personalization system”, “Collaborative filtering” and “