Question: Detecting students at risk in massive open online courses.

This paper presents a novel technique for transforming classifier scores into precise multiclass probability estimates, which is crucial for various data mining applications where classification outputs are integrated with other information sources for decision-making processes. These sources might include example-dependent misclassification costs, outputs from other classifiers, or domain-specific knowledge. Earlier calibration methods were limited to binary classification problems. In contrast, this work demonstrates how to generate accurate probability estimates for multiclass scenarios by merging calibrated binary probability predictions. Additionally, the paper introduces an innovative method for obtaining calibrated two-class probability estimates applicable to any classifier capable of ranking examples. Through experiments with naive Bayes and support vector machine classifiers, the study provides empirical evidence from diverse domains such as direct marketing, text categorization, and digit recognition, showcasing the effectiveness of the proposed method in both two-class and multiclass classification tasks.

Predicting student academic performance has long been an important research topic in many academic disciplines. The present study is the first study that develops and compares four types of mathematical models to predict student academic performance in engineering dynamics – a high-enrollment, highimpact, and core course that many engineering undergraduates are required to take. The four types of mathematical models include the multiple linear regression model, the multilayer perception network model, the radial basis function network model, and the support vector machine model. The inputs (i.e., predictor variables) of the models include student’s cumulative GPA, grades earned in four pre-requisite courses (statics, calculus I, calculus II, and physics), and scores on three dynamics mid-term exams (i.e., the exams given to students during the semester and before the final exam). The output of the models is students’ scores on the dynamics final comprehensive exam. A total of 2907 data points were collected from 323 undergraduates in four semesters. Based on the four types of mathematical models and six different combinations of predictor variables, a total of 24 predictive mathematical models were developed from the present study. The analysis reveals that the type of mathematical model has only a slight effect on the average prediction accuracy (APA, which indicates on average how well a model predicts the final exam scores of all students in the dynamics course) and on the percentage of accurate predictions (PAP, which is calculated as the number of accurate predictions divided by the total number of predictions). The combination of predictor variables has only a slight effect on the APA, but a profound effect on the PAP. In general, the support vector machine models have the highest PAP as compared to the other three types of mathematical models. The research findings from the present study imply that if the goal of the instructor is to predict the average academic performance of his/her dynamics class as a whole, the instructor should choose the simplest mathematical model, which is the multiple linear regression model, with student’s cumulative GPA as the only predictor variable. Adding more predictor variables does not help improve the average prediction accuracy of any mathematical model. However, if the goal of the instructor is to predict the academic performance of individual students, the instructor should use the support vector machine model with the first six predictor variables as the inputs of the model, because this particular predictor combination increases the percentage of accurate predictions, and most importantly, allows sufficient time for the instructor to implement subsequent educational interventions to improve student learning. 2012 Elsevier Ltd. All rights reserved.

The goal of this paper is to address the question: ‘why do lurkers lurk?’ Lurkers reportedly makeup the majority of members in online groups, yet little is known about them. Without insight into lurkers, our understanding of online groups is incomplete. Ignoring, dismissing, or misunderstanding lurking distorts knowledge of life online and may lead to inappropriate design of online environments. To investigate lurking, the authors carried out a study of lurking using in-depth, semi-structured interviews with ten members of online groups. 79 reasons for lurking and seven lurkers’ needs are identified from the interview transcripts. The analysis reveals that lurking is a strategic activity involving more than just reading posts. Reasons for lurking are categorized and a gratification model is proposed to explain lurker behavior.

The overall goal of our Software Engineering Teamwork Assessment and Prediction (SETAP) project is to develop effective machine-learning-based methods for assessment and early prediction of student learning effectiveness in software engineering teamwork. Specifically, we use the Random Forest (RF) machine learning (ML) method to predict the effectiveness of software engineering teamwork learning based on data collected during student team project development. These data include over 100 objective and quantitative Team Activity Measures (TAM) obtained from monitoring and measuring activities of student teams during the creation of their final class project in our joint software engineering classes which ran concurrently at San Francisco State University (SFSU), Fulda University (Fulda) and Florida Atlantic University (FAU). In this paper we provide the first RF analysis results done at SFSU on our full data set covering four years of our joint SE classes. These data include 74 student teams with over 380 students, totaling over 30000 discrete data points. These data are grouped into 11 time intervals, each measuring important phases of project development during the class (e.g. early requirement gathering and design, development, testing and delivery). We briefly elaborate on the methods of data collection and describe the data itself. We then show prediction results of the RF analysis applied to this full data set. Results show that we are able to detect student teams who are bound to fail or need attention in early class time with good (about 70%) accuracy. Moreover, the variable importance analysis shows that the features (TAM measures) with high predictive power make intuitive sense, such as late delivery/late responses, time used to help each other, and surprisingly statistics on commit messages to the code repository, etc. In summary, we believe we demonstrate the viability of using ML on objective and quantitative team activity measures to predict student learning of software engineering teamwork, and point to easy-to-measure factors that can be used to guide educators and software engineering managers to implement early intervention for teams bound to fail. Details about the project and the complete ML training database are downloadable from the project web site.

While games are a popular social media for children, there is a real risk that these children are exposed to potential sexual assault. A number of studies have already addressed this issue, however, the data used in previous research did not properly represent the real chats found in multiplayer online games. To address this issue, we obtained real chat data from MovieStarPlanet, a massively multiplayer online game for children. The research described in this paper aimed to detect predatory behaviours in the chats using machine learning methods. In order to achieve a high accuracy on this task, extensive preprocessing was necessary. We describe three different strategies for data selection and preprocessing, and extensively compare the performance of different learning algorithms on the different datasets and features.

This paper presents a data mining methodology to analyze the careers of University graduated students. We present different approaches based on clustering and sequential patterns techniques in order to identify strategies for improving the performance of students and the scheduling of exams. We introduce an ideal career as the career of an ideal student which has taken each examination just after the end of the corresponding course, without delays. We then compare the career of a generic student with the ideal one by using the different techniques just introduced. Finally, we apply the methodology to a real case study and interpret the results which underline that the more students follow the order given by the ideal career the more they get good performance in terms of graduation time and final grade. 2015 Elsevier Ltd. All rights reserved.

Flipped classroom approach has been increasingly adopted in higher institutions. Although this approach has many advantages, there are also many challenges that should be considered. In this paper, we discuss the suitability of this approach to teach computer programming, and we report on our pilot experience of using this approach at Qatar University to teach one subject of computer programming course. It is found that students has positive attitude to this approach, it improves their learning. However, the main challenge was how to involve some of the students in online learning activities.

According to UNESCO, education is a fundamental human right and every nation's citizens should be granted universal access with equal quality to it. Because this goal is yet to be achieved in most countries, in particular in the developing and underdeveloped countries, it is extremely important to find more effective ways to improve education. This paper presents a model based on the application of computational intelligence (data mining and data science) that leads to the development of the student's knowledge profile and that can help educators in their decision making for best orienting their students. This model also tries to establish key performance indicators to monitor objectives' achievement within individual strategic planning assembled for each student. The model uses random forest for classification and prediction, graph description for data structure visualization and recommendation systems to present relevant information to stakeholders. The results presented were built based on the real dataset obtained from a Brazilian private k-9 (elementary school). The obtained results include correlations among key data, a model to predict student performance and recommendations that were generated for the stakeholders.

School of Computing and Mathematics, Faculty of Engineering, University of Ulster at Jordanstown, Shore Road, Newtownabbey, County Antrim BT37 0QB, Northern Ireland, UK The Applied Software Systems Laboratory (TASSL), Electrical and Computer Engineering, Rutgers, The State University of New Jersey, 94 Brett Road, Piscataway, NJ 08854-8058, USA School of Computing and Mathematical Sciences, The SRIF/SHEFC Centre for Virtual Organization Technology Enabling Research (VOTER), Glasgow Caledonian University, 70 Cowcaddens Road, Glasgow G4 0BA, UK Institute for Computer Science and Business Information Systems (ICB), University of Duisburg-Essen, Schützenbahn 70, 45117 Essen, Germany

In this research paper we consider formative assessment (FA) and discuss ways in which it has been implemented in four different university courses. We illustrate the different aspects of FA by deconstructing it and then demonstrating effectiveness in improving both teaching and student achievement. It appears that specifically “what is done” was less important since there were positive achievement gains in each study. While positive gains were realized with use of technology, gains were also realized with implementation of nontechnology dependent techniques. Further, gains were independent of class size or subject matter.

With the increase in available educational data, it is expected that Learning Analytics will become a powerful means to inform and support learners, teachers and their institutions in better understanding and predicting personal learning needs and performance. However, the processes and requirements behind the beneficial application of Learning and Knowledge Analytics as well as the consequences for learning and teaching are still far from being understood. In this paper, we explore the key dimensions of Learning Analytics (LA), the critical problem zones, and some potential dangers to the beneficial exploitation of educational data. We propose and discuss a generic design framework that can act as a useful guide for setting up Learning Analytics services in support of educational practice and learner guidance, in quality assurance, curriculum development, and in improving teacher effectiveness and efficiency. Furthermore, the presented article intends to inform about soft barriers and limitations of Learning Analytics. We identify the required skills and competences that make meaningful use of Learning Analytics data possible to overcome gaps in interpretation literacy among educational stakeholders. We also discuss privacy and ethical issues and suggest ways in which these issues can be addressed through policy guidelines and best practice examples.


Question: Machine Learning Techniques for Identifying Failure Types and Prognostic Maintenance.

This tutorial aims to provide a comprehensive guide to the fundamental concepts of Support Vector (SV) machines as applied to function estimation, particularly focusing on regression tasks. We delve into the essential ideas that underpin SV machines for regression, offering a detailed exploration of their theoretical foundations. Additionally, we present an up-to-date review of the algorithms currently employed for training SV machines, encompassing both the optimization of quadratic programming and innovative strategies for handling large-scale datasets. Our discussion also includes various modifications and extensions to the standard SV algorithm, highlighting their impact on performance and applicability. Moreover, we address the critical topic of regularization from a SV perspective, elucidating its role in enhancing the robustness and generalization capabilities of SV machines in regression analysis.

Earthquakes around the world have been a cause of major destruction and loss of life and property. An early detection and prediction system using machine learning classification models can prove to be very useful for disaster management teams. The earthquake stations continuously collect data even when there is no event. From this data, we need to distinguish earthquake and non-earthquake. Machine learning techniques can be used to analyze continuous time series data to detect earthquakes effectively. Furthermore, the earthquake data can be used to predict the P-wave and S-wave arrival times.

Despite the great developments in information technology, particularly the Internet, computer networks, global information exchange, and its positive impact in all areas of daily life, it has also contributed to the development of penetration and intrusion which forms a high risk to the security of information organizations, government agencies, and causes large economic losses. There are many techniques designed for protection such as firewall and intrusion detection systems (IDS). IDS is a set of software and/or hardware techniques used to detect hacker's activities in computer systems. Two types of anomalies are used in IDS to detect intrusive activities different from normal user behavior. Misuse relies on the knowledge base that contains all known attack techniques and intrusion is discovered through research in this knowledge base. Artificial intelligence techniques have been introduced to improve the performance of these systems. The importance of IDS is to identify unauthorized access attempting to compromise confidentiality, integrity or availability of the computer network. This paper investigates the Intrusion Detection (ID) problem using three machine learning algorithms namely, BayesNet algorithm, Multi-Layer Perceptron (MLP), and Support Vector Machine (SVM). The algorithms are applied on a real, Management Information Based (MIB) dataset that is collected from real life environment. To enhance the detection process accuracy, a set of feature selection approaches is used; Infogain (IG), ReleifF (RF), and Genetic Search (GS). Our experiments show that the three feature selection methods have enhanced the classification performance. GS with bayesNet, MLP and SVM give high accuracy rates, more specifically the BayesNet with the GS accuracy rate is 99.9%.

An important issue in anomaly detection in smart manufacturing systems is the lack of consistency in the formal definitions of anomalies, faults, and attacks. The term anomaly is used to cover a wide range of situations that are addressed by different types of solutions. In this letter, we categorize anomalies in machines, controllers, and networks along with their detection mechanisms, and unify them under a common framework to aid in the identification of potential solutions. The main contribution of the proposed categorization is that it allows the identification of gaps in anomaly detection in smart manufacturing systems.

Service providing companies including telecommunication companies often receive substantial damage from customers’ fraudulent behaviors. One of the common types of fraud is subscription fraud in which usage type is in contradiction with subscription type. This study aimed at identifying customers’ subscription fraud by employing data mining techniques and adopting knowledge discovery process. To this end, a hybrid approach consisting of preprocessing, clustering, and classification phases was applied, and appropriate tools were employed commensurate to each phase. Specifically, in the clustering phase SOM and K-means were combined, and in the classification phase decision tree (C4.5), neural networks, and support vector machines as single classifiers and bagging, boosting, stacking, majority and consensus voting as ensembles were examined. In addition to using clustering to identify outlier cases, it was also possible – by defining new features – to maintain the results of clustering phase for the classification phase. This, in turn, contributed to better classification results. A real dataset provided by Telecommunication Company of Tehran was applied to demonstrate the effectiveness of the proposed method. The efficient use of synergy among these techniques significantly increased prediction accuracy. The performance of all single and ensemble classifiers is evaluated based on various metrics and compared by statistical tests. The results showed that support vector machines among single classifiers and boosted trees among all classifiers have the best performance in terms of various metrics. The research findings show that the proposed model has a high accuracy, and the resulting outcomes are significant both theoretically and practically. & 2010 Elsevier Ltd. All rights reserved.

Much of the intrusion detection research focuses on signature (misuse) detection, where models are built to recognize known attacks. However, signature detection, by its nature, cannot detect novel attacks. Anomaly detection focuses on modeling the normal behavior and identifying significant deviations, which could be novel attacks. In this paper we explore two machine learning methods that can construct anomaly detection models from past behavior. The first method is a rule learning algorithm that characterizes normal behavior in the absence of labeled attack data. The second method uses a clustering algorithm to identify outliers.

We review in this paper several methods from Statistical Learning Theory (SLT) for the performance assessment and uncertainty quantification of predictive models. Computational issues are addressed so to allow the scaling to large datasets and the application of SLT to Big Data analytics. The effectiveness of the application of SLT to manufacturing systems is exemplified by targeting the derivation of a predictive model for quality forecasting of products on an assembly line.

Context. Software testing is the process of finding faults in software while executing it. The results of the testing are used to find and correct faults. Software defect prediction estimates where faults are likely to occur in source code. The results from the defect prediction can be used to optimize testing and ultimately improve software quality. Machine learning, that concerns computer programs learning from data, is used to build prediction models which then can be used to classify data. Objectives. In this study we, in collaboration with Ericsson, investigated whether software metrics from source code files combined with metrics from their respective tests predicts faults with better prediction performance compared to using only metrics from the source code files. Methods. A literature review was conducted to identify inputs for an experiment. The experiment was applied on one repository from Ericsson to identify the best performing set of metrics. Results. The prediction performance results of three metric sets are presented and compared with each other. Wilcoxon’s signed rank tests are performed on four different performance measures for each metric set and each machine learning algorithm to demonstrate significant differences of the results. Conclusions. We conclude that metrics from tests can be used to predict faults. However, the combination of source code metrics and test metrics do not outperform using only source code metrics. Moreover, we conclude that models built with metrics from the test metric set with minimal information of the source code can in fact predict faults in the source code.

Diagnosis, or the method to connect causes to its effects, is an important reasoning task for obtaining insight on cities and reaching the concept of sustainable and smarter cities that is envisioned nowadays. This paper, focusing on transportation and its road traffic, presents how road traffic congestions can be detected and diagnosed in quasi real-time. We adapt pure Artificial Intelligence diagnosis techniques to fully exploit knowledge which is captured through relevant semantics-augmented stream and static data from various domains. Our prototype of semantic-aware diagnosis of road traffic congestions, experimented in Dublin Ireland, works efficiently with large, heterogeneous information sources and delivers value-added services to citizens and city managers in quasi real-time.

Today, machine learning techniques play a significant role in data analysis, predictive modeling and visualization. The main aim of the machine learning algorithms is that they first learn from the empirical data and can be utilized in cases for which the modeled phenomenon is hidden or not yet described. Several significant techniques like Artificial Neural Network, Support Vector Regression, k-Nearest Neighbour, Bayesian Classifiers and Decision Trees are developed in past years to acheive this task. The first and most complicated problem concerns is the amount of available data. This paper reviews state-of-the-art in the domain of spatial data analysis by employing machine learning approaches. First various methods have been summarized, which exist in the literature. Further, the current research scenarios which are going on in this area are described. Based on the research done in past years, identification of the problem in the existing system is also presented in this paper and have given future research directions.

DC-link capacitors are an important part in the majority of power electronic converters which contribute to cost, size and failure rate on a considerable scale. From capacitor users' viewpoint, this paper presents a review on the improvement of reliability of dc link in power electronic converters from two aspects: 1) reliability-oriented dc-link design solutions; 2) conditioning monitoring of dc-link capacitors during operation. Failure mechanisms, failure modes and lifetime models of capacitors suitable for the applications are also discussed as a basis to understand the physics-of-failure. This review serves to provide a clear picture of the state-of-the-art research in this area and to identify the corresponding challenges and future research directions for capacitors and their dc-link applications.


Question: Machine Learning Techniques for Identifying Failure Types and Predictive Maintenance Strategies

This paper presents a method for creating orthonormal bases of compactly supported wavelets, featuring customizable levels of smoothness. The degree of regularity is directly proportional to the width of the support, allowing for precise control over the wavelets' properties. The development of these bases begins with an overview of multiresolution analysis, alongside a discussion of various algorithms used in vision decomposition and reconstruction techniques. The synthesis of these methodologies leads to the construction of the orthonormal bases, integrating the benefits of each approach to achieve enhanced flexibility and performance. However, in the context of predictive maintenance and failure type detection, wavelet analysis can be a powerful tool for feature extraction and anomaly detection in machine learning models. By decomposing signals into different frequency components, wavelets can help in identifying patterns that are indicative of impending failures, thus enabling proactive maintenance strategies. This approach complements traditional statistical methods and can significantly improve the accuracy of predictive models in complex industrial environments.

Context. Software testing is the process of finding faults in software while executing it. The results of the testing are used to find and correct faults. Software defect prediction estimates where faults are likely to occur in source code. The results from the defect prediction can be used to optimize testing and ultimately improve software quality. Machine learning, that concerns computer programs learning from data, is used to build prediction models which then can be used to classify data. Objectives. In this study we, in collaboration with Ericsson, investigated whether software metrics from source code files combined with metrics from their respective tests predicts faults with better prediction performance compared to using only metrics from the source code files. Methods. A literature review was conducted to identify inputs for an experiment. The experiment was applied on one repository from Ericsson to identify the best performing set of metrics. Results. The prediction performance results of three metric sets are presented and compared with each other. Wilcoxon’s signed rank tests are performed on four different performance measures for each metric set and each machine learning algorithm to demonstrate significant differences of the results. Conclusions. We conclude that metrics from tests can be used to predict faults. However, the combination of source code metrics and test metrics do not outperform using only source code metrics. Moreover, we conclude that models built with metrics from the test metric set with minimal information of the source code can in fact predict faults in the source code.

An important issue in anomaly detection in smart manufacturing systems is the lack of consistency in the formal definitions of anomalies, faults, and attacks. The term anomaly is used to cover a wide range of situations that are addressed by different types of solutions. In this letter, we categorize anomalies in machines, controllers, and networks along with their detection mechanisms, and unify them under a common framework to aid in the identification of potential solutions. The main contribution of the proposed categorization is that it allows the identification of gaps in anomaly detection in smart manufacturing systems.

In this paper, we examine the use of machine learning to improve the robustness of systems for image analysis on the task of roof detection. We review the problem of analyzing aerial photographs, and describe an existing vision system that attempts to automate the identiication of buildings in aerial images. After this, we brieey review several well-known learning algorithms that represent a wide variety of inductive biases. We report three experiments designed to illuminate facets of applying machine learning methods to the image analysis task; one experiment focuses on within-image learning, another deals with the cost of diierent errors, and a third addresses between-image learning. Experimental results demonstrate that machine-learned classiiers meet or exceed the accuracy of handcrafted solutions and that useful generalization occurs when training and testing on data derived from diierent images.

Machine-learning technology powers many aspects of modern society: from web searches to content filtering on social networks to recommendations on e-commerce websites, and it is increasingly present in consumer products such as cameras and smartphones. Machine-learning systems are used to identify objects in images, transcribe speech into text, match news items, posts or products with users’ interests, and select relevant results of search. Increasingly, these applications make use of a class of techniques called deep learning. Conventional machine-learning techniques were limited in their ability to process natural data in their raw form. For decades, constructing a pattern-recognition or machine-learning system required careful engineering and considerable domain expertise to design a feature extractor that transformed the raw data (such as the pixel values of an image) into a suitable internal representation or feature vector from which the learning subsystem, often a classifier, could detect or classify patterns in the input. Representation learning is a set of methods that allows a machine to be fed with raw data and to automatically discover the representations needed for detection or classification. Deep-learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but non-linear modules that each transform the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. With the composition of enough such transformations, very complex functions can be learned. For classification tasks, higher layers of representation amplify aspects of the input that are important for discrimination and suppress irrelevant variations. An image, for example, comes in the form of an array of pixel values, and the learned features in the first layer of representation typically represent the presence or absence of edges at particular orientations and locations in the image. The second layer typically detects motifs by spotting particular arrangements of edges, regardless of small variations in the edge positions. The third layer may assemble motifs into larger combinations that correspond to parts of familiar objects, and subsequent layers would detect objects as combinations of these parts. The key aspect of deep learning is that these layers of features are not designed by human engineers: they are learned from data using a general-purpose learning procedure. Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years. It has turned out to be very good at discovering intricate structures in high-dimensional data and is therefore applicable to many domains of science, business and government. In addition to beating records in image recognition and speech recognition, it has beaten other machine-learning techniques at predicting the activity of potential drug molecules, analysing particle accelerator data, reconstructing brain circuits, and predicting the effects of mutations in non-coding DNA on gene expression and disease. Perhaps more surprisingly, deep learning has produced extremely promising results for various tasks in natural language understanding, particularly topic classification, sentiment analysis, question answering and language translation. We think that deep learning will have many more successes in the near future because it requires very little engineering by hand, so it can easily take advantage of increases in the amount of available computation and data. New learning algorithms and architectures that are currently being developed for deep neural networks will only accelerate this progress.

This paper describes a transient event classification scheme, system identification techniques, and implementation for use in nonintrusive load monitoring. Together, these techniques form a system that can determine the operating schedule and find parameters of physical models of loads that are connected to an AC or DC power distribution system. The monitoring system requires only off-the-shelf hardware and recognizes individual transients by disaggregating the signal from a minimal number of sensors that are installed at a central location in the distribution system. Implementation details and field tests for AC and DC systems are presented.

This paper proposes an empirical-mode decomposition (EMD) and Hilbert transform (HT)-based method for the classification of power quality (PQ) events. Nonstationary power signal disturbance waveforms are considered as the superimposition of various undulating modes, and EMD is used to separate out these intrinsic modes known as intrinsic mode functions (IMFs). The HT is applied on all the IMFs to extract instantaneous amplitude and frequency components. This time-frequency analysis results in the clear visual detection, localization, and classification of the different power signal disturbances. The required feature vectors are extracted from the time-frequency distribution to perform the classification. A balanced neural tree is constructed to classify the power signal patterns. Finally, the proposed method is compared with an S-transform-based classifier to show the efficacy of the proposed technique in classifying the PQ disturbances.

We review in this paper several methods from Statistical Learning Theory (SLT) for the performance assessment and uncertainty quantification of predictive models. Computational issues are addressed so to allow the scaling to large datasets and the application of SLT to Big Data analytics. The effectiveness of the application of SLT to manufacturing systems is exemplified by targeting the derivation of a predictive model for quality forecasting of products on an assembly line.

As a result of the recent trend towards digitization -- which increasingly affects evidence-based medicine, accountable care, personalized medicine, and medical "Big Data" analysis -- growing amounts of clinical data are becoming available for analysis. In this paper, we follow the idea that one can model clinical processes based on clinical data, which can then be the basis for many useful applications. We model the whole clinical evolution of each individual patient, which is composed of thousands of events such as ordered tests, lab results and diagnoses. Specifically, we base our work on a dataset provided by the Charité University Hospital of Berlin which is composed of patients that suffered from kidney failure and either obtained an organ transplant or are still waiting for one. These patients face a lifelong treatment and periodic visits to the clinic. Our goal is to develop a system to predict the sequence of events recorded in the electronic medical record of each patient, and thus to develop the basis for a future clinical decision support system. For modelling, we use machine learning approaches which are based on a combination of the embedding of entities and events in a multidimensional latent space, in combination with Neural Network predictive models. Similar approaches have been highly successful in statistical models for recommendation systems, language models, and knowledge graphs. We extend existing embedding models to the clinical domain, in particular with respect to temporal sequences, long-term memories and personalization. We compare the performance of our proposed models with standard approaches such as K-nearest neighbors method, Naïve Bayes classifier and Logistic Regression, and obtained favorable results with our proposed model.

The bankruptcy prediction research domain continues to evolve with many new different predictive models developed using various tools. Yet many of the tools are used with the wrong data conditions or for the wrong situation. Using the Web of Science, Business Source Complete and Engineering Village databases, a systematic review of 49 journal articles published between 2010 and 2015 was carried out. This review shows how eight popular and promising tools perform based on 13 key criteria within the bankruptcy prediction models research area. These tools include two statistical tools: multiple discriminant analysis and Logistic regression; and six artificial intelligence tools: artificial neural network, support vector machines, rough sets, case based reasoning, decision tree and genetic algorithm. The 13 criteria identified include accuracy, result transparency, fully deterministic output, data size capability, data dispersion, variable selection method required, variable types applicable, and more. Overall, it was found that no single tool is predominantly better than other tools in relation to the 13 identified criteria. A tabular and a diagrammatic framework are provided as guidelines for the selection of tools that best fit different situations. It is concluded that an overall better performance model can only be found by informed integration of tools to form a hybrid model. This paper contributes towards a thorough understanding of the features of the tools used to develop bankruptcy prediction models and their related shortcomings.

0026-2714/$ see front matter 2009 Published by doi:10.1016/j.microrel.2009.07.055 * Corresponding author. Tel.: +44 (0)20 8331 8660 E-mail address: c.bailey@gre.ac.uk (C. Bailey). Power electronics uses semiconductor technology to convert and control electrical power. Demands for efficient energy management, conversion and conservation, and the increasing take-up of electronics in transport systems has resulted in tremendous growth in the use of power electronics devices such as Insulated Gate Bipolar Transistors (IGBT’s). The packaging of power electronics devices involves a number of challenges for the design engineer in terms of reliability. For example, IGBT modules will contain a number of semiconductor dies within a small footprint bonded to substrates with aluminum wires and wide area solder joints. To a great extent, the reliability of the package will depend on the thermo-mechanical behavior of these materials. This paper details a physics of failure approach to reliability predictions of IGBT modules. It also illustrates the need for a probabilistic approach to reliability predictions that include the effects of design variations. Also discussed are technologies for predicting the remaining life of the package when subjected to qualification stresses or in service stresses using prognostics methods. 2009 Published by Elsevier Ltd.


Question: Machine Learning Techniques for Identifying Failure Types and Prognostic Maintenance.

This paper provides an overview of the foundational principles of hidden Markov models (HMMs), a probabilistic framework that has gained significant traction in the field of speech processing over the past several decades. Despite the underlying theory of Markov chains being established nearly eight decades ago, its explicit application to speech-related challenges is a relatively recent development. A pivotal factor hindering the integration of Markov chain-based models into speech processing was the absence of a robust method to calibrate the model's parameters in alignment with empirical data patterns. This obstacle was surmounted in the late 1960s with the introduction of a parameter optimization technique that was swiftly adopted by various research institutions for speech processing applications. The subsequent evolution of Markov modeling methodologies, characterized by continuous theoretical enhancements and implementation refinements, has catalyzed a proliferation of applications for these models across diverse domains. This paper aims to demystify the theoretical underpinnings of Markov models and elucidate their practical utility in the realm of speech recognition, underscoring their pivotal role in advancing the field.

Requirement engineers are not able to elicit and analyze the security requirements clearly, that are essential for the development of secure and reliable software. Proper identification of security requirements present in the Software Requirement Specification (SRS) document has been a problem being faced by the developers. As a result, they are not able to deliver the software free from threats and vulnerabilities. Thus, in this paper, we intend to mine the descriptions of security requirements present in the SRS document and thereafter develop the classification models. The security-based descriptions are analyzed using text mining techniques and are then classified into four types of security requirements viz. authentication-authorization, access control, cryptography-encryption and data integrity using J48 decision tree method. Corresponding to each type of security requirement, a prediction model has been developed. The effectiveness of the prediction models is evaluated against requirement specifications collected from 15 projects which have been developed by MS students at DePaul University. The result analysis indicated that all the four models have performed very well in predicting their respective type of security requirements.

We show in this paper that multi-layer dependability is an indispensable way to cope with the increasing amount of technology-induced dependability problems that threaten to proceed further scaling. We introduce the definition of multi-layer dependability and present our design flow within this paradigm that seamlessly integrates techniques starting at circuit layer all the way up to application layer and thereby accounting for ASIC-based architectures as well as for reconfigurable-based architectures. At the end, we give evidence that the paradigm of multi-layer dependability bears a large potential for significantly increasing dependability at reasonable effort.

It is estimated that over 90% of all new information produced in the world is being stored on magnetic media, most of it on hard disk drives. Despite their importance, there is relatively little published work on the failure patterns of disk drives, and the key factors that affect their lifetime. Most available data are either based on extrapolation from accelerated aging experiments or from relatively modest sized field studies. Moreover, larger population studies rarely have the infrastructure in place to collect health signals from components in operation, which is critical information for detailed failure analysis. We present data collected from detailed observations of a large disk drive population in a production Internet services deployment. The population observed is many times larger than that of previous studies. In addition to presenting failure statistics, we analyze the correlation between failures and several parameters generally believed to impact longevity. Our analysis identifies several parameters from the drive’s self monitoring facility (SMART) that correlate highly with failures. Despite this high correlation, we conclude that models based on SMART parameters alone are unlikely to be useful for predicting individual drive failures. Surprisingly, we found that temperature and activity levels were much less correlated with drive failures than previously reported.

This paper presents a review of the literature on the application of data mining techniques for the detection of insurance fraud. Academic literature were analyzed and classified into three types of insurance fraud (automobile insurance, crop insurance and healthcare insurance) and six classes of data mining techniques (classification, regression, clustering, prediction, outlier detection, and visualization). The findings of this review clearly show that automobile insurance fraud detection have also attracted a great deal of attention in recent years. The main data mining techniques used for insurance fraud detection are logistic models, Decision tree, the Naïve Bayes, and support vector machine.

The paper describes the mobile information and communication aspects of a next generation train maintenance and diagnosis system, discusses the working prototype features, and research results. Wearable/ Mobile computers combined with the wireless technology improve efficiency and accuracy of the maintenance work. This technology enables maintenance personnel at the site to communicate with a remote helpdesk / expertise center through digital data, audio, and image.

Machine Learning is the technology, in which algorithms which are capable of learning from previous cases and past experiences are designed. It is implemented using various algorithms which reiterate over the same data repeatedly to analyze the pattern of data. The techniques of data mining are no far behind and are widely used to extract data from large databases to discover some patterns making decisions. This paper presents the Naïve Bayes improved K-Nearest Neighbor method (NBKNN) for Fraud Detection of Credit Card. Experimental results illustrate that both classifiers work differently for the same dataset. The purpose is to enhance the accuracy and enhance the flexibility of the algorithm.

The state machine approach is a general method for implementing fault-tolerant services in distributed systems. This paper reviews the approach and describes protocols for two different failure models—Byzantine and fail stop. Systems reconfiguration techniques for removing faulty components and integrating repaired components are also discussed.

On-line fault detection in sensor networks is of paramount importance due to the convergence of a variety of challenging technological, application, conceptual, and safety related factors. We introduce a taxonomy for classi£cation of faults in sensor networks and the £rst on-line model-based testing technique. The approach is generic in the sense that it can be applied on an arbitrary system of heterogeneous sensors with an arbitrary type of fault model, while it provides a ¤exible tradeoff between accuracy and latency. The key idea is to formulate on-line testing as a set of instances of a non-linear function minimization and consequently apply nonparametric statistical methods to identify the sensors that have the highest probability to be faulty. The optimization is conducted using the Powell nonlinear function minimization method. The effectiveness of the approach is evaluated in the presence of random noise using a system of light sensors.

Earthquakes around the world have been a cause of major destruction and loss of life and property. An early detection and prediction system using machine learning classification models can prove to be very useful for disaster management teams. The earthquake stations continuously collect data even when there is no event. From this data, we need to distinguish earthquake and non-earthquake. Machine learning techniques can be used to analyze continuous time series data to detect earthquakes effectively. Furthermore, the earthquake data can be used to predict the P-wave and S-wave arrival times.

The paper is devoted to supervise method approach to identify the vessel anomaly behaviour in waterways using the Automated Identification System (AIS) vessel reporting data. In this work, we describe the use of SVMs to detect the vessel anomaly behaviour. The SVMs is a supervised method that needs some pre knowledge to extract the maritime movement patterns of AIS raw data into information. This is the basis to remodel information into a meaningful and valuable form. The result of this work shows that the SVMs technique is applicable to be used for the identification of vessel anomaly behaviour. It is proved that the best accuracy result is obtained from dividing raw data into 70% for training and 30% for testing stages.


Question: Machine Learning Techniques for Identifying Failure Types and Predictive Maintenance Strategies.

In the realm of failure type detection and predictive maintenance, statistical learning theory plays a pivotal role. This theoretical framework provides the foundation for understanding how machine learning models can be applied to predict equipment failures and categorize their types. Statistical learning theory emphasizes the importance of model generalization and the balance between bias and variance to ensure that predictive models perform well on unseen data. In the context of predictive maintenance, this theory guides the selection of appropriate algorithms, such as support vector machines or random forests, which are capable of handling complex, high-dimensional data often encountered in industrial settings. Furthermore, it informs the process of feature selection and engineering, helping to identify the most relevant data attributes for predicting different failure modes. By leveraging the principles of statistical learning theory, maintenance strategies can evolve from reactive to proactive, minimizing downtime and maintenance costs while maximizing equipment lifespan. This approach requires careful model validation through cross-validation techniques to ensure reliability and robustness in real-world applications. Overall, statistical learning theory is crucial for developing accurate and efficient machine learning models tailored to the specific challenges of failure type detection and predictive maintenance in various industries.

5G represents the next generation of communication networks and services, and will bring a new set of use cases and scenarios. These in turn will address a new set of challenges from the network and service management perspective, such as network traffic and resource management, big data management and energy efficiency. Consequently, novel techniques and strategies are required to address these challenges in a smarter way. In this paper, we present the limitations of the current network and service management and describe in detail the challenges that 5G is expected to face from a management perspective. The main contribution of this paper is presenting a set of use cases and scenarios of 5G in which machine learning can aid in addressing their management challenges. It is expected that machine learning can provide a higher and more intelligent level of monitoring and management of networks and applications, improve operational efficiencies and facilitate the requirements of the future 5G network.

Modern healthcare service records, called Claims, record the medical treatments by a Provider (Doctor/Clinic), medication advised etc., along with the charges, and payments to be made by the patient and the Payer (insurance provider). Denial and rejection of healthcare claims is a significant administrative burden and source of loss to various healthcare providers and payers as well. Automating the identification of Claims prone to denial by reason, source, cause and other deciding factors is critical to lowering this burden of rework. We present classification methods based on Machine Learning (ML) to fully automate identification of such claims prone to rejection or denial with high accuracy, investigate the reasons for claims denial and recommend methods to engineer features using Claim Adjustment Reason Codes (CARC) as features with high Information Gain. The ML engine reported is first of its kind in Claims risk identification and represents a novel, significant enhancement to the state of practice of using ML for automating and containing claims denial risks.

Customer churn prediction has gathered greater interest in business especially in telecommunications industries. Many authors have presented different versions of the churn prediction models greatly based on the data mining concepts employing the machine learning and meta-heuristic algorithms. This aim of this paper is to study some of the most important churn prediction techniques developed over the recent years. The primary objective is on the churn in telecom industries to accurately estimate the customer survival and customer hazard functions to gain the complete knowledge of churn over the customer tenure. Another objective is the identification of the customers who are at the blink of churn and approximating the time they will churn. This paper focuses on analyzing the churn prediction techniques to identify the churn behavior and validate the reasons for customer churn. This paper summarizes the churn prediction techniques in order to have a deeper understanding of the customer churn and it shows that most accurate churn prediction is given by the hybrid models rather than single algorithms so that telecom industries become aware of the needs of high risk customers and enhance their services to overturn the churn decision.

Internet of things (IoT) is nowadays one of the fastest growing technologies for both private and business purposes. Due to a big number of IoT devices and their rapid introduction to the market, security of things and their services is often not at the expected level. Recently, machine learning algorithms, techniques, and methods are used in research papers to enhance IoT security. In this paper, we systematically review the state-of-the art to classify the research on machine learning for the IoT security. We analysed the primary studies, identify types of studies and publication fora. Next, we have extracted all machine learning algorithms and techniques described in primary studies, and identified the most used ones to tackle IoT security issues. We classify the research into three main categories (intrusion detection, authentication and other) and describe the primary studies in detail to analyse existing relevant works and propose topics for future research.

ISSN 2250 – 1061 | © 2011 Bonfring Abstract--In machine learning, classification refers to a step by step procedure for designating a given piece of input data into any one of the given categories. There are many classification problem occurs and need to be solved. Different types are classification algorithms like tree-based, rule-based, etc are widely used. This work studies the effectiveness of Rule-Based classifiers for classification by taking a sample data set from UCI machine learning repository using the open source machine learning tool. A comparison of different rulebased classifiers used in Data Mining and a practical guideline for selecting the most suited algorithm for a classification is presented and some empirical criteria for describing and evaluating the classifiers are given.

Mechanical devices such as engines, vehicles, aircrafts, etc., are typically instrumented with numerous sensors to capture the behavior and health of the machine. However, there are often external factors or variables which are not captured by sensors leading to time-series which are inherently unpredictable. For instance, manual controls and/or unmonitored environmental conditions or load may lead to inherently unpredictable time-series. Detecting anomalies in such scenarios becomes challenging using standard approaches based on mathematical models that rely on stationarity, or prediction models that utilize prediction errors to detect anomalies. We propose a Long Short Term Memory Networks based Encoder-Decoder scheme for Anomaly Detection (EncDec-AD) that learns to reconstruct ‘normal’ time-series behavior, and thereafter uses reconstruction error to detect anomalies. We experiment with three publicly available quasi predictable time-series datasets: power demand, space shuttle, and ECG, and two realworld engine datasets with both predictive and unpredictable behavior. We show that EncDecAD is robust and can detect anomalies from predictable, unpredictable, periodic, aperiodic, and quasi-periodic time-series. Further, we show that EncDec-AD is able to detect anomalies from short time-series (length as small as 30) as well as long time-series (length as large as 500).

This research focuses on Text Classification. Text classification is the task of automatically sorting a set of documents into categories from a predefined set. The domain of this research is the combination of information ret rieval (IR) technology, Data mining and machine learning (ML) technology. This research will outline the fundamental traits of the technologies involved. This research uses three text classification algorithms (Naive Bayes, VSM for text classificatio n and the new technique -Use of Stanford Tagger for text classific ation) to classify documents into different categories, which is trained on two different datasets (20 Newsgroups and New news dataset for five categories).In regards to the above classifica tion strategies, Naïve Bayes is potentially good at serving as a tex classification model due to its simplicity.

Support tickets from customers contain much hidden information. Unsupervised machine learning methods are able to discover this hidden information. In this paper we propose the categorization of support tickets using clustering methods in combination with topic models. Furthermore label generation techniques are used to generate meaningful names for these categories. The results are compared with related

We review in this paper several methods from Statistical Learning Theory (SLT) for the performance assessment and uncertainty quantification of predictive models. Computational issues are addressed so to allow the scaling to large datasets and the application of SLT to Big Data analytics. The effectiveness of the application of SLT to manufacturing systems is exemplified by targeting the derivation of a predictive model for quality forecasting of products on an assembly line.

Timely detection and identification of faults in railway track circuits are crucial for the safety and availability of railway networks. In this paper, the use of the long-short-term memory (LSTM) recurrent neural network is proposed to accomplish these tasks based on the commonly available measurement signals. By considering the signals from multiple track circuits in a geographic area, faults are diagnosed from their spatial and temporal dependences. A generative model is used to show that the LSTM network can learn these dependences directly from the data. The network correctly classifies 99.7% of the test input sequences, with no false positive fault detections. In addition, the t-Distributed Stochastic Neighbor Embedding (t-SNE) method is used to examine the resulting network, further showing that it has learned the relevant dependences in the data. Finally, we compare our LSTM network with a convolutional network trained on the same task. From this comparison, we conclude that the LSTM network architecture is better suited for the railway track circuit fault detection and identification tasks than the convolutional network.


Question: Machine Learning Techniques for Identifying Failure Types and Proactive Maintenance Strategies

This article explores the shortcomings of time-based maintenance strategies and highlights the benefits of predictive or online maintenance methodologies in detecting the early signs of equipment failure. It categorizes the primary predictive maintenance techniques based on their data sources into three groups: 1) the traditional sensor-based approach; 2) the test-sensor-based strategy (encompassing wireless sensor technology); and 3) the test-signal-based method (incorporating techniques like loop current step response, time-domain reflectometry, and inductance-capacitance-resistance testing). Case studies are provided, showcasing the effectiveness of sensor-based techniques in identifying obstructions within pressure sensing lines and the validation of calibration through direct current output measurements using existing sensors. The paper also outlines three initiatives funded by the Department of Energy (DOE), all geared towards the development of online and wireless systems designed for predictive maintenance of critical equipment. These projects target applications in nuclear power plants, DOE research reactors, and a broader range of industrial settings, demonstrating a commitment to advancing maintenance practices through cutting-edge technology.

The bankruptcy prediction research domain continues to evolve with many new different predictive models developed using various tools. Yet many of the tools are used with the wrong data conditions or for the wrong situation. Using the Web of Science, Business Source Complete and Engineering Village databases, a systematic review of 49 journal articles published between 2010 and 2015 was carried out. This review shows how eight popular and promising tools perform based on 13 key criteria within the bankruptcy prediction models research area. These tools include two statistical tools: multiple discriminant analysis and Logistic regression; and six artificial intelligence tools: artificial neural network, support vector machines, rough sets, case based reasoning, decision tree and genetic algorithm. The 13 criteria identified include accuracy, result transparency, fully deterministic output, data size capability, data dispersion, variable selection method required, variable types applicable, and more. Overall, it was found that no single tool is predominantly better than other tools in relation to the 13 identified criteria. A tabular and a diagrammatic framework are provided as guidelines for the selection of tools that best fit different situations. It is concluded that an overall better performance model can only be found by informed integration of tools to form a hybrid model. This paper contributes towards a thorough understanding of the features of the tools used to develop bankruptcy prediction models and their related shortcomings.

Text mining is a process of extracting information of interest from text. Such a method includes techniques from various areas such as Information Retrieval (IR), Natural Language Processing (NLP), and Information Extraction (IE). In this study, text mining methods are applied to extract causal relations from maritime accident investigation reports collected from the Marine Accident Investigation Branch (MAIB). These causal relations provide information on various mechanisms behind accidents, including human and organizational factors relating to the accident. The objective of this study is to facilitate the analysis of the maritime accident investigation reports, by means of extracting contributory causes with more feasibility. A careful investigation of contributory causes from the reports provide opportunity to improve safety in future. Two methods have been employed in this study to extract the causal relations. They are 1) Pattern classification method and 2) Connectives method. The earlier one uses naı̈ve Bayes and Support Vector Machines (SVM) as classifiers. The latter simply searches for the words connecting cause and effect in sentences. The causal patterns extracted using these two methods are compared to the manual (human expert) extraction. The pattern classification method showed a fair and sensible performance with F-measure(average) = 65% when compared to connectives method with F-measure(average) = 58%. This study is an evidence, that text mining methods could be employed in extracting causal relations from marine accident investigation reports.

During building operation, a significant amount of energy is wasted due to equipment and humanrelated faults. To reduce waste, today’s smart buildings monitor energy usage with the aim of identifying abnormal consumption behaviour and notifying the building manager to implement appropriate energysaving procedures. To this end, this research proposes a new pattern-based anomaly classifier, the collective contextual anomaly detection using sliding window (CCAD-SW) framework. The CCAD-SW framework identifies anomalous consumption patterns using overlapping sliding windows. To enhance the anomaly detection capacity of the CCAD-SW, this research also proposes the ensemble anomaly detection (EAD) framework. The EAD is a generic framework that combines several anomaly detection classifiers

Timely detection and identification of faults in railway track circuits are crucial for the safety and availability of railway networks. In this paper, the use of the long-short-term memory (LSTM) recurrent neural network is proposed to accomplish these tasks based on the commonly available measurement signals. By considering the signals from multiple track circuits in a geographic area, faults are diagnosed from their spatial and temporal dependences. A generative model is used to show that the LSTM network can learn these dependences directly from the data. The network correctly classifies 99.7% of the test input sequences, with no false positive fault detections. In addition, the t-Distributed Stochastic Neighbor Embedding (t-SNE) method is used to examine the resulting network, further showing that it has learned the relevant dependences in the data. Finally, we compare our LSTM network with a convolutional network trained on the same task. From this comparison, we conclude that the LSTM network architecture is better suited for the railway track circuit fault detection and identification tasks than the convolutional network.

This paper presents a fault diagnosis (FD) method for on line monitoring of stator winding partial inter-turn fault and severe faults, in permanent magnet synchronous generator (PMSG) used in shaft generator system in marine applications. The faulty machine is represented in state space model based on the dq-frame transformation. The Extended Kalman Filter (EKF) parameters estimation technique is used to estimate the value of stator winding short-circuit parameters of the machine. The proposed technique has been simulated for different fault scenarios using Matlab®/Simulink®. Simulation results show that the proposed technique is robust and effective. Moreover it can be applied at different operating conditions to prevent development of complete failure.

As a result of the recent trend towards digitization -- which increasingly affects evidence-based medicine, accountable care, personalized medicine, and medical "Big Data" analysis -- growing amounts of clinical data are becoming available for analysis. In this paper, we follow the idea that one can model clinical processes based on clinical data, which can then be the basis for many useful applications. We model the whole clinical evolution of each individual patient, which is composed of thousands of events such as ordered tests, lab results and diagnoses. Specifically, we base our work on a dataset provided by the Charité University Hospital of Berlin which is composed of patients that suffered from kidney failure and either obtained an organ transplant or are still waiting for one. These patients face a lifelong treatment and periodic visits to the clinic. Our goal is to develop a system to predict the sequence of events recorded in the electronic medical record of each patient, and thus to develop the basis for a future clinical decision support system. For modelling, we use machine learning approaches which are based on a combination of the embedding of entities and events in a multidimensional latent space, in combination with Neural Network predictive models. Similar approaches have been highly successful in statistical models for recommendation systems, language models, and knowledge graphs. We extend existing embedding models to the clinical domain, in particular with respect to temporal sequences, long-term memories and personalization. We compare the performance of our proposed models with standard approaches such as K-nearest neighbors method, Naïve Bayes classifier and Logistic Regression, and obtained favorable results with our proposed model.

Intelligent systems rely on pattern recognition and signaturebased approaches for a wide range of sensors enhancing situational awareness. For example, autonomous systems depend on environmental sensors to perform their tasks and secure systems depend on anomaly detection methods. The availability of large amount of data requires the processing of data in a “streaming” fashion with online algorithms. Yet, just as online learning can enhance adaptability to a non-stationary environment, it introduces vulnerabilities that can be manipulated by adversaries to achieve their goals while evading detection. Although human intelligence might have evolved from social interactions, machine intelligence has evolved as a human intelligence artifact and been kept isolated to avoid ethical dilemmas. As our adversaries become sophisticated, it might be time to revisit this question and examine how we can combine online learning and reasoning leading to the science of deceptive and counter-deceptive machines.

Business process automation technologies are being increasingly used by many companies to improve the efficiency of both internal processes as well as of e-services offered to customers. In order to satisfy customers and employees, business processes need to be executed with a high and predictable quality. In particular, it is crucial for organizations to meet the Service Level Agreements (SLAs) stipulated with the customers and to foresee as early as possible the risk of missing SLAs, in order to set the right expectations and to allow for corrective actions. In this paper we focus on a critical issue in business process quality: that of analyzing, predicting and preventing the occurrence of exceptions, i.e., of deviations from the desired or acceptable behavior. We characterize the problem and propose a solution, based on data warehousing and mining techniques. We then describe the architecture and implementation of a tool suite that enables exception analysis, prediction, and prevention. Finally, we show experimental results obtained by using the tool suite to analyze internal HP processes.

Systems with several components interacting to accomplish challenging tasks are ubiquitous; examples include large server clusters providing “cloud computing”, manufacturing plants, automobiles, etc. Our relentless efforts to improve the capabilities of these systems inevitably increase their complexity as we add more components or introduce more dependencies between existing ones. To tackle this surge in distributed system complexity, system operators collect continuous monitoring data from various sources including hardware and software-based sensors. A large amount of this data is either in the form of time-series or contained in logs, e.g., operators’ activity, system event, and error logs, etc. In this paper, we propose a framework for mining system operational intelligence from massive amount of monitoring data that combines the time series data with information extracted from text-based logs. Our work is aimed at systems where logs capture the context of a system’s operations and the time-series data record the state of different components. This category includes a wide variety of systems including IT systems (compute clouds, web services’ infrastructure, enterprise computing infrastructure, etc.) and complex physical systems such as manufacturing plants. We instantiate our framework for Hadoop. Our preliminary results using both real and synthetic datasets show that the proposed context-aware approach is more effective for detecting anomalies compared to a time series only approach.

BACKGROUND
Mental illness is quickly becoming one of the most prevalent public health problems worldwide. Social network platforms, where users can express their emotions, feelings, and thoughts, are a valuable source of data for researching mental health, and techniques based on machine learning are increasingly used for this purpose.

OBJECTIVE
The objective of this review was to explore the scope and limits of cutting-edge techniques that researchers are using for predictive analytics in mental health and to review associated issues, such as ethical concerns, in this area of research.

METHODS
We performed a systematic literature review in March 2017, using keywords to search articles on data mining of social network data in the context of common mental health disorders, published between 2010 and March 8, 2017 in medical and computer science journals.

RESULTS
The initial search returned a total of 5386 articles. Following a careful analysis of the titles, abstracts, and main texts, we selected 48 articles for review. We coded the articles according to key characteristics, techniques used for data collection, data preprocessing, feature extraction, feature selection, model construction, and model verification. The most common analytical method was text analysis, with several studies using different flavors of image analysis and social interaction graph analysis.

CONCLUSIONS
Despite an increasing number of studies investigating mental health issues using social network data, some common problems persist. Assembling large, high-quality datasets of social media users with mental disorder is problematic, not only due to biases associated with the collection methods, but also with regard to managing consent and selecting appropriate analytics techniques.


Question: Argumentative zoning to enhance citation indexing efficiency.

Digital Libraries and Enhanced Citation Indexing through Argumentative Zoning

The internet has fundamentally transformed the landscape of scientific literature access, yet the online repository of such literature remains largely unstructured and chaotic. To address this issue, the concept of autonomous citation indexing has emerged, aiming to streamline the organization of scientific works by automating the creation of citation indices. This approach not only enhances the dissemination and retrieval of scholarly articles but also offers significant advancements in terms of cost-effectiveness, accessibility, comprehensiveness, operational efficiency, and promptness. By integrating argumentative zoning techniques, autonomous citation indexing can further refine the indexing process, enabling more accurate and context-sensitive identification of citations within documents. This enhancement ensures that the system can distinguish between various types of citations, such as supportive, critical, or merely acknowledging references, thereby providing a more nuanced understanding of the scholarly discourse.

Web search engines use highly optimized compression schemes to decrease inverted index size and improve query throughput, and many index compression techniques have been studied in the literature. One approach taken by several recent studies first performs a renumbering of the document IDs in the collection that groups similar documents together, and then applies standard compression techniques. It is known that this can significantly improve index compression compared to a random document ordering. We study index compression and query processing techniques for such reordered indexes. Previous work has focused on determining the best possible ordering of documents. In contrast, we assume that such an ordering is already given, and focus on how to optimize compression methods and query processing for this case. We perform an extensive study of compression techniques for document IDs and present new optimizations of existing techniques which can achieve significant improvement in both compression and decompression performances. We also propose and evaluate techniques for compressing frequency values for this case. Finally, we study the effect of this approach on query processing performance. Our experiments show very significant improvements in index size and query processing speed on the TREC GOV2 collection of 25.2 million web pages.

This article statistically analyses how the citatio n impact of articles deposited in the Condensed Matter section of the preprint server ArX iv (hosted by Cornell University), and subsequently published in a scientific journal, compares to that of articles in the same journal that were not deposited in that archive. Its principal aim is to further illustrate and roughly estimate the effect of two factors, ‘early view’ and ‘quality bias’, upon differences in citation impact between these two se ts of papers, using citation data from Thomson Scientific’s Web of Science. It presents es imates for a number of journals in the field of condensed matter physics. In order to discriminate between an ‘open access’ effect and an early view effect, longitudinal citat ion data was analysed covering a time period as long as 7 years. Quality bias was measure d by calculating ArXiv citation impact differentials at the level of individual aut hors publishing in a journal, taking into account co-authorship. The analysis provided evide nce of a strong quality bias and early view effect. Correcting for these effects, there is in a sample of 6 condensed matter physics journals studied in detail, no sign of a general ‘open access advantage’ of paper s deposited in ArXiv. The study does provide evidence that ArXiv accelerates citation, due to the fact that that ArXiv makes papers arlier available rather than that it makes papers freely available.

Existing methods for searching and exploring large document collections focus on surface-level matches to user queries, ignoring higher-level semantic structure. In this paper we show how topic modeling - a technique for identifying latent themes across a large collection of documents - can support semantic exploration. We present TopicViz: an interactive environment which combines traditional search and citation-graph exploration with a force-directed layout that links documents to the latent themes discovered by the topic model. We describe usage scenarios in which TopicViz supports rapid sensemaking on large document collections.

Many practical tasks require accessing specific types of information in scientific literature; e.g. information about the objective, methods, results or conclusions of the study in question. Several schemes have been developed to characterize such information in full journal papers. Yet many tasks focus on abstracts instead. We take three schemes of different type and granularity (those based on section names, argumentative zones and conceptual structure of documents) and investigate their applicability to biomedical abstracts. We show that even for the finest-grained of these schemes, the majority of categories appear in abstracts and can be identified relatively reliably using machine learning. We discuss the impact of our results and the need for subsequent task-based evaluation of the schemes.

Two well-known indexing methods are inverted files and signature files. We have undertaken a detailed comparison of these two approaches in the context of text indexing, paying particular attention to query evaluation speed and space requirements. We have examined their relative performance using both experimentation and a refined approach to modeling of signature files, and demonstrate that inverted files are distinctly superior to signature files. Not only can inverted files be used to evaluate typical queries in less time than can signature files, but inverted files require less space and provide greater functionality. Our results also show that a synthetic text database can provide a realistic indication of the behavior of an actual text database. The tools used to generate the synthetic database have been made publicly available

Tags are user-generated labels for entities. Existing research on tag recommendation either focuses on improving its accuracy or on automating the process, while ignoring the efficiency issue. We propose a highly-automated novel framework for real-time tag recommendation. The tagged training documents are treated as triplets of (words, docs, tags), and represented in two bipartite graphs, which are partitioned into clusters by Spectral Recursive Embedding (SRE). Tags in each topical cluster are ranked by our novel ranking algorithm. A two-way Poisson Mixture Model (PMM) is proposed to model the document distribution into mixture components within each cluster and aggregate words into word clusters simultaneously. A new document is classified by the mixture model based on its posterior probabilities so that tags are recommended according to their ranks. Experiments on large-scale tagging datasets of scientific documents (CiteULike) and web pages del.icio.us) indicate that our framework is capable of making tag recommendation efficiently and effectively. The average tagging time for testing a document is around 1 second, with over 88% test documents correctly labeled with the top nine tags we suggested.


Question: Enhanced citation indexing through the implementation of argumentative zoning.

This work explores the challenge of categorizing documents based on sentiment rather than subject matter, specifically assessing whether a review conveys a positive or negative sentiment. Utilizing movie reviews as a dataset, we ascertain that conventional machine learning algorithms significantly surpass human-generated benchmarks in sentiment classification. Among the machine learning approaches tested—Naive Bayes, maximum entropy classification, and support vector machines—while they excel in sentiment analysis compared to human judgment, they exhibit lesser efficacy in this context than in traditional topic-oriented classification tasks. The study concludes with an examination of the complexities inherent in sentiment classification, highlighting factors that contribute to its heightened difficulty. Conference details: Proceedings of EMNLP 2002, pages 79–86.

In this paper, we describe a case study of a sentence-level categorization in which tagging instructions are developed and used by four judges to classify clauses from the Wall Street Journal as either subjective or objective. Agreement among the four judges is analyzed, and, based on that analysis, each clause is given a nal classiication. To provide empirical support for the classiications, correlations are assessed in the data between the subjective category and a basic semantic class posited by Quirk et al. (1985).

Due to the explosion of social networking and the information sharing among their users, the interest in analyzing social networks has increased over the recent years. Two general interests in this kind of studies are community detection and visualization. In the first case, most of the classic algorithms for community detection use only the structural information to identify groups, that is, how clusters are formed according to the topology of the relationships. However, these methods do not take into account any semantic information which could guide the clustering process, and which may add elements to conduct further analyses. In the second case most of the layout algorithms for clustered graphs have been designed to differentiate the groups within the graph, but they are not designed to analyze the interactions between such groups. Identifying these interactions gives an insight into the way different communities exchange messages or information, and allows the social network researcher to identify key actors, roles, and paths from one community to another.
 This article presents a novel model to use, in a conjoint way, the semantic information from the social network and its structural information to, first, find structurally and semantically related groups of nodes, and second, a layout algorithm for clustered graphs which divides the nodes into two types, one for nodes with edges connecting other communities and another with nodes connecting nodes only within their own community. With this division the visualization tool focuses on the connections between groups facilitating deep studies of augmented social networks.

We approach the problem of academic literature search by considering an unpublished manuscript as a query to a search system. We use the text of previous literature as well as the citation graph that connects it to find relevant related material. We evaluate our technique with manual and automatic evaluation methods, and find an order of magnitude improvement in mean average precision as compared to a text similarity baseline.

Argumentation represents the study of views and opinions that humans express with the goal of reaching a conclusion through logical reasoning. Since the 1950’s, several models have been proposed to capture the essence of informal argumentation in different settings. With the emergence of the Web, and then the Semantic Web, this modeling shifted towards ontologies, while from the development perspective, we witnessed an important increase in Web 2.0 human-centered collaborative deliberation tools. Through a review of more than 150 scholarly papers, this article provides a comprehensive and comparative overview of approaches to modeling argumentation for the Social Semantic Web. We start from theoretical foundational models and investigate how they have influenced Social Web tools. We also look into Semantic Web argumentation models. Finally we end with Social Web tools for argumentation, including online applications combining Web 2.0 and Semantic Web technologies, following the path to a global World Wide Argument Web.

A technical standard is an established norm or requirement in regard to technical systems. It is usually a formal document that establishes uniform engineering or technical criteria, methods, processes and practices. Every year, more than 10K standards are released by standardization organizations, such as International Standardization Organization (ISO), American National Standards Institute (ANSI), and China National Institute of Standardization (CNIS). Thereinto, China publishes more than 4K national and industrial standards per year. Confronted with the large amount of standards, how to manage and make full use of these standards efficiently and effectively has become a major issue to be addressed. In this paper, we introduce a mining and visualizing system for large-scale Chinese technical standards from spatial and temporal perspectives. Specifically, we propose an s-index metric to measure the contribution of a drafting unit. Moreover, we develop a multiple dimensional mining and visualizing analysis system for standardization experts and users to explore regional differences and temporal trends of standards in China. Our system has run on 95K Chinese standards spanning from 2001 to 2016 and provided online service. The extensive experiments and case study demonstrate the effectiveness of our proposed system.

Evaluating a scientist’s past and future potential impact is key in decision making concerning with recruitment and funding, and is increasingly linked to publication citation count. Meanwhile, timely identifying those valuable work with great potential before they receive wide recognition and become highly cited papers is both useful for readers and authors in many regards. We propose a method for predicting the citation counts of individual publications, over an arbitrary time period. Our approach explores paper-specific covariates, and a point process model to account for the aging effect and triggering role of recent citations, through which papers lose and gain their popularity, respectively. Empirical results on the Microsoft Academic Graph data suggests that our model can be useful for both prediction and interpretability.

Sentiment analysis seeks to identify the viewpoint(s) underlying a text span; an example application is classifying a movie review as “thumbs up” or “thumbs down”. To determine this sentiment polarity, we propose a novel machine-learning method that applies text-categorization techniques to just the subjective portions of the document. Extracting these portions can be implemented using efficient techniques for finding minimum cuts in graphs; this greatly facilitates incorporation of cross-sentence contextual constraints. Publication info: Proceedings of the ACL, 2004.

Citation sentiment analysis is an important task in scientific paper analysis. Existing machine learning techniques for citation sentiment analysis are focusing on labor-intensive feature engineering, which requires large annotated corpus. As an automatic feature extraction tool, word2vec has been successfully applied to sentiment analysis of short texts. In this work, I conducted empirical research with the question: how well does word2vec work on the sentiment analysis of citations? The proposed method constructed sentence vectors (sent2vec) by averaging the word embeddings, which were learned from Anthology Collections (ACL-Embeddings). I also investigated polarity-specific word embeddings (PS-Embeddings) for classifying positive and negative citations. The sentence vectors formed a feature space, to which the examined citation sentence was mapped to. Those features were input into classifiers (support vector machines) for supervised classification. Using 10-cross-validation scheme, evaluation was conducted on a set of annotated citations. The results showed that word embeddings are effective on classifying positive and negative citations. However, hand-crafted features performed better for the overall classification.

Given a large-scale linked document collection, such as a collection of blog posts or a research literature archive, there are two fundamental problems that have generated a lot of interest in the research community. One is to identify a set of high-level topics covered by the documents in the collection; the other is to uncover and analyze the social network of the authors of the documents. So far these problems have been viewed as separate problems and considered independently from each other. In this paper we argue that these two problems are in fact inter-dependent and should be addressed together. We develop a Bayesian hierarchical approach that performs topic modeling and author community discovery in one unified framework. The effectiveness of our model is demonstrated on two blog data sets in different domains and one research paper citation data from CiteSeer.

The network structure of a hyperlinked environment can be a rich source of information about the content of the environment, provided we have effective means for understanding it. We develop a set of algorithmic tools for extracting information from the link structures of such environments, and report on experiments that demonstrate their effectiveness in a variety of context on the World Wide Web. The central issue we address within our framework is the distillation of broad search topics, through the discovery of “authorative” information sources on such topics. We propose and test an algorithmic formulation of the notion of authority, based on the relationship between a set of relevant authoritative pages and the set of “hub pages” that join them together in the link structure. Our formulation has connections to the eigenvectors of certain matrices associated with the link graph; these connections in turn motivate additional heuristrics for link-based analysis.