Question: An overview of clinical prediction models in medical decision-making.

This comprehensive review examines the evolution and variety of cost-sensitive decision tree induction algorithms developed over the past ten years. The focus has increasingly shifted towards creating decision trees that consider the costs associated with misclassification and feature acquisition in the decision-making process. The review identifies and analyzes more than 50 algorithms, encompassing direct adaptations of accuracy-based methods, genetic algorithm approaches, anytime methods, and techniques utilizing boosting and bagging. By consolidating these diverse studies and innovative methods for cost-sensitive decision tree learning, the review offers a structured taxonomy, a chronological perspective on the field's development, and serves as a valuable resource for guiding future research in this area.

Procedures for examining whether treatment effects on an outcome are mediated and/or moderated have been well developed and are routinely applied. The mediation question focuses on the intervening mechanism that produces the treatment effect. The moderation question focuses on factors that affect the magnitude of the treatment effect. It is important to note that these two processes may be combined in informative ways, such that moderation is mediated or mediation is moderated. Although some prior literature has discussed these possibilities, their exact definitions and analytic procedures have not been completely articulated. The purpose of this article is to define precisely both mediated moderation and moderated mediation and provide analytic strategies for assessing each.

Software defect prediction strives to improve software quality and testing efficiency by constructing predictive classification models from code attributes to enable a timely identification of fault-prone modules. Several classification models have been evaluated for this task. However, due to inconsistent findings regarding the superiority of one classifier over another and the usefulness of metric-based classification in general, more research is needed to improve convergence across studies and further advance confidence in experimental results. We consider three potential sources for bias: comparing classifiers over one or a small number of proprietary data sets, relying on accuracy indicators that are conceptually inappropriate for software defect prediction and cross-study comparisons, and, finally, limited use of statistical testing procedures to secure empirical findings. To remedy these problems, a framework for comparative software defect prediction experiments is proposed and applied in a large-scale empirical comparison of 22 classifiers over 10 public domain data sets from the NASA Metrics Data repository. Overall, an appealing degree of predictive accuracy is observed, which supports the view that metric-based classification is useful. However, our results indicate that the importance of the particular classification algorithm may be less than previously assumed since no significant performance differences could be detected among the top 17 classifiers.

This paper presents the case of a non-traditional use of the Delphi method for theory evaluation. On the basis of experience gained through secondary and primary research, a generic decision toolkit for Delphi studies is proposed, comprising of taxonomy of Delphi design choices, a stage model and critical methodological decisions. These research tools will help to increase confidence when adopting the Delphi alternative and allow for a wider and more comprehensive recognition of the method within both scientific and interpretivist studies.

Clinical notes are text documents that are created by clinicians for each patient encounter. They are typically accompanied by medical codes, which describe the diagnosis and treatment. Annotating these codes is labor intensive and error prone; furthermore, the connection between the codes and the text is not annotated, obscuring the reasons and details behind specific diagnoses and treatments. We present an attentional convolutional network that predicts medical codes from clinical text. Our method aggregates information across the document using a convolutional neural network, and uses an attention mechanism to select the most relevant segments for each of the thousands of possible codes. The method is accurate, achieving precision@8 of 0.71 and a Micro-F1 of 0.54, which are both better than the prior state of the art. Furthermore, through an interpretability evaluation by a physician, we show that the attention mechanism identifies meaningful explanations for each code assignment.

Decision trees and ensembles of decision trees are very popular in machine learning and often achieve state-of-the-art performance on black-box prediction tasks. However, popular variants such as C4.5, CART, boosted trees and random forests lack a probabilistic interpretation since they usually just specify an algorithm for training a model. We take a probabilistic approach where we cast the decision tree structures and the parameters associated with the nodes of a decision tree as a probabilistic model; given labeled examples, we can train the probabilistic model using a variety of approaches (Bayesian learning, maximum likelihood, etc). The probabilistic approach allows us to encode prior assumptions about tree structures and share statistical strength between node parameters; furthermore, it offers a principled mechanism to obtain probabilistic predictions which is crucial for applications where uncertainty quantification is important. Existing work on Bayesian decision trees relies on Markov chain Monte Carlo which can be computationally slow and suffer from poor mixing. We propose a novel sequential Monte Carlo algorithm that computes a particle approximation to the posterior over trees in a top-down fashion. We also propose a novel sampler for Bayesian additive regression trees by combining the above top-down particle filtering algorithm with the Particle Gibbs (Andrieu et al., 2010) framework. Finally, we propose Mondrian forests (MFs), a computationally efficient hybrid solution that is competitive with non-probabilistic counterparts in terms of speed and accuracy, but additionally produces well-calibrated uncertainty estimates. MFs use the Mondrian process (Roy and Teh, 2009) as the randomization mechanism and hierarchically smooth the node parameters within each tree (using a hierarchical probabilistic model and approximate Bayesian updates), but combine the trees in a non-Bayesian fashion. MFs can be grown in an incremental/online fashion and remarkably, the distribution of online MFs is the same as that of batch MFs.

Historically, suicide risk assessment has relied on question-and-answer type tools. These tools, built on psychometric advances, are widely used because of availability. Yet there is no known tool based on biologic and cognitive evidence. This absence often cause a vexing clinical problem for clinicians who question the value of the result as time passes. The purpose of this paper is to describe one experiment in a series of experiments to develop a tool that combines Biological Markers ( Bm) with Thought Markers ( Tm), and use machine learning to compute a real-time index for assessing the likelihood repeated suicide attempt in the next six-months. For this study we focus using unsupervised machine learning to distinguish between actual suicide notes and newsgroups. This is important because it gives us insight into how well these methods discriminate between real notes and general conversation.

Traditional machine learning algorithms for pattern recognition just output simple predictions, without any associated confidence values. Confidence values are an indication of how likely each prediction is of being correct. In the ideal case, a confidence of 99% or higher for all examples in a set, means that the percentage of erroneous predictions in that set will not exceed 1%. Knowing the likelihood of each prediction enables us to assess the extent to which we can rely on it. For this reason, predictions that are associated with some kind of confidence values are highly desirable in many risk-sensitive applications, such as those used for medical diagnosis or financial analysis. In fact, such information can benefit any application that requires human-computer interaction, as confidence values can be used to determine the way in which each prediction will be treated. For instance, a filtering mechanism can be employed so that only predictions which satisfy a certain level of confidence will be taken into account, while the rest can be discarded or passed on to a human for judgement. There are two main areas in mainstream machine learning that can be used in order to obtain some kind of confidence values; the Bayesian framework and the theory of Probably Approximately Correct learning (PAC theory). Quite often the Bayesian framework is used for producing algorithms that complement individual predictions with probabilistic measures of their quality. On the other hand, PAC theory can be used for producing upper bounds on the probability of error for a given algorithm with respect to some confidence level 1 − δ. Both of these approaches however, have their drawbacks. In order to apply the Bayesian framework one is required to have some prior knowledge about the distribution that generates the data. When the correct prior is known, Bayesian methods provide optimal decisions. For real world data sets though, as the required knowledge is not available, one has to assume the existence of an arbitrarily chosen prior. In this case, if the assumed prior is incorrect, the resulting confidence levels may also be “incorrect”; for example the predictive regions output for the 95% confidence level may contain the true label in much less than 95% of the cases. This signifies a major failure, as we would expect confidence levels to bound the percentage of expected errors. An experimental demonstration of how misleading Bayesian methods can become when their assumptions are violated can be found in (Melluish et al., 2001).

Multiple choice questions (MCQs) that can be generated from a domain ontology can significantly reduce human effort & time re quired for authoring & administering assessments in an e-Learning envir o ment. Even though there are various methods for generating MCQs from ontologi es, methods for determining the difficulty-levels of such MCQs are less explor ed. In this paper, we study various aspects and factors that are involved in deter mining the difficultyscore of an MCQ, and propose an ontology-based model for the p rediction. This model characterizes the difficulty values associated with t he s em and choice set of the MCQs, and describes a measure which combines both the s cor s. Furthermore, the notion of assigning difficultly-scores based on th e skill level of the test taker is utilized for predicating difficulty-score of a stem . We studied the effectiveness of the predicted difficulty-scores with the help of a psychometric model from the Item Response Theory, by involving real-students a d domain experts. Our results show that, the predicated difficulty-levels of t he MCQs are having high correlation with their actual difficulty-levels.

Health informatics has emerged as a growing domain of interest among researchers world-wide owing to its major implications on society. Applications of machine learning in healthcare range from disease prediction to patient-level personalized services. The prevalence of big data in healthcare has paved the way for applications based on deep learning techniques in the past few years. This paper reviews recent trends and applications of deep learning applied to the healthcare domain. We highlight recent research work, identify challenges and suggest possible future directions that could be pursued further in this domain.

Decision Tree Classifiers (DTC's) are used successfully in many diverse areas such as radar signal classification, character recognition, remote sensing, medical diagnosis, expert systems, and speech recognition, to name only a few. Perhaps, the most important feature of DTC's is their capability to break down a complex decision-making process into a collection of simpler decisions, thus providing a solution which is often easier to interpret. This paper presents a survey of current methods for DTC designs and the various existing issues. After considering potential advantages of DTC's over single stage classifiers, subjects of tree structure design, feature selection at each internal node, and decision and search strategies are discussed_ Finally, several remarks are made concerning possible future research directions.


Question: An overview of clinical prediction models in medical decision-making.

Adaptation of Tree-Structured Recursive Partitioning for Survival Analysis

This paper presents an extension of tree-structured recursive partitioning methodologies to accommodate survival analysis with censored data. Unlike traditional approaches, this method does not require the restrictive assumptions of the well-known semi-parametric model introduced by Cox. Instead, it relies on assumptions that ensure the identifiability of conditional distributions of survival times given the presence of covariates. This adaptation broadens the application scope of tree-structured models, making them suitable for a wider range of scenarios in survival analysis.

The diagnosis of diseases is a crucial and difficult job in medicine. The recognition of heart disease from diverse features or signs is a multi-layered problem that is not free from false assumptions and is frequently accompanied by hasty effects [1]. Thus an attempt to exploit knowledge and experience of several specialists and clinical screening data of patients composed in databases to assist the diagnosis procedure is regarded as a great challenge. The healthcare industry gathers enormous amounts of heart disease data that unfortunately, are not mined to determine concealed information for effective diagnosing [2]. In this paper, an efficient approach for the intelligent heart disease prediction based on Probabilistic Neural Network (PNN) technique is proposed. Initially, the data set containing 13 medical attributes were obtained from the Cleveland heart disease database. The dataset is clustered with the aid of k-means clustering algorithm. The PNN with radial basis function is trained using the selected data sets. The comparison between existing approaches and proposed approach with respect to accuracy of prediction, sensitivity and specificity is recorded in this paper. The results from experiments returned with diminishing fact that there is considerable improvement in classification and prediction. The proposed PNN works as promising tool for prediction of heart disease.

Brain-imaging research has predominantly generated insight by means of classical statistics, including regression-type analyses and null-hypothesis testing using t-test and ANOVA. Throughout recent years, statistical learning methods enjoy increasing popularity especially for applications in rich and complex data, including cross-validated out-of-sample prediction using pattern classification and sparsity-inducing regression. This concept paper discusses the implications of inferential justifications and algorithmic methodologies in common data analysis scenarios in neuroimaging. It is retraced how classical statistics and statistical learning originated from different historical contexts, build on different theoretical foundations, make different assumptions, and evaluate different outcome metrics to permit differently nuanced conclusions. The present considerations should help reduce current confusion between model-driven classical hypothesis testing and data-driven learning algorithms for investigating the brain with imaging techniques.

We performed a Monte Carlo study to evaluate the effect of the number of events per variable (EPV) analyzed in logistic regression analysis. The simulations were based on data from a cardiac trial of 673 patients in which 252 deaths occurred and seven variables were cogent predictors of mortality; the number of events per predictive variable was (252/7 =) 36 for the full sample. For the simulations, at values of EPV = 2, 5, 10, 15, 20, and 25, we randomly generated 500 samples of the 673 patients, chosen with replacement, according to a logistic model derived from the full sample. Simulation results for the regression coefficients for each variable in each group of 500 samples were compared for bias, precision, and significance testing against the results of the model fitted to the original sample. For EPV values of 10 or greater, no major problems occurred. For EPV values less than 10, however, the regression coefficients were biased in both positive and negative directions; the large sample variance estimates from the logistic model both overestimated and underestimated the sample variance of the regression coefficients; the 90% confidence limits about the estimated values did not have proper coverage; the Wald statistic was conservative under the null hypothesis; and paradoxical associations (significance in the wrong direction) were increased. Although other factors (such as the total number of events, or sample size) may influence the validity of the logistic model, our findings indicate that low EPV can lead to major problems.

This article presents a survey on diagnosis code prediction from various information in Electronic Health Records (EHR): both unstructured free text and structured data. Particularly, our interests are in casting the problem as text classification with multiple sources and using neural network based models. We will first present previous work in this area and describe some simple baseline models for the relevant tasks.

As an integral part of Electronic Health Records (EHRs), clinical notes pose special challenges for analyzing EHRs due to their unstructured nature. In this paper, we present a general mining framework SympGraph for modeling and analyzing symptom relationships in clinical notes.
 A SympGraph has symptoms as nodes and co-occurrence relations between symptoms as edges, and can be constructed automatically through extracting symptoms over sequences of clinical notes for a large number of patients. We present an important clinical application of SympGraph: symptom expansion, which can expand a given set of symptoms to other related symptoms by analyzing the underlying SympGraph structure. We further propose a matrix update algorithm which provides a significant computational saving for dynamic updates to the graph. Comprehensive evaluation on 1 million longitudinal clinical notes over 13K patients shows that static symptom expansion can successfully expand a set of known symptoms to a disease with high agreement rate with physician input (average precision 0.46), a 31% improvement over baseline co-occurrence based methods. The experimental results also show that the expanded symptoms can serve as useful features for improving AUC measure for disease diagnosis prediction, thus confirming the potential clinical value of our work.

In some real-world areas, it is important to enrich the data with external background knowledge so as to provide context and to facilitate pattern recognition. These areas may be described as data rich but knowledge poor. There are two challenges to incorporate this biological knowledge into the data mining cycle: (1) generating the ontologies; and (2) adapting the data mining algorithms to make use of the ontologies. This chapter presents the state-of-the-art in bringing the background ontology knowledge into the pattern recognition task for biomedical data.

This paper evaluates the performance of a number of modelling approaches for future mortgage default status. Boosted regression trees, random forests, penalised linear and semi-parametric logistic regression models are applied to four portfolios of over 300,000 Irish owner-occupier mortgages. The main findings are that the selected approaches have varying degrees of predictive power and that boosted regression trees significantly outperform logistic regression. This suggests that boosted regression trees can be a useful addition to the current toolkit for mortgage credit risk assessment by banks and regulators.

It is an important issue to utilize large amount of medical records which are being accumulated on medical information systems to improve the quality of medical treatment. The process of medical treatment can be considered as a sequential interaction process between doctors and patients. From this viewpoint, we have been modeling medical records using Markov decision processes (MDPs). Using our model, we can simulate the future of each patient and evaluate each treatment. In order to do so, the reward function of the MDP should be specified. However, there is no explicit information regarding the reward value in medical records. In this study we report our results of applying an inverse reinforcement learning (IRL) algorithm to medical records of diabetes treatment to explore the reward function that doctors have in mind during their treatments.

The Accreditation Council for Graduate Medical Education sets forth a number of required educational topics that must be addressed in residency and fellowship programs. We sought to provide a primer on some of the important basic statistical concepts to consider when examining the medical literature. It is not essential to understand the exact workings and methodology of every statistical test encountered, but it is necessary to understand selected concepts such as parametric and nonparametric tests, correlation, and numerical versus categorical data. This working knowledge will allow you to spot obvious irregularities in statistical analyses that you encounter.

We study the construction of prediction algorithms in a situation in which a learner faces a sequence of trials with a prediction to be made in each and the goal of the learner is to make few mistakes We are interested in the case that the learner has reason to believe that one of some pool of known algorithms will perform well but the learner does not know which one A simple and e ective method based on weighted voting is introduced for constructing a compound algorithm in such a circumstance We call this method the Weighted Majority Algorithm We show that this algorithm is robust in the presence of errors in the data We discuss various versions of the Weighted Majority Algorithm and prove mistake bounds for them that are closely related to the mistake bounds of the best algorithms of the pool For example given a sequence of trials if there is an algorithm in the pool A that makes at most m mistakes then the Weighted Majority Algorithm will make at most c log jAj m mistakes on that sequence where c is xed constant


Question: An overview of clinical prediction models.

This concise document delves into the concept of boosting, a versatile technique aimed at enhancing the precision of any learning algorithm. Specifically, it focuses on the AdaBoost algorithm, detailing the foundational principles of boosting and addressing the common concern of overfitting. It elucidates why boosting mechanisms typically avoid this pitfall, maintaining robust performance. Furthermore, the document highlights several contemporary applications of boosting, showcasing its widespread relevance and effectiveness in various domains.

**Question**: An Empirical Analysis of Machine Learning Algorithms for Text Classification

**Documents**: A Comparative Study of Feature Selection Methods for Text Classification
This study compares the performance of several feature selection methods in the context of text classification. The methods include chi-squared, information gain, and mutual information. The results show that information gain and mutual information outperform chi-squared in most cases, although the difference is not always statistically significant. The study also suggests that the choice of feature selection method should be guided by the specific characteristics of the dataset.

**Similar Documents**: This comprehensive analysis evaluates the efficacy of various feature selection techniques in the realm of text classification. It juxtaposes chi-squared, information gain, and mutual information approaches, revealing that information gain and mutual information typically surpass chi-squared in effectiveness, albeit with occasional marginal statistical variance. The findings underscore the importance of tailoring the feature selection strategy to the unique attributes of the dataset, emphasizing context-specific optimization for enhanced classification accuracy.

Nowadays there are many risks related to bank loans, especially for the banks so as to reduce their capital loss. The analysis of risks and assessment of default becomes crucial thereafter. Banks hold huge volumes of customer behaviour related data from which they are unable to arrive at a judgement if an applicant can be defaulter or not. Data Mining is a promising area of data analysis which aims to extract useful knowledge from tremendous amount of complex data sets. This work aims to develop a model and construct a prototype for the same using a data set available in the UCI repository. The model is a decision tree based classification model that uses the functions available in the R Package. Prior to building the model, the dataset is pre-processed, reduced and made ready to provide efficient predictions. The final model is used for prediction with the test dataset and the experimental results prove the efficiency of the built model.

Machine learning models have been adapted in biomedical research and practice for knowledge discovery and decision support. While mainstream biomedical informatics research focuses on developing more accurate models, the importance of data preprocessing draws less attention. We propose the Generalized Logistic (GL) algorithm that scales data uniformly to an appropriate interval by learning a generalized logistic function to fit the empirical cumulative distribution function of the data. The GL algorithm is simple yet effective; it is intrinsically robust to outliers, so it is particularly suitable for diagnostic/classification models in clinical/medical applications where the number of samples is usually small; it scales the data in a nonlinear fashion, which leads to potential improvement in accuracy. To evaluate the effectiveness of the proposed algorithm, we conducted experiments on 16 binary classification tasks with different variable types and cover a wide range of applications. The resultant performance in terms of area under the receiver operation characteristic curve (AUROC) and percentage of correct classification showed that models learned using data scaled by the GL algorithm outperform the ones using data scaled by the Min-max and the Z-score algorithm, which are the most commonly used data scaling algorithms. The proposed GL algorithm is simple and effective. It is robust to outliers, so no additional denoising or outlier detection step is needed in data preprocessing. Empirical results also show models learned from data scaled by the GL algorithm have higher accuracy compared to the commonly used data scaling algorithms.

Omer Gottesman, Fredrik Johansson, Joshua Meier, Jack Dent, Donghun Lee, Srivatsan Srinivasan, Linying Zhang, Yi Ding, David Wihl, Xuefeng Peng, Jiayu Yao, Isaac Lage, Christopher Mosch, Li-wei H. Lehman, Matthieu Komorowski, Aldo Faisal, Leo Anthony Celi, David Sontag, and Finale Doshi-Velez Paulson School of Engineering and Applied Sciences, Harvard University Institute for Medical Engineering and Science, MIT T.H. Chan School of Public Health, Harvard University Department of Statistics, Harvard University Laboratory for Computational Physiology, Harvard-MIT Health Sciences & Technology, MIT Department of Surgery and Cancer, Faculty of Medicine, Imperial College London Department of Bioengineering, Imperial College London Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center MIT Critical Data

Review websites, such as TripAdvisor and Yelp, allow users to post online reviews for various businesses, products and services, and have been recently shown to have a significant influence on consumer shopping behaviour. An online review typically consists of free-form text and a star rating out of 5. The problem of predicting a user’s star rating for a product, given the user’s text review for that product, is called Review Rating Prediction and has lately become a popular, albeit hard, problem in machine learning. In this paper, we treat Review Rating Prediction as a multi-class classification problem, and build sixteen different prediction models by combining four feature extraction methods, (i) unigrams, (ii) bigrams, (iii) trigrams and (iv) Latent Semantic Indexing, with four machine learning algorithms, (i) logistic regression, (ii) Näıve Bayes classification, (iii) perceptrons, and (iv) linear Support Vector Classification. We analyse the performance of each of these sixteen models to come up with the best model for predicting the ratings from reviews. We use the dataset provided by Yelp for training and testing the models.

BACKGROUND
To facilitate the addressing of spirituality in clinical practice, several authors have created instruments for obtaining a spiritual history. However, in only a few studies have authors compared these instruments. The aim of this study was to compare the most commonly used instruments for taking a spiritual history in a clinical setting.

METHODS
A systematic review of spiritual history assessment was conducted in five stages: identification of instruments used in the literature (databases searching); relevant articles from title and initial abstract review; exclusion and Inclusion criteria; full text retrieval and final analysis of each instrument.

RESULTS
A total of 2,641 articles were retrieved and after the analysis, 25 instruments were included. The authors independently evaluated each instrument on 16 different aspects. The instruments with the greatest scores in the final analysis were FICA, SPIRITual History, FAITH, HOPE, and the Royal College of Psychiatrists. Concerning all 25 instruments, 20 of 25 inquire about the influence of spirituality on a person's life and 17 address religious coping. Nevertheless, only four inquire about medical practices not allowed, six deal with terminal events, nine have mnemonics to facilitate their use, and five were validated.

CONCLUSIONS
FICA, SPIRITual History, FAITH, HOPE, and Royal College of Psychiatrists scored higher in our analysis. The use of each instrument must be individualized, according to the professional reality, time available, patient profile, and settings.

Analysis of clinical records contributes to the Traditional Chinese Medicine (TCM) experience expansion and techniques promotion. More than two diagnostic classes (diagnostic syndromes) in the clinical records raise a popular data mining problem: multi-value classification. In this paper, we propose a novel multi-class classifier, named Multiple Asymmetric Partial Least Squares Classifier (MAPLSC). MAPLSC attempts to be robust facing imbalanced data distribution in the multi-value classification. Elaborated comparisons with other seven state-of-the-art methods on two TCM clinical datasets and four public microarray datasets demonstrate MAPLSC's remarkable improvements.

0950-5849/$ see front matter 2009 Elsevier B.V. A doi:10.1016/j.infsof.2009.11.005 * Corresponding author. Tel.: +44 (0) 1782 734090; E-mail address: m.turner@cs.keele.ac.uk (M. Turne Context: The technology acceptance model (TAM) was proposed in 1989 as a means of predicting technology usage. However, it is usually validated by using a measure of behavioural intention to use (BI) rather than actual usage. Objective: This review examines the evidence that the TAM predicts actual usage using both subjective and objective measures of actual usage. Method: We performed a systematic literature review based on a search of six digital libraries, along with vote-counting meta-analysis to analyse the overall results. Results: The search identified 79 relevant empirical studies in 73 articles. The results show that BI is likely to be correlated with actual usage. However, the TAM variables perceived ease of use (PEU) and perceived usefulness (PU) are less likely to be correlated with actual usage. Conclusion: Care should be taken using the TAM outside the context in which it has been validated. 2009 Elsevier B.V. All rights reserved.

© International Association for Applied Psychology, 2004. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. Blackwell Publishing, Ltd. Oxford, UK APPS pplied Psychology: an International Review 0269-994X © Blackwell Publishing 2004 ct ber 2004 53 4 riginal Arti le PERFORMANCE MANAGEMENT D N HARTOG ET AL. Performance Management: A Model and Research Agenda

In this age of computer science each and every thing becomes intelligent and perform task as human. For that purpose there are various tools, techniques and methods are proposed. Support vector machine is a model for statistics and computer science, to perform supervised learning, methods that are used to make analysis of data and recognize patterns. SVM is mostly used for classification and regression analysis. And in the same way k-nearest neighbor algorithm is a classification algorithm used to classify data using training examples. In this paper we use SVM and KNN algorithm to classify data and get prediction (find hidden patterns) for target. Here we use medical patients nominal data to classify and discover the data pattern to predict future disease, Uses data mining which is use to classify text analysis in future.

Software engineering researchers have long been interested in where and why bugs occur in code, and in predicting where they might turn up next. Historical bug-occurence data has been key to this research. Bug tracking systems, and code version histories, record when, how and by whom bugs were fixed; from these sources, datasets that relate file changes to bug fixes can be extracted. These historical datasets can be used to test hypotheses concerning processes of bug introduction, and also to build statistical bug prediction models. Unfortunately, processes and humans are imperfect, and only a fraction of bug fixes are actually labelled in source code version histories, and thus become available for study in the extracted datasets. The question naturally arises, are the bug fixes recorded in these historical datasets a fair representation of the full population of bug fixes? In this paper, we investigate historical data from several software projects, and find strong evidence of systematic bias. We then investigate the potential effects of "unfair, imbalanced" datasets on the performance of prediction techniques. We draw the lesson that bias is a critical problem that threatens both the effectiveness of processes that rely on biased datasets to build prediction models and the generalizability of hypotheses tested on biased data.


Question: An overview of clinical prediction models.

This paper explores the application of tree-based batch mode reinforcement learning techniques in determining optimal control policies. Reinforcement learning seeks to derive the best control strategy through interaction or observation of a system. In batch mode, this is achieved by estimating the Q-function, a key component in reinforcement learning, using sets of four-tuples (xt, ut, rt, xt+1). Here, xt represents the system state at a given time t, ut the action taken, rt the immediate reward, and xt+1 the subsequent state. The Q-function is approximated through a series of supervised learning tasks in batch mode. We discuss the application of various tree-based supervised learning methods, including CART, Kd-tree, and tree bagging, alongside two innovative ensemble algorithms: extremely randomized trees and totally randomized trees. Performance evaluations across multiple examples reveal that ensemble methods, particularly those based on regression trees, are adept at extracting pertinent information about optimal control policies from four-tuple datasets. Specifically, totally randomized trees exhibit strong performance while ensuring the convergence of the sequence. However, by slightly relaxing the convergence requirement, extremely randomized trees offer even higher accuracy in identifying optimal control policies.

Credit risk models are used by financial companies to evaluate in advance the insolvency risk caused by credits that enter into default. Many models for credit risk have been developed over the past few decades. In this paper, we focus on those models that can be formulated in terms of the probability of default by using survival analysis techniques. With this objective three different mechanisms are proposed based on the key idea of writing the default probability in terms of the conditional distribution function of the time to default. The first method is based on a Cox’s regression model, the second approach uses generalized linear models under censoring and the third one is based on nonparametric kernel estimation, using the product-limit conditional distribution function estimator by Beran. The resulting nonparametric estimator of the default probability is proved to be consistent and asymptotically normal. An empirical study, based on modified real data, illustrates the three methods. MSC: 62P20, 91B30, 62G05, 62N01

In February 2013, Google Flu Trends (GFT) made headlines but not for a reason that Google executives or the creators of the flu tracking system would have hoped. Nature reported that GFT was predicting more than double the proportion of doctor visits for influenza-like illness (ILI) than the Centers for Disease Control and Prevention (CDC), which bases its estimates on surveillance reports from laboratories across the United States (1, 2). This happened despite the fact that GFT was built to predict CDC reports. Given that GFT is often held up as an exemplary use of big data (3, 4), what lessons can we draw from this error?

The Accreditation Council for Graduate Medical Education sets forth a number of required educational topics that must be addressed in residency and fellowship programs. We sought to provide a primer on some of the important basic statistical concepts to consider when examining the medical literature. It is not essential to understand the exact workings and methodology of every statistical test encountered, but it is necessary to understand selected concepts such as parametric and nonparametric tests, correlation, and numerical versus categorical data. This working knowledge will allow you to spot obvious irregularities in statistical analyses that you encounter.

The researchers and practitioners of today create models, algorithms, functions, and other constructs defined in abstract spaces. The research of the future will likely be data driven. Symbolic and numeric data that are becoming available in large volumes will define the need for new data analysis techniques and tools. Data mining is an emerging area of computational intelligence that offers new theories, techniques, and tools for analysis of large data sets. In this paper, a novel approach for autonomous decision-making is developed based on the rough set theory of data mining. The approach has been tested on a medical data set for patients with lung abnormalities referred to as solitary pulmonary nodules (SPNs). The two independent algorithms developed in this paper either generate an accurate diagnosis or make no decision. The methodology discussed in the paper depart from the developments in data mining as well as current medical literature, thus creating a variable approach for autonomous decision-making.

The proliferation of information technology (IT) in supporting highly specialized tasks and services has made it increasingly important to understand the factors essential to technology acceptance by individuals. In a typical professional setting, the essential characteristics of user, technology, and context may differ considerably from those in ordinary business settings. This study examined physicians’ acceptance of telemedicine technology. Following a theory comparison approach, it evaluated the extent to which prevailing intention-based models, including the technology acceptance model (TAM), the theory of planned behavior (TPB) and an integrated model, could explain individual physicians’ technology acceptance decisions. Based on responses from more than 400 physicians, both models were evaluated in terms of overall fit, explanatory power, and their causal links. Overall, findings suggest that TAM may be more appropriate than TPB for examining technology acceptance by individual professionals and that the integrated model, although more fully depicting physicians’ technology acceptance, may not provide significant additional explanatory power. Also, instruments developed and repeatedly tested in prior studies involving conventional end-users and business managers may not be valid in professional settings. Several interesting implications are also discussed. # 2002 Elsevier Science B.V. All rights reserved.

As with many other sectors the amount of agriculture data based are increasing on a daily basis. However, the application of data mining methods and techniques to discover new insights or knowledge is a relatively a novel research area. In this paper we provide a brief review of a variety of Data Mining techniques that have been applied to model data from or about the agricultural domain. The Data Mining techniques applied on Agricultural data include k-means, bi clustering, k nearest neighbor, Neural Networks (NN) Support Vector Machine (SVM), Naive Bayes Classifier and Fuzzy c-means. As can be seen the appropriateness of data mining techniques is to a certain extent determined by the different types of agricultural data or the problems being addressed. This survey summarize the application of data mining techniques and predictive modeling application in the agriculture field.

Few issues can be more important to physicians or patients than that treatment decisions are based on reliable information about benefits and harms. While randomized clinical trials (RCTs) are generally regarded as the most valid source of evidence about benefits and some harms, concerns about their generalizability, costs, and heterogeneity of treatment effects have led to the search for other sources of information to augment or possibly replace trials. This is embodied in the recently passed 21st Century Cures Act, which mandates that the US Food and Drug Administration develop rules for the use of “real world evidence” in drug approval, defined as “data...derived from sources other than randomized clinical trials.”1 A second push toward the use of nontrial evidence is based on the perception that the torrent of electronic health-related data—medical record, genomic, and lifestyle (ie, “Big Data”)—can be transformed into reliable evidence with the use of powerful modern analytic tools. The use of nontrial evidence puts experts who weigh medical evidence, no less a practicing clinician, in a quandary, as the evaluation of such information requires skills well beyond those of basic critical review. The didactic tutorial by Agoritsas et al2 in this issue of JAMA partly addresses that problem, providing a clear, concise, and accessible introduction to several statistical approaches to adjust analyses of observational studies to minimize or eliminate confounding: matching, standard regression, propensity scores, and instrumental variable analyses. While understanding these technical tools is valuable, valid causal inference from nonrandomized studies about treatment effects depends on many factors other than confounding. These include whether the causal question motivating the study is clearly specified, whether the design matches that question and avoids design biases, whether the analysis matches the design, the appropriateness and quality of the data, the fit of adjustment models, and the potential for model searching to find spurious patterns in vast data streams. Some of these issues are recognizable and remediable, whereas others may defy solely analytic solutions, leading to the rejection of some nonrandomized studies to guide treatment. The hallmark of a well-posed causal question is that one can describe an RCT that would answer it. An RCT requires specifying at minimum an eligible population, an intervention, a comparator, and a start time. As posed by Hernán and Taubman,3 an example of a noncausal question is “Does obesity shorten life?” This is not causal because “obesity” is not a randomizable intervention, nor is weight loss. Randomizable interventions—the goal of which might be weight loss—include diet, exercise, gastric bypass surgery, liposuction, or drugs. Once the causal question is specified, the experimental design must mirror it, creating the foundation for a “like vs like” causal contrast on which a valid analysis is based. When observational data not collected for research are analyzed to estimate treatment effects, sometimes they are fed into statistical software without an explicit design and with a causal question that is either imprecise or not stated. This sets the stage for invalid results, even when the statistical methods appear to be properly implemented.4,5 The example of postmenopausal hormone therapy (HT) illustrates the criticality, when analyzing observational data, of posing a precise causal question, with a design and analysis to match. Observational studies, including one from the Nurses’ Health Study, using some of the adjustment techniques described by Agoritsas et al, showed a strong cardiovascular protective association with HT. This result was famously contravened by a subsequent RCT, the Women’s Health Initiative, which showed cardiovascular harm from HT. A subsequent reanalysis of the Nurses’ Health Study data conformed with the RCT results.6 The reanalysis was structured to mirror the clinical question asked in the trial; viz, how does initiating HT in a postmenopausal woman affect her cardiovascular risk? The original observational analysis, using standard regression, effectively excluded women with coronary heart disease events that occurred shortly after initiation of HT, a remediable design bias known in pharmacoepidemiology as “depletion of susceptibles”7 or in other fields as “survivorship bias.” No amount of adjustment for baseline factors between ongoing HT users and nonusers can overcome that bias, but by refining the question, and then using the appropriately tailored design, an appropriate analysis can be crafted. Following the causal thinking reflected in RCT design, most successful examples of observational research that produce findings similar to those from RCTs have compared new drug users with new users of a comparator agent.8,9 The disproven nonrandomized studies referenced at the end of the article by Agoritsas et al all compared users with nonusers. In an RCT, a nonuser is always someone who was eligible and considered for treatment, but that is not guaranteed in an observational study. If instead 2 active treatments indicated for the same condition are compared, it is likely Related article page 748 Opinion

Disease diagnosis is one of the most important applications of such system as it is one of the leading causes of deaths all over the world. Almost all system predicting disease use inputs from complex tests conducted in labs and none of the system predicts disease based on the risk factors such as tobacco smoking, alcohol intake, age, family history, diabetes, hypertension, high cholesterol, physical inactivity, obesity. Researchers have been using several data mining techniques to help health care professionals in the diagnosis of heart disease. K-Nearest-Neighbour (KNN) is one of the successful data mining techniques used in classification problems. However, it is less used in the diagnosis of heart disease patients. Recently, researchers are showing that combining different classifiers through voting is outperforming other single classifiers. This paper investigates applying KNN to help healthcare professionals in the diagnosis of disease specially heart disease. It also investigates if integrating voting with KNN can enhance its accuracy in the diagnosis of heart disease patients. The results show that applying KNN could achieve higher accuracy than neural network ensemble in the diagnosis of heart disease patients. The results also show that applying voting could not enhance the KNN accuracy in the diagnosis of heart disease.

Omer Gottesman, Fredrik Johansson, Joshua Meier, Jack Dent, Donghun Lee, Srivatsan Srinivasan, Linying Zhang, Yi Ding, David Wihl, Xuefeng Peng, Jiayu Yao, Isaac Lage, Christopher Mosch, Li-wei H. Lehman, Matthieu Komorowski, Aldo Faisal, Leo Anthony Celi, David Sontag, and Finale Doshi-Velez Paulson School of Engineering and Applied Sciences, Harvard University Institute for Medical Engineering and Science, MIT T.H. Chan School of Public Health, Harvard University Department of Statistics, Harvard University Laboratory for Computational Physiology, Harvard-MIT Health Sciences & Technology, MIT Department of Surgery and Cancer, Faculty of Medicine, Imperial College London Department of Bioengineering, Imperial College London Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center MIT Critical Data

The 2010 i2b2/VA Workshop on Natural Language Processing Challenges for Clinical Records presented three tasks: a concept extraction task focused on the extraction of medical concepts from patient reports; an assertion classification task focused on assigning assertion types for medical problem concepts; and a relation classification task focused on assigning relation types that hold between medical problems, tests, and treatments. i2b2 and the VA provided an annotated reference standard corpus for the three tasks. Using this reference standard, 22 systems were developed for concept extraction, 21 for assertion classification, and 16 for relation classification. These systems showed that machine learning approaches could be augmented with rule-based systems to determine concepts, assertions, and relations. Depending on the task, the rule-based systems can either provide input for machine learning or post-process the output of machine learning. Ensembles of classifiers, information from unlabeled data, and external knowledge sources can help when the training data are inadequate.


Question: A dual prediction neural network model for generating image captions.

In this paper, we explore the challenges of learning long-term dependencies in recurrent neural networks (RNNs) and propose a novel approach called Long Short-Term Memory (LSTM) to address these issues. Traditional RNNs struggle with the vanishing gradient problem, which significantly hinders the learning process over extended time intervals. We analyze Hochreiter's (1991) critique of this limitation and introduce LSTM as a gradient-based solution. By selectively truncating the gradient where it does not impact learning, LSTM is designed to overcome long time-lag challenges. It achieves this through the introduction of special units called constant error carousels, which facilitate the constant flow of error backpropagation. Multiplicative gate units are employed to learn when to allow or block this error flow, enabling the network to selectively retain or forget information over time. LSTM maintains local spatial and temporal computation, ensuring computational efficiency with a complexity of O.1 per time step and weight. Our experimental results using artificial data, which include local, distributed, real-valued, and noisy pattern representations, highlight the superiority of LSTM. Compared to other recurrent network algorithms like real-time recurrent learning, backpropagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM consistently demonstrates faster learning and a higher success rate in solving complex, long-time-lag tasks that were previously unsolvable.

As a fundamental task in the natural language processing field, language modeling aims to estimate the distribution of the word sequences. However, the most existing algorithms have focused on the main texts while often ignoring the vastly-accessible semantic attributes of the documents, e.g., titles, authors, sentiments and tags. To address this issue, we propose Semantic Attribute Modulated (SAM) language modeling, a novel language modeling framework that incorporates the various semantic attributes. Attributes are selected automatically with an attribute attention mechanism. We build three text datasets with a diversity of semantic attributes. On the three text datasets, we empirically examine the language model perplexities of several typical corpora, and then demonstrate the superiority of our model with the different combinations of the attributes. Extensive qualitative results, including word semantic analysis, attention values and an interesting lyric generation, further demonstrate the effectiveness of our SAMmethod.

We present the first user study on neural interactive translation prediction. Our neural translation aid was built into existing online translation software. Depending on the prefix already typed by the user the system suggests a list of continuations. We assessed the impact of the system on human translators in a user study with both professional and non-professional translators. The analysis was done with respect to translation speed and translation accuracy as assessed by human judges. We find that our neural translation aid enables relatively fast translations and does not compromise on quality.

We propose a semi-supervised model which segments and annotates images using very few labeled images and a large unaligned text corpus to relate image regions to text labels. Given photos of a sports event, all that is necessary to provide a pixel-level labeling of objects and background is a set of newspaper articles about this sport and one to five labeled images. Our model is motivated by the observation that words in text corpora share certain context and feature similarities with visual objects. We describe images using visual words, a new region-based representation. The proposed model is based on kernelized canonical correlation analysis which finds a mapping between visual and textual words by projecting them into a latent meaning space. Kernels are derived from context and adjective features inside the respective visual and textual domains. We apply our method to a challenging dataset and rely on articles of the New York Times for textual features. Our model outperforms the state-of-the-art in annotation. In segmentation it compares favorably with other methods that use significantly more labeled training data.

Modeling the distribution of natural images is a landmark problem in unsupervised learning. This task requires an image model that is at once expressive, tractable and scalable. We present a deep neural network that sequentially predicts the pixels in an image along the two spatial dimensions. Our method models the discrete probability of the raw pixel values and encodes the complete set of dependencies in the image. Architectural novelties include fast twodimensional recurrent layers and an effective use of residual connections in deep recurrent networks. We achieve log-likelihood scores on natural images that are considerably better than the previous state of the art. Our main results also provide benchmarks on the diverse ImageNet dataset. Samples generated from the model appear crisp, varied and globally coherent.

Together with the development of more accurate methods in Computer Vision and Natural Language Understanding, holistic architectures that answer on questions about the content of real-world images have emerged. In this tutorial, we build a neural-based approach to answer questions about images. We base our tutorial on two datasets: (mostly on) DAQUAR, and (a bit on) VQA. With small tweaks the models that we present here can achieve a competitive performance on both datasets, in fact, they are among the best methods that use a combination of LSTM with a global, full frame CNN representation of an image. We hope that after reading this tutorial, the reader will be able to use Deep Learning frameworks, such as Keras and introduced Kraino, to build various architectures that will lead to a further performance improvement on this challenging task. 1 Preface In this tutorial1 we build a few architectures that can answer questions about images. The architectures are based on our two papers on this topic: Malinowski et al. [2015] and Malinowski et al. [2016]; and more broadly, on our project towards a Visual Turing Test2. In particular, an encoder-decoder perspective of Malinowski et al. [2016] allows us to effectively experiment with various design choices. For the sake of simplicity, we only consider a classification-based approach to answer questions about images, although an approach that generate answers word-by-word is also studied in the community [Malinowski et al., 2015]. In the tutorial, we mainly focus on the DAQUAR dataset [Malinowski and Fritz, 2014a], but a few possible directions to apply learnt techniques to VQA [Antol et al., 2015] are also pointed. First, we will get familiar with the task of answering questions about images, and a dataset that implements the task (due to a small size, we mainly use DAQUAR as it better serves an educational purpose that we aim at this tutorial). Next, we build a few blind models that answer questions about images without actually seeing such images. Such models already exhibit a reasonable performance as they can effectively learn various biases that exist in a dataset, which we also interpret as learning a common sense knowledge [Malinowski et al., 2015, 2016]. Subsequently, we build a few language+vision models that answer questions based on both a textual and a visual inputs. Finally, we leave the tutorial with a few possible research directions. Technical aspects The tutorial is originally written using Python Notebook, which the reader is welcome to download and use through the tutorial. Instructions necessary to run the Notebook version of this tutorial are provided in the following: https://github.com/mateuszmalinowski/visual_turing_test-tutorial. In this tutorial, we heavily use a Python code, and therefore it is expected the reader either already knows this language, or can quickly learn it. However, we made an effort to make this tutorial approachable to a wider audience. We use Kraino that is a framework prepared for this tutorial in order to simplify the development of the question answering architectures. Under the hood, it uses Theano4 [Bastien et al., 2012] and Keras5 [Chollet, 2015] – two frameworks to build Deep Learning models. We also use various CNNs representations extracted from images that can be downloaded as explained at the beginning of our Notebook tutorial. We also highlight exercises that a curious reader may attempt to solve. This tutorial was presented for the first time during the 2nd Summer School on Integrating Vision and Language: Deep Learning. http://mpii.de/visual_turing_test https://github.com/mateuszmalinowski/visual_turing_test-tutorial/blob/master/visual_ turing_test.ipynb http://deeplearning.net/software/theano/ https://keras.io 1 ar X iv :1 61 0. 01 07 6v 1 [ cs .C V ] 4 O ct 2 01 6

We consider the problem of learning to summarize images by text and visualize text utilizing images, which we call Mutual-Summarization. We divide the web image-text data space into three subspaces, namely pure image space (PIS), pure text space (PTS) and image-text joint space (ITJS). Naturally, we treat the ITJS as a knowledge base.
 For summarizing images by sentence issue, we map images from PIS to ITJS via image classification models and use text summarization on the corresponding texts in ITJS to summarize images. For text visualization problem, we map texts from PTS to ITJS via text categorization models and generate the visualization by choosing the semantic related images from ITJS, where the selected images are ranked by their confidence. In above approaches images are represented by color histograms, dense visual words and feature descriptors at different levels of spatial pyramid; and the texts are generated according to the Latent Dirichlet Allocation (LDA) topic model. Multiple Kernel (MK) methodologies are used to learn classifiers for image and text respectively. We show the Mutual-Summarization results on our newly collected dataset of six big events ("Gulf Oil Spill", "Haiti Earthquake", etc.) as well as demonstrate improved cross-media retrieval performance over existing methods in terms of MAP, Precision and Recall.

We present sketch-rnn, a recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on thousands of crude human-drawn images representing hundreds of classes. We outline a framework for conditional and unconditional sketch generation, and describe new robust training methods for generating coherent sketch drawings in a vector format.

Prediction without justification has limited applicability. As a remedy, we learn to extract pieces of input text as justifications – rationales – that are tailored to be short and coherent, yet sufficient for making the same prediction. Our approach combines two modular components, generator and encoder, which are trained to operate well together. The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction. Rationales are never given during training. Instead, the model is regularized by desiderata for rationales. We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases. Our approach outperforms attention-based baseline by a significant margin. We also successfully illustrate the method on the question retrieval task.1

Neural net advances improve computers' language ability in many fields.

In this work, we present a simple, highly efficient and modularized Dual Path Network (DPN) for image classification which presents a new topology of connection paths internally. By revealing the equivalence of the state-of-the-art Residual Network (ResNet) and Densely Convolutional Network (DenseNet) within the HORNN framework, we find that ResNet enables feature re-usage while DenseNet enables new features exploration which are both important for learning good representations. To enjoy the benefits from both path topologies, our proposed Dual Path Network shares common features while maintaining the flexibility to explore new features through dual path architectures. Extensive experiments on three benchmark datasets, ImagNet-1k, Places365 and PASCAL VOC, clearly demonstrate superior performance of the proposed DPN over state-of-the-arts. In particular, on the ImagNet-1k dataset, a shallow DPN surpasses the best ResNeXt-101(64× 4d) with 26% smaller model size, 25% less computational cost and 8% lower memory consumption, and a deeper DPN (DPN-131) further pushes the state-of-the-art single model performance with about 2 times faster training speed. Experiments on the Places365 large-scale scene dataset, PASCAL VOC detection dataset, and PASCAL VOC segmentation dataset also demonstrate its consistently better performance than DenseNet, ResNet and the latest ResNeXt model over various applications.


Question: A dual prediction network approach for generating captions for images.

This paper presents the Spatial and Channel-Wise Attention Convolutional Neural Network (SCA-CNN), a novel architecture designed to enhance visual attention in tasks such as image captioning and question answering. While traditional visual attention models primarily focus on spatial probabilities to reweight the final convolutional layer's feature map of a CNN for input image encoding, SCA-CNN takes a more comprehensive approach. It argues that attention mechanisms should not be limited to spatial aspects, considering the multi-faceted nature of CNN features which are inherently spatial, channel-wise, and multi-layered. SCA-CNN integrates both spatial and channel-wise attentions, dynamically adjusting the sentence generation context across multiple layers of feature maps. This allows the network to selectively focus on where (attentive spatial locations across multiple layers) and what (attentive channels) aspects of the visual attention are crucial for the task at hand. The effectiveness of the SCA-CNN architecture is demonstrated through extensive evaluations on three major image captioning datasets: Flickr8K, Flickr30K, and MSCOCO. Results consistently show that SCA-CNN outperforms existing state-of-the-art visual attention-based image captioning methods by a significant margin.

We extend the SKIP-GRAM model of Mikolov et al. (2013a) by taking visual information into account. Like SKIP-GRAM, our multimodal models (MMSKIP-GRAM) build vector-based word representations by learning to predict linguistic contexts in text corpora. However, for a restricted set of words, the models are also exposed to visual representations of the objects they denote (extracted from natural images), and must predict linguistic and visual features jointly. The MMSKIP-GRAM models achieve good performance on a variety of semantic benchmarks. Moreover, since they propagate visual information to all words, we use them to improve image labeling and retrieval in the zero-shot setup, where the test concepts are never seen during model training. Finally, the MMSKIP-GRAM models discover intriguing visual properties of abstract words, paving the way to realistic implementations of embodied theories of meaning.

A large amount of annotated training images is critical for training accurate and robust deep network models but the collection of a large amount of annotated training images is often time-consuming and costly. Image synthesis alleviates this constraint by generating annotated training images automatically by machines which has attracted increasing interest in the recent deep learning research. We develop an innovative image synthesis technique that composes annotated training images by realistically embedding foreground objects of interest (OOI) into background images. The proposed technique consists of two key components that in principle boost the usefulness of the synthesized images in deep network training. The first is context-aware semantic coherence which ensures that the OOI are placed around semantically coherent regions within the background image. The second is harmonious appearance adaptation which ensures that the embedded OOI are agreeable to the surrounding background from both geometry alignment and appearance realism. The proposed technique has been evaluated over two related but very different computer vision challenges, namely, scene text detection and scene text recognition. Experiments over a number of public datasets demonstrate the effectiveness of our proposed image synthesis technique the use of our synthesized images in deep network training is capable of achieving similar or even better scene text detection and scene text recognition performance as compared with using real images.

Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP are not expressive enough, in the sense that all input sentences share the same learned (and static) set of filters. Motivated by this problem, we propose an adaptive convolutional filter generation framework for natural language understanding, by leveraging a meta network to generate inputaware filters. We further generalize our framework to model question-answer sentence pairs and propose an adaptive question answering (AdaQA) model; a novel two-way feature abstraction mechanism is introduced to encapsulate co-dependent sentence representations. We investigate the effectiveness of our framework on document categorization and answer sentence-selection tasks, achieving state-of-the-art performance on several

CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart) are increasingly used in many applications for machine and human identification. Compared with traditional English and digital characters based CAPTCHAs, Chinese characters contain more complicated characters which greatly enhance difficulty of automatic recognition. To solve that problem, we proposed a Convolution Neural Network (CNN) based approach. This approach greatly improves the recognition accuracy of Chinese Character CAPTCHAs with distortion, rotation and background noise. Our experiment results show that this approach achieves more than 95% accuracy for single character and 84% accuracy for three types of Chinese Character CAPTCHAs with four characters. This encouraging result indicates that deep neural network is useful in complicated structure perception of Chinese Character CAPTCHAs.

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

We introduce a novel Recurrent Neural Network-based algorithm for future video feature generation and action anticipation called feature mapping RNN . Our novel RNN architecture builds upon three effective principles of machine learning, namely parameter sharing, Radial Basis Function kernels and adversarial training. Using only some of the earliest frames of a video, the feature mapping RNN is able to generate future features with a fraction of the parameters needed in traditional RNN. By feeding these future features into a simple multilayer perceptron facilitated with an RBF kernel layer, we are able to accurately predict the action in the video. In our experiments, we obtain 18% improvement on JHMDB-21 dataset, 6% on UCF101-24 and 13% improvement on UT-Interaction datasets over prior stateof-the-art for action anticipation.

In this paper, we investigate an approach for reconstructing storyline graphs from large-scale collections of Internet images, and optionally other side information such as friendship graphs. The storyline graphs can be an effective summary that visualizes various branching narrative structure of events or activities recurring across the input photo sets of a topic class. In order to explore further the usefulness of the storyline graphs, we leverage them to perform the image sequential prediction tasks, from which photo recommendation applications can benefit. We formulate the storyline reconstruction problem as an inference of sparse time-varying directed graphs, and develop an optimization algorithm that successfully addresses a number of key challenges of Web-scale problems, including global optimality, linear complexity, and easy parallelization. With experiments on more than 3.3 millions of images of 24 classes and user studies via Amazon Mechanical Turk, we show that the proposed algorithm improves other candidate methods for both storyline reconstruction and image prediction tasks.

Social media platforms allow rapid information diffusion, and serve as a source of information to many of the users. Particularly, in Twitter information provided by tweets diffuses over the users through retweets. Hence, being able to predict the retweet count of a given tweet is important for understanding and controlling information diffusion on Twitter. Since the length of a tweet is limited to 140 characters, extracting relevant features to predict the retweet count is a challenging task. However, visual features of images linked in tweets may provide predictive features. In this study, we focus on predicting the expected retweet count of a tweet by using visual cues of an image linked in that tweet in addition to content and structure-based features.

Recursive structure is commonly found in the inputs of different modalities such as natural scene images or natural language sentences. Discovering this recursive structure helps us to not only identify the units that an image or sentence contains but also how they interact to form a whole. We introduce a max-margin structure prediction architecture based on recursive neural networks that can successfully recover such structure both in complex scene images as well as sentences. The same algorithm can be used both to provide a competitive syntactic parser for natural language sentences from the Penn Treebank and to outperform alternative approaches for semantic scene segmentation, annotation and classification. For segmentation and annotation our algorithm obtains a new level of state-of-theart performance on the Stanford background dataset (78.1%). The features from the image parse tree outperform Gist descriptors for scene classification by 4%.

With the exponential growth of web meta-data, exploiting multimodal online sources via standard search engine has become a trend in visual recognition as it effectively alleviates the shortage of training data. However, the web meta-data such as text data is usually not as cooperative as expected due to its unstructured nature. To address this problem, this paper investigates the numerical representation of web text data. We firstly adopt convolutional neural network (CNN) for web text modeling on top of word vectors. Combined with CNN for image, we present a multimodal fusion to maximize the discriminative power of visual and textual modality data for decision level and feature level simultaneously. Experimental results show that the proposed framework achieves significant improvement in large-scale image classification on Pascal VOC-2007 and VOC-2012 datasets.


Question: A dual prediction network approach for generating captions for images.

This paper introduces a novel framework for enhancing the training of deep neural networks, specifically addressing the challenge of training significantly deeper networks than those previously utilized. We propose a residual learning approach, which involves reformulating the layers to learn residual functions relative to the input layers, rather than learning functions independently. This strategy significantly simplifies the optimization process and enables the extraction of more accurate representations from increased network depth. On the ImageNet dataset, our residual networks reach depths of up to 152 layers, 8 times deeper than VGG nets [40], yet with reduced complexity. An ensemble of these residual networks achieved an error rate of 3.57% on the ImageNet test set, securing the top position in the ILSVRC 2015 classification task. Additionally, we conduct a detailed analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations plays a crucial role in various visual recognition tasks. Leveraging this depth, we observe a 28% relative improvement in performance on the COCO object detection dataset. Deep residual networks form the core of our submissions to the ILSVRC & COCO 2015 competitions, where we emerged victorious in the ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation tasks.

We propose a method for embedding twodimensional locations in a continuous vector space using a neural network-based model incorporating mixtures of Gaussian distributions, presenting two model variants for text-based geolocation and lexical dialectology. Evaluated over Twitter data, the proposed model outperforms conventional regression-based geolocation and provides a better estimate of uncertainty. We also show the effectiveness of the representation for predicting words from location in lexical dialectology, and evaluate it using the DARE dataset.

We introduce an effective and efficient method that grounds (i.e., localizes) natural sentences in long, untrimmed video sequences. Specifically, a novel Temporal GroundNet (TGN)1 is proposed to temporally capture the evolving fine-grained frame-by-word interactions between video and sentence. TGN sequentially scores a set of temporal candidates ended at each frame based on the exploited frameby-word interactions, and finally grounds the segment corresponding to the sentence. Unlike traditional methods treating the overlapping segments separately in a sliding window fashion, TGN aggregates the historical information and generates the final grounding result in one single pass. We extensively evaluate our proposed TGN on three public datasets with significant improvements over the stateof-the-arts. We further show the consistent effectiveness and efficiency of TGN through an ablation study and a runtime test.

We propose a deep learning method for single image superresolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deep convolutional neural network (CNN) [15] that takes the lowresolution image as the input and outputs the high-resolution one. We further show that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network. But unlike traditional methods that handle each component separately, our method jointly optimizes all layers. Our deep CNN has a lightweight structure, yet demonstrates state-of-the-art restoration quality, and achieves fast speed for practical on-line usage.

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach is demonstrated for text (where the data are discrete) and online handwriting (where the data are real-valued). It is then extended to handwriting synthesis by allowing the network to condition its predictions on a text sequence. The resulting system is able to generate highly realistic cursive handwriting in a wide variety of styles.

People often try to find an image using a short query and images are usually indexed using short annotations. Matching the query vocabulary with the indexing vocabulary is a difficult problem when little text is available. Textual user generated content in Web 2.0 platforms contains a wealth of data that can help solve this problem. Here we describe how to use Wikipedia and Flickr content to improve this match. The initial query is launched in Flickr and we create a query model based on co-occurring terms. We also calculate nearby concepts using Wikipedia and use these to expand the query. The final results are obtained by ranking the results for the expanded query using the similarity between their annotation and the Flickr model. Evaluation of these expansion and ranking techniques, over the Image CLEF 2010 Wikipedia Collection containing 237,434 images and their multilingual textual annotations, shows that a consistent improvement compared to state of the art methods.

Automatic image annotation is an effective way for managing and retrieving abundant images on the internet. In this paper, a bipartite graph reinforcement model (BGRM) is proposed for web image annotation. Given a web image, a set of candidate annotations is extracted from its surrounding text and other textual information in the hosting web page. As this set is often incomplete, it is extended to include more potentially relevant annotations by searching and mining a large-scale image database. All candidates are modeled as a bipartite graph. Then a reinforcement algorithm is performed on the bipartite graph to re-rank the candidates. Only those with the highest ranking scores are reserved as the final annotations. Experimental results on real web images demonstrate the effectiveness of the proposed model.

This paper has two parts. In the first part we discuss word embeddings. We discuss the need for them, some of the methods to create them, and some of their interesting properties. We also compare them to image embeddings and see how word embedding and image embedding can be combined to perform different tasks. In the second part we implement a convolutional neural network trained on top of pre-trained word vectors. The network is used for several sentence-level classification tasks, and achieves state-of-art (or comparable) results, demonstrating the great power of pre-trainted word embeddings over random ones.

Nonparametric Bayesian Models for Joint Analysis of Imagery and Text

We propose “Areas of Attention”, a novel attentionbased model for automatic image captioning. Our approach models the dependencies between image regions, caption words, and the state of an RNN language model, using three pairwise interactions. In contrast to previous attentionbased approaches that associate image regions only to the RNN state, our method allows a direct association between caption words and image regions. During training these associations are inferred from image-level captions, akin to weakly-supervised object detector training. These associations help to improve captioning by localizing the corresponding regions during testing. We also propose and compare different ways of generating attention areas: CNN activation grids, object proposals, and spatial transformers nets applied in a convolutional fashion. Spatial transformers give the best results. They allow for image specific attention areas, and can be trained jointly with the rest of the network. Our attention mechanism and spatial transformer attention areas together yield state-of-the-art results on the MSCOCO dataset.

Prediction without justification has limited applicability. As a remedy, we learn to extract pieces of input text as justifications – rationales – that are tailored to be short and coherent, yet sufficient for making the same prediction. Our approach combines two modular components, generator and encoder, which are trained to operate well together. The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction. Rationales are never given during training. Instead, the model is regularized by desiderata for rationales. We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases. Our approach outperforms attention-based baseline by a significant margin. We also successfully illustrate the method on the question retrieval task.1


Question: A dual prediction neural network for generating image captions.

This paper presents CIDEr, a novel consensus-based evaluation metric for assessing the quality of image descriptions generated by automated systems. The task of automatically generating descriptive sentences for images has gained renewed interest due to advancements in object detection, attribute classification, and action recognition. However, evaluating the accuracy and appropriateness of these descriptions has remained a significant challenge. The proposed CIDEr metric addresses this issue by focusing on human consensus, a key aspect of natural language understanding and generation. The evaluation framework is comprised of three core components: a triplet-based annotation method to measure consensus among human annotators, a new automated metric that quantifies this consensus, and two extensive datasets—PASCAL-50S and ABSTRACT-50S—featuring 50 descriptions for each image. The effectiveness of CIDEr in reflecting human judgment is demonstrated through comparisons with existing metrics, showing superior performance across descriptions produced by various sources. Additionally, the paper evaluates five leading image description approaches using this new protocol, establishing a benchmark for future research. A variant of CIDEr, known as CIDEr-D, is integrated into the Microsoft COCO evaluation server, facilitating systematic assessment and benchmarking of image captioning models.

Automatically describing videos with natural language is a crucial challenge of video understanding. Compared to images, videos have specific spatial-temporal structure and various modality information. In this paper, we propose a Multirate Multimodal Approach for video captioning. Considering that the speed of motion in videos varies constantly, we utilize a Multirate GRU to capture temporal structure of videos. It encodes video frames with different intervals and has a strong ability to deal with motion speed variance. As videos contain different modality cues, we design a particular multimodal fusion method. By incorporating visual, motion, and topic information together, we construct a well-designed video representation. Then the video representation is fed into a RNN-based language model for generating natural language descriptions. We evaluate our approach for video captioning on "Microsoft Research - Video to Text" (MSR-VTT), a large-scale video benchmark for video understanding. And our approach gets great performance on the 2nd MSR Video to Language Challenge.

In this paper, we investigate an approach for reconstructing storyline graphs from large-scale collections of Internet images, and optionally other side information such as friendship graphs. The storyline graphs can be an effective summary that visualizes various branching narrative structure of events or activities recurring across the input photo sets of a topic class. In order to explore further the usefulness of the storyline graphs, we leverage them to perform the image sequential prediction tasks, from which photo recommendation applications can benefit. We formulate the storyline reconstruction problem as an inference of sparse time-varying directed graphs, and develop an optimization algorithm that successfully addresses a number of key challenges of Web-scale problems, including global optimality, linear complexity, and easy parallelization. With experiments on more than 3.3 millions of images of 24 classes and user studies via Amazon Mechanical Turk, we show that the proposed algorithm improves other candidate methods for both storyline reconstruction and image prediction tasks.

A large amount of annotated training images is critical for training accurate and robust deep network models but the collection of a large amount of annotated training images is often time-consuming and costly. Image synthesis alleviates this constraint by generating annotated training images automatically by machines which has attracted increasing interest in the recent deep learning research. We develop an innovative image synthesis technique that composes annotated training images by realistically embedding foreground objects of interest (OOI) into background images. The proposed technique consists of two key components that in principle boost the usefulness of the synthesized images in deep network training. The first is context-aware semantic coherence which ensures that the OOI are placed around semantically coherent regions within the background image. The second is harmonious appearance adaptation which ensures that the embedded OOI are agreeable to the surrounding background from both geometry alignment and appearance realism. The proposed technique has been evaluated over two related but very different computer vision challenges, namely, scene text detection and scene text recognition. Experiments over a number of public datasets demonstrate the effectiveness of our proposed image synthesis technique the use of our synthesized images in deep network training is capable of achieving similar or even better scene text detection and scene text recognition performance as compared with using real images.

Neural machine translation (NMT) becomes a new state-ofthe-art and achieves promising translation results using a simple encoder-decoder neural network. This neural network is trained once on the parallel corpus and the fixed network is used to translate all the test sentences. We argue that the general fixed network cannot best fit the specific test sentences. In this paper, we propose the dynamic NMT which learns a general network as usual, and then fine-tunes the network for each test sentence. The fine-tune work is done on a small set of the bilingual training data that is obtained through similarity search according to the test sentence. Extensive experiments demonstrate that this method can significantly improve the translation performance, especially when highly similar sentences are available.

Context-predicting models (more commonly known as embeddings or neural language models) are the new kids on the distributional semantics block. Despite the buzz surrounding these models, the literature is still lacking a systematic comparison of the predictive models with classic, count-vector-based distributional semantic approaches. In this paper, we perform such an extensive evaluation, on a wide range of lexical semantics tasks and across many parameter settings. The results, to our own surprise, show that the buzz is fully justified, as the context-predicting models obtain a thorough and resounding victory against their count-based counterparts.

We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel form of semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in state-of-the-art-performance.

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach is demonstrated for text (where the data are discrete) and online handwriting (where the data are real-valued). It is then extended to handwriting synthesis by allowing the network to condition its predictions on a text sequence. The resulting system is able to generate highly realistic cursive handwriting in a wide variety of styles.

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

– Section 1: converting between different formats of ground-truth annotations (Section 3.1 in the main text) – Section 2: details of the datasets (Section 4.1 in the main text) – Section 3: details of our LSTM-based models, including the learning objective for dppLSTM and the generating process of shot-based summaries for both vsLSTM and dppLSTM (Section 3.4 and 3.5 in the main text) – Section 4: comparing different network structures for dppLSTM (Section 3.4 in the main text) – Section 5: Other implementation details – Section 6: Additional discussions on video summarization

Attention mechanism has been a key component in Recurrent Neural Networks (RNNs) based sequence to sequence learning framework, which has been adopted in many text understanding tasks, such as neural machine translation and abstractive summarization. In these tasks, the attention mechanism models how important each part of the source sentence is to generate a target side word. To compute such importance scores, the attention mechanism summarizes the source side information in the encoder RNN hidden states (i.e., ht), and then builds a context vector for a target side word upon a subsequence representation of the source sentence, since ht actually summarizes the information of the subsequence containing the first t-th words in the source sentence. We in this paper, show that an additional attention mechanism called word attention, that builds itself upon word level representations, significantly enhances the performance of sequence to sequence learning. Our word attention can enrich the source side contextual representation by directly promoting the clean word level information in each step. Furthermore, we propose to use contextual gates to dynamically combine the subsequence level and word level contextual information. Experimental results on abstractive summarization and neural machine translation show that word attention significantly improve over strong baselines. In particular, we achieve the state-of-the-art result on WMT’14 English-French translation task with 12M training data.