Question:
SGDR: Stochastic Gradient Descent with Restart Mechanism

**Question**: A novel multi-objective optimization algorithm for solving the economic load dispatch problem

**Similar Question**: An innovative multi-objective optimization technique for addressing the economic load dispatch issue

This paper explores the repurposing of features derived from the activation of a deep convolutional network, originally trained in a fully supervised manner on a large, static set of object recognition tasks, for use in generic visual recognition tasks. These generic tasks may vary considerably from the initial training tasks and may lack sufficient labeled or unlabeled data to conventionally train or adapt a deep architecture for the new tasks. We delve into and visually represent the semantic clustering of deep convolutional features across a range of such tasks, encompassing scene recognition, domain adaptation, and fine-grained recognition challenges. Our investigation includes an assessment of the effectiveness of utilizing different network levels to define a fixed feature representation. We present cutting-edge results that markedly surpass the current state-of-the-art on multiple significant vision challenges. Accompanying this research, we are making DeCAF, an open-source implementation of these deep convolutional activation features, along with all associated network parameters, available to the vision research community. This resource will facilitate experimentation with deep representations across various visual concept learning scenarios.

We propose a new stochastic algorithm (generalized simulated annealing) for computationally finding the global minimum of a given (not necessarily convex) energy/cost function defined in a continuous D-dimensional space. This algorithm recovers, as particular cases, the so called classical (“Boltzmann machine”) and fast (“Cauchy machine”) simulated annealings, and can be quicker than both. Key-words: Simulated annealing; Nonconvex optimization; Gradient descent; Generalized Statistical Mechanics.

The long short-term memory (LSTM) network trained by gradient descent solves difficult problems which traditional recurrent neural networks in general cannot. We have recently observed that the decoupled extended Kalman filter training algorithm allows for even better performance, reducing significantly the number of training steps when compared to the original gradient descent training algorithm. In this paper we present a set of experiments which are unsolvable by classical recurrent networks but which are solved elegantly and robustly and quickly by LSTM combined with Kalman filters.

Artificial neural networks (ANNs) have been widely applied in electricity price forecasts due to their nonlinear modeling capabilities. However, it is well known that in general, traditional training methods for ANNs such as back-propagation (BP) approach are normally slow and it could be trapped into local optima. In this paper, a fast electricity market price forecast method is proposed based on a recently emerged learning method for single hidden layer feed-forward neural networks, the extreme learning machine (ELM), to overcome these drawbacks. The new approach also has improved price intervals forecast accuracy by incorporating bootstrapping method for uncertainty estimations. Case studies based on chaos time series and Australian National Electricity Market price series show that the proposed method can effectively capture the nonlinearity from the highly volatile price data series with much less computation time compared with other methods. The results show the great potential of this proposed approach for online accurate price forecasting for the spot market prices analysis.

Load disaggregation based on aided linear integer programming (ALIP) is proposed. We start with a conventional linear integer programming (IP)-based disaggregation and enhance it in several ways. The enhancements include additional constraints, correction based on a state diagram, median filtering, and linear-programming-based refinement. With the aid of these enhancements, the performance of IP-based disaggregation is significantly improved. The proposed ALIP system relies only on the instantaneous load samples instead of waveform signatures and, hence, works well on low-frequency data. Experimental results show that the proposed ALIP system performs better than conventional IP-based load disaggregation.

This paper proposes a scheme that integrates rate of change of frequency (ROCOF) and under frequency (UF) elements to design more effective load shedding response. A new load block activation algorithm is also introduced that improves the overall load shedding (LS) performance. The combination of ROCOF and UF criteria is used to overcome the LS block discrimination problem and to provide a flexible and decentralised solution to mitigation. The proposed method is verified through simulations on the IEEE 39-bus test system.

Adaptive gradient-based optimizers such as AdaGrad and Adam are among the methods of choice in modern machine learning. These methods maintain second-order statistics of each parameter, thus doubling the memory footprint of the optimizer. In behemoth-size applications, this memory overhead restricts the size of the model being used as well as the number of examples in a mini-batch. We describe a novel, simple, and flexible adaptive optimization method with sublinear memory cost that retains the benefits of per-parameter adaptivity while allowing for larger models and mini-batches. We give convergence guarantees for our method and demonstrate its effectiveness in training very large deep models.

In this paper, we consider learning a Bayesian collaborative filtering model on a shared cluster of commodity machines. Two main challenges arise: (1) How can we parallelize and distribute Bayesian collaborative filtering? (2) How can our distributed inference system handle elasticity events common in a shared, resource managed cluster, including resource ramp-up, preemption, and stragglers? To parallelize Bayesian inference, we adapt ideas from both matrix factorization partitioning schemes used with stochastic gradient descent and stale synchronous programming used with parameter servers. To handle elasticity events we offer a generalization of previous partitioning schemes that gives increased flexibility during system disruptions. We additionally describe two new scheduling algorithms to dynamically route work at runtime. In our experiments, we compare the effectiveness of both scheduling algorithms and demonstrate their robustness to system failure.

The supervisory control strategy of a hybrid vehicle coordinates the operation of vehicle sub-systems to achieve performance targets such as maximizing fuel economy and reducing exhaust emissions. This high-level control problem is commonly referred as the power management problem. In the past, many supervisory control strategies were developed on the basis of a few pre-defined driving cycles, using intuition and heuristics. The resulting control strategy was often inherently cycle-beating and lacked a guaranteed level of optimality. In this study, the power management problem is tackled from a stochastic viewpoint. An infinite-horizon stochastic dynamic optimization problem is formulated. The power demand from the driver is modeled as a random Markov process. The optimal control strategy is then obtained by using stochastic dynamic programming (SDP). The obtained control law is in the form of a stationary full-state feedback and can be directly implemented. Simulation results over standard driving cycles and random driving cycles are presented to demonstrate the effectiveness of the proposed stochastic approach. It was found that the obtained SDP control algorithm outperforms a sub-optimal rule-based control strategy trained from deterministic DP results.

In this paper, we propose a method for solving the distributed optimization problem in which the objective function is the sum of separable convex functions with linear constraints. In our approach, the primal variable is partially updated to make the Method of Multiplier algorithm distributed which is based on the suitable scaling of constraint matrix. The algorithm is then viewed as a dynamical system the convergence analysis of which is done using the passivity concepts of nonlinear control theory. The convexity of the function is related to the passivity of the non-linear functions which is in feedback with the positive real linear system.

We study the resilience to Byzantine failures of distributed implementations of Stochastic Gradient Descent (SGD). So far, distributed machine learning frameworks have largely ignored the possibility of failures, especially arbitrary (i.e., Byzantine) ones. Causes of failures include software bugs, network asynchrony, biases in local datasets, as well as attackers trying to compromise the entire system. Assuming a set of n workers, up to f being Byzantine, we ask how resilient can SGD be, without limiting the dimension, nor the size of the parameter space. We first show that no gradient aggregation rule based on a linear combination of the vectors proposed by the workers (i.e, current approaches) tolerates a single Byzantine failure. We then formulate a resilience property of the aggregation rule capturing the basic requirements to guarantee convergence despite f Byzantine workers. We propose Krum, an aggregation rule that satisfies our resilience property, which we argue is the first provably Byzantine-resilient algorithm for distributed SGD. We also report on experimental evaluations of Krum.


Question: SGDR: Stochastic Gradient Descent with Restart Mechanism.

This paper introduces a novel framework for subgradient methods that dynamically integrates insights from previous iterations' data geometry to enhance the effectiveness of gradient-based learning. Inspired by recent advancements in stochastic optimization and online learning, these methods leverage proximal functions to govern the gradient steps, enabling the discovery of highly predictive yet infrequently encountered features, akin to finding needles in a haystack. Our approach revolves around an adaptive mechanism for adjusting the proximal function, which significantly streamlines the learning rate setting process while ensuring regret bounds that match those achievable with the optimal proximal function in hindsight. We outline and analyze several efficient algorithms designed for empirical risk minimization tasks, focusing on prevalent regularization functions and domain constraints. Through experimental validation, we corroborate our theoretical findings, demonstrating that adaptive subgradient methods outperform their non-adaptive counterparts, which are currently considered state-of-the-art in the field.

{ Adding Gradient Noise Improves Learning for Very Deep Networks. Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens. International Conference on Learning Representations Workshop (ICLR WS), 2016 { Generating Sentences from a Continuous Space. Samuel Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, Samy Bengio. International Conference on Learning Representations Workshop (ICLR WS), 2016 { Bethe Projections for Non-Local Inference. Luke Vilnis, David Belanger, Daniel Sheldon, Andrew McCallum. Uncertainty in Artificial Intelligence (UAI), 2015 { Learning Dynamic Feature Selection for Fast Sequential Prediction. Emma Strubell, Luke Vilnis, Kate Silverstein, Andrew McCallum. Annual Meeting of the Association for Computational Linguistics (ACL), 2015 { Word Representations via Gaussian Embedding. Luke Vilnis, Andrew McCallum. International Conference on Learning Representations (ICLR), 2015 { Generalized Eigenvectors for Large Multiclass Problems. Luke Vilnis, Nikos Karampatziakis, Paul Mineiro. Neural Information Processing Systems Workshop on Representation and Learning Methods for Complex Outputs (NIPS WS), 2014 { Training for Fast Sequential Prediction Using Dynamic Feature Selection. Emma Strubell, Luke Vilnis, Andrew McCallum. Neural Information Processing Systems Workshop on Modern Machine Learning and Natural Language Processing (NIPS WS), 2014

We propose a new weight initialization suited for arbitrary nonlinearities by generalizing previous weight initializations. The initialization corrects for the influence of dropout rates and an arbitrary nonlinearity’s influence on variance through simple corrective scalars. Consequently, this initialization does not require computing mini-batch statistics nor weight pre-initialization. This simple method enables improved accuracy over previous initializations, and it allows for training highly regularized neural networks where previous initializations lead to poor convergence.

In this paper, we analyze the numerics of common algorithms for training Generative Adversarial Networks (GANs). Using the formalism of smooth two-player games we analyze the associated gradient vector field of GAN training objectives. Our findings suggest that the convergence of current algorithms suffers due to two factors: i) presence of eigenvalues of the Jacobian of the gradient vector field with zero real-part, and ii) eigenvalues with big imaginary part. Using these findings, we design a new algorithm that overcomes some of these limitations and has better convergence properties. Experimentally, we demonstrate its superiority on training common GAN architectures and show convergence on GAN architectures that are known to be notoriously hard to train.

Large deep neural network models have recently demonstrated state-of-the-art accuracy on hard visual recognition tasks. Unfortunately such models are extremely time consuming to train and require large amount of compute cycles. We describe the design and implementation of a distributed system called Adam comprised of commodity server machines to train such models that exhibits world-class performance, scaling and task accuracy on visual recognition tasks. Adam achieves high efficiency and scalability through whole system co-design that optimizes and balances workload computation and communication. We exploit asynchrony throughout the system to improve performance and show that it additionally improves the accuracy of trained models. Adam is significantly more efficient and scalable than was previously thought possible and used 30x fewer machines to train a large 2 billion connection model to 2x higher accuracy in comparable time on the ImageNet 22,000 category image classification task than the system that previously held the record for this benchmark. We also show that task accuracy improves with larger models. Our results provide compelling evidence that a distributed systems-driven approach to deep learning using current training algorithms is worth pursuing.

Network pruning is widely used for reducing the heavy computational cost of 1 deep models. A typical pruning algorithm is a three-stage pipeline, i.e., training (a 2 large model), pruning and fine-tuning. In this work, we make a rather surprising 3 observation: fine-tuning a pruned model only gives comparable or even worse 4 performance than training that model with randomly initialized weights. Our 5 results have several implications: 1) training a large, over-parameterized model is 6 not necessary to obtain an efficient final model, 2) learned “important” weights 7 of the large model are not necessarily useful for the small pruned model, 3) the 8 pruned architecture itself, rather than a set of inherited weights, is what leads to the 9 efficiency benefit in the final model, which suggests that some pruning algorithms 10 could be seen as performing network architecture search. 11

In this paper, we propose a new model called deep network cascade (DNC) to gradually upscale low-resolution images layer by layer, each layer with a small scale factor. DNC is a cascade of multiple stacked collaborative local auto-encoders. In each layer of the cascade, non-local self-similarity search is first performed to enhance high-frequency texture details of the partitioned patches in the input image. The enhanced image patches are then input into a collaborative local auto-encoder (CLA) to suppress the noises as well as collaborate the compatibility of the overlapping patches. By closing the loop on non-local self-similarity search and CLA in a cascade layer, we can refine the super-resolution result, which is further fed into next layer until the required image scale. Experiments on image super-resolution demonstrate that the proposed DNC can gradually upscale a low-resolution image with the increase of network layers and achieve more promising results in visual quality as well as quantitative performance.

Modern convolutional networks, incorporating rectifiers and max-pooling, are neither smooth nor convex; standard guarantees therefore do not apply. Nevertheless, methods from convex optimization such as gradient descent and Adam are widely used as building blocks for deep learning algorithms. This paper provides the first convergence guarantee applicable to modern convnets, which furthermore matches a lower bound for convex nonsmooth functions. The key technical tool is the neural Taylor approximation – a straightforward application of Taylor expansions to neural networks – and the associated Taylor loss. Experiments on a range of optimizers, layers, and tasks provide evidence that the analysis accurately captures the dynamics of neural optimization. The second half of the paper applies the Taylor approximation to isolate the main difficulty in training rectifier nets – that gradients are shattered – and investigates the hypothesis that, by exploring the space of activation configurations more thoroughly, adaptive optimizers such as RMSProp and Adam are able to converge to better solutions.

Stacking-based deep neural network (S-DNN), in general, denotes a deep neural network (DNN) resemblance in terms of its very deep, feedforward network architecture. The typical S-DNN aggregates a variable number of individually learnable modules in series to assemble a DNN-alike alternative to the targeted object recognition tasks. This work likewise devises an S-DNN instantiation, dubbed deep analytic network (DAN), on top of the spectral histogram (SH) features. The DAN learning principle relies on ridge regression, and some key DNN constituents, specifically, rectified linear unit, fine-tuning, and normalization. The DAN aptitude is scrutinized on three repositories of varying domains, including FERET (faces), MNIST (handwritten digits), and CIFAR10 (natural objects). The empirical results unveil that DAN escalates the SH baseline performance over a sufficiently deep layer.

We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within the pooling region. The approach is hyper-parameter free and can be combined with other regularization approaches, such as dropout and data augmentation. We achieve state-of-the-art performance on four image datasets, relative to other approaches that do not utilize data augmentation.

We introduce a hybrid CPU/GPU version of the Asynchronous Advantage ActorCritic (A3C) algorithm, currently the state-of-the-art method in reinforcement learning for various gaming tasks. We analyze its computational traits and concentrate on aspects critical to leveraging the GPU’s computational power. We introduce a system of queues and a dynamic scheduling strategy, potentially helpful for other asynchronous algorithms as well. Our hybrid CPU/GPU version of A3C, based on TensorFlow, achieves a significant speed up compared to a CPU implementation; we make it publicly available to other researchers at https://github.com/NVlabs/GA3C.


Question: SGDR: Stochastic Gradient Descent with Restart Mechanism.

This paper introduces an innovative method, t-SNE (t-distributed Stochastic Neighbor Embedding), designed to visualize high-dimensional data in two or three-dimensional maps. As a refined version of Stochastic Neighbor Embedding (Hinton and Roweis, 2002), t-SNE is more efficient to optimize and yields markedly superior visualizations by mitigating the issue of data points clustering excessively in the center of the map. One of t-SNE's key strengths is its ability to create a single map that effectively reveals structure across multiple scales, a critical feature for high-dimensional data that may exist on various related low-dimensional manifolds, such as images of objects from different classes viewed from diverse angles. In managing the visualization of extensive datasets, t-SNE incorporates random walks on neighborhood graphs, enabling the entire dataset's structure to influence the representation of a subset of the data. We showcase t-SNE's capabilities through a range of datasets, demonstrating its superiority over other non-parametric visualization techniques, including Sammon mapping, Isomap, and Locally Linear Embedding, in almost all instances.

A momentum term is usually included in the simulations of connectionist learning algorithms. Although it is well known that such a term greatly improves the speed of learning, there have been few rigorous studies of its mechanisms. In this paper, I show that in the limit of continuous time, the momentum parameter is analogous to the mass of Newtonian particles that move through a viscous medium in a conservative force field. The behavior of the system near a local minimum is equivalent to a set of coupled and damped harmonic oscillators. The momentum term improves the speed of convergence by bringing some eigen components of the system closer to critical damping. Similar results can be obtained for the discrete time case used in computer simulations. In particular, I derive the bounds for convergence on learning-rate and momentum parameters, and demonstrate that the momentum term can increase the range of learning rate over which the system converges. The optimal condition for convergence is also analyzed.

We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gradient method with constant minibatch converges to a stationary point. To tackle this issue, we develop fast stochastic algorithms that provably converge to a stationary point for constant minibatches. Furthermore, using a variant of these algorithms, we obtain provably faster convergence than batch proximal gradient descent. Our results are based on the recent variance reduction techniques for convex optimization but with a novel analysis for handling nonconvex and nonsmooth functions. We also prove global linear convergence rate for an interesting subclass of nonsmooth nonconvex functions, which subsumes several recent works.

We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a blackbox differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.

Knowledge Distillation (KD) consists of transferring “knowledge” from one machine learning model (the teacher) to another (the student). Commonly, the teacher is a high-capacity model with formidable performance, while the student is more compact. By transferring knowledge, one hopes to benefit from the student’s compactness, without sacrificing too much performance. We study KD from a new perspective: rather than compressing models, we train students parameterized identically to their teachers. Surprisingly, these Born-Again Networks (BANs), outperform their teachers significantly, both on computer vision and language modeling tasks. Our experiments with BANs based on DenseNets demonstrate state-of-the-art performance on the CIFAR-10 (3.5%) and CIFAR-100 (15.5%) datasets, by validation error. Additional experiments explore two distillation objectives: (i) Confidence-Weighted by Teacher Max (CWTM) and (ii) Dark Knowledge with Permuted Predictions (DKPP). Both methods elucidate the essential components of KD, demonstrating the effect of the teacher outputs on both predicted and nonpredicted classes.

Bayesian optimization is a prominent method for optimizing expensive-to-evaluate black-box functions that is widely applied to tuning the hyperparameters of machine learning algorithms. Despite its successes, the prototypical Bayesian optimization approach – using Gaussian process models – does not scale well to either many hyperparameters or many function evaluations. Attacking this lack of scalability and flexibility is thus one of the key challenges of the field. We present a general approach for using flexible parametric models (neural networks) for Bayesian optimization, staying as close to a truly Bayesian treatment as possible. We obtain scalability through stochastic gradient Hamiltonian Monte Carlo, whose robustness we improve via a scale adaptation. Experiments including multi-task Bayesian optimization with 21 tasks, parallel optimization of deep neural networks and deep reinforcement learning show the power and flexibility of this approach.

Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a crucial role in determining the network’s behavior during training. Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy. We also demonstrate an energy-efficient hardware accelerator that implements low-precision fixed-point arithmetic with stochastic rounding.

In many regards, tuning deep-learning networks is still more an art than it is a technique. Choosing the correct hyper-parameters and getting a complex network to learn properly can be daunting to people not well versed in that art. Taking the example of entailment classification, this work will analyze different methods and tools that can be used in order to diagnose learning problems. It will aim at providing an deep understanding of some key hyper-parameters, sketch out their effects and explain how to identify recurrent problems when tuning them.

We propose a new fast algorithm for solving one of the standard formulations of image restoration and reconstruction which consists of an unconstrained optimization problem where the objective includes an l2 data-fidelity term and a nonsmooth regularizer. This formulation allows both wavelet-based (with orthogonal or frame-based representations) regularization or total-variation regularization. Our approach is based on a variable splitting to obtain an equivalent constrained optimization formulation, which is then addressed with an augmented Lagrangian method. The proposed algorithm is an instance of the so-called alternating direction method of multipliers, for which convergence has been proved. Experiments on a set of image restoration and reconstruction benchmark problems show that the proposed algorithm is faster than the current state of the art methods.

Faced with continuously increasing scale of data, original back-propagation neural network based machine learning algorithm presents two non-trivial challenges: huge amount of data makes it difficult to maintain both efficiency and accuracy; redundant data aggravates the system workload. This project is mainly focused on the solution to the issues above, combining deep learning algorithm with cloud computing platform to deal with large-scale data. A MapReduce-based handwriting character recognizer will be designed in this project to verify the efficiency improvement this mechanism will achieve on training and practical large-scale data. Careful discussion and experiment will be developed to illustrate how deep learning algorithm works to train handwritten digits data, how MapReduce is implemented on deep learning neural network, and why this combination accelerates computation. Besides performance, the scalability and robustness will be mentioned in this report as well. Our system comes with two demonstration software that visually illustrates our handwritten digit recognition/encoding application. 1

In this paper we introduce a restart-CMA-evolution strategy, where the population size is increased for each restart (IPOP). By increasing the population size the search characteristic becomes more global after each restart. The IPOP-CMA-ES is evaluated on the test suit of 25 functions designed for the special session on real-parameter optimization of CEC 2005. Its performance is compared to a local restart strategy with constant small population size. On unimodal functions the performance is similar. On multi-modal functions the local restart strategy significantly outperforms IPOP in 4 test cases whereas IPOP performs significantly better in 29 out of 60 tested cases.


Question: Combining text mining, data mining, and network analysis to identify trends in genetic breast cancer research.

This paper explores the potential of Social Network Analysis (SNA) as a potent investigative tool within the realm of information sciences, emphasizing its versatility and applicability across various domains. Unlike a rigid theoretical framework, SNA is a flexible strategy for examining the intricate patterns and dynamics of social structures. Given its broad scope and adaptability, we specifically analyze its significance and impact within the information sciences sector. Information scientists engage in the exploration of publication networks, citation and co-citation patterns, collaborative frameworks, and other social interaction networks. Moreover, the advent of the Internet has introduced a social network of unparalleled magnitude. We assert that SNA holds considerable promise in elucidating the complexities of these networks. Additionally, SNA shares connections with contemporary theories on free market economics, geographical networks, and transportation systems. The paper documents the evolution of SNA and presents a co-author network analysis, highlighting the key players in this field through the calculation of centrality measures. * All correspondence should be directed to the corresponding author.

**Question**: The Role of Machine Learning in Enhancing Cybersecurity

**Documents**: Machine Learning for Cybersecurity: An Overview
The application of machine learning (ML) techniques to cybersecurity is a rapidly growing field, driven by the increasing sophistication of cyber threats and the need for automated defense mechanisms. This paper provides an overview of the current landscape of ML in cybersecurity, exploring various use cases and challenges. ML algorithms can detect anomalies, identify patterns of malicious behavior, and predict potential threats. However, the implementation of ML in cybersecurity is not without its hurdles, including data privacy concerns, the need for large and diverse datasets, and the risk of adversarial attacks on ML models. The paper discusses these challenges and suggests strategies for addressing them. It also highlights the importance of interdisciplinary collaboration between cybersecurity experts and data scientists to develop robust and effective ML-based solutions. * For further information, please contact the lead author.

**Similar Documents**: An Insight into Machine Learning's Impact on Cybersecurity: Opportunities and Challenges
The integration of machine learning (ML) into cybersecurity measures is gaining momentum as a critical response to the escalating complexity of cyber threats. This review delves into the current state of ML applications in cybersecurity, examining diverse scenarios and the associated obstacles. ML methodologies are instrumental in anomaly detection, pattern recognition of nefarious activities, and forecasting emerging threats. Yet, the practical deployment of ML in cybersecurity is fraught with challenges, encompassing issues of data confidentiality, the demand for extensive and varied datasets, and the vulnerability of ML models to adversarial manipulation. The review

In this paper, we present methods for automatically finding abstract, legally relevant concepts in case texts and demonstrate how they can be used to make predictions of case outcomes, given the texts as inputs.In a set of experiments to test these methods, we focus on the open question of how best to represent legal text for finding abstract concepts. We compare different ways of representing legal case texts in order to test whether adding domain knowledge and some linguistic information can improve performance.We found that replacing individual names by roles in the case texts led to better indexing, and that adding certain syntactic and semantic information, in the form of Propositional Patterns that capture a sense of "who did what", led to better prediction. Our experiments also showed that of three learning algorithms, Nearest Neighbor worked best in learning how to identify indexing concepts in texts.In these experiments, we introduced a prototype system that can reason with text cases; it analyzes a case, predicts its outcome considering other cases in the database, and explains the prediction, all starting with a textual description of the case's facts as input.

We have designed and developed a data integration and visualization platform that provides evidence about the association of known and potential drug targets with diseases. The platform is designed to support identification and prioritization of biological targets for follow-up. Each drug target is linked to a disease using integrated genome-wide data from a broad range of data sources. The platform provides either a target-centric workflow to identify diseases that may be associated with a specific target, or a disease-centric workflow to identify targets that may be associated with a specific disease. Users can easily transition between these target- and disease-centric workflows. The Open Targets Validation Platform is accessible at https://www.targetvalidation.org.

Many studies demonstrated that the DNA methylation, which occurs in the context of a CpG, has strong correlation with diseases, including cancer. There is a strong interest in analyzing the DNA methylation data to find how to distinguish different subtypes of the tumor. However, the conventional statistical methods are not suitable for analyzing the highly dimensional DNA methylation data with bounded support. In order to explicitly capture the properties of the data, we design a deep neural network, which composes of several stacked binary restricted Boltzmann machines, to learn the low-dimensional deep features of the DNA methylation data. Experimental results show that these features perform best in breast cancer DNA methylation data cluster analysis, compared with some state-of-the-art methods.

Online social and news media generate rich and timely information about real-world events of all kinds. However, the huge amount of data available, along with the breadth of the user base, requires a substantial effort of information filtering to successfully drill down to relevant topics and events. Trending topic detection is therefore a fundamental building block to monitor and summarize information originating from social sources. There are a wide variety of methods and variables and they greatly affect the quality of results. We compare six topic detection methods on three Twitter datasets related to major events, which differ in their time scale and topic churn rate. We observe how the nature of the event considered, the volume of activity over time, the sampling procedure and the pre-processing of the data all greatly affect the quality of detected topics, which also depends on the type of detection method used. We find that standard natural language processing techniques can perform well for social streams on very focused topics, but novel techniques designed to mine the temporal distribution of concepts are needed to handle more heterogeneous streams containing multiple stories evolving in parallel. One of the novel topic detection methods we propose, based on -grams cooccurrence and topic ranking, consistently achieves the best performance across all these conditions, thus being more reliable than other state-of-the-art techniques.

Breast cancer is one of the diseases that represent a large number of incidence and mortality in the world. Data mining classifications techniques will be effective tools for classifying data of cancer to facilitate decision-making.
 The objective of this paper is to compare the performance of different machine learning algorithms in the diagnosis of breast cancer, to define exactly if this type of cancer is a benign or malignant tumor.
 Six machine learning algorithms were evaluated in this research Bayes Network (BN), Support Vector Machine (SVM), k-nearest neighbors algorithm (Knn), Artificial Neural Network (ANN), Decision Tree (C4.5) and Logistic Regression. The simulation of the algorithms is done using the WEKA tool (The Waikato Environment for Knowledge Analysis) on the Wisconsin breast cancer dataset available in UCI machine learning repository.

An important aspect of the work of researchers as well as librarians is to manage collections of scientific literature. Social research networks, such as Mendeley and CiteULike, provide services that support this task. Meta-data plays an important role in providing services to retrieve and organise the articles. In such settings, meta-data is rarely explicitly provided, leading to the need for automatically extracting this valuable information. The TeamBeam algorithm analyses a scientific article and extracts out structured meta-data, such as the title, journal name and abstract, as well as information about the article’s authors (e.g. names, e-mail addresses, affiliations). The input of the algorithm is a set of blocks generated from the article text. A classification algorithm, which takes the sequence of the input into account, is then applied in two consecutive phases. In the evaluation the performance of the algorithm is compared against two heuristics and three existing meta-data extraction systems. Three different data sets with varying characteristics are used to assess the quality of the extraction results. TeamBeam performs well under testing and compares favourably with existing approaches.

In recent years we have seen the emergence of a variety of scholarly datasets. Typically these capture ‘standard’ scholarly entities and their connections, such as authors, affiliations, venues, publications, citations, and others. However, as the repositories grow and the technology improves, researchers are adding new entities to these repositories to develop a richer model of the scholarly domain. In this paper, we introduce TechMiner, a new approach, which combines NLP, machine learning and semantic technologies, for mining technologies from research publications and generating an OWL ontology describing their relationships with other research entities. The resulting knowledge base can support a number of tasks, such as: richer semantic search, which can exploit the technology dimension to support better retrieval of publications; richer expert search; monitoring the emergence and impact of new technologies, both within and across scientific fields; studying the scholarly dynamics associated with the emergence of new technologies; and others. TechMiner was evaluated on a manually annotated gold standard and the results indicate that it significantly outperforms alternative NLP approaches and that its semantic features improve performance significantly with respect to both recall and precision.

Given a large-scale linked document collection, such as a collection of blog posts or a research literature archive, there are two fundamental problems that have generated a lot of interest in the research community. One is to identify a set of high-level topics covered by the documents in the collection; the other is to uncover and analyze the social network of the authors of the documents. So far these problems have been viewed as separate problems and considered independently from each other. In this paper we argue that these two problems are in fact inter-dependent and should be addressed together. We develop a Bayesian hierarchical approach that performs topic modeling and author community discovery in one unified framework. The effectiveness of our model is demonstrated on two blog data sets in different domains and one research paper citation data from CiteSeer.

It is well known that post-secondary science and engineering programs attract fewer female students. In this paper, we analyze gender differences through text mining of over 30,000 applications to the engineering faculty of a large North American university. We use syntactic and semantic analysis methods to highlight differences in motivation, interests and background. Our analysis leads to three main findings. First, female applicants demonstrate a wider breadth of experience, whereas male applicants put a greater emphasis on technical depth. Second, more female applicants demonstrate a greater desire to serve society. Third, female applicants are more likely to mention personal influences for studying engineering.

Weblogs are now a key part of online culture, and social scientists are interested in characterising the networks formed by bloggers and measuring their extent and impact in areas such as politics. However, researchers wishing to conduct quantitative social science analysis of the blogging phenomenon are faced with the challenge of using new methods of data collection and analysis largely derived from fields outside of the social sciences, such as the information sciences. This paper presents an overview of one new approach for collecting and analysing weblog data, and illustrates this approach in the context of a preliminary quantitative analysis of online networks formed by a sample of North-American “A-list” political bloggers. There are two aims to this paper. First is to assess (using different data and methods) the conclusion of Adamic and Glance (2005) that there are significant differences in the behaviour of liberal and conservative bloggers, with the latter forming more dense patterns of linkages. We find broad support for this conclusion, and empirically assess the implications of differences in conservative/liberal linking behaviour for the online visibility of different political messages or ideologies. The second aim is to highlight the role of web mining and data visualisation in the analysis of weblogs, and the opportunities and challenges inherent in this new field of research.


Question: Combining text mining, data mining, and network analysis to identify trends in genetic breast cancer.

This paper outlines a methodology for classifying the semantic relationships within noun compounds at an intermediate level of specificity. Our goal is to achieve a level of detail more nuanced than that provided by case frames, yet broader than the relations typically captured by traditional knowledge representation systems. To this end, we present a classification algorithm designed to pinpoint the relationships between elements in two-word noun compounds. Our approach, which combines a machine learning algorithm with a domain-specific lexical hierarchy, proves effective in generalizing from training data. We demonstrate that this method outperforms a baseline strategy that relies solely on training with the words themselves, by achieving higher accuracy when applied to previously unseen noun compounds. This advancement in identifying semantic relations within noun compounds has implications for enhancing the precision of natural language processing tasks, particularly in specialized domains where the complexity of language requires a deeper understanding of the relationships between terms.

Mining cluster evolution from multiple correlated time-varying text corpora is important in exploratory text analytics. In this paper, we propose an approach called evolutionary hierarchical Dirichlet processes (EvoHDP) to discover interesting cluster evolution patterns from such text data. We formulate the EvoHDP as a series of hierarchical Dirichlet processes~(HDP) by adding time dependencies to the adjacent epochs, and propose a cascaded Gibbs sampling scheme to infer the model. This approach can discover different evolving patterns of clusters, including emergence, disappearance, evolution within a corpus and across different corpora. Experiments over synthetic and real-world multiple correlated time-varying data sets illustrate the effectiveness of EvoHDP on discovering cluster evolution patterns.

Data Mining is taking out of hidden patterns from huge database. It is commonly used in a marketing, surveillance, fraud detection and scientific discovery. In data mining, machine learning is mainly focused as research which is automatically learnt to recognize complex patterns and make intelligent decisions based on data. Nowadays traffic accidents are the major causes of death and injuries in this world. Roadway patterns are useful in the development of

The emergence of online social networks in the past few years has generated an enormous amount of information about potentially any subject. Valuable data containing users' opinions and thoughts are available on those repositories and several sentiment analysis techniques have been proposed that address the problem of understanding the opinion orientation of the users' postings. In this paper, we take a different perspective to the problem through a user centric approach, which uses a graph to model users (and their postings) and applies link mining techniques to infer opinions of users. Preliminary experiments on a Twitter corpus have shown promising results.

This paper presents a new method for extracting unique features of items based on their textual reviews. The method is built of two similar iterations of applying a weighting scheme and then clustering the resultant set of vectors. In the first iteration, restaurants of similar food genres are grouped together into clusters. The second iteration reduces the importance of common terms in each such cluster, and highlights those that are unique to each specific restaurant. Clustering the restaurants again, now according to their unique features, reveals very interesting connections between the restaurants.

Applications of clustering algorithms in biomedical research are ubiquitous, with typical examples including gene expression data analysis, genomic sequence analysis, biomedical document mining, and MRI image analysis. However, due to the diversity of cluster analysis, the differing terminologies, goals, and assumptions underlying different clustering algorithms can be daunting. Thus, determining the right match between clustering algorithms and biomedical applications has become particularly important. This paper is presented to provide biomedical researchers with an overview of the status quo of clustering algorithms, to illustrate examples of biomedical applications based on cluster analysis, and to help biomedical researchers select the most suitable clustering algorithms for their own applications.

This article reports on a detailed investigation of PubMed users' needs and behavior as a step toward improving biomedical information retrieval. PubMed is providing free service to researchers with access to more than 19 million citations for biomedical articles from MEDLINE and life science journals. It is accessed by millions of users each day. Efficient search tools are crucial for biomedical researchers to keep abreast of the biomedical literature relating to their own research. This study provides insight into PubMed users' needs and their behavior. This investigation was conducted through the analysis of one month of log data, consisting of more than 23 million user sessions and more than 58 million user queries. Multiple aspects of users' interactions with PubMed are characterized in detail with evidence from these logs. Despite having many features in common with general Web searches, biomedical information searches have unique characteristics that are made evident in this study. PubMed users are more persistent in seeking information and they reformulate queries often. The three most frequent types of search are search by author name, search by gene/protein, and search by disease. Use of abbreviation in queries is very frequent. Factors such as result set size influence users' decisions. Analysis of characteristics such as these plays a critical role in identifying users' information needs and their search habits. In turn, such an analysis also provides useful insight for improving biomedical information retrieval.Database URL:http://www.ncbi.nlm.nih.gov/PubMed.

We propose an approach for extracting relations between entities from biomedical literature based solely on shallow linguistic information. We use a combination of kernel functions to integrate two different information sources: (i) the whole sentence where the relation appears, and (ii) the local contexts around the interacting entities. We performed experiments on extracting gene and protein interactions from two different data sets. The results show that our approach outperforms most of the previous methods based on syntactic and semantic information.

Biomedical Natural Language Processing (BioNLP) attempts to capture biomedical phenomena from texts by extracting relations between biomedical entities (i.e. proteins and genes). Traditionally, only binary relations have been extracted from large numbers of published papers. Recently, more complex relations (biomolecular events) have also been extracted. Such events may include several entities or other relations. To evaluate the performance of the text mining systems, several shared task challenges have been arranged for the BioNLP community. With a common and consistent task setting, the BioNLP'09 shared task evaluated complex biomolecular events such as binding and regulation.Finding these events automatically is important in order to improve biomedical event extraction systems. In the present paper, we propose an automatic event extraction system, which contains a model for complex events, by solving a classification problem with rich features. The main contributions of the present paper are: (1) the proposal of an effective bio-event detection method using machine learning, (2) provision of a high-performance event extraction system, and (3) the execution of a quantitative error analysis. The proposed complex (binding and regulation) event detector outperforms the best system from the BioNLP'09 shared task challenge.

Complex biological systems have been successfully modeled by biochemical and genetic interaction networks, typically gathered from high-throughput (HTP) data. These networks can be used to infer functional relationships between genes or proteins. Using the intuition that the topological role of a gene in a network relates to its biological function, local or diffusionbased “guilt-by-association” and graph-theoretic methods have had success in inferring gene functions. Here we seek to improve function prediction by integrating diffusion-based methods with a novel dimensionality reduction technique to overcome the incomplete and noisy nature of network data. In this paper, we introduce diffusion component analysis (DCA), a framework that plugs in a diffusion model and learns a low-dimensional vector representation of each node to encode the topological properties of a network. As a proof of concept, we demonstrate DCA’s substantial improvement over state-of-the-art diffusion-based approaches in predicting protein function from molecular interaction networks. Moreover, our DCA framework can integrate multiple networks from heterogeneous sources, consisting of genomic information, biochemical experiments and other resources, to even further improve function prediction. Yet another layer of performance gain is achieved by integrating the DCA framework with support vector machines that take our node vector representations as features. Overall, our DCA framework provides a novel representation of nodes in a network that can be used as a plug-in architecture to other machine learning algorithms to decipher topological properties of and obtain novel insights into interactomes. This paper was selected for oral presentation at RECOMB 2015 and an abstract is published in the conference proceedings. ar X iv :1 50 4. 02 71 9v 1 [ qbi o. M N ] 1 0 A pr 2 01 5

In this paper, we present overview of our approach for clustering tweets. Due to short text of tweets, traditional text clustering mechanisms alone may not produce optimal results. We believe that there is an underlying theme/topic present in majority of tweets which is evident in growing usage of hashtag feature in the Twitter network. Clustering tweets based on these themes seems a more natural way for grouping. We propose to use Wikipedia topic taxonomy to discover the themes from the tweets and use the themes along with traditional word based similarity metric for clustering. We show some of our initial results to demonstrate the effectiveness of our approach.


Question: Combining text mining, data mining, and network analysis to identify trends in genetic breast cancer.

This paper delves into the methodology and significance of Social Network Analysis (SNA) within the realm of online communities. SNA, a powerful framework for examining the structure and dynamics of relationships, focuses on the interplay between individuals or entities. Central to this approach is the recognition of the units of analysis—nodes—and the connections—ties—between them. The concept of multiplexity, the multiple roles and types of ties that exist within a network, is a key aspect of SNA. Composition analysis, which explores the attributes of nodes, and the study of relations beyond the immediate ties, further enriches the understanding of social networks. Two perspectives are adopted: ego-centered networks, focusing on individual nodes and their immediate connections, and whole networks, which provide a comprehensive view of the network as a whole. Network characteristics, including density, centrality, and clustering, are crucial for understanding the structure and function of networks. The paper also discusses the partitioning of networks into groups and the analysis of networks of networks. In the context of computer-mediated communication (CMC), SNA offers a unique lens through which to study online social networks. The integration of SNA with CMC allows for a deeper understanding of how online interactions are structured and how they influence various aspects of social life. Collecting network data for CMC studies requires careful consideration of sampling methods and data collection techniques. The analysis of network data in CMC involves both ego-centered and whole network approaches, offering insights into the patterns of interaction and the emergent properties of the network. The paper concludes by highlighting the potential of SNA in CMC studies and the need for further research in this area. A list of references and information about the authors is provided for further exploration. However, the original link to the article by Laura Garton is no longer accessible, necessitating a search for alternative sources or updated versions of the material. The integration of SNA with CMC studies holds significant promise for advancing our understanding of online social dynamics and their implications for various fields, including genetics and health, as demonstrated in the context of identifying trends in genetic breast cancer. By applying SNA to the analysis of online discussions and data related to genetic breast cancer, researchers can uncover patterns, networks of support, and information diffusion mechanisms that are crucial for disease management and public health interventions. This interdisciplinary approach, combining text mining, data mining, and network analysis, opens up new avenues for identifying and understanding the multifaceted trends in genetic breast cancer.

Researchers and scientists increasingly find themselves in the position of having to quickly understand large amounts of technical material. Our goal is to effectively serve this need by using bibliometric text mining and summarization techniques to generate summaries of scientific literature. We show how we can use citations to produce automatically generated, readily consumable, technical extractive summaries. We first propose C-LexRank, a model for summarizing single scientific articles based on citations, which employs community detection and extracts salient information-rich sentences. Next, we further extend our experiments to summarize a set of papers, which cover the same scientific topic. We generate extractive summaries of a set of Question Answering (QA) and Dependency Parsing (DP) papers, their abstracts, and their citation sentences and show that citations have unique information amenable to creating a summary.

Network data is ubiquitous; e-mail traffic between persons, telecommunication, transport and financial networks are some examples. Often these networks are large and multivariate, besides the topological structure of the network, multivariate data on the nodes and links is available. Currently, exploration and analysis methods are focused on a single aspect; the network topology or the multivariate data. In addition, tools and techniques are highly domain specific and require expert knowledge. We focus on the non-expert user and propose a novel solution for multivariate network exploration and analysis that tightly couples structural and multivariate analysis. In short, we go from Detail to Overview via Selections and Aggregations (DOSA): users are enabled to gain insights through the creation of selections of interest (manually or automatically), and producing high-level, infographic-style overviews simultaneously. Finally, we present example explorations on real-world datasets that demonstrate the effectiveness of our method for the exploration and understanding of multivariate networks where presentation of findings comes for free.

Data mining technique has attracted attention in the information industry and society as a whole, because of the big amount of data and the imminent need to transform that data into useful information and knowledge. Recently conducted studies with successfully demarcated results using this technique, to estimate several parameters in a variety of domains. However, the effective use of data in some areas is still developing, as is the case of sports, which has shown moderate growth. In this context, the objective of this article is to present a systematic review of the literature about research involving sports data mining. As systematic searches were made out in five databases, resulting in 21 articles that answered a question that grounded this article.

Diabetes is a major health problem in the United States. There is a long history of diabetic registries and databases with systematically collected patient information. We examine one such diabetic data warehouse, showing a method of applying data mining techniques, and some of the data issues, analysis problems, and results. The diabetic data warehouse is from a large integrated health care system in the New Orleans area with 30,383 diabetic patients. Methods for translating a complex relational database with time series and sequencing information to a flat file suitable for data mining are challenging. We discuss two variables in detail, a comorbidity index and the HgbA1c, a measure of glycemic control related to outcomes. We used the classification tree approach in Classification and Regression Trees (CART) with a binary target variable of HgbA1c >9.5 and 10 predictors: age, sex, emergency department visits, office visits, comorbidity index, dyslipidemia, hypertension, cardiovascular disease, retinopathy, end-stage renal disease. Unexpectedly, the most important variable associated with bad glycemic control is younger age, not the comorbiditity index or whether patients have related diseases. If we want to target diabetics with bad HgbA1c values, the odds of finding them is 3.2 times as high in those <65 years of age than those older. Data mining can discover novel associations that are useful to clinicians and administrators [corrected].

Opinion miningrefers to the use of natural language processing, text analysis and computational linguistics to identify and extract the subjective information. Opinion Mining has become an indispensible part of online reviews which is in the present scenario. In the field of information retrieval, a various kinds of probabilistic topic modeling techniques have been used to analyze contents present in a document. A topic model is a generative technique for document. All

In this paper, we present overview of our approach for clustering tweets. Due to short text of tweets, traditional text clustering mechanisms alone may not produce optimal results. We believe that there is an underlying theme/topic present in majority of tweets which is evident in growing usage of hashtag feature in the Twitter network. Clustering tweets based on these themes seems a more natural way for grouping. We propose to use Wikipedia topic taxonomy to discover the themes from the tweets and use the themes along with traditional word based similarity metric for clustering. We show some of our initial results to demonstrate the effectiveness of our approach.

This paper presents a new method for extracting unique features of items based on their textual reviews. The method is built of two similar iterations of applying a weighting scheme and then clustering the resultant set of vectors. In the first iteration, restaurants of similar food genres are grouped together into clusters. The second iteration reduces the importance of common terms in each such cluster, and highlights those that are unique to each specific restaurant. Clustering the restaurants again, now according to their unique features, reveals very interesting connections between the restaurants.

Paper presents a survey of methods and approaches for keyword extraction task. In addition to the systematization of methods, the paper gathers a comprehensive review of existing research. Related work on keyword extraction is elaborated for supervised and unsupervised methods, with special emphasis on graphbased methods as well as Croatian keyword extraction. Selectivity-based keyword extraction method is proposed as a new unsupervised graph-based keyword extraction method which extracts nodes from a complex network as keyword candidates. The paper provides guidelines for future research and development of new graph-based approaches for keyword extraction.

We describe here the use of nonnegative matrix factorization (NMF), an algorithm based on decomposition by parts that can reduce the dimension of expression data from thousands of genes to a handful of metagenes. Coupled with a model selection mechanism, adapted to work for any stochastic clustering algorithm, NMF is an efficient method for identification of distinct molecular patterns and provides a powerful method for class discovery. We demonstrate the ability of NMF to recover meaningful biological information from cancer-related microarray data. NMF appears to have advantages over other methods such as hierarchical clustering or self-organizing maps. We found it less sensitive to a priori selection of genes or initial conditions and able to detect alternative or context-dependent patterns of gene expression in complex biological systems. This ability, similar to semantic polysemy in text, provides a general method for robust molecular pattern discovery.

The Internet has become a rich and large repository of information about us as individuals. Anything from the links and text on a user's homepage to the mailing lists the user subscribes to are reflections of social interactions a user has in the real world. In this paper we devise techniques to mine this information in order to predict relationships between individuals. Further we show that some pieces of information are better indicators of social connections than others, and that these indicators vary between user populations and provide a glimpse into the social lives of individuals in different communities. Our techniques provide potential applications in automatically inferring real-world connections and discovering, labeling, and characterizing communities.


Question: Combining text mining, data mining, and network analysis to identify trends in genetic breast cancer research.

Introducing the Multi-Asymmetric Partial Least Squares Classifier (MAPLSC): An Innovative Multi-Class Classifier for Medical Diagnosis
The integration of clinical records into the realm of Traditional Chinese Medicine (TCM) serves as a valuable tool for expanding medical knowledge and enhancing diagnostic techniques. The presence of multiple diagnostic classes (syndromes) within these records poses a significant challenge for data mining: the multi-value classification problem. In response to this, we present the Multi-Asymmetric Partial Least Squares Classifier (MAPLSC), a novel classifier designed to address the complexities of multi-value classification, particularly when dealing with imbalanced data distributions. To validate the effectiveness of MAPLSC, we conducted extensive comparisons against seven leading methods in the field. These evaluations were performed using not only two TCM clinical datasets but also four publicly available microarray datasets. The results unequivocally demonstrate MAPLSC's superior performance and its potential to significantly advance medical diagnosis in TCM and beyond.

-The huge amounts of data generated by healthcare transactions are too complex and voluminous to be processed and analyzed by traditional methods. Data mining provides the methodology and technology to transform these mounds of data into useful information for decision making. The Healthcare industry is generally “information rich”, which is not feasible to handle manually. These large amounts of data are very important in the field of data mining to extract useful information and generate relationships amongst the attributes. Kidney disease is a complex task which requires much experience and knowledge. Kidney disease is a silent killer in developed countries and one of the main contributors to disease burden in developing countries. In the health care industry the data mining is mainly used for predicting the diseases from the datasets. The Data mining classification techniques, namely Decision trees, ANN, Naive Bayes are analyzed on Kidney disease data set. Keywords--Data Mining, Kidney Disease, Decision tree, Naive Bayes, ANN, K-NN, SVM, Rough Set, Logistic Regression, Genetic Algorithms (GAs) / Evolutionary Programming (EP), Clustering

Many applications such as news group filtering, text crawling, and document organization require real time clustering and segmentation of text data records. The categorical data stream clustering problem also has a number of applications to the problems of customer segmentation and real time trend analysis. We will present an online approach for clustering massive text and categorical data streams with the use of a statistical summarization methodology. We present results illustrating the effectiveness of the technique.

BACKGROUND
The management of metastatic breast cancer requires monitoring of the tumor burden to determine the response to treatment, and improved biomarkers are needed. Biomarkers such as cancer antigen 15-3 (CA 15-3) and circulating tumor cells have been widely studied. However, circulating cell-free DNA carrying tumor-specific alterations (circulating tumor DNA) has not been extensively investigated or compared with other circulating biomarkers in breast cancer.

METHODS
We compared the radiographic imaging of tumors with the assay of circulating tumor DNA, CA 15-3, and circulating tumor cells in 30 women with metastatic breast cancer who were receiving systemic therapy. We used targeted or whole-genome sequencing to identify somatic genomic alterations and designed personalized assays to quantify circulating tumor DNA in serially collected plasma specimens. CA 15-3 levels and numbers of circulating tumor cells were measured at identical time points.

RESULTS
Circulating tumor DNA was successfully detected in 29 of the 30 women (97%) in whom somatic genomic alterations were identified; CA 15-3 and circulating tumor cells were detected in 21 of 27 women (78%) and 26 of 30 women (87%), respectively. Circulating tumor DNA levels showed a greater dynamic range, and greater correlation with changes in tumor burden, than did CA 15-3 or circulating tumor cells. Among the measures tested, circulating tumor DNA provided the earliest measure of treatment response in 10 of 19 women (53%).

CONCLUSIONS
This proof-of-concept analysis showed that circulating tumor DNA is an informative, inherently specific, and highly sensitive biomarker of metastatic breast cancer. (Funded by Cancer Research UK and others.).

The ACL Anthology Network (AAN)1 is a comprehensive manually curated networked database of citations and collaborations in the field of Computational Linguistics. Each citation edge in AAN is associated with one or more citing sentences. A citing sentence is one that appears in a scientific article and contains an explicit reference to another article. In this paper, we shed the light on the usefulness of AAN citing sentences for understanding research trends and summarizing previous discoveries and contributions. We also propose and motivate several different uses and applications of citing sentences.

In this paper, we define and study a novel text mining problem, which we refer to as Comparative Text Mining (CTM). Given a set of comparable text collections, the task of comparative text mining is to discover any latent common themes across all collections as well as summarize the similarity and differences of these collections along each common theme. This general problem subsumes many interesting applications, including business intelligence and opinion summarization. We propose a generative probabilistic mixture model for comparative text mining. The model simultaneously performs cross-collection clustering and within-collection clustering, and can be applied to an arbitrary set of comparable text collections. The model can be estimated efficiently using the Expectation-Maximization (EM) algorithm. We evaluate the model on two different text data sets (i.e., a news article data set and a laptop review data set), and compare it with a baseline clustering method also based on a mixture model. Experiment results show that the model is quite effective in discovering the latent common themes across collections and performs significantly better than our baseline mixture model.

We present an evolutionary multi-branch tree clustering method to model hierarchical topics and their evolutionary patterns overtime. The method builds evolutionary trees in a Bayesian online filtering framework. The tree construction is formulated as an online posterior estimation problem, which well balances both the fitness of the current tree and the smoothness between trees. The state-of-the-art multi-branch clustering method, Bayesian rose trees, is employed to generate a topic tree with a high fitness value. A constraint model is also introduced to preserve the smoothness between trees. A set of comprehensive experiments on real world news data demonstrates that the proposed method better incorporates historical tree information and is more efficient and effective than the traditional evolutionary hierarchical clustering algorithm. In contrast to our previous method [31], we implement two additional baseline algorithms to compare them with our algorithm. We also evaluate the performance of the clustering algorithm based on multiple constraint trees. Furthermore, two case studies are conducted to demonstrate the effectiveness and usefulness of our algorithm in helping users understand the major hierarchical topic evolutionary patterns in text data.

Many of the text mining applications contain a huge amount of information from document in the form of text. This text can be very helpful for Text Clustering. This text also includes various kind of other information known as Side Information or Metadata. Examples of this side information include links to other web pages, title of the document, author name or date of Publication which are present in the text document. Such metadata may possess a lot of information for the clustering purposes. But this Side information may be sometimes noisy. Using such Side Information for producing clusters without filtering it, can result to bad quality of Clusters. So we use an efficient Feature Selection method to perform the mining process to select that Side Information which is useful for Clustering so as to maximize the advantages from using it. The proposed technique, CCSI (Co-Clustering with Side Information) system makes use of the process of Co-Clustering or Two-mode clustering which is a data mining technique that allows concurrently clustering of the rows and columns of a matrix.

Most objects and data in the real world are of multiple types, interconnected, forming complex, heterogeneous but often semi-structured information networks. However, most network science researchers are focused on homogeneous networks, without distinguishing different types of objects and links in the networks. We view interconnected, multityped data, including the typical relational database data, as heterogeneous information networks, study how to leverage the rich semantic meaning of structural types of objects and links in the networks, and develop a structural analysis approach on mining semi-structured, multi-typed heterogeneous information networks. In this article, we summarize a set of methodologies that can effectively and efficiently mine useful knowledge from such information networks, and point out some promising research directions.

Weblogs are now a key part of online culture, and social scientists are interested in characterising the networks formed by bloggers and measuring their extent and impact in areas such as politics. However, researchers wishing to conduct quantitative social science analysis of the blogging phenomenon are faced with the challenge of using new methods of data collection and analysis largely derived from fields outside of the social sciences, such as the information sciences. This paper presents an overview of one new approach for collecting and analysing weblog data, and illustrates this approach in the context of a preliminary quantitative analysis of online networks formed by a sample of North-American “A-list” political bloggers. There are two aims to this paper. First is to assess (using different data and methods) the conclusion of Adamic and Glance (2005) that there are significant differences in the behaviour of liberal and conservative bloggers, with the latter forming more dense patterns of linkages. We find broad support for this conclusion, and empirically assess the implications of differences in conservative/liberal linking behaviour for the online visibility of different political messages or ideologies. The second aim is to highlight the role of web mining and data visualisation in the analysis of weblogs, and the opportunities and challenges inherent in this new field of research.

Keywords in the ACM Digital Library and IEEE Xplore digital library and in NSF grants anticipate future CS research.


Question: DeepFood: A deep learning-based method for recognizing food images in computer-aided dietary assessment.

This paper presents a groundbreaking deep convolutional neural network architecture, referred to as Inception, which has set new benchmarks for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The Inception architecture's standout feature is its optimized utilization of computational resources within the network. Through a meticulously designed structure, we expanded the network's depth and breadth while maintaining a constant computational budget. Quality enhancements were guided by the Hebbian principle and the concept of multi-scale processing. A specific version of this architecture, known as GoogLeNet, is a 22-layer deep network. The quality of GoogLeNet was rigorously evaluated in the context of both classification and detection tasks, showcasing its superior performance.

In light of the aforementioned, the following document is proposed as a similar document for the question regarding "DeepFood: Deep Learning-Based Food Image Recognition for Computer-Aided Dietary Assessment":

In the realm of computer-aided dietary assessment, a novel deep learning model, coined DeepFood, has been developed to recognize food images with unprecedented accuracy. Inspired by the advancements in deep convolutional neural networks, particularly the Inception architecture, DeepFood utilizes a multi-scale processing strategy to enhance the recognition of food items in images. By adapting the principles of the Hebbian theory and incorporating a sophisticated design akin to that of GoogLeNet, DeepFood has achieved significant improvements in food image classification. This model not only identifies a wide variety of food types but also estimates portion sizes accurately, making it a valuable tool for dietary monitoring and analysis. DeepFood's performance was assessed against a large dataset of food images, demonstrating its capability to support healthier eating habits through precise dietary assessment.

In this paper we propose a deep learning solution to age estimation from a single face image without the use of facial landmarks and introduce the IMDB-WIKI dataset, the largest public dataset of face images with age and gender labels. If the real age estimation research spans over decades, the study of apparent age estimation or the age as perceived by other humans from a face image is a recent endeavor. We tackle both tasks with our convolutional neural networks (CNNs) of VGG-16 architecture which are pre-trained on ImageNet for image classification. We pose the age estimation problem as a deep classification problem followed by a softmax expected value refinement. The key factors of our solution are: deep learned models from large data, robust face alignment, and expected value formulation for age regression. We validate our methods on standard benchmarks and achieve state-of-the-art results for both real and apparent age estimation.

Over the past decade, functional Magnetic Resonance Imaging (fMRI) has emerged as a powerful new instrument to collect vast quantities of data about activity in the human brain. A typical fMRI experiment can produce a three-dimensional image related to the human subject's brain activity every half second, at a spatial resolution of a few millimeters. As in other modern empirical sciences, this new instrumentation has led to a flood of new data, and a corresponding need for new data analysis methods. We describe recent research applying machine learning methods to the problem of classifying the cognitive state of a human subject based on fRMI data observed over a single time interval. In particular, we present case studies in which we have successfully trained classifiers to distinguish cognitive states such as (1) whether the human subject is looking at a picture or a sentence, (2) whether the subject is reading an ambiguous or non-ambiguous sentence, and (3) whether the word the subject is viewing is a word describing food, people, buildings, etc. This learning problem provides an interesting case study of classifier learning from extremely high dimensional (105 features), extremely sparse (tens of training examples), noisy data. This paper summarizes the results obtained in these three case studies, as well as lessons learned about how to successfully apply machine learning methods to train classifiers in such settings.

Object Classification is an important task within the field of computer vision. Image classification refers to the labelling of images into one of a number of predefined categories. Classification includes image sensors, image pre-processing, object detection, object segmentation, feature extraction and object classification.Many classification techniques have been developed for image classification. In this survey various classification techniques are considered; Artificial Neural Network(ANN), Decision Tree(DT), Support Vector Machine(SVM) and Fuzzy Classification

This paper presents and evaluates a deep learning architecture for automated basal cell carcinoma cancer detection that integrates (1) image representation learning, (2) image classification and (3) result interpretability. A novel characteristic of this approach is that it extends the deep learning architecture to also include an interpretable layer that highlights the visual patterns that contribute to discriminate between cancerous and normal tissues patterns, working akin to a digital staining which spotlights image regions important for diagnostic decisions. Experimental evaluation was performed on set of 1,417 images from 308 regions of interest of skin histopathology slides, where the presence of absence of basal cell carcinoma needs to be determined. Different image representation strategies, including bag of features (BOF), canonical (discrete cosine transform (DCT) and Haar-based wavelet transform (Haar)) and proposed learned-from-data representations, were evaluated for comparison. Experimental results show that the representation learned from a large histology image data set has the best overall performance (89.4% in F-measure and 91.4% in balanced accuracy), which represents an improvement of around 7% over canonical representations and 3% over the best equivalent BOF representation.

In this paper, the automatic medical annotation task of the 2007 CLEF cross-language image retrieval campaign (ImageCLEF) is described. The paper focusses on the images used, the task setup, and the results obtained in the evaluation campaign. Since 2005, the medical automatic image annotation task exists in ImageCLEF with increasing complexity to evaluate the performance of state-of-the-art methods for completely automatic annotation of medical images based on visual properties. The paper also describes the evolution of the task from its origin in 2005 to 2007. The 2007 task, comprising 11,000 fully annotated training images and 1,000 test images to be annotated, is a realistic task with a large number of possible classes at different levels of detail. Detailed analysis of the methods across participating groups is presented with respect to the (i) image representation, (ii) classification method, and (iii) use of the class hierarchy. The results show that methods which build on local image descriptors and discriminative models are able to provide good predictions of the image classes, mostly by using techniques that were originally developed in the machine learning and computer vision domain for object recognition in non-medical images.

This paper extends research on automated image captioning in the dimension of language, studying how to generate Chinese sentence descriptions for unlabeled images. To evaluate image captioning in this novel context, we present Flickr8k-CN, a bilingual extension of the popular Flickr8k set. The new multimedia dataset can be used to quantitatively assess the performance of Chinese captioning and English-Chinese machine translation. The possibility of re-using existing English data and models via machine translation is investigated. Our study reveals to some extent that a computer can master two distinct languages, English and Chinese, at a similar level for describing the visual world. Data is publicly available at http://tinyurl.com/flickr8kcn

With the advent of the Internet, billions of images are now freely available online and constitute a dense sampling of the visual world. Using a variety of non-parametric methods, we explore this world with the aid of a large dataset of 79,302,017 images collected from the Internet. Motivated by psychophysical results showing the remarkable tolerance of the human visual system to degradations in image resolution, the images in the dataset are stored as 32 x 32 color images. Each image is loosely labeled with one of the 75,062 non-abstract nouns in English, as listed in the Wordnet lexical database. Hence the image database gives a comprehensive coverage of all object categories and scenes. The semantic information from Wordnet can be used in conjunction with nearest-neighbor methods to perform object classification over a range of semantic levels minimizing the effects of labeling noise. For certain classes that are particularly prevalent in the dataset, such as people, we are able to demonstrate a recognition performance comparable to class-specific Viola-Jones style detectors.

Deep learning has emerged as a hot topic due to extensive application and high accuracy. In this paper this efficient method is used for vehicle detection and classification. We extract visual features from the activation of a deep convolutional network, large-scale sparse learning and other distinguishing features in order to compare their accuracy. When compared to the leading methods in the challenging ImageNet dataset, our deep learning approach obtains highly competitive results. Through the experiments with in short of training data, the features extracted by deep learning method outperform those generated by traditional approaches.

Attribute-based identification systems are essential for forensic investigations because they help in identifying individuals. An item such as clothing is a visual attribute because it can usually be used to describe people. The method proposed in this article aims to identify people based on the visual information derived from their attire. Deep learning is used to train the computer to classify images based on clothing content. We first demonstrate clothing classification using a large scale dataset, where the proposed model performs relatively poorly. Then, we use clothing classification on a dataset containing popular logos and famous brand images. The results show that the model correctly classifies most of the test images with a success rate that is higher than 70%. Finally, we evaluate clothing classification using footage from surveillance cameras. The system performs well on this dataset, labelling 70% of the test images correctly.

The continually increasing number of complex datasets each year necessitates ever improving machine learning methods for robust and accurate categorization of these data. This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble, deep learning approach for classification. Deep learning models have achieved state-of-the-art results across many domains. RMDL solves the problem of finding the best deep learning structure and architecture while simultaneously improving robustness and accuracy through ensembles of deep learning architectures. RDML can accept as input a variety data to include text, video, images, and symbolic. This paper describes RMDL and shows test results for image and text data including MNIST, CIFAR-10, WOS, Reuters, IMDB, and 20newsgroup. These test results show that RDML produces consistently better performance than standard methods over a broad range of data types and classification problems.