Question: DeepLogic: An End-to-End System for Logical Reasoning Tasks

This paper advocates for the development of a comprehensive suite of toy tasks aimed at evaluating a machine learning system's capacity for reading comprehension and logical reasoning through question answering. Inspired by the broader goal of creating AI agents capable of engaging in intelligent dialogue, these tasks are specifically designed to test various aspects of understanding, such as the ability to chain facts, perform simple induction and deduction, and more. Each task is intended as a building block, essential for any system aspiring to converse meaningfully with humans. The authors posit that many contemporary learning systems are likely to struggle with these tasks, thus highlighting the need for researchers to pinpoint and address the specific deficiencies in their models. In an effort to advance the field, the paper also presents an enhanced version of the Memory Networks model, demonstrating its effectiveness in solving a subset of the proposed tasks while acknowledging its limitations in tackling others. This work serves as a call to action for the AI community to refine and expand their models' reasoning abilities, using the outlined tasks as benchmarks for progress.

In this paper, we extend the state-of-the-art NLIDB system and present a dialog interface to relational databases. Dialog interface enables users to automatically exploit the semantic context of the conversation while asking natural language queries over RDBMS, thereby making it simpler to express complex questions in a natural, piece-wise manner. We propose novel ontology-driven techniques for addressing each of the dialog-specific challenges such as co-reference resolution, ellipsis resolution, and query disambiguation, and use them in determining the overall intent of the user query. We demonstrate the applicability and usefulness of dialog interface over two different domains viz. finance and healthcare.

In this thesis, we present a novel approach for modeling textual entailment using lexicalsemantic information on the level of predicate-argument structure. To this end, we adopt information provided by the Berkeley FrameNet repository and embed it into an implemented end-to-end system. The two main goals of this thesis are the following: (i) to provide an analysis of the potential contribution of frame semantic information to the recognition textual entailment and (ii) to present a robust system architecture that can serve as basis for future experiments, research, and improvement. Our work was carried out in the context of the textual entailment initiative, which since 2005 has set the stage for the broad investigation of inference in natural-language processing tasks, including empirical evaluation of its coverage and reliability. In short, textual entailment describes inferential relations between (entailing) texts and (entailed) hypotheses as interpreted by typical language users. This pre-theoretic notion captures a natural range of inferences as compared to logical entailment, which has traditionally been used within theoretical approaches to natural language semantics. Various methods for modeling textual entailment have been proposed in the literature, ranging from shallow techniques like lexical overlap to shallow syntactic parsing and the exploitation of WordNet relations. Recently, there has been a move towards more structured meaning representations. In particular, the level of predicate-argument structure has gained much attention, which seems to be a natural and straightforward choice. Predicate-argument structure allows annotating sentences or texts with nuclear meaning representations (“who did what to whom”), which are of obvious relevance for this task. For example, it can account for paraphrases like “Ghosts scare John” vs. “John is scared by ghosts”. In this thesis, we present an approach to textual entailment that is centered around the analysis of predicate-argument structure. It combines LFG grammatical analysis, predicate-argument structure in the FrameNet paradigm, and taxonomic information from WordNet into tripartite graph structures. By way of a declarative graph matching algorithm, the “structural and semantic” similarity of hypotheses and texts is computed and the result is represented as feature vectors. A supervised machine learning architecture trained on entailment corpora is used to check textual entailment for new text/hypothesis pairs. The approach is implemented in the SALSA RTE system, which successfully participated in the second and third RTE challenge. While system performance is on a par with that of comparable systems, the intuitively expected strong positive effect of using FrameNet information has not yet been confirmed. In order to evaluate different system components and to assess the potential contribution of FrameNet information for checking textual entailment, we conducted a number of experiments. For example, with the help of a gold-standard corpus, we

Identifying incorrect content (i.e., semantic error) in text is a difficult task because of the ambiguous nature of written natural language and themany factors that canmake a statement semantically erroneous. Current methods identify semantic errors in a sentence by determining whether it contradicts the domain to which the sentence belongs. However, because thesemethods are constructedon expected logic contradictions, they cannot handle new or unexpected semantic errors. In this paper, we propose a new method for detecting semantic errors that is based on logic reasoning. Our proposed method converts text into logic clauses, which are later analyzed against a domain ontology by an automatic reasoner to determine its consistency. This approach can provide a complete analysis of the text, since it can analyze a single sentence or sets of multiple sentences. When there are multiple sentences to analyze, in order to avoid the high complexity of reasoning over a large set of logic clauses, we propose rules that reduce the set of sentences to analyze, based on the logic relationships between sentences. In our evaluation, we have found that our proposed method can identify a significant percentage of semantic errors and, in the case of multiple sentences, it does so without significant computational cost. We have also found that both the quality of the information extraction output and modeling elements B Dejing Dou dou@cs.uoregon.edu Fernando Gutierrez fernando@cs.uoregon.edu Nisansa de Silva nisansa@cs.uoregon.edu Stephen Fickas fickas@cs.uoregon.edu 1 Department of Computer and Information Science, 1202 University of Oregon, Eugene, OR 97403, USA of the ontology (i.e., property domain and range) affect the capability of detecting errors.

The gap between unstructured natural language and structured data makes it challenging to build a system that supports using natural language to query large knowledge graphs. Many existing methods construct a structured query for the input question based on a syntactic parser. Once the input question is parsed incorrectly, a false structured query will be generated, which may result in false or incomplete answers. The problem gets worse especially for complex questions. In this paper, we propose a novel systematic method to understand natural language questions by using a large number of binary templates rather than semantic parsers. As sufficient templates are critical in the procedure, we present a low-cost approach that can build a huge number of templates automatically. To reduce the search space, we carefully devise an index to facilitate the online template decomposition. Moreover, we design effective strategies to perform the two-level disambiguations (i.e., entity-level ambiguity and structure-level ambiguity) by considering the query semantics. Extensive experiments over several benchmarks demonstrate that our proposed approach is effective as it significantly outperforms state-of-the-art methods in terms of both precision and recall. PVLDB Reference Format: Weiguo Zheng, Jeffrey Xu Yu, Lei Zou, and Hong Cheng. Question Answering Over Knowledge Graphs: Question Understanding Via Template Decomposition. PVLDB, 11 (11): 1373-1386, 2018. DOI: https://doi.org/10.14778/3236187.3236192.

Together with the development of more accurate methods in Computer Vision and Natural Language Understanding, holistic architectures that answer on questions about the content of real-world images have emerged. In this tutorial, we build a neural-based approach to answer questions about images. We base our tutorial on two datasets: (mostly on) DAQUAR, and (a bit on) VQA. With small tweaks the models that we present here can achieve a competitive performance on both datasets, in fact, they are among the best methods that use a combination of LSTM with a global, full frame CNN representation of an image. We hope that after reading this tutorial, the reader will be able to use Deep Learning frameworks, such as Keras and introduced Kraino, to build various architectures that will lead to a further performance improvement on this challenging task. 1 Preface In this tutorial1 we build a few architectures that can answer questions about images. The architectures are based on our two papers on this topic: Malinowski et al. [2015] and Malinowski et al. [2016]; and more broadly, on our project towards a Visual Turing Test2. In particular, an encoder-decoder perspective of Malinowski et al. [2016] allows us to effectively experiment with various design choices. For the sake of simplicity, we only consider a classification-based approach to answer questions about images, although an approach that generate answers word-by-word is also studied in the community [Malinowski et al., 2015]. In the tutorial, we mainly focus on the DAQUAR dataset [Malinowski and Fritz, 2014a], but a few possible directions to apply learnt techniques to VQA [Antol et al., 2015] are also pointed. First, we will get familiar with the task of answering questions about images, and a dataset that implements the task (due to a small size, we mainly use DAQUAR as it better serves an educational purpose that we aim at this tutorial). Next, we build a few blind models that answer questions about images without actually seeing such images. Such models already exhibit a reasonable performance as they can effectively learn various biases that exist in a dataset, which we also interpret as learning a common sense knowledge [Malinowski et al., 2015, 2016]. Subsequently, we build a few language+vision models that answer questions based on both a textual and a visual inputs. Finally, we leave the tutorial with a few possible research directions. Technical aspects The tutorial is originally written using Python Notebook, which the reader is welcome to download and use through the tutorial. Instructions necessary to run the Notebook version of this tutorial are provided in the following: https://github.com/mateuszmalinowski/visual_turing_test-tutorial. In this tutorial, we heavily use a Python code, and therefore it is expected the reader either already knows this language, or can quickly learn it. However, we made an effort to make this tutorial approachable to a wider audience. We use Kraino that is a framework prepared for this tutorial in order to simplify the development of the question answering architectures. Under the hood, it uses Theano4 [Bastien et al., 2012] and Keras5 [Chollet, 2015] – two frameworks to build Deep Learning models. We also use various CNNs representations extracted from images that can be downloaded as explained at the beginning of our Notebook tutorial. We also highlight exercises that a curious reader may attempt to solve. This tutorial was presented for the first time during the 2nd Summer School on Integrating Vision and Language: Deep Learning. http://mpii.de/visual_turing_test https://github.com/mateuszmalinowski/visual_turing_test-tutorial/blob/master/visual_ turing_test.ipynb http://deeplearning.net/software/theano/ https://keras.io 1 ar X iv :1 61 0. 01 07 6v 1 [ cs .C V ] 4 O ct 2 01 6

Most recently proposed methods for Neural Program Induction work under the assumption of having a large set of input/output (I/O) examples for learning any underlying input-output mapping. This paper aims to address the problem of data and computation efficiency of program induction by leveraging information from related tasks. Specifically, we propose two approaches for cross-task knowledge transfer to improve program induction in limited-data scenarios. In our first proposal, portfolio adaptation, a set of induction models is pretrained on a set of related tasks, and the best model is adapted towards the new task using transfer learning. In our second approach, meta program induction, a k-shot learning approach is used to make a model generalize to new tasks without additional training. To test the efficacy of our methods, we constructed a new benchmark of programs written in the Karel programming language [17]. Using an extensive experimental evaluation on the Karel benchmark, we demonstrate that our proposals dramatically outperform the baseline induction method that does not use knowledge transfer. We also analyze the relative performance of the two approaches and study conditions in which they perform best. In particular, meta induction outperforms all existing approaches under extreme data sparsity (when a very small number of examples are available), i.e., fewer than ten. As the number of available I/O examples increase (i.e. a thousand or more), portfolio adapted program induction becomes the best approach. For intermediate data sizes, we demonstrate that the combined method of adapted meta program induction has the strongest performance.

1Student,Dept. of Computer Engineering, VESIT, Maharashtra, India ---------------------------------------------------------------------------***-------------------------------------------------------------------Abstract Deep Learning has come into existence as a new area for research in Machine Learning. It aims to act like a human brain, having the ability to learn and process from complex data and also tries solving intricate tasks as well. Due to this capability, its been used in various fields like text, sound, images etc. Natural language process has started to being impacted by the deep learning techniques. This research paper highlights Deep Learning’s recent developments and applications in Natural Language Processing.

The goal of this paper is to generate insights about the differences between graph-based and logic-based approaches to multi-relational data mining by performing a case study of graph-based system, Subdue and the inductive logic programming system, CProgol. We identify three key factors for comparing graph-based and logic-based multi-relational data mining; namely, the ability to discover structurally large concepts, the ability to discover semantically complicated concepts and the ability to effectively utilize background knowledge. We perform an experimental comparison of Subdue and CProgol on the Mutagenesis domain and various artificially generated Bongard problems. Experimental results indicate that Subdue can significantly outperform CProgol while discovering structurally large multi-relational concepts. It is also observed that CProgol is better at learning semantically complicated concepts and it tends to use background knowledge more effectively than Subdue.

Robots assisting humans in complex domains need the ability to represent, reason with, and learn from, different descriptions of incomplete domain knowledge and uncertainty. This paper focuses on the challenge of incrementally and interactively discovering previously unknown axioms governing domain dynamics, and describes an architecture that integrates declarative programming and relational reinforcement learning to address this challenge. Answer Set Prolog (ASP), a declarative programming paradigm, is used to represent and reason with incomplete domain knowledge for planning and diagnostics. For any given goal, unexplained failure of plans created by ASP-based inference is taken to indicate the existence of unknown domain axioms. The task of discovering these axioms is formulated as a reinforcement learning problem, and a relational representation is used to incrementally generalize from specific axioms identified over time. These generic axioms are then added to the ASP-based representation for subsequent inference. The architecture's capabilities are demonstrated and evaluated in two domains, Blocks World and Robot Butler.

In many regards, tuning deep-learning networks is still more an art than it is a technique. Choosing the correct hyper-parameters and getting a complex network to learn properly can be daunting to people not well versed in that art. Taking the example of entailment classification, this work will analyze different methods and tools that can be used in order to diagnose learning problems. It will aim at providing an deep understanding of some key hyper-parameters, sketch out their effects and explain how to identify recurrent problems when tuning them.


Question: DeepLogic: An end-to-end framework for logical reasoning.

Introducing Lifted Relational Neural Networks: A Synergy of Logic and Learning

This paper presents a novel framework that integrates the precision of relational logic with the adaptability of neural network learning, marking a significant advancement in the field of artificial intelligence. Our method employs a lifted architecture, which may be derived from background knowledge or learned from data, to create a bridge between logic and neural networks. This architecture is described through relational rules, which can either be manually crafted or automatically learned. These rules act as a blueprint for the construction of deep neural networks that dynamically adjust their structures to match the relational patterns found in specific training or testing examples.

In this framework, networks derived from different examples share common weights, facilitating a collaborative learning process. This shared learning is achieved through the stochastic gradient descent algorithm, which simultaneously optimizes weights across various network instances. The framework supports hierarchical relational modeling, allowing for the discovery and learning of latent relational concepts through the shared weights in the hidden layers corresponding to the rules.

Our experiments highlight the method's capability to uncover significant relational concepts. Moreover, the performance of our approach is evaluated on 78 relational learning benchmarks, where it demonstrates superior outcomes compared to traditional methods. This work showcases the potential of combining relational logic and neural networks to solve complex relational learning tasks with enhanced efficiency and accuracy.

This paper summarizes results on a recently introduced family of Datalog-based languages, called Datalog+/-, which is a new framework for tractable ontology querying, and for a variety of other applications. Datalog+/- extends plain Datalog by features such as existentially quantified rule heads and, at the same time, restricts the rule syntax so as to achieve decidability and tractability. In particular, we discuss three paradigms ensuring decidability: chase termination, guardedness, and stickiness.

Statistical Spoken Dialogue Systems have been around for many years. However, access to these systems has always been difficult as there is still no publicly available end-to-end system implementation. To alleviate this, we present PyDial, an opensource end-to-end statistical spoken dialogue system toolkit which provides implementations of statistical approaches for all dialogue system modules. Moreover, it has been extended to provide multidomain conversational functionality. It offers easy configuration, easy extensibility, and domain-independent implementations of the respective dialogue system modules. The toolkit is available for download under the Apache 2.0 license.

Systems for symbolic complex event recognition detect occurrences of events in time using a set of event definitions in the form of logical rules. The Event Calculus is a temporal logic that has been used as a basis in event recognition applications, providing among others, connections to techniques for learning such rules from data. We advance the state-of-the-art by combining an existing online algorithm for learning crisp relational structure with an online method for weight learning in Markov Logic Networks (MLN). The result is an algorithm that learns complex event patterns in the form of Event Calculus theories in the MLN semantics. We evaluate our approach on a challenging real-world application for activity recognition and show that it outperforms both its crisp predecessor and competing online MLN learners in terms of predictive performance, at the price of a small increase in training time.

Deep networks are state-of-the-art models used for understanding the content of images, videos, audio and raw input data. Current computing systems are not able to run deep network models in real-time with low power consumption. In this paper we present nn-X: a scalable, low-power coprocessor for enabling real-time execution of deep neural networks. nn-X is implemented on programmable logic devices and comprises an array of configurable processing elements called collections. These collections perform the most common operations in deep networks: convolution, subsampling and non-linear functions. The nn-X system includes 4 high-speed direct memory access interfaces to DDR3 memory and two ARM Cortex-A9 processors. Each port is capable of a sustained throughput of 950 MB/s in full duplex. nn-X is able to achieve a peak performance of 227 G-ops/s, a measured performance in deep learning applications of up to 200 G-ops/s while consuming less than 4 watts of power. This translates to a performance per power improvement of 10 to 100 times that of conventional mobile and desktop processors.

Apache Spark is an open-source cluster computing framework. It has emerged as the next generation big data processing engine, overtaking Hadoop MapReduce which helped ignite the big data revolution. Spark maintains MapReduce’s linear scalability and fault tolerance, but extends it in a few important ways: it is much faster (100 times faster for certain applications), much easier to program in due to its rich APIs in Python, Java, Scala, SQL and R (MapReduce has 2 core calls) , and its core data abstraction, the distributed data frame. In addition, it goes far beyond batch applications to support a variety of compute-intensive tasks, including interactive queries, streaming, machine learning, and graph processing. With massive amounts of computational power, deep learning has been shown to produce state-of-the-art results on various tasks in different fields like computer vision, automatic speech recognition, natural language processing and online advertising targeting. Thanks to the open-source frameworks, e.g. Torch, Theano, Caffe, MxNet, Keras and TensorFlow, we can build deep learning model in a much easier way. Among all these framework, TensorFlow is probably the most popular open source deep learning library. TensorFlow 1.0 was released recently, which provide a more stable, flexible and powerful computation tool for numerical computation using data flow graphs. Keras is a highlevel neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. This tutorial will provide an accessible introduction to large-scale distributed machine learning and data mining, and to Spark and its potential to revolutionize academic and commercial data science practices. It is divided into three parts: the first part will cover fundamental Spark concepts, including Spark Core, functional programming ala map-reduce, data frames, the Spark Shell, Spark Streaming, Spark SQL, MLlib, and more; the second part will focus on hands-on algorithmic design and development with Spark (developing algorithms from scratch such as decision tree learning, association rule mining (aPriori), graph processing algorithms such as pagerank/shortest path, gradient descent algorithms such as support vectors machines and matrix factorization. Industrial applications and deployments of Spark will also be presented.; the third part will introduce deep learning concepts, how to implement a deep learning model through TensorFlow, Keras and run the model on Spark. Example code will be made available in python (pySpark) notebooks.

Neural network architectures with memory and attention mechanisms exhibit certain reasoning capabilities required for question answering. One such architecture, the dynamic memory network (DMN), obtained high accuracy on a variety of language tasks. However, it was not shown whether the architecture achieves strong results for question answering when supporting facts are not marked during training or whether it could be applied to other modalities such as images. Based on an analysis of the DMN, we propose several improvements to its memory and input modules. Together with these changes we introduce a novel input module for images in order to be able to answer visual questions. Our new DMN+ model improves the state of the art on both the Visual Question Answering dataset and the bAbI-10k text question-answering dataset without supporting fact supervision.

Deep models are the defacto standard in visual decision problems due to their impressive performance on a wide array of visual tasks. However, they are frequently seen as opaque and are unable to explain their decisions. In contrast, humans can justify their decisions with natural language and point to the evidence in the visual world which supports their decisions. We propose a method which incorporates a novel explanation attention mechanism; our model is trained using textual rationals, and infers latent attention to visually ground explanations. We collect two novel datasets in domains where it is interesting and challenging to explain decisions. First, we extend the visual question answering task to not only provide an answer but also visual and natural language explanations for the answer. Second, we focus on explaining human activities in a contemporary activity recognition dataset. We extensively evaluate our model, both on the justification and pointing tasks, by comparing it to prior models and ablations using both automatic and human evaluations.

Deep generative models have been enjoying success in modeling continuous data. However it remains challenging to capture the representations for discrete structures with formal grammars and semantics, e.g., computer programs and molecular structures. How to generate both syntactically and semantically correct data still remains largely an open problem. Inspired by the theory of compiler where the syntax and semantics check is done via syntax-directed translation (SDT), we propose a novel syntax-directed variational autoencoder (SD-VAE) by introducing stochastic lazy attributes. This approach converts the offline SDT check into on-the-fly generated guidance for constraining the decoder. Comparing to the state-of-the-art methods, our approach enforces constraints on the output space so that the output will be not only syntactically valid, but also semantically reasonable. We evaluate the proposed model with applications in programming language and molecules, including reconstruction and program/molecule optimization. The results demonstrate the effectiveness in incorporating syntactic and semantic constraints in discrete generative models, which is significantly better than current state-of-the-art approaches.

Optimizing task-related mathematical model is one of the most fundamental methodologies in statistic and learning areas. However, generally designed schematic iterations may hard to investigate complex data distributions in real-world applications. Recently, training deep propagations (i.e., networks) has gained promising performance in some particular tasks. Unfortunately, existing networks are often built in heuristic manners, thus lack of principled interpretations and solid theoretical supports. In this work, we provide a new paradigm, named Propagation and Optimization based Deep Model (PODM), to bridge the gaps between these different mechanisms (i.e., model optimization and deep propagation). On the one hand, we utilize PODM as a deeply trained solver for model optimization. Different from these existing network based iterations, which often lack theoretical investigations, we provide strict convergence analysis for PODM in the challenging nonconvex and nonsmooth scenarios. On the other hand, by relaxing the model constraints and performing end-to-end training, we also develop a PODM based strategy to integrate domain knowledge (formulated as models) and real data distributions (learned by networks), resulting in a generic ensemble framework for challenging real-world applications. Extensive experiments verify our theoretical results and demonstrate the superiority of PODM against these state-of-the-art approaches.

Software frameworks for neural networks play key roles in the development and application of deep learning methods. However, as new types of deep learning models are developed, existing frameworks designed for convolutional neural networks are becoming less useful. In this paper, we introduce Chainer, a Pythonbased, standalone open source framework for deep learning models. Chainer provides a flexible, intuitive, and high performance means of implementing a full range of deep learning models, including state-of-the-art models such as recurrent neural networks and variational autoencoders.


Question: DeepLogic: An End-to-End System for Logical Reasoning Tasks

Presenting AdaGradX: An Enhanced Stochastic Optimization Framework
We unveil AdaGradX, a cutting-edge algorithm designed for efficient first-order gradient-based optimization of stochastic objectives, with a focus on adaptive estimation of lower-order moments. This method is distinguished by its ease of implementation, computational efficiency, minimal memory footprint, robustness to diagonal gradient rescaling, and suitability for large-scale problems in terms of data and parameters. AdaGradX is particularly advantageous for non-stationary objectives and scenarios characterized by highly noisy or sparse gradients. The algorithm's hyper-parameters are intuitively designed and typically require minimal adjustment. We explore the connections between AdaGradX and related optimization techniques that have influenced its development. A theoretical analysis of AdaGradX's convergence properties is provided, along with a regret bound on its convergence rate that is competitive with the state-of-the-art under the online convex optimization paradigm. Empirical evaluations confirm AdaGradX's practical efficacy and its superior performance relative to conventional stochastic optimization strategies. Additionally, we introduce AdaGradXmax, a variant of AdaGradX that employs the infinity norm for enhanced optimization capabilities.

Neural network architectures with memory and attention mechanisms exhibit certain reasoning capabilities required for question answering. One such architecture, the dynamic memory network (DMN), obtained high accuracy on a variety of language tasks. However, it was not shown whether the architecture achieves strong results for question answering when supporting facts are not marked during training or whether it could be applied to other modalities such as images. Based on an analysis of the DMN, we propose several improvements to its memory and input modules. Together with these changes we introduce a novel input module for images in order to be able to answer visual questions. Our new DMN+ model improves the state of the art on both the Visual Question Answering dataset and the bAbI-10k text question-answering dataset without supporting fact supervision.

In this work, we want to teach the computers to do reading comprehension. More specifically, the task is to let computers read a short passage and then answer some simple questions based on the passage.[1] proposes an end-to-end, RNN-like approach which, according to them, needs significantly less supervision but performs comparatively good to benchmarks for many applications. However, we found that their Q&A result is based on a small synthetic data set defined in [2]. We could see that the dataset is quite small and simple with a vocabulary size of 177. Here is a sample from the dataset.

The capability of processing and digesting raw data is one of the key features of a humanlike artificial intelligence system. For instance, real-time machine translation should be able to process and understand spoken natural language, and autonomous driving relies on the comprehension of visual inputs. Representation learning is a class of machine learning techniques that autonomously learn to derive latent features from raw data. These new features are expected to represent the data instances in a vector space that facilitates the machine learning task. This thesis studies two specific data situations that require efficient representation learning: knowledge graph data and high dimensional sequences. In the first part of this thesis, we first review multiple relational learning models based on tensor decomposition for knowledge graphs. We point out that relational learning is in fact a means of learning representations through one-hot mapping of entities. Furthermore, we generalize this mapping function to consume a feature vector that encodes all known facts about each entity. It enables the relational model to derive the latent representation instantly for a new entity, without having to re-train the tensor decomposition. In the second part, we focus on learning representations from high dimensional sequential data. Sequential data often pose the challenge that they are of variable lengths. Electronic health records, for instance, could consist of clinical event data that have been collected at subsequent time steps. But each patient may have a medical history of variable length. We apply recurrent neural networks to produce fixed-size latent representations from the raw feature sequences of various lengths. By exposing a prediction model to these learned representations instead of the raw features, we can predict the therapy prescriptions more accurately as a means of clinical decision support. We further propose Tensor-Train recurrent neural networks. We give a detailed introduction to the technique of tensorizing and decomposing large weight matrices into a few smaller tensors. We demonstrate the specific algorithms to perform the forward-pass and the back-propagation in this setting. Then we apply this approach to the input-to-hidden weight matrix in recurrent neural networks. This novel architecture can process extremely high dimensional sequential features such as video data. The model also provides a promising solution to processing sequential features with high sparsity. This is, for instance, the case with electronic health records, since they are often of categorical nature and have to be binary-coded. We incorporate a statistical survival model with this representation learning model, which shows superior prediction quality.

Empirically, neural networks that attempt to learn programs from data have exhibited poor generalizability. Moreover, it has traditionally been difficult to reason about the behavior of these models beyond a certain level of input complexity. In order to address these issues, we propose augmenting neural architectures with a key abstraction: recursion. As an application, we implement recursion in the Neural Programmer-Interpreter framework on four tasks: grade-school addition, bubble sort, topological sort, and quicksort. We demonstrate superior generalizability and interpretability with small amounts of training data. Recursion divides the problem into smaller pieces and drastically reduces the domain of each neural network component, making it tractable to prove guarantees about the overall system’s behavior. Our experience suggests that in order for neural architectures to robustly learn program semantics, it is necessary to incorporate a concept like recursion.

This paper proposes an improvement to the existing data-driven Neural Belief Tracking (NBT) framework for Dialogue State Tracking (DST). The existing NBT model uses a hand-crafted belief state update mechanism which involves an expensive manual retuning step whenever the model is deployed to a new dialogue domain. We show that this update mechanism can be learned jointly with the semantic decoding and context modelling parts of the NBT model, eliminating the last rule-based module from this DST framework. We propose two different statistical update mechanisms and show that dialogue dynamics can be modelled with a very small number of additional model parameters. In our DST evaluation over three languages, we show that this model achieves competitive performance and provides a robust framework for building resource-light DST models.

We describe SLING, a framework for parsing natural language into semantic frames. SLING supports general transition-based, neural-network parsing with bidirectional LSTM input encoding and a Transition Based Recurrent Unit (TBRU) for output decoding. The parsing model is trained end-to-end using only the text tokens as input. The transition system has been designed to output frame graphs directly without any intervening symbolic representation. The SLING framework includes an efficient and scalable frame store implementation as well as a neural network JIT compiler for fast inference during parsing. SLING is implemented in C++ and it is available for download on GitHub.

We present a generative model to map natural language questions into SQL queries. Existing neural network based approaches typically generate a SQL query wordby-word, however, a large portion of the generated results is incorrect or not executable due to the mismatch between question words and table contents. Our approach addresses this problem by considering the structure of table and the syntax of SQL language. The quality of the generated SQL query is significantly improved through (1) learning to replicate content from column names, cells or SQL keywords; and (2) improving the generation of WHERE clause by leveraging the column-cell relation. Experiments are conducted on WikiSQL, a recently released dataset with the largest questionSQL pairs. Our approach significantly improves the state-of-the-art execution accuracy from 69.0% to 74.4%.

Interacting with computers is a ubiquitous activity for millions of people. Repetitive or specialized tasks often require creation of small, often one-off, programs. End-users struggle with learning and using the myriad of domain-specific languages (DSLs) to effectively accomplish these tasks.
 We present a general framework for constructing program synthesizers that take natural language (NL) inputs and produce expressions in a target DSL. The framework takes as input a DSL definition and training data consisting of NL/DSL pairs. From these it constructs a synthesizer by learning optimal weights and classifiers (using NLP features) that rank the outputs of a keyword-programming based translation. We applied our framework to three domains: repetitive text editing, an intelligent tutoring system, and flight information queries. On 1200+ English descriptions, the respective synthesizers rank the desired program as the top-1 and top-3 for 80% and 90% descriptions respectively.

Populating large-scale structured databases from unstructured sources is a critical and challenging task in data analytics. As automated feature engineering methods grow increasingly prevalent, constructing sufficiently large labeled training sets has become the primary hurdle in building machine learning information extraction systems. In light of this, we have taken a new approach called data programming [7]. Rather than hand-labeling data, in the data programming paradigm, users generate large amounts of noisy training labels by programmatically encoding domain heuristics as simple rules. Using this approach over more traditional distant supervision methods and fully supervised approaches using labeled data, we have been able to construct knowledge base systems more rapidly and with higher quality. Since the ability to quickly prototype, evaluate, and debug these rules is a key component of this paradigm, we introduce DDLite, an interactive development framework for data programming. This paper reports feedback collected from DDLite users across a diverse set of entity extraction tasks. We share observations from several DDLite hackathons in which 10 biomedical researchers prototyped information extraction pipelines for chemicals, diseases, and anatomical named entities. Initial results were promising, with the disease tagging team obtaining an F1 score within 10 points of the state-of-the-art in only a single day-long hackathon's work. Our key insights concern the challenges of writing diverse rule sets for generating labels, and exploring training data. These findings motivate several areas of active data programming research.

The increasing use of deep neural networks for safety-critical applications, such as autonomous driving and flight control, raises concerns about their safety and reliability. Formal verification can address these concerns by guaranteeing that a deep learning system operates as intended, but the state-of-the-art is limited to small systems. In this work-in-progress report we give an overview of our work on mitigating this difficulty, by pursuing two complementary directions: devising scalable verification techniques, and identifying design choices that result in deep learning systems that are more amenable to verification. ACM Reference Format: Lindsey Kuper, Guy Katz, Justin Gottschlich, Kyle Julian, Clark Barrett, and Mykel J. Kochenderfer. 2018. Toward Scalable Verification for SafetyCritical Deep Networks. In Proceedings of SysML Conference (SysML). ACM, New York, NY, USA, 3 pages.


Question: DeepLogic: An End-to-End System for Logical Reasoning and Inference

Introducing TensorLog: A Differentiable Deductive Database for Deep Learning Integration

The integration of large-scale knowledge bases (KBs) into deep learning architectures, particularly those based on gradient descent, has been a challenge due to the disparate nature of symbolic reasoning and neural computation. To bridge this gap, we present TensorLog, a probabilistic deductive database that enables differentiable reasoning. TensorLog translates logical clauses into a specific form of factor graphs, facilitating the integration of logical theories into neural network frameworks. For each query type, TensorLog unrolls the necessary belief propagation (BP) steps into a differentiable function, allowing for the recursive composition of these functions to perform inference in complex logical theories with interconnected clauses and predicates.

The efficiency of TensorLog is a key feature, with both compilation and inference processes designed for scalability. Compilation is linear in relation to the size of the logical theory and the depth of proofs, while inference scales linearly with the size of the knowledge base and the number of BP iterations. We showcase the effectiveness of TensorLog through experimental results and discuss its compatibility with other probabilistic first-order logics, highlighting its potential for enhancing deep learning models with logical reasoning capabilities.

Common-sense or background knowledge is required to understand natural language, but in most neural natural language understanding (NLU) systems, the requisite background knowledge is indirectly acquired from static corpora. We develop a new reading architecture for the dynamic integration of explicit background knowledge in NLU models. A new task-agnostic reading module provides refined word representations to a task-specific NLU architecture by processing background knowledge in the form of free-text statements, together with the taskspecific inputs. Strong performance on the tasks of document question answering (DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness and flexibility of our approach. Analysis shows that our models learn to exploit knowledge selectively and in a semantically appropriate way.

This paper describes a uniform formalization of much of the current work in artificial intelligence on inference systems. We show that many of these systems, including first-order theorem provers, assumption-based truth maintenance systems (ATMSS), and unimplemented formal systems such as default logic or circumscription, can be subsumed under a single general framework. We begin by defining this framework, which is based on a mathematical structure known as a bilattice. We present a formal definition of inference using this structure and show that this definition generalizes work involving ATMSS and some simple nonmonotonic logics. Following the theoretical description, we describe a constructive approach to inference in this setting; the resulting generalization of both conventional inference and ATMSS is achieved without incurring any substantial computational overhead. We show that our approach can also be used to implement a default reasoner, and discuss a combination of default and ATMS methods that enables us to formally describe an “incremental” default reasoning system. This incremental system does not need to perform consistency checks before drawing tentative conclusions, but can instead adjust its beliefs when a default premise or conclusion is overturned in the face of convincing contradictory evidence. The system is therefore much more computationally viable than earlier approaches. Finally, we discuss the implementation of our ideas. We begin by considering general issues that need to be addressed when implementing a multivalued approach such as that we are proposing, and then turn to specific examples showing the results of an existing implementation. This single implementation is used to solve a digital simulation task using first-order logic, a diagnostic task using ATMSS as suggested by de Kleer and Williams, a problem in default reasoning as in Reiter’s default logic or McCarthy’s circumscription, and to solve the same problem more efficiently by combining default methods with justification information. All of these applications use the same general-purpose bilattice theorem prover and differ only in the choice of bilattice being considered.

Answering questions correctly from standardized eighth-grade science tests is itself a test of machine intelligence.

Understanding argumentation and its role in human reasoning has been a continuous subject of investigation for scholars from the ancient Greek philosophers to current researchers in philosophy, logic and artificial intelligence. In recent years, argumentation models have been used in different areas such as knowledge representation, explanation, proof elaboration, commonsense reasoning, logic programming, legal reasoning, decision making, and negotiation. However, these models address quite specific needs and there is need for a conceptual framework that would organize and compare existing argumentation-based models and methods. Such a framework would be very useful especially for researchers and practitioners who want to select appropriate argumentation models or techniques to be incorporated in new software systems with argumentation capabilities. In this paper, we propose such a conceptual framework, based on taxonomy of the most important argumentation models, approaches and systems found in the literature. This framework highlights the similarities and differences between these argumentation models. As an illustration of the practical use of this framework, we present a case study which shows how we used this framework to select and enrich an argumentation model in a knowledge acquisition project which aimed at representing argumentative knowledge contained in texts critiquing military courses of action.

Speaker: Mrinmaya Sachan Thesis Committee: Eric P. Xing, Chair Jaime Carbonell Tom Mitchell Dan Roth (University of Pennsylvania) Thesis Proposal Jan. 22, 2018 1:00pm 8102 GHC Link to draft document: http://www.cs.cmu.edu/~mrinmays/thesis_proposal.pdf Standardized tests are often used to test students as they progress in the formal education system. These tests are widely available and measurable with clear evaluation procedures and metrics. Hence, these can serve as good tests for AI. We propose approaches for solving some of these tests. We broadly categorize these tests into two categories: open domain question answering tests such as reading comprehensions and elementary school science tests, and closed domain question answering tests such as intermediate or advanced math and science tests. We present alignment based approach with multi-task learning for the former. For closed domain tests, we propose a parsing to programs approach which can be seen as a natural language interface to expert systems. We also describe approaches for question generation based on instructional material in both open domain as well as closed domain settings. Finally, we show that we can improve both the question answering and question generation models by learning them jointly. This mechanism also allows us to leverage cheap unlabelled data for learning the two models. Our work can potentially be applied for the social good in the education domain. We perform studies on human subjects who found our approaches useful as assistive tools in education.

We propose a novel mixtures of Gaussian processes model in which the gating function is interconnected with a probabilistic logical model, in our case Markov logic networks. In this way, the resulting mixed graphical model, called Markov logic mixtures of Gaussian processes (MLxGP), solves joint Bayesian non-parametric regression and probabilistic relational inference tasks. In turn, MLxGP facilitates novel, interesting tasks such as regression based on logical constraints or drawing probabilistic logical conclusions about regression data, thus putting “machines reading regression data” in reach.

Deep generative models have been enjoying success in modeling continuous data. However it remains challenging to capture the representations for discrete structures with formal grammars and semantics, e.g., computer programs and molecular structures. How to generate both syntactically and semantically correct data still remains largely an open problem. Inspired by the theory of compiler where the syntax and semantics check is done via syntax-directed translation (SDT), we propose a novel syntax-directed variational autoencoder (SD-VAE) by introducing stochastic lazy attributes. This approach converts the offline SDT check into on-the-fly generated guidance for constraining the decoder. Comparing to the state-of-the-art methods, our approach enforces constraints on the output space so that the output will be not only syntactically valid, but also semantically reasonable. We evaluate the proposed model with applications in programming language and molecules, including reconstruction and program/molecule optimization. The results demonstrate the effectiveness in incorporating syntactic and semantic constraints in discrete generative models, which is significantly better than current state-of-the-art approaches.

Empirically, neural networks that attempt to learn programs from data have exhibited poor generalizability. Moreover, it has traditionally been difficult to reason about the behavior of these models beyond a certain level of input complexity. In order to address these issues, we propose augmenting neural architectures with a key abstraction: recursion. As an application, we implement recursion in the Neural Programmer-Interpreter framework on four tasks: grade-school addition, bubble sort, topological sort, and quicksort. We demonstrate superior generalizability and interpretability with small amounts of training data. Recursion divides the problem into smaller pieces and drastically reduces the domain of each neural network component, making it tractable to prove guarantees about the overall system’s behavior. Our experience suggests that in order for neural architectures to robustly learn program semantics, it is necessary to incorporate a concept like recursion.

This paper proposes a framework for modeling and controlling systems described by interdependent physical laws, logic rules, and operating constraints, denoted as Mixed Logical Dynamical (MLD) systems. These are described by linear dynamic equations subject to linear inequalities involving real and integer variables. MLD systems include constrained linear systems, finite state machines, some classes of discrete event systems, and nonlinear systems which can be approximated by piecewise linear functions. A predictive control scheme is proposed which is able to stabilize MLD systems on desired reference trajectories while fulfilling operating constraints, and possibly take into account previous qualitative knowledge in the form of heuristic rules. Due to the presence of integer variables, the resulting on-line optimization procedures are solved through Mixed Integer Quadratic Programming (MIQP), for which efficient solvers have been recently developed. Some examples and a simulation case study on a complex gas supply system are reported.

A major challenge in computer vision is scaling activity understanding to the long tail of complex activities without requiring collecting large quantities of data for new actions. The task of video retrieval using natural language descriptions seeks to address this through rich, unconstrained supervision about complex activities. However, while this formulation offers hope of leveraging underlying compositional structure in activity descriptions, existing approaches typically do not explicitly model compositional reasoning. In this work, we introduce an approach for explicitly and dynamically reasoning about compositional natural language descriptions of activity in videos. We take a modular neural network approach that, given a natural language query, extracts the semantic structure to assemble a compositional neural network layout and corresponding network modules. We show that this approach is able to achieve state-of-the-art results on the DiDeMo video retrieval dataset.


Question: A non-intrusive method for monitoring water usage in residential buildings.

This paper offers an in-depth exploration of the HydroSense technology, a pioneering, cost-effective, and straightforward-to-install single-point pressure sensor designed for the automatic disaggregation of water usage activities within households. Building upon our prior research (Froehlich et al., 2009 [53]), we provide a detailed review of existing and emerging methodologies for water disaggregation. Additionally, we delve deeper into the theoretical underpinnings of our approach and offer an extended analysis that encompasses the differentiation between hot and cold water valve usage. The analysis also contrasts two classification methodologies: the template-based matching scheme previously utilized in Froehlich et al. (2009) [53] and a novel stochastic approach employing a Hidden Markov Model. Our findings indicate that both methods are highly effective, achieving accuracies exceeding 90% in identifying water events at the valve and fixture levels. The paper concludes with a critical assessment of the limitations in our experimental design and outlines future research directions to address these challenges.

The aim of this research survey note is to provide a comprehensive overview of Smart Home Environments with a focus on their architectures and application areas, as well as utilized technologies, infrastructures, and standards. The main source of information was provided by past and existing research and development projects carried out in this area, while the main result of our survey is a classification of Smart Home Environments, as revealed by this projects’ survey.

At present, labor-saving and water-saving technology is a key issue in irrigation. A wireless solution for intelligent field irrigation system dedicated to Jew's-ear planting in Lishui, Zhejiang, China, based on ZigBee technology was proposed in this paper. Instead of conventional wired connection, the wireless design made the system easy installation and maintenance. The hardware architecture and software algorithm of wireless sensor/actuator node and portable controller, acting as the end device and coordinator in ZigBee wireless sensor network respectively, were elaborated in detail. The performance of the whole system was evaluated in the end. The long-time smooth and proper running of the system in the field proved its high reliability and practicability. As an explorative application of wireless sensor network in irrigation management, this paper offered a methodology to establish large-scale remote intelligent irrigation system.

Emergence of Controlled Environment Agriculture (CEA) ranging from computer controlled water irrigation system to lightning and ventilation has changed the conventional scenario of farming. This paper proposes and demonstrate an economical and easy to use arduino based controlled irrigation system. The designed system deals with various environmental factors such as moisture, temperature and amount of water required by the crops using sensors like water flow sensor, temperature sensor and soil moisture sensor. Datas are collected and received by arduino which can be linked to an interactive website which show the real time values along with the standard values of different factor required by a crop. This allows user to control irrigation pumps and sprinklers from far distance through a website and to meet the standard values which would help the farmer to yield maximum and quality crops. Studies conducted on laboratory prototype suggested the designed system to be applicable which can be implemented.

In this paper, we examine at-home activity rhythms and present a dozen of behavioral patterns obtained from an activity monitoring pilot study of 22 residents in an assisted living setting with four case studies. Established behavioral patterns have been captured using custom software based on a statistical predictive algorithm that models circadian activity rhythms (CARs) and their deviations. The CAR was statistically estimated based on the average amount of time a resident spent in each room within their assisted living apartment, and also on the activity level given by the average n.umber of motion events per room. A validated in-home monitoring system (IMS) recorded the monitored resident's movement data and established the occupancy period and activity level for each room. Using these data, residents' circadian behaviors were extracted, deviations indicating anomalies were detected, and the latter were correlated to activity reports generated by the IMS as well as notes of the facility's professional caregivers on the monitored residents. The system could be used to detect deviations in activity patterns and to warn caregivers of such deviations, which could reflect changes in health status, thus providing caregivers with the opportunity to apply standard of care diagnostics and to intervene in a timely manner.


Question: A method for nonintrusive water load monitoring.

This paper delves into the concept of instantaneous frequency (IF), a critical parameter for analyzing nonstationary signals, which are prevalent in various fields including seismic, radar, sonar, communications, and biomedical applications. Unlike the frequency of sinusoidal signals, which is straightforward, the IF is necessary for signals that do not conform to a simple sinusoidal structure. The paper examines the historical development of IF, exploring the successive definitions and models that have been proposed to represent this concept. It also investigates the correspondence between these mathematical models and the physical reality they aim to describe. The relationships between IF and other key signal processing concepts such as group-delay, analytic signals, and the bandwidth-time (BT) product are highlighted. Additionally, the paper discusses the connection between IF and time-frequency distributions, illustrating how these distributions can provide insights into the time-varying nature of signals. The distinctions between monocomponent and multicomponent signals are analyzed, along with the concept of instantaneous bandwidth, showing how these concepts are integral to the theoretical framework presented. Overall, the paper offers a comprehensive exploration of the instantaneous frequency, its definitions, and its relevance in practical applications, making it a valuable resource for researchers and practitioners in signal processing.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An enhanced version of the teaching-learning-based optimization algorithm for economic dispatch with valve-point effect
This paper presents an enhanced version of the teaching-learning-based optimization (TLBO) algorithm for solving the economic dispatch problem with valve-point effect. The valve-point effect is a nonlinear characteristic of the thermal power plants that causes a non-smooth, multimodal objective function, which makes the problem challenging to solve. The proposed algorithm, named ETLBO, introduces a mutation operator to enhance the exploration ability of the basic TLBO algorithm. The mutation operator is applied to the teacher and learner phases of the TLBO algorithm to improve the convergence rate and the quality of the solutions. The effectiveness of the proposed algorithm is demonstrated through its application to three standard test systems with valve-point effect. The results are compared with those obtained by other state-of-the-art optimization algorithms, and it is shown that the proposed ETLBO algorithm outperforms them in terms of solution quality and robustness. The proposed algorithm can be a useful tool for power system operators to solve the economic dispatch problem with valve-point effect.

**Similar Documents**: This article introduces an improved Teaching-Learning-Based Optimization (TLBO) algorithm, designated as ETLBO, specifically designed to address

A quasi-digital dc current sensor with rated current of ±600 A while overload current of about ±750 A is proposed in this paper. The new sensor is based on the open-loop self-oscillating fluxgate technology, but its originality is using a microcontroller unit to detect the duty cycle of the exciting voltage of the fluxgate. Compared with the published similar method, the whole signal chain of the new sensor is quasi-digital and without low-pass filter and analog-to-digital converter required when connected to digital systems. A precisely theoretical equation with respect to the linear dependence between the duty cycle and the current to be measured is established. Based on the equation, factors affecting the sensor sensitivity, accuracy, and resolution are determined, which constitutes the theoretical basis on the optimization design of the new sensor. The sensor linearity is improved using the least-squares polynomial fitting method. Some key specifications including the linearity, repeatability, and power supply effect of the sensor are characterized. The measurement results show that the linearity of the new sensor with the theoretical equation is better than 1.7% in the full scale of ±600 A, and can be improved to 0.3% when the fifth-order polynomial fitting method is used.

A power grid is a complex system connecting electric power generators to consumers through power transmission and distribution networks across a large geographical area. System monitoring is necessary to ensure the reliable operation of power grids, and state estimation is used in system monitoring to best estimate the power grid state through analysis of meter measurements and power system models. Various techniques have been developed to detect and identify bad measurements, including interacting bad measurements introduced by arbitrary, nonrandom causes. At first glance, it seems that these techniques can also defeat malicious measurements injected by attackers.
 In this article, we expose an unknown vulnerability of existing bad measurement detection algorithms by presenting and analyzing a new class of attacks, called false data injection attacks, against state estimation in electric power grids. Under the assumption that the attacker can access the current power system configuration information and manipulate the measurements of meters at physically protected locations such as substations, such attacks can introduce arbitrary errors into certain state variables without being detected by existing algorithms. Moreover, we look at two scenarios, where the attacker is either constrained to specific meters or limited in the resources required to compromise meters. We show that the attacker can systematically and efficiently construct attack vectors in both scenarios to change the results of state estimation in arbitrary ways. We also extend these attacks to generalized false data injection attacks, which can further increase the impact by exploiting measurement errors typically tolerated in state estimation. We demonstrate the success of these attacks through simulation using IEEE test systems, and also discuss the practicality of these attacks and the real-world constraints that limit their effectiveness.

This paper describes a research project to develop a network of high-precision phasor measurement units, termed micro-synchrophasors or μPMUs, and explore the applications of μPMU data for electric power distribution systems.

Energy and environmental sustainability can benefit a lot from advances in data mining and machine learning techniques. However, those advances rely on the availability of relevant datasets required to develop, improve and validate new techniques. Only recently the first datasets were made publicly available for the energy and sustainability research community. In this paper we present a freely available dataset containing power usage and related information from 50 homes. Here we describe our dataset, the hardware and software setups used when collecting the data and how others can access it. We then discuss potential uses of this data in the future of energy ecofeedback and demand side management research.


Question: A method for nonintrusive water load monitoring.

This article outlines a novel approach to nonintrusive load monitoring (NLM) in power systems, encompassing transient event classification, system identification methodologies, and practical implementation strategies. The proposed system is designed to accurately discern the operational patterns and model parameters of various loads connected to alternating current (AC) or direct current (DC) power distribution networks. A key feature of this monitoring solution is its reliance on readily available hardware, enabling the recognition of individual transient events through the disaggregation of signals collected by a limited set of sensors positioned at a central point within the distribution infrastructure. The paper provides an in-depth look at the implementation specifics and showcases real-world testing results for both AC and DC systems, demonstrating the system's efficacy in NLM applications.

Over the past few years, dozens of new techniques have been proposed for more accurate energy disaggregation, but the jury is still out on whether these techniques can actually save energy and, if so, whether higher accuracy translates into higher energy savings. In this paper, we explore both of these questions. First, we develop new techniques that use disaggregated power data to provide actionable feedback to residential users. We evaluate these techniques using power traces from 240 homes and find that they can detect homes that need feedback with as much as 84% accuracy. Second, we evaluate whether existing energy disaggregation techniques provide power traces with sufficient fidelity to support the feedback techniques that we created and whether more accurate disaggregation results translate into more energy savings for the users. Results show that feedback accuracy is very low even while disaggregation accuracy is high. These results indicate a need to revisit the metrics by which disaggregation is evaluated.

In this paper we have proposed and designed FindingHuMo (Finding Human Motion), a real-time user tracking system for Smart Environments. FindingHuMo can perform device-free tracking of multiple (unknown and variable number of) users in the Hallway Environments, just from non-invasive and anonymous (not user specific) binary motion sensor data stream. The significance of our designed system are as follows: (a) fast tracking of individual targets from binary motion data stream from a static wireless sensor network in the infrastructure. This needs to resolve unreliable node sequences, system noise and path ambiguity, (b) Scaling for multi-user tracking where user motion trajectories may crossover with each other in all possible ways. This needs to resolve path ambiguity to isolate overlapping trajectories, FindingHumo applies the following techniques on the collected motion data stream: (i) a proposed motion data driven adaptive order Hidden Markov Model with Viterbi decoding (called Adaptive-HMM), and then (ii) an innovative path disambiguation algorithm (called CPDA). Using this methodology the system accurately detects and isolates motion trajectories of individual users. The system performance is illustrated with results from real-time system deployment experience in a Smart Environment.

Demand-side management technology is a key element of the proposed smart grid, which will help utilities make more efficient use of their generation assets by reducing consumers' energy demand during peak load periods. However, although some modern appliances can respond to price signals from the utility companies, there is a vast stock of older appliances that cannot. For such appliances, utilities must infer what appliances are operating in a home, given only the power signals on the main feeder to the home (i.e., the home's power consumption must be disaggregated into individual appliances). We report on an in-depth investigation of multi-label classification algorithms for disaggregating appliances in a power signal. A systematic review of this research topic shows that this class of algorithms has received little attention in the literature, even though it is arguably a more natural fit to the disaggregation problem than the traditional single-label classifiers used to date. We examine a multi-label meta-classification framework (RAkEL), and a bespoke multi-label classification algorithm (MLkNN), employing both time-domain and wavelet-domain feature sets. We test these classifiers on two real houses from the Reference Energy Disaggregation Dataset. We found that the multilabel algorithms are effective and competitive with published results on the datasets.


Question: A method for nonintrusive water load monitoring.

This paper introduces a novel methodology for recognizing Activities of Daily Living (ADLs) by leveraging interactions with objects, a critical component in pervasive computing aimed at understanding and assisting human behavior in their natural environment. A significant challenge lies in accurately identifying the activities being performed, the manner in which they are executed, and their progression stages, particularly relevant in the context of elder care. A recent approach to tackling this issue involves the integration of Radio-Frequency Identification (RFID) technology, advanced data mining techniques, and a probabilistic inference engine. The goal is to identify ADLs based on the objects individuals interact with during their execution. Our proposed solution innovatively addresses the complexities of ADL monitoring by focusing on the sequence of object interactions, which we have found to reliably distinguish not only the type of ADL but also the proficiency of its execution. To operationalize this concept, we have developed the Proactive Activity Toolkit (PROACT), which holds promise in automating the monitoring of ADLs in a non-intrusive and efficient manner.

A distributed detection method is proposed to detect single stage multi-point (SSMP) attacks on a Cyber Physical System (CPS). Such attacks aim at compromising two or more sensors or actuators at any one stage of a CPS and could totally compromise a controller and prevent it from detecting the attack. However, as demonstrated in this work, using the flow properties of water from one stage to the other, a neighboring controller was found effective in detecting such attacks. The method is based on physical invariants derived for each stage of the CPS from its design. The attack detection effectiveness of the method was evaluated experimentally against an operational water treatment testbed containing 42 sensors and actuators. Results from the experiments point to high effectiveness of the method in detecting a variety of SSMP attacks but also point to its limitations. Distributing the attack detection code among various controllers adds to the scalability of the proposed method.

This paper presents a novel unsupervised approach to detect cyber attacks in Cyber-Physical Systems (CPS). We describe an unsupervised learning approach using a Recurrent Neural network which is a time series predictor as our model. We then use the Cumulative Sum method to identify anomalies in a replicate of a water treatment plant. The proposed method not only detects anomalies in the CPS but also identifies the sensor that was attacked. The experiments were performed on a complex dataset which is collected through a Secure Water Treatment Testbed (SWaT). Through the experiments, we show that the proposed technique is able to detect majority of the attacks designed by our research team with low false positive rates.


Question: A method for nonintrusive water load monitoring.

Introducing NAWMS: An Innovative Nonintrusive Autonomous Water Monitoring System for Conservation
Facing the escalating strain on fresh water resources due to increasing demand, innovative technological solutions are imperative to promote conservation and minimize waste. Inspired by the successful application of real-time, detailed data in power and gasoline conservation, we present NAWMS, a pioneering water monitoring system designed to empower users with information about their water consumption patterns.
The primary objective of our endeavor is to offer a user-friendly, self-calibrating system that seamlessly integrates into households, providing insights into the timing, location, and volume of water usage. The system's design is centered around wireless vibration sensors, which are effortlessly attached to water pipes, eliminating the need for plumbing expertise or alterations. This non-invasive, autonomous, and adaptable system, built with off-the-shelf components, is economically viable and scalable for widespread deployment.
NAWMS complements the existing household water flow meter, known for its accuracy but lacking spatial detail, by adding vibration sensors on individual water pipes to estimate flow rates at each outlet. To address the variability in sensor performance due to manufacturing, installation, and material differences, we have developed an adaptive auto-calibration routine. This process solves a dual-phase optimization problem, combining linear programming and mixed linear geometric programming, to ensure a high level of accuracy.
Through rigorous testing on a three-pipe prototype, we have confirmed the system's feasibility and its ability to dynamically adjust, significantly reducing errors in water usage estimates. Our results demonstrate an impressive accuracy across typical domestic flow-rate scenarios, maintaining long-term stability with a mean absolute error of 7%. This breakthrough system holds the potential to revolutionize water conservation efforts by equipping consumers with the knowledge to make informed choices about their water consumption.

Recently, with the explosive increase of automobiles in cities, parking problems are serious and even worsen in many cities. This paper proposes a new algorithm for parking occupancy detection based on the use of anisotropic magnetoresistive sensors. Parking occupancy detection is abstracted as binary pattern recognition problem. According to the status of the parking space, the recognition result contains two categories: vacant and occupied. The feature extraction method of the parking magnetic signal is proposed. In addition, the classification criteria are derived based on the distance discriminate analysis method. Eighty-two sensor nodes are deployed on the roadside parking spaces. By running the system for six months, we observed that the accuracy rate of the proposed parking occupancy detection algorithm is better than 98%.

This study investigates a prototyping of integrated system of Internet of Things based Wetting front detector (IOT-WFD) which focuses on how to enhance the IOT based Wetting front detector design for smart irrigation system. The empirical study was conducted with 2 sensors type to detect the wetting fronts which are the Frequency Domain Reflectrometry sensor (FDR) and Resistor-based sensor (RB) integrated and design with low-cost WFD. The results of this study point toward the IOT-WFD as an appropriated technology providing real time wetting front information in soil positively for application in terms of agricultural water management, with precision agriculture and efficient irrigation domain with a related decision knowledge that matches with the technology trend and smart farmer requirements. Evidence of positive results of this prototyping summary has been provided.

Smart grid aims to improve control and monitoring routines to ensure reliable and efficient supply of electricity. The rapid advancements in information and communication technologies of Supervisory Control And Data Acquisition (SCADA) networks, however, have resulted in complex cyber physical systems. This added complexity has broadened the attack surface of power-related applications, amplifying their susceptibility to cyber threats. A particular class of system integrity attacks against the smart grid is False Data Injection (FDI). In a successful FDI attack, an adversary compromises the readings of grid sensors in such a way that errors introduced into estimates of state variables remain undetected. This paper presents an end-to-end case study of how to instantiate real FDI attacks to the Alternating Current (AC) --nonlinear-- State Estimation (SE) process. The attack is realized through firmware modifications of the microprocessor-based remote terminal systems, falsifying the data transmitted to the SE routine, and proceeds regardless of perfect or imperfect knowledge of the current system state. The case study concludes with an investigation of an attack on the IEEE 14 bus system using load data from the New York Independent System Operator (NYISO).


Question: CAES Cryptosystem: Detailed Security Evaluations and Findings

This paper delves into an in-depth examination of the Advanced Encryption Standard (AES) cryptosystem, specifically focusing on its resilience against differential cryptanalysis, a sophisticated form of attack that exploits the differences in output caused by variations in the input. The study presents a comprehensive analysis of the AES's security measures, highlighting its ability to withstand such targeted attacks. Through a series of rigorous tests, the researchers have verified the effectiveness of AES in maintaining data integrity and confidentiality, even under conditions designed to simulate real-world security threats. The findings not only underscore the robustness of AES but also contribute to the ongoing discourse on cryptographic security, offering valuable insights for the development of future encryption algorithms.

Bit coin is the first digital currency to see widespread adoption. While payments are conducted between pseudonyms, Bit coin cannot offer strong privacy guarantees: payment transactions are recorded in a public decentralized ledger, from which much information can be deduced. Zero coin (Miers et al., IEEE S&P 2013) tackles some of these privacy issues by unlinking transactions from the payment's origin. Yet, it still reveals payments' destinations and amounts, and is limited in functionality. In this paper, we construct a full-fledged ledger-based digital currency with strong privacy guarantees. Our results leverage recent advances in zero-knowledge Succinct Non-interactive Arguments of Knowledge (zk-SNARKs). First, we formulate and construct decentralized anonymous payment schemes (DAP schemes). A DAP scheme enables users to directly pay each other privately: the corresponding transaction hides the payment's origin, destination, and transferred amount. We provide formal definitions and proofs of the construction's security. Second, we build Zero cash, a practical instantiation of our DAP scheme construction. In Zero cash, transactions are less than 1 kB and take under 6 ms to verify - orders of magnitude more efficient than the less-anonymous Zero coin and competitive with plain Bit coin.

Malwares are becoming increasingly stealthy, more and more malwares are using cryptographic algorithms (e.g., packing, encrypting C&C communication) to protect themselves from being analyzed. The use of cryptographic algorithms and truly transient cryptographic secrets inside the malware binary imposes a key obstacle to effective malware analysis and defense. To enable more effective malware analysis, forensics, and reverse engineering, we have developed CipherXRay - a novel binary analysis framework that can automatically identify and recover the cryptographic operations and transient secrets from the execution of potentially obfuscated binary executables. Based on the avalanche effect of cryptographic functions, CipherXRay is able to accurately pinpoint the boundary of cryptographic operation and recover truly transient cryptographic secrets that only exist in memory for one instant in between multiple nested cryptographic operations. CipherXRay can further identify certain operation modes (e.g., ECB, CBC, CFB) of the identified block cipher and tell whether the identified block cipher operation is encryption or decryption in certain cases. We have empirically validated CipherXRay with OpenSSL, popular password safe KeePassX, the ciphers used by malware Stuxnet, Kraken and Agobot, and a number of third party softwares with built-in compression and checksum. CipherXRay is able to identify various cryptographic operations and recover cryptographic secrets that exist in memory for only a few microseconds. Our results demonstrate that current software implementations of cryptographic algorithms hardly achieve any secrecy if their execution can be monitored.

Securing digital data has become tedious task as the technology is increasing. Existing encryption algorithms such as AES,DES and Blowfish ensure information security but consume lot of time as the security level increases. In this paper, Byte Rotation Encryption Algorithm (BREA) has implemented using parallel processing and multi-core utilization. BREA divides data into fixed size blocks. Each block is processed parallely using random number key. So the set of blocks are executed in parallel by utilizing all the available CPU cores. Finally, from the experimental analysis, it is observed that the proposed BREA algorithm reduces execution time when the number of cores has increased.

Software cache-based side channel attacks present serious threats to modern computer systems. Using caches as a side channel, these attacks are able to derive secret keys used in cryptographic operations through legitimate activities. Among existing countermeasures, software solutions are typically application specific and incur substantial performance overhead. Recent hardware proposals including the Partition-Locked cache (PLcache) and Random-Permutation cache (RPcache) [23], although very effective in reducing performance overhead while enhancing the security level, may still be vulnerable to advanced cache attacks. In this paper, we propose three hardware-software approaches to defend against software cache-based attacks - they present different tradeoffs between hardware complexity and performance overhead. First, we propose to use preloading to secure the PLcache. Second, we leverage informing loads, which is a lightweight architectural support originally proposed to improve memory performance, to protect the RPcache. Third, we propose novel software permutation to replace the random permutation hardware in the RPcache. This way, regular caches can be protected with hardware support for informing loads. In our experiments, we analyze various processor models for their vulnerability to cache attacks and demonstrate that even to the processor model that is most vulnerable to cache attacks, our proposed software-hardware integrated schemes provide strong security protection.

A central tool in constructing pseudorandom generators, secure encryption functions, and in other areas are “hard-core” predicates <italic>b</italic> of functions (permutations) ƒ, discovered in [Blum Micali 82]. Such <italic>b</italic>(<italic>x</italic>) cannot be efficiently guessed (substantially better than 50-50) given only ƒ(<italic>x</italic>). Both <italic>b</italic>, ƒ are computable in polynomial time.
[Yao 82] transforms any one-way function ƒ into a more complicated one, ƒ<supscrpt>*</supscrpt>, which has a hard-core predicate. The construction applies the original ƒ to many small pieces of the input to ƒ<supscrpt>*</supscrpt> just to get one “hard-core” bit. The security of this bit may be smaller than any constant positive power of the security of ƒ. In fact, for inputs (to ƒ<supscrpt>*</supscrpt>) of practical size, the pieces effected by ƒ are so small that ƒ can be inverted (and the “hard-core” bit computed) by exhaustive search.
In this paper we show that every one-way function, padded to the form ƒ(<italic>p</italic>, <italic>x</italic>) = (<italic>p</italic>, <italic>g</italic>(<italic>x</italic>)), ‖‖<italic>p</italic>‖‖ = ‖<italic>x</italic>‖, has by itself a hard-core predicate of the same (within a polynomial) security. Namely, we prove a conjecture of [Levin 87, sec. 5.6.2] that the scalar product of Boolean vectors <italic>p</italic>, <italic>x</italic> is a hard-core of every one-way function ƒ(<italic>p</italic>, <italic>x</italic>) = (<italic>p</italic>, <italic>g</italic>(<italic>x</italic>)). The result extends to multiple (up to the logarithm of security) such bits and to any distribution on the <italic>x</italic>'s for which ƒ is hard to invert.

This research study proposes Hybrid Encryption System using new public key algorithm and private key algorithm. A hybrid cryptosystem is one which combines the convenience of a public-key cryptosystem with the efficiency of a symmetrickey cryptosystem. Here, we propose a provably two way secured data encryption system, which addresses the concerns of user’s privacy, authentication and accuracy. This system has two different encryption algorithms have been used both in the Encryption and decryption sequence. One is public key cryptography based on linear block cipher another one is private key cryptography based on simple symmetric algorithm. This cryptography algorithm provides more security as well as authentication comparing to other existing hybrid algorithm.

Encryption is the process of scrambling a message so that only the intended recipient can read it. With the fast progression of digital data exchange in electronic way, Information Security is becoming much more important in data storage and transmission. Caesar cipher is a mono alphabetic cipher. It is a type of substitution cipher in which each letter in the plaintext is replaced by a letter. In this paper, author modified the traditional Caesar cipher and fixed the key size as one. Another thing alphabet index is checked if the alphabet index is even then increase the value by one else alphabet index is odd decrease the key value by one. Encryption and scrambling of the letters in the Cipher Text.

New Cryptographic protocols which take full advantage of the unique properties of public key cryptosystems are now evolving. Several protocols for public key distribution and for digital signatures are briefly compared with each other and with the conventional alternative.

In an attribute-based encryption (ABE) scheme, a ciphertext is associated with an ℓ-bit public index ind and a message m, and a secret key is associated with a Boolean predicate P. The secret key allows decrypting the ciphertext and learning m if and only if P(ind) = 1. Moreover, the scheme should be secure against collusions of users, namely, given secret keys for polynomially many predicates, an adversary learns nothing about the message if none of the secret keys can individually decrypt the ciphertext.
 We present attribute-based encryption schemes for circuits of any arbitrary polynomial size, where the public parameters and the ciphertext grow linearly with the depth of the circuit. Our construction is secure under the standard learning with errors (LWE) assumption. Previous constructions of attribute-based encryption were for Boolean formulas, captured by the complexity class NC1.
 In the course of our construction, we present a new framework for constructing ABE schemes. As a by-product of our framework, we obtain ABE schemes for polynomial-size branching programs, corresponding to the complexity class LOGSPACE, under quantitatively better assumptions.

The one-time pad, the mother of all encryption schemes, is well known to be information-theoretically secure, in contrast to most encryption schemes used in practice, which are at most computationally secure. In this paper, we focus on another, completely different aspect in which the one-time pad is superior to normal encryption, and which surfaces only when the receiver (not only the eavesdropper) is considered potentially dishonest, as can be the case in a larger protocol context in which encryption is used as a sub-protocol. For example, such a dishonest receiver (who is, say, coerced by the eavesdropper) can in normal encryption verifiably leak the message to the eavesdropper by revealing the secret key. While this leakage feature can provably not be avoided completely, it is more limited if the one-time pad is used. We use the constructive cryptography framework to make these statements precise.