Question: A biologically plausible deep learning algorithm for neural networks in the brain.

This study reveals the impact of feature-based attention on motion processing within the macaque visual cortex, extending our understanding of how attention modifies neural responses beyond spatial focus. Prior research has established that attention enhances neural responses to stimuli at attended locations while diminishing those elsewhere, underscoring attention's neural correlate as an amplification mechanism. Our investigation introduces a non-spatial dimension to attentional modulation by illustrating how attention specifically augments the gain of direction-selective neurons in the MT visual cortical area, without altering the breadth of their direction-tuning curves. This finding offers crucial insights into the neural mechanisms underlying attention and supports the development of a comprehensive 'feature similarity gain model' of attention, which integrates the effects of spatial location, motion direction, and other characteristics of attended stimuli into a unified framework.

Giving robots the ability to classify surface textures requires appropriate sensors and algorithms. Inspired by the biology of human tactile perception, we implement a neurorobotic texture classifier with a recurrent spiking neural network, using a novel semisupervised approach for classifying dynamic stimuli. Input to the network is supplied by accelerometers mounted on a robotic arm. The sensor data are encoded by a heterogeneous population of neurons, modeled to match the spiking activity of mechanoreceptor cells. This activity is convolved by a hidden layer using bandpass filters to extract nonlinear frequency information from the spike trains. The resulting high-dimensional feature representation is then continuously classified using a neurally implemented support vector machine. We demonstrate that our system classifies 18 metal surface textures scanned in two opposite directions at a constant velocity. We also demonstrate that our approach significantly improves upon a baseline model that does not use the described feature extraction. This method can be performed in real-time using neuromorphic hardware, and can be extended to other applications that process dynamic stimuli online.

To perform nontrivial, real-time computations on a sensory input stream, biological systems must retain a short-term memory trace of their recent inputs. It has been proposed that generic high-dimensional dynamical systems could retain a memory trace for past inputs in their current state. This raises important questions about the fundamental limits of such memory traces and the properties required of dynamical systems to achieve these limits. We address these issues by applying Fisher information theory to dynamical systems driven by time-dependent signals corrupted by noise. We introduce the Fisher Memory Curve (FMC) as a measure of the signal-to-noise ratio (SNR) embedded in the dynamical state relative to the input SNR. The integrated FMC indicates the total memory capacity. We apply this theory to linear neuronal networks and show that the capacity of networks with normal connectivity matrices is exactly 1 and that of any network of N neurons is, at most, N. A nonnormal network achieving this bound is subject to stringent design constraints: It must have a hidden feedforward architecture that superlinearly amplifies its input for a time of order N, and the input connectivity must optimally match this architecture. The memory capacity of networks subject to saturating nonlinearities is further limited, and cannot exceed square root N. This limit can be realized by feedforward structures with divergent fan out that distributes the signal across neurons, thereby avoiding saturation. We illustrate the generality of the theory by showing that memory in fluid systems can be sustained by transient nonnormal amplification due to convective instability or the onset of turbulence.

Several approaches to implementing symbol-like representations in neurally plausible models have been proposed. These approaches include binding through synchrony (Shastri & Ajjanagadde, ), "mesh" binding (van der Velde & de Kamps, ), and conjunctive binding (Smolensky, ). Recent theoretical work has suggested that most of these methods will not scale well, that is, that they cannot encode structured representations using any of the tens of thousands of terms in the adult lexicon without making implausible resource assumptions. Here, we empirically demonstrate that the biologically plausible structured representations employed in the Semantic Pointer Architecture (SPA) approach to modeling cognition (Eliasmith, ) do scale appropriately. Specifically, we construct a spiking neural network of about 2.5 million neurons that employs semantic pointers to successfully encode and decode the main lexical relations in WordNet, which has over 100,000 terms. In addition, we show that the same representations can be employed to construct recursively structured sentences consisting of arbitrary WordNet concepts, while preserving the original lexical structure. We argue that these results suggest that semantic pointers are uniquely well-suited to providing a biologically plausible account of the structured representations that underwrite human cognition.

1College of Computer Science, Sichuan Normal University, Chengdu, Sichuan, China 2Developmental Neurocognition Lab, Department of Psychological Sciences, Birkbeck, University of London, London, UK 3College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China Correspondence Juan Yang, College of Computer Science, Sichuan Normal University, Chengdu, Sichuan, China. Email: jkxy_yjuan@sicnu.edu.cn Funding information National Natural Science Foundation of China, Grant/Award Number: (61402309); UK Economic and Social Research Council, Grant/Award Number: RES-062-23-2721 Summary Mapping problems are typical research topics related to natural language learning, and they include not only classification mappings but also nonclassification mappings, such as verbs and their past tenses. Connectionist computational models are one of the most popular approaches for simulating those mapping problems; however, their lack of explanatory ability has prevented them from being further used to understand the language learning process. Therefore, the work of extracting rational rules from a connectionist model is as important as simulating the mapping behaviors. Unfortunately, there is no available technique that can be directly applied in those computational models to simulate nonclassification problems. In this paper, an autoencoder-based connectionist computational model is proposed to derive a rule extraction method that can construct “If-Then” rational relations with high fidelity for nonclassification mapping problems. To demonstrate its generalizability, this computational model is extended to a modified version to address a multi-label classification mapping problem related to cognitive style prediction. Experiments prove this computational model's simulation ability and its explanatory ability on nonclassification problems by comparing its fidelity performances with those of the classical connectionist computational model (multilayer perceptron artificial neural network), and its similar ability on a multi-label classification problem (Felder-Silverman learning style classification) by comparing its prediction accuracy with those of other rule induction techniques.

In this review, we compare methods for temporal sequence learning (TSL) across the disciplines machine-control, classical conditioning, neuronal models for TSL as well as spike-timing-dependent plasticity (STDP). This review introduces the most influential models and focuses on two questions: To what degree are reward-based (e.g., TD learning) and correlation-based (Hebbian) learning related? and How do the different models correspond to possibly underlying biological mechanisms of synaptic plasticity? We first compare the different models in an open-loop condition, where behavioral feedback does not alter the learning. Here we observe that reward-based and correlation-based learning are indeed very similar. Machine control is then used to introduce the problem of closed-loop control (e.g., actor-critic architectures). Here the problem of evaluative (rewards) versus nonevaluative (correlations) feedback from the environment will be discussed, showing that both learning approaches are fundamentally different in the closed-loop condition. In trying to answer the second question, we compare neuronal versions of the different learning architectures to the anatomy of the involved brain structures (basal-ganglia, thalamus, and cortex) and the molecular biophysics of glutamatergic and dopaminergic synapses. Finally, we discuss the different algorithms used to model STDP and compare them to reward-based learning rules. Certain similarities are found in spite of the strongly different timescales. Here we focus on the biophysics of the different calcium-release mechanisms known to be involved in STDP.

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.

What if we could effectively read the mind and transfer human visual capabilities to computer vision methods? In this paper, we aim at addressing this question by developing the first visual object classifier driven by human brain signals. In particular, we employ EEG data evoked by visual object stimuli combined with Recurrent Neural Networks (RNN) to learn a discriminative brain activity manifold of visual categories in a reading the mind effort. Afterward, we transfer the learned capabilities to machines by training a Convolutional Neural Network (CNN)–based regressor to project images onto the learned manifold, thus allowing machines to employ human brain–based features for automated visual classification. We use a 128-channel EEG with active electrodes to record brain activity of several subjects while looking at images of 40 ImageNet object classes. The proposed RNN-based approach for discriminating object classes using brain signals reaches an average accuracy of about 83%, which greatly outperforms existing methods attempting to learn EEG visual object representations. As for automated object categorization, our human brain–driven approach obtains competitive performance, comparable to those achieved by powerful CNN models and it is also able to generalize over different visual datasets.

Much has been learned about plasticity of biological synapses from empirical studies. Hebbian plasticity is driven by correlated activity of presynaptic and postsynaptic neurons. Synapses that converge onto the same neuron often behave as if they compete for a fixed resource; some survive the competition while others are eliminated. To provide computational interpretations of these aspects of synaptic plasticity, we formulate unsupervised learning as a zero-sum game between Hebbian excitation and anti-Hebbian inhibition in a neural network model. The game formalizes the intuition that Hebbian excitation tries to maximize correlations of neurons with their inputs, while anti-Hebbian inhibition tries to decorrelate neurons from each other. We further include a model of synaptic competition, which enables a neuron to eliminate all connections except those from its most strongly correlated inputs. Through empirical studies, we show that this facilitates the learning of sensory features that resemble parts of objects. Much has been learned about plasticity of biological synapses from empirical studies. One line of research has explored Hebbian plasticity, which is triggered by correlated presynaptic and postsynaptic activity. Many kinds of excitatory synapses are strengthened by correlated activity, and are said to be Hebbian [Bi and Poo, 2001]. Strengthening of inhibitory synapses by correlated activity has also been observed [Gaiarsa et al., 2002]. The phenomenon is sometimes said to be anti-Hebbian, because strengthening an inhibitory synapse is like making a negative number more negative. Another line of research has shown that synapses converging onto the same neuron often behave as if they compete for shares of a fixed “pie”; some survive the competition while others are eliminated [Lichtman and Colman, 2000]. According to this viewpoint, Hebbian plasticity does not increase or decrease the pie of synaptic resources; it only allocates resources across convergent synapses [Miller, 1996] Theoretical neuroscientists have proposed a number of computational functions for Hebbian excitation, anti-Hebbian inhibition, and synaptic competition/elimination. Hebbian excitation has long been invoked as a mechanism for learning features from sensory input [Von der Malsburg, 1973]. Lateral inhibition has been used to sparsen neural activity, thereby facilitating Hebbian feature learning [Von der Malsburg, 1973, Fukushima, 1980, Rumelhart and Zipser, 1985, Kohonen, 1990]. Endowing the lateral inhibition with anti-Hebbian plasticity can give more robust control over sparseness of activity. Földiák [1990] demonstrated this with numerical experiments, but did not provide an interpretation in terms of an optimization principle. Less relevant here are antiHebbian models without reference to sparse activity in linear [Foldiak, 1989, Rubner and Tavan, 1989, Rubner and Schulten, 1990] and nonlinear [Carlson, 1990, Girolami and Fyfe, 1997] networks. Also less relevant is the application of anti-Hebbian plasticity to feedforward rather than lateral connections [Hyvärinen and Oja, 1998]. 1 ar X iv :1 70 4. 00 64 6v 1 [ cs .N E ] 3 A pr 2 01 7 Leen [1991] performed a stability analysis for a linear network with Hebbian feedforward connections and anti-Hebbian lateral connections. Plumbley [1993] derived a linear network with antiHebbian lateral inhibition (but no plasticity of feedforward connections) from the principle of information maximization with a power constraint. Pehlevan et al. [2015] showed that a linear network with Hebbian feedforward connections and anti-Hebbian lateral inhibition can be interpreted as online gradient optimization of a “similarity matching” cost function. Pehlevan and Chklovskii [2014] and Hu et al. [2014] went on to extend the similarity matching principle to derive nonlinear neural networks for unsupervised learning. Synaptic competition and elimination have been studied in models of cortical development, and have been shown to play an important role in the emergence of feature selectivity [Miller, 1996]. The subject of the present work is a mathematical formalism that provides computational interpretations of Hebbian excitation, anti-Hebbian inhibition, and synaptic competition/elimination in nonlinear neural networks. We start by formulating unsupervised learning as the maximization of output-input correlations subject to upper bound constraints on output-output correlations. We motivate our formulation by describing its relation to previous theoretical frameworks for unsupervised learning, such as maximization [Linsker, 1988, Atick and Redlich, 1990, Plumbley, 1993, Bell and Sejnowski, 1995] or minimization of mutual information [Hyvärinen and Oja, 2000], (2) projection onto a subspace that maximizes a moment-based statistic such as variance [Oja, 1982, Linsker, 1988] or kurtosis [Huber, 1985], and the (3) similarity matching principle [Pehlevan et al., 2015]. To solve our constrained maximization problem, we introduce Lagrange multipliers. This Lagrangian dual formulation of unsupervised learning can in turn be solved by a nonlinear neural network with Hebbian excitation and anti-Hebbian inhibition. The network is very similar to the original model of Földiák [1990], differing mainly by its use of rectification rather than sigmoidal nonlinearity. (The latter can also be handled by our formalism, as shown in Appendix B.) Lagrange multipliers were also used to study anti-Hebbian plasticity by Plumbley [1993], but only for linear networks. Effectively, excitation and inhibition behave like players in a game, and the inhibitory connections can be interpreted as Lagrange multipliers. The game is zero-sum, in that excitation tries to maximize a payoff function and inhibition tries to minimize exactly the same payoff function. Roughly speaking, however, one could say that excitation aims to maximize the correlation of each output neuron with its inputs, while inhibition aims to decorrelate the output neurons from each other. Our term “correlation game” is derived from this intuitive picture. Within our mathematical formalism, we also consider a dynamics of synaptic competition and elimination that is drawn from models of cortical development [Miller and MacKay, 1994]. Competition between the excitatory synapses convergent on a single neuron is capable of driving the strengths of some synapses to zero. In numerical experiments with the MNIST dataset, we show that synapse elimination has the computational function of facilitating the learning of features that resemble “parts” of objects. Theoretical analysis shows that the surviving synapses converging onto an output neuron come from its most strongly correlated inputs; synapses from weakly correlated inputs are eliminated. Our correlation game is closely related to the similarity matching principle of Pehlevan and Chklovskii [2014] and Pehlevan et al. [2015]. The major novelty is the introduction of decorrelation as a constraint for the optimization. Paralleling our work, Pehlevan et al. [2017] have shown that the similarity matching principle leads to a game theoretic formulation through Hubbard-Stratonovich duality. Again our novelty is the use of decorrelation as a constraint, which leads to our correlation game through Lagrangian duality. Our model of synaptic competition and elimination was borrowed with slight modification from the literature on modeling neural development [Miller and MacKay, 1994]. It can be viewed as a more biologically plausible alternative to previous unsupervised learning algorithms that sparsen features. For example, Hoyer [2004] is similar to ours because it can be interpreted as independently sparsening each set of convergent synapses, rather than applying a global L1 regularizer to all synapses.

The first successful FPGA implementation [1] of artificial neural networks (ANNs) was published a little over a decade ago. It is timely to review the progress that has been made in this research area. This brief survey provides a taxonomy for classifying FPGA implementations of ANNs. Different implementation techniques and design issues are discussed. Future research trends are also presented.

Approximation properties of a class of artificial neural networks are established. It is shown that feedforward networks with one layer of sigmoidal nonlinearities achieve inte grated squared error of order O(l/n), where n is the number of nodes. The function appruximated is assumed to have a bound on the first moment of the magnitude distribution of the Fourier transform. The nonlinear parameters associated with the sigmoidal nodes, as well as the parameters of linear combination, are adjusted in the approximation. In contrast, it is shown that for series expansions with n terms, in which only the parameters of linear combination are adjusted, the integrated squared approximation error cannot be made smaller than order 1/n2/d uniformly for functions satisfying the same smoothness assumption, where d is the dimension of the input to the function. For the class of functions examined here, the approximation rate and the parsimony of the parameterization of the networks are surprisingly advantageous in high-dimensional settings.


Question: A biologically plausible deep learning algorithm for neural learning processes in the brain.

This paper explores the application of gradient-based learning techniques to the realm of document recognition, demonstrating a significant advancement in the field. Unlike traditional learning methods, the approach presented here employs a gradient-based mechanism, which is capable of optimizing the parameters of a model through the computation of gradients. This method has been particularly effective in enhancing the performance of document recognition systems, enabling them to accurately identify and interpret various forms of textual information. By leveraging gradient-based learning, the system can adapt to different document types, handwriting styles, and even subtle variations in text formatting, making it highly versatile and efficient. The success of this approach in document recognition underscores its potential for broader application in pattern recognition tasks, suggesting that gradient-based learning could be a key component in developing more sophisticated and robust artificial intelligence systems capable of understanding complex visual and textual data. This research not only contributes to the field of document recognition but also opens up new possibilities for the application of gradient-based learning in other areas of artificial intelligence, potentially leading to breakthroughs in machine learning and computer vision.

This paper explores the complexity of deep feedforward networks with linear presynaptic couplings and rectified linear activations. This is a contribution to the growing body of work contrasting the representational power of deep and shallow network architectures. In particular, we offer a framework for comparing deep and shallow models that belong to the family of piecewise linear functions based on computational geometry. We look at a deep rectifier multi-layer perceptron (MLP) with linear outputs units and compare it with a single layer version of the model. In the asymptotic regime, when the number of inputs stays constant, if the shallow model has kn hidden units and n0 inputs, then the number of linear regions is O(k0n0). For a k layer model with n hidden units on each layer it is Ω(bn/n0c n0). The number bn/n0c grows faster than k0 when n tends to infinity or when k tends to infinity and n ≥ 2n0. Additionally, even when k is small, if we restrict n to be 2n0, we can show that a deep model has considerably more linear regions that a shallow one. We consider this as a first step towards understanding the complexity of these models and specifically towards providing suitable mathematical tools for future analysis.

Deep Learning has attracted significant attention in recent years. Here I present a brief overview of my first Deep Learner of 1991, and its historic context, with a timeline of Deep Learning highlights. Note: As a machine learning researcher I am obsessed with proper credit assignment. This draft is the result of an experiment in rapid massive open online peer review. Since 20 September 2013, subsequent revisions published under www.deeplearning.me have absorbed many suggestions for improvements by experts. The abbreviation “TL” is used to refer to subsections of the timeline section. Figure 1: My first Deep Learning system of 1991 used a deep stack of recurrent neural networks (a Neural Hierarchical Temporal Memory) pre-trained in unsupervised fashion to accelerate subsequent supervised learning [79, 81, 82]. 1 ar X iv :1 31 2. 55 48 v1 [ cs .N E ] 1 9 D ec 2 01 3 In 2009, our Deep Learning Artificial Neural Networks became the first Deep Learners to win official international pattern recognition competitions [40, 83] (with secret test sets known only to the organisers); by 2012 they had won eight of them (TL 1.13), including the first contests on object detection in large images (ICPR 2012) [2, 16] and image segmentation (ISBI 2012) [3, 15]. In 2011, they achieved the world’s first superhuman visual pattern recognition results [20, 19]. Others have implemented very similar techniques, e.g., [51], and won additional contests or set benchmark records since 2012, e.g., (TL 1.13, TL 1.14). The field of Deep Learning research is far older though—compare the timeline (TL) further down. My first Deep Learner dates back to 1991 [79, 81, 82] (TL 1.7). It can perform credit assignment across hundreds of nonlinear operators or neural layers, by using unsupervised pre-training for a stack of recurrent neural networks (RNN) (deep by nature) as in Figure 1. Such RNN are general computers more powerful than normal feedforward NN, and can encode entire sequences of input vectors. The basic idea is still relevant today. Each RNN is trained for a while in unsupervised fashion to predict its next input. From then on, only unexpected inputs (errors) convey new information and get fed to the next higher RNN which thus ticks on a slower, self-organising time scale. It can easily be shown that no information gets lost. It just gets compressed (much of machine learning is essentially about compression). We get less and less redundant input sequence encodings in deeper and deeper levels of this hierarchical temporal memory, which compresses data in both space (like feedforward NN) and time. There also is a continuous variant [84]. One ancient illustrative Deep Learning experiment of 1993 [82] required credit assignment across 1200 time steps, or through 1200 subsequent nonlinear virtual layers. The top level code of the initially unsupervised RNN stack, however, got so compact that (previously infeasible) sequence classification through additional supervised learning became possible. There is a way of compressing higher levels down into lower levels, thus partially collapsing the hierarchical temporal memory. The trick is to retrain lower-level RNN to continually imitate (predict) the hidden units of already trained, slower, higher-level RNN, through additional predictive output neurons [81, 79, 82]. This helps the lower RNN to develop appropriate, rarely changing memories that may bridge very long time lags. The Deep Learner of 1991 was a first way of overcoming the Fundamental Deep Learning Problem identified and analysed in 1991 by my very first student (now professor) Sepp Hochreiter (TL 1.6): the problem of vanishing or exploding gradients [46, 47, 10]. The latter motivated all our subsequent Deep Learning research of the 1990s and 2000s. Through supervised LSTM RNN (1997) (e.g., [48, 32, 39, 36, 37, 40, 38, 83], TL 1.8) we could eventually perform similar feats as with the 1991 system [81, 82], overcoming the Fundamental Deep Learning Problem without any unsupervised pre-training. Moreover, LSTM could also learn tasks unlearnable by the partially unsupervised 1991 chunker [81, 82]. Particularly successful are stacks of LSTM RNN [40] trained by Connectionist Temporal Classification (CTC) [36]. On faster computers of 2009, this became the first RNN system ever to win an official international pattern recognition competition [40, 83], through the work of my PhD student and postdoc Alex Graves, e.g., [40]. To my knowledge, this also was the first Deep Learning system ever (recurrent or not) to win such a contest (TL 1.10). (In fact, it won three different ICDAR 2009 contests on connected handwriting in three different languages, e.g., [83, 40], TL 1.10.) A while ago, Alex moved on to Geoffrey Hinton’s lab (Univ. Toronto), where a stack [40] of our bidirectional LSTM RNN [39] also broke a famous TIMIT speech recognition record [38] (TL 1.14), despite thousands of man years previously spent on HMM-based speech recognition research. CTC-LSTM also helped to score first at NIST’s OpenHaRT2013 evaluation [11]. Recently, well-known entrepreneurs also got interested [43, 52] in such hierarchical temporal memories [81, 82] (TL 1.7). The expression Deep Learning actually got coined relatively late, around 2006, in the context of unsupervised pre-training for less general feedforward networks [44] (TL 1.9). Such a system reached 1.2% error rate [44] on the MNIST handwritten digits [54, 55], perhaps the most famous benchmark of Machine Learning. Our team first showed that good old backpropagation (TL 1.2) on GPUs (with training pattern distortions [6, 86] but without any unsupervised pre-training) can actually achieve a three times better result of 0.35% [17] back then, a world record (a previous standard net achieved 0.7% [86]; a backprop-trained [54, 55] Convolutional NN (CNN or convnet) [29, 30, 54, 55] got 0.39% [70](TL 1.9);

This paper describes an ultralow-power neural recording amplifier. The amplifier appears to be the lowest power and most energy-efficient neural recording amplifier reported to date. We describe low-noise design techniques that help the neural amplifier achieve input-referred noise that is near the theoretical limit of any amplifier using a differential pair as an input stage. Since neural amplifiers must include differential input pairs in practice to allow robust rejection of common-mode and power supply noise, our design appears to be near the optimum allowed by theory. The bandwidth of the amplifier can be adjusted for recording either neural spikes or local field potentials (LFPs). When configured for recording neural spikes, the amplifier yielded a midband gain of 40.8 dB and a -3-dB bandwidth from 45 Hz to 5.32 kHz; the amplifier's input-referred noise was measured to be 3.06 muVrms while consuming 7.56 muW of power from a 2.8-V supply corresponding to a noise efficiency factor (NEF) of 2.67 with the theoretical limit being 2.02. When configured for recording LFPs, the amplifier achieved a midband gain of 40.9 dB and a -3-dB bandwidth from 392 mHz to 295 Hz; the input-referred noise was 1.66 muVrms while consuming 2.08 muW from a 2.8-V supply corresponding to an NEF of 3.21. The amplifier was fabricated in AMI's 0.5-mum CMOS process and occupies 0.16 mm2 of chip area. We obtained successful recordings of action potentials from the robust nucleus of the arcopallium (RA) of an anesthesized zebra finch brain with the amplifier. Our experimental measurements of the amplifier's performance including its noise were in good accord with theory and circuit simulations.

-A neural network learning procedure has been applied to the classification ~/sonar returns [kom two undersea targets, a metal cylinder and a similarly shaped rock. Networks with an intermediate layer ~/ hidden processing units achieved a classification accuracy as high as 100% on a training set of l04 returns. These net~orks correctly classified up to 90.4% of 104 test returns not contained in the training set. This perfi~rmance was better than that of a nearest neighbor classifier, which was 82.7%. and was close to that of an optimal Bayes classifie~ Specific signal features extracted by hidden units in a trained network were identified and related to coding schemes in the pattern of connection strengths between the input and the hidden units. Network perlbrmance and class[/~cation strategy was comparable to that of trained human listeners. Keywords--Learning algorithms, Hidden units. Multilayered neural network, Sonar, Signal processing.

Statistical machine learning is concerned with learning models that describe observations. We train our models from data on tasks like machine translation or object recognition because we cannot explicitly write down programs to solve such problems. A statistical model is only useful when it generalizes to unseen data. Solomonoff114 has proved that one should choose the model that agrees with the observed data, while preferring the model that can be compressed the most, because such a choice guarantees the best possible generalization. The size of the best possible compression of the model is called the Kolmogorov complexity of the model. We define an algorithm as a function with small Kolmogorov complexity. This Ph.D. thesis outlines the problem of learning algorithms from data and shows several partial solutions to it. Our data model is mainly neural networks as they have proven to be successful in various domains like object recognition67,109,122, language modelling90, speech recognition48,39 and others. First, we examine empirical trainability limits for classical neural networks. Then, we extend them by providing interfaces, which provide a way to read memory, access the input, and postpone predictions. The model learns how to use them with reinforcement learning techniques like REINFORCE and Q-learning. Next, we ex-

OBJECTIVE
Brain-computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional neural networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible.

APPROACH
In this work we introduce EEGNet, a compact convolutional neural network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR).

MAIN RESULTS
We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, the reference algorithms when only limited training data is available across all tested paradigms. In addition, we demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features.

SIGNIFICANCE
Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks. Our models can be found at: https://github.com/vlawhern/arl-eegmodels.

Human agents draw a variety of inferences effortlessly, spontaneously, and with remarkable efficiency — as though these inferences are a reflex response of their cognitive apparatus. Furthermore, these inferences are drawn with reference to a large body of background knowledge. This remarkable human ability seems paradoxical given the results about the complexity of reasoning reported by researchers in artificial intelligence. It also poses a challenge for cognitive science and computational neuroscience: How can a system of simple and slow neuron-like elements represent a large body of systematic knowledge and perform a range of inferences with such speed? We describe a computational model that is a step toward addressing the cognitive science challenge and resolving the artificial intelligence paradox. We show how a connectionist network can encode millions of facts and rules involving n-ary predicates and variables, and perform a class of inferences in a few hundred msec. Efficient reasoning requires the rapid representation and propagation of dynamic bindings. Our model achieves this by i) representing dynamic bindings as the synchronous firing of appropriate nodes, ii) rules as interconnection patterns that direct the propagation of rhythmic activity, iii) and long-term facts as temporal pattern-matching sub-networks. The model is consistent with recent neurophysiological findings which suggest that synchronous activity occurs in the brain and may play a representational role in neural information processing. The model also makes specific predictions about the nature of reflexive reasoning that are psychologically significant. It identifies constraints on the form of rules that may participate in such reasoning and relates the capacity of the working memory underlying reflexive reasoning to biological parameters such as the lowest frequency at which nodes can sustain synchronous oscillations and the coarseness of synchronization.

Neuromorphic algorithms, which are comprised of highly complex, large-scale networks of artificial neurons, are increasingly used for a variety of recognition, classification, search and vision tasks. However, their computational and energy requirements can be quite high, and hence their energy-efficient implementation is of great interest.
 We propose a new approach to design energy-efficient hardware implementations of large-scale neural networks (NNs) using approximate computing. Our work is motivated by the observations that (i) NNs are used in applications where less-than-perfect results are acceptable, and often inevitable, and (ii) they are highly resilient to inexactness in many (but not all) of their constituent computations. We make two key contributions. First, we propose a method to transform any given NN into an Approximate Neural Network (AxNN). This is performed by (i) adapting the backpropagation technique, which is commonly used to train these networks, to quantify the impact of approximating each neuron to the overall network quality (e.g., classification accuracy), and (ii) selectively approximating those neurons that impact network quality the least. Further, we make the key observation that training is a naturally error-healing process that can be used to mitigate the impact of approximations to neurons. Therefore, we incrementally retrain the network with the approximations in-place, reclaiming a significant portion of the quality ceded by approximations. As a second contribution, we propose a programmable and quality-configurable neuromorphic processing engine (qcNPE), which utilizes arrays of specialized processing elements that execute neuron computations with dynamically configurable accuracies and can be used to execute AxNNs from diverse applications. We evaluated the proposed approach by constructing AXNNs for 6 recognition applications (ranging in complexity from 12-47,818 neurons and 160-3,155,968 connections) and executing them on two different platforms--qcNPE implementation containing 272 processing elements in 45nm technology and a commodity Intel Xeon server. Our results demonstrate 1.14X-1.92X energy benefits for virtually no loss (< 0.5%) in output quality, and even higher improvements (upto 2.3X) when some loss (upto 7.5%) in output quality is acceptable.

We report the design of an ultra-low-power 32-channel neural-recording integrated circuit (chip) in a 0.18 μ m CMOS technology. The chip consists of eight neural recording modules where each module contains four neural amplifiers, an analog multiplexer, an A/D converter, and a serial programming interface. Each amplifier can be programmed to record either spikes or LFPs with a programmable gain from 49-66 dB. To minimize the total power consumption, an adaptive-biasing scheme is utilized to adjust each amplifier's input-referred noise to suit the background noise at the recording site. The amplifier's input-referred noise can be adjusted from 11.2 μVrms (total power of 5.4 μW) down to 5.4 μVrms (total power of 20 μW) in the spike-recording setting. The ADC in each recording module digitizes the a.c. signal input to each amplifier at 8-bit precision with a sampling rate of 31.25 kS/s per channel, with an average power consumption of 483 nW per channel, and, because of a.c. coupling, allows d.c. operation over a wide dynamic range. It achieves an ENOB of 7.65, resulting in a net efficiency of 77 fJ/State, making it one of the most energy-efficient designs for neural recording applications. The presented chip was successfully tested in an in vivo wireless recording experiment from a behaving primate with an average power dissipation per channel of 10.1 μ W. The neural amplifier and the ADC occupy areas of 0.03 mm2 and 0.02 mm2 respectively, making our design simultaneously area efficient and power efficient, thus enabling scaling to high channel-count systems.

We present the soft exponential activation function for artificial neural networks that continuously interpolates between logarithmic, linear, and exponential functions. This activation function is simple, differentiable, and parameterized so that it can be trained as the rest of the network is trained. We hypothesize that soft exponential has the potential to improve neural network learning, as it can exactly calculate many natural operations that typical neural networks can only approximate, including addition, multiplication, inner product, distance, and sinusoids.


Question: Interfering with the delivery of blocks and transactions within the Bitcoin network.

From Bitter to Better: Enhancing Bitcoin's Currency Potential
As a decentralized digital currency, Bitcoin has amassed a significant user base, achieving a level of success that eluded previous cryptographic e-cash systems. To understand Bitcoin's triumph and its potential for longevity as a stable currency, we conduct a thorough analysis. In the process, we pinpoint various challenges and vulnerabilities within the Bitcoin ecosystem, including attacks that could undermine its integrity. To fortify Bitcoin's position as a viable and enduring currency, we propose a set of innovative solutions aimed at mitigating these issues. By addressing these concerns, we envision a refined version of Bitcoin that can maintain stability and security, enhancing its appeal to a broader audience.

With the growing realization that current Internet protocols are reaching the limits of their senescence, several ongoing research efforts aim to design potential next-generation Internet architectures. Although they vary in maturity and scope, in order to avoid past pitfalls, these efforts seek to treat security and privacy as fundamental requirements. Resilience to Denialof-Service (DoS) attacks that plague today’s Internet is a major issue for any new architecture and deserves full attention. In this paper, we focus on DoS in Named Data Networking (NDN) – a specific candidate for next-generation Internet architecture designs. By naming data instead of its locations, NDN transforms data into a first-class entity and makes itself an attractive and viable approach to meet the needs for many current and emerging applications. It also incorporates some basic security features that mitigate classes of attacks that are commonly seen today. However, NDN’s resilience to DoS attacks has not been analyzed to-date. This paper represents a first step towards assessment and possible mitigation of DoS in NDN. After identifying and analyzing several new types of attacks, it investigates their variations, effects and counter-measures. This paper also sheds some light on the debate about relative virtues of self-certifying, as opposed to human-readable, names in the context of content-centric networking.

Various cryptographic puzzle schemes are proposed as a defense mechanism against denial of service attack. But, all these puzzle schemes face a dilemma when there is a large disparity between the computational power of attackers and legitimate clients: increasing the difficulty of puzzles might unnecessarily restrict legitimate clients too much, and lower difficulty puzzles cannot sufficiently block attackers with large computational resources. In this paper, we introduce guided tour puzzle, a novel puzzle scheme that is not affected by such resource disparity. A guided tour puzzle requires a client to visit a predefined set of nodes, called tour guides, in a certain sequential order to retrieve an n piece answer, one piece from each tour guide that appears in the tour. This puzzle solving process is non-parallelizable, thus cheating by trying to solve the puzzle in parallel is not possible. Guided tour puzzle not only achieves all previously defined desired properties of a cryptographic puzzle scheme, but it also satisfies more important requirements, such as puzzle fairness and minimum interference, that we identified. The number of tour guides required by the scheme can be as few as two, and this extra cost can be amortized by sharing the same set of tour guides among multiple servers.

As Multiprocessor System-on-Chips (MPSoCs) continue to scale, security for Network-on-Chips (NoCs) is a growing concern as rogue agents threaten to infringe on the hardware's trust and maliciously implant Hardware Trojans (HTs) to undermine their reliability. The trustworthiness of MPSoCs will rely on our ability to detect Denial-of-Service (DoS) threats posed by the HTs and mitigate HTs in a compromised NoC to permit graceful network degradation. In this paper, we propose a new light-weight target-activated sequential payload (TASP) HT model that performs packet inspection and injects faults to create a new type of DoS attack. Faults injected are used to trigger a response from error correction code (ECC) schemes and cause repeated retransmission to starve network resources and create deadlocks capable of rendering single-application to full chip failures. To circumvent the threat of HTs, we propose a heuristic threat detection model to classify faults and discover HTs within compromised links. To prevent further disruption, we propose several switch-to-switch link obfuscation methods to avoid triggering of HTs in an effort to continue using links instead of rerouting packets with minimal overhead (1-3 cycles). Our proposed modifications complement existing fault detection and obfuscation methods and only adds 2% in area overhead and 6% in excess power consumption in the NoC micro-architecture.

1 Manuscript submitted to ICDCS 2005 on October 8, 2004. All rights reserved by the authors. This research was supported by an NSF ITR Research Grant under contract number ACI-0325409. Corresponding Author: Kai Hwang, Email: kaihwang@usc.edu, Tel: 213-740-4470, Fax: 213-740-4418. Abstract—High availability in network services is crucial for effective large-scale distributed computing. While denial-ofservice (DoS) attacks through massive packet flooding have baffled researchers for years, a new type of even more detrimental attack—shrew attacks (periodic intensive packet bursts with low average rate)—has recently been identified. Shrew attacks can significantly degrade well-behaved TCP sessions and are very difficult to detect, not to mention defend against, due to its low average rate. We propose a new stateful adaptive queue management technique called HAWK (Halting Anomaly with Weighted choKing) which works by judiciously identifying malicious shrew packet flows using a small flow table and dropping such packets decisively to halt the attack such that well-behaved TCP sessions can re-gain their bandwidth shares. Our NS-2 based extensive performance results indicate that HAWK is highly agile in the sense that it can swiftly (e.g., within 5 seconds) identify and then halt shrew attacks with a flow table of tiny size (e.g., 600 bytes). Thus, HAWK is readily amenable to practical implementation in a real environment.

If Bitcoin becomes the prevalent payment system on the Internet, crime fighters will join forces with regulators and enforce blacklisting of transaction prefixes at the parties who offer real products and services in exchange for bitcoin. Blacklisted bitcoins will be hard to spend and therefore less liquid and less valuable. This requires every recipient of Bitcoin payments not only to check all incoming transactions for possible blacklistings, but also to assess the risk of a transaction being blacklisted in the future. We elaborate this scenario, specify a risk model, devise a prediction approach using public knowledge, and present preliminary results using data from selected known thefts. We discuss the implications on markets where bitcoins are traded and critically revisit Bitcoin’s ability to serve as a unit of account.

This paper presents efficient off-line anonymous e-cash schemes where a user can withdraw a wallet containing 2 coins each of which she can spend unlinkably. Our first result is a scheme, secure under the strong RSA and the y-DDHI assumptions, where the complexity of the withdrawal and spend operations is O( + k) and the user’s wallet can be stored using O( +k) bits, where k is a security parameter. The best previously known schemes require at least one of these complexities to be O(2 · k). In fact, compared to previous e-cash schemes, our whole wallet of 2 coins has about the same size as one coin in these schemes. Our scheme also offers exculpability of users, that is, the bank can prove to third parties that a user has double-spent. We then extend our scheme to our second result, the first e-cash scheme that provides traceable coins without a trusted third party. That is, once a user has double spent one of the 2 coins in her wallet, all her spendings of these coins can be traced. However, the price for this is that the complexity of the spending and of the withdrawal protocols becomes O( · k) and O( · k + k) bits, respectively, and wallets take O( · k) bits of storage. All our schemes are secure in the random oracle model.

The blockchain-based digital content distribution system was developed. Decentralized and pear-to-pear authentication mechanism can be considered as the ideal rights management mechanism. The blockchain has the potential to realize this ideal content distribution system. This is the successful model of the Superdistribution concept which was announced almost 30 years ago. The proposed system was demonstrated and got a lot of feedback for the future practical system.

In 2008, Beck and Tews have proposed a practical attack on WPA. Their attack (called the Beck-Tews attack) can recover plaintext from an encrypted short packet, and can falsify it. The execution time of the Beck-Tews attack is about 12-15 minutes. However, the attack has the limitation, namely, the targets are only WPA implementations those support IEEE802.11e QoS features. In this paper, we propose a practical message falsification attack on any WPA implementation. In order to ease targets of limitation of wireless LAN products, we apply the Beck-Tews attack to the man-in-the-middle attack. In the man-inthe-middle attack, the user’s communication is intercepted by an attacker until the attack ends. It means that the users may detect our attack when the execution time of the attack is large. Therefore, we give methods for reducing the execution time of the attack. As a result, the execution time of our attack becomes about one minute in the best case.

A frequently proposed method of reducing unsolicited bulk email (“spam”) is for senders to pay for each email they send. Proof-ofwork schemes avoid charging real money by requiring senders to demonstrate that they have expended processing time in solving a cryptographic puzzle. We attempt to determine how difficult that puzzle should be so as to be effective in preventing spam. We analyse this both from an economic perspective, “how can we stop it being cost-effective to send spam”, and from a security perspective, “spammers can access insecure end-user machines and will steal processing cycles to solve puzzles”. Both analyses lead to similar values of puzzle difficulty. Unfortunately, realworld data from a large ISP shows that these difficulty levels would mean that significant numbers of senders of legitimate email would be unable to continue their current levels of activity. We conclude that proof-of-work will not be a solution to the problem of spam.

Blockchain is an emerging technology for decentralized and transactional data sharing across a large network of untrusted participants. It enables new forms of distributed software architectures, where components can find agreements on their shared states without trusting a central integration point or any particular participating components. Considering the blockchain as a software connector helps make explicitly important architectural considerations on the resulting performance and quality attributes (for example, security, privacy, scalability and sustainability) of the system. Based on our experience in several projects using blockchain, in this paper we provide rationales to support the architectural decision on whether to employ a decentralized blockchain as opposed to other software solutions, like traditional shared data storage. Additionally, we explore specific implications of using the blockchain as a software connector including design trade-offs regarding quality attributes.


Question: Interfering with the delivery of blocks and transactions in the Bitcoin network.

This paper offers a comprehensive review, titled "State of Knowledge (SoK): Research Perspectives and Challenges for Bitcoin and Cryptocurrencies," exploring the evolution and impact of Bitcoin, the pioneering cryptographic currency. Since its inception in 2009, Bitcoin has rapidly grown in economic significance, amassing billions of dollars in value despite initial underanalysis of its system design. The subsequent academic literature has delved into the nuances of the system, revealing its intricate properties, potential vulnerabilities, and innovative alternatives, while also outlining formidable challenges for its future. Simultaneously, an active open-source community has contributed to the system's development through various modifications and enhancements. This work is the first to systematically analyze Bitcoin and the plethora of alternative cryptocurrencies, or 'altcoins.' By disentangling the three fundamental design components of Bitcoin, it facilitates a more nuanced understanding of its characteristics and long-term viability. The paper explores the design landscape for various modifications, comparing different consensus algorithms, mechanisms for currency distribution, computational puzzles, and key management techniques. It also delves into privacy concerns within the Bitcoin ecosystem, presenting an evaluation framework for assessing privacy-enhancing solutions. Additionally, the paper introduces and examines "disintermediation protocols," which eliminate the need for trusted intermediaries in a range of applications. It outlines three primary strategies for disintermediation and provides a detailed comparative analysis.

Bitcoin and Ethereum are novel mechanisms for decentralizing the concept of money and computation. Extending decentralization to the human identity concept, we can think of using blockchain for creating a list of verified human identities with a one-person-one-ID property. UniqueID is a Decentralized Autonomous Organization(DAO) for maintaining human identities such that every physical human entity can have no more that one account. One part of this identity is simply the user’s claim on one of his unique, permanent, and measurable characteristics -biometrics. Blockchain has proved its integrity as a platform for storing and performing computations on such claims. The biggest challenge here is to ensure that the user has submitted his own valid biometric data. Human verifiers can check if there is any inconsistency in other users’ data, by peer-to-peer checks. For preventing bad behavior and centralization in the verification process, UniqueID benefits from novel governance mechanisms to choose verifiers and punish unjust ones. Also, there are incentives for honest verifiers and users by newly generated tokens. We show how the users’ privacy can be preserved by using stateof-the-art cryptographic techniques, and so they can use their identity without any concerns for votings, financial and banking purposes, social media accounts, reputation systems etc.

This paper proposes a distributed authorization architecture for Internet of Things adopting a hyperledger fabric framework. This paper also applies Genetic Algorithm to enhance the consensus component, traditionally used for Kafka, in order to determine the best configuration over different parameters, such as input transactions and their success rates. The experimental results confirm our superior performance in that the transaction transfer rate of our optimization, called GA Kafka, outperforms the traditional Kafka by 69.43%, and 89.97% for transaction transfer success rate.

Bitcoin has enjoyed wider adoption than any previous crypto-currency; yet its success has also attracted the attention of fraudsters who have taken advantage of operational insecurity and transaction irreversibility. We study the risk investors face from Bitcoin exchanges, which convert between Bitcoins and hard currency. We examine the track record of 40 Bitcoin exchanges established over the past three years, and find that 18 have since closed, with customer account balances often wiped out. Fraudsters are sometimes to blame, but not always. Using a proportional hazards model, we find that an exchange’s transaction volume indicates whether or not it is likely to close. Less popular exchanges are more likely to be shut than popular ones. We also present a logistic regression showing that popular exchanges are more likely to suffer a security breach.

There have been many incidents of prefix hijacking in the Internet. The hijacking AS can blackhole the hijacked traffic. Alternatively, it can transparently intercept the hijacked traffic by forwarding it onto the owner. This paper presents a study of such prefix hijacking and interception with the following contributions: (1). We present a methodology for prefix interception, (2). We estimate the fraction of traffic to any prefix that can be hijacked and intercepted in the Internet today, (3). The interception methodology is implemented and used to intercept real traffic to our prefix, (4). We conduct a detailed study to detect ongoing prefix interception.
 We find that: Our hijacking estimates are in line with the impact of past hijacking incidents and show that ASes higher up in the routing hierarchy can hijack a significant amount of traffic to any prefix, including popular prefixes. A less apparent result is that the same holds for prefix interception too. Further, our implementation shows that intercepting traffic to a prefix in the Internet is almost as simple as hijacking it. Finally, while we fail to detect ongoing prefix interception, the detection exercise highlights some of the challenges posed by the prefix interception problem.

The Bitcoin system (https://bitcoin.org) is a pseudo-anonymous currency that can dissociate a user from any real-world identity. In that context, a successful breach of the virtual and physical divide represents a signficant aw in the Bit-coin system [1]. In this project we demonstrate how to glean information about the real-world users behind Bitcoin transactions. We analyze publicly available data about the cryptocurrency. In particular, we focus on determining information about a Bitcoin user's physical location by examining that user's spending habits.

Bitcoin has recently attracted considerable attention in the fields of economics, cryptography, and computer science due to its inherent nature of combining encryption technology and monetary units. This paper reveals the effect of Bayesian neural networks (BNNs) by analyzing the time series of Bitcoin process. We also select the most relevant features from Blockchain information that is deeply involved in Bitcoin’s supply and demand and use them to train models to improve the predictive performance of the latest Bitcoin pricing process. We conduct the empirical study that compares the Bayesian neural network with other linear and non-linear benchmark models on modeling and predicting the Bitcoin process. Our empirical studies show that BNN performs well in predicting Bitcoin price time series and explaining the high volatility of the recent Bitcoin price.

Distributed Infrastructure Initiatives (DIIs) are communities that collaboratively produce and consume infrastructure. To develop a healthy ecosystem, DIIs require an economic model that balances supply and demand, but there is currently a lack of tooling to support implementing these. In this research, we propose an architecture for a platform that enables DIIs to implement such models, focused around a digital currency based on blockchain technology. The currency is issued according to the amount participants contribute to the initiative, which is quantified based on operational metrics gathered from the infrastructure. Furthermore, the platform enables participants to deploy smart contracts which encode self-enforcing agreements about the infrastructure services they exchange. The architecture has been evaluated through a case study at The Things Network (TTN) a global distributed crowdsourced Internet of Things initiative. The case study revealed that the architecture is effective for the selected case at TTN. In addition, the results motivate future research lines to support scalability (i.e., to deploy the architecture on a larger scale) and security.

Dense, unmanaged IEEE 802.11 deployments tempt saboteurs into launching jamming attacks by injecting malicious interference. Nowadays, jammers can be portable devices that transmit intermittently at low power in order to conserve energy. In this paper, we first conduct extensive experiments on an indoor 802.11 network to assess the ability of two physical-layer functions, rate adaptation and power control, in mitigating jamming. In the presence of a jammer, we find that: 1) the use of popular rate adaptation algorithms can significantly degrade network performance; and 2) appropriate tuning of the carrier sensing threshold allows a transmitter to send packets even when being jammed and enables a receiver to capture the desired signal. Based on our findings, we build ARES, an Anti-jamming REinforcement System, which tunes the parameters of rate adaptation and power control to improve the performance in the presence of jammers. ARES ensures that operations under benign conditions are unaffected. To demonstrate the effectiveness and generality of ARES, we evaluate it in three wireless test-beds: 1) an 802.11n WLAN with MIMO nodes; 2) an 802.11a/g mesh network with mobile jammers; and 3) an 802.11a WLAN with TCP traffic. We observe that ARES improves the network throughput across all testbeds by up to 150%.

The SSL protocol has been widely used for verifying digital identities and to secure Internet traffic since the early days of the web. Although X.509 certificates have been in existence for more than two decades, individual user uptake has been low due to the high cost of issuance and maintenance of such certs. This has led to a situation whereby users are able to verify the identity of an organization or e-commerce retailer via their digital certificate, but organizations have to rely on weak username and password combinations to verify the identity of customers registered with their service. We propose the X509Cloud framework which enables organizations to issue certificates to their users at zero cost, and allows them to securely store and disseminate client certificates using the Bitcoin inspired blockchain protocol. This in turn will enable organizations and individuals to authenticate and to securely communicate with other users on the Internet.

Transactive energy paradigms will enable the exchange of energy from a distributed set of prosumers. While prosumers have access to distributed energy resources, these resources are intermittently available. There is a need for distributed markets to enable the exchange of energy in transactive environments, however, the large number of potential prosumers introduces challenges in the establishment of trust between prosumers. Markets for transactive environments create other challenges, such as establishing clearing prices for energy and exchanging money between prosumers. Blockchains provide a unique technology to address this distributed trust problem through the use of a distributed ledger, cryptocurrencies, and the execution of smart contracts. This paper introduces a smart contract that implements a transactive energy auction that operates without the need for a trusted entitys oversight. The auction mechanism implements a Vickrey second price auction, which guarantees bidders will submit honest bids. The contract is implemented on transactive agents on the WSU campus interacting with a 72kW PV array and the Ethereum blockchain. The contract is then used to execute auctions based on the energy from the the PV array and simulated building loads to demonstrate the auctions operations.


Question: Interfering with the delivery of blocks and transactions in the Bitcoin network.

In the realm of cryptocurrency security, it has been revealed that the consensus of a majority is not a sufficient safeguard against potential attacks on Bitcoin mining operations. This finding challenges the widely held belief that the decentralized nature of Bitcoin inherently protects it from malicious activities. The study highlights vulnerabilities that could be exploited by attackers, even without controlling the majority of the network's hash power. These vulnerabilities stem from the complex interplay between block propagation, transaction confirmation times, and the strategic behavior of miners. By manipulating the delivery of blocks and transactions, an attacker can potentially gain an advantage, disrupting the fairness and efficiency of the network. This revelation underscores the need for further research and the development of enhanced security measures to protect the integrity of Bitcoin and other cryptocurrencies.

In 2008, Beck and Tews have proposed a practical attack on WPA. Their attack (called the Beck-Tews attack) can recover plaintext from an encrypted short packet, and can falsify it. The execution time of the Beck-Tews attack is about 12-15 minutes. However, the attack has the limitation, namely, the targets are only WPA implementations those support IEEE802.11e QoS features. In this paper, we propose a practical message falsification attack on any WPA implementation. In order to ease targets of limitation of wireless LAN products, we apply the Beck-Tews attack to the man-in-the-middle attack. In the man-inthe-middle attack, the user’s communication is intercepted by an attacker until the attack ends. It means that the users may detect our attack when the execution time of the attack is large. Therefore, we give methods for reducing the execution time of the attack. As a result, the execution time of our attack becomes about one minute in the best case.

Distributed Denial of Service (DDoS) attacks are an ongoing problem in today's Internet. In this paper we focus on DDoS attacks in Named Data Networking (NDN). NDN is a specific candidate for next-generation Internet architecture designs. In NDN locations are named instead of data, So NDN transforms data into a first-class entity and makes itself an attractive and practical approach to meet the needs for many applications. In a Named Data Networking (NDN), end users request data by sending Interest packets, and the network delivers Data packets upon request only, effectively eliminating many existing DDoS attacks. However, an NDN network faces a new type of DDoS attack, namely Interest packet flooding. In this paper we try to alleviate the Interest flooding using token bucket with per interface fairness algorithm and also we try to find countermeasures for Interest flooding attacks

Security of financial transactions in E-Commerce is difficult to implement and there is a risk that user’s confidential data over the internet may be accessed by hackers. Unfortunately, interacting with an online service such as a banking web application often requires certain degree of technical sophistication that not all Internet users possess. For the last couple of year such naive users have been increasingly targeted by phishing attacks that are launched by miscreants who are aiming to make an easy profit by means of illegal financial transactions. In this paper, we have proposed an idea for securing e-commerce transaction from phishing attack. An approach already exists where phishing attack is prevented using one time password which is sent on user’s registered mobile via SMS for authentication. But this method can be counter attacked by “Man in the Middle”. In our paper, a new idea is proposed which is more secure compared to the existing online payment system using OTP. In this mechanism OTP is combined with the secure key and is then passed through RSA algorithm to generate the Transaction password. A Copy of this password is maintained at the server side and is being generated at the user side using a mobile application; so that it is not transferred over the insecure network leading to a fraudulent transaction. Keywords—Phishing, Replay attack, MITM attack, RSA, Random Generator.

Secure access is one of the fundamental problems in wireless mobile networks. Digital signature is a widely used technique to protect messages’ authenticity and nodes’ identities. From the practical perspective, to ensure the quality of services in wireless mobile networks, ideally the process of signature verification should introduce minimum delay. Batch cryptography technique is a powerful tool to reduce verification time. However, most of the existing works focus on designing batch verification algorithms for wireless mobile networks without sufficiently considering the impact of invalid signatures, which can lead to verification failures and performance degradation. In this paper, we propose a Batch Identification Game Model (BIGM) in wireless mobile networks, enabling nodes to find invalid signatures with reasonable delay no matter whether the game scenario is complete information or incomplete information. Specifically, we analyze and prove the existence of Nash Equilibriums (NEs) in both scenarios, to select the dominant algorithm for identifying invalid signatures. To optimize the identification algorithm selection, we propose a self-adaptive auto-match protocol which estimates the strategies and states of attackers based on historical information. Comprehensive simulation results in terms of NE reasonability, algorithm selection accuracy, and identification delay are provided to demonstrate that BIGM can identify invalid signatures more efficiently than existing algorithms.

Various cryptographic puzzle schemes are proposed as a defense mechanism against denial of service attack. But, all these puzzle schemes face a dilemma when there is a large disparity between the computational power of attackers and legitimate clients: increasing the difficulty of puzzles might unnecessarily restrict legitimate clients too much, and lower difficulty puzzles cannot sufficiently block attackers with large computational resources. In this paper, we introduce guided tour puzzle, a novel puzzle scheme that is not affected by such resource disparity. A guided tour puzzle requires a client to visit a predefined set of nodes, called tour guides, in a certain sequential order to retrieve an n piece answer, one piece from each tour guide that appears in the tour. This puzzle solving process is non-parallelizable, thus cheating by trying to solve the puzzle in parallel is not possible. Guided tour puzzle not only achieves all previously defined desired properties of a cryptographic puzzle scheme, but it also satisfies more important requirements, such as puzzle fairness and minimum interference, that we identified. The number of tour guides required by the scheme can be as few as two, and this extra cost can be amortized by sharing the same set of tour guides among multiple servers.

Dear editor, As data are one of the most important factors of today’s intelligent systems but are relatively scarce, people try exchanging data with other organizations or public marketplaces. However, data exchange is now lack of a secure execution mechanism to automatically and fairly protect the rights which are seriously concerned by data owners. For instance, data owners cannot limit the identity of a data transferee who is banned to take the data by any way. Moreover, data transferees are also afraid that their applications for exchange may be unfairly treated. For instance, data owners could dynamically change data price when multiple transferees submit their applications at the same time. A data marketplace can take over the role of a trusted third party during data exchange, but single point failure is always a serious problem in a large-scale distributed system. Furthermore, these marketplaces usually use contracts based on natural languages which may be maliciously misinterpreted and cannot be automatically executed. Numerous researches have been presented to provide digital rights management (DRM) [1]. For instance, Zhu et al. [2] presented a privacypreserving video subscription scheme with the limitation of expire date. However, rights of data owners are more complex than rights of digital content owners. Decentralized and trustworthy enforcement for access control policies is a long-term issue. Han et al. [3] proposed an optimized mechanism which is a trusted and decentralized access control framework for the client/server architecture. These studies still need third party authorities which are used to execute or supervise the rights control, leading to a few trust and security issues. Existing studies [4] on attribute-based access control have been applied to many fields, such as e-commerce and Internet of Things. This article introduces attribute-based access control and proposes a secure framework for data exchange based on smart contracts (DESC) to enable automatic and fair access control. Featuring with decentralized, fair, transparent, immutable and traceable advantages, blockchain 2.0 [5] provides a generalized framework for implementing decentralized compute resources named smart contracts which define rights and policies in mathematical and programming forms. Once the predefined smart contract is triggered by a transaction, it can automatically execute the specific contractual clauses. To find out which data rights are seriously concerned by data owners, we investigate the DRM mechanisms and several data exchange platforms: GBDEX, national engineering lab for big data distribution and exchange technologies. Then,

Smart contracts are programs running on cryptocurrency (e.g., Ethereum) blockchains, whose popularity stem from the possibility to perform financial transactions, such as payments and auctions, in a distributed environment without need for any trusted third party. Given their financial nature, bugs or vulnerabilities in these programs may lead to catastrophic consequences, as witnessed by recent attacks. Unfortunately, programming smart contracts is a delicate task that requires strong expertise: Ethereum smart contracts are written in Solidity, a dedicated language resembling JavaScript, and shipped over the blockchain in the EVM bytecode format. In order to rigorously verify the security of smart contracts, it is of paramount importance to formalize their semantics as well as the security properties of interest, in particular at the level of the bytecode being executed. In this paper, we present the first complete small-step semantics of EVM bytecode, which we formalize in the F* proof assistant, obtaining executable code that we successfully validate against the official Ethereum test suite. Furthermore, we formally define for the first time a number of central security properties for smart contracts, such as call integrity, atomicity, and independence from miner controlled parameters. This formalization relies on a combination of hyperand safety properties. Along this work, we identified various mistakes and imprecisions in existing semantics and verification tools for Ethereum smart contracts, thereby demonstrating once more the importance of rigorous semantic foundations for the design of security verification techniques.

Building trustless cross-blockchain trading protocols is challenging. Therefore, centralized liquidity providers remain the preferred route to execute transfers across chains — which fundamentally contradicts the purpose of permissionless ledgers to replace trusted intermediaries. Enabling crossblockchain trades could not only enable currently competing blockchain projects to better collaborate, but seems of particular importance to decentralized exchanges as those are currently limited to the trade of digital assets within their respective blockchain ecosystem. In this paper we systematize the notion of cryptocurrencybacked tokens, an approach towards trustless cross-chain communication. We propose XCLAIM, a protocol for issuing, trading, and redeeming e.g. Bitcoin-backed tokens on Ethereum. We provide implementations for three possible protocol versions and evaluate their security and on-chain costs. With XCLAIM, it costs at most USD 1.17 to issue an arbitrary amount of Bitcoinbacked tokens on Ethereum, given current blockchain transaction fees. Our protocol requires no modifications to Bitcoin’s and Ethereum’s consensus rules and is general enough to support other cryptocurrencies.

Today, most of the services one may think of are based on a collaborative paradigm (e.g., social media services, IoT-based services, etc.). One of the most relevant representative of such class of services are inter-organizational processes, where an organized group of joined activities is carried out by two or more organizations to achieve a common business goal. Inter-organizational processes are therefore vital to achieve business partnerships among different organizations. However, they may also pose serious security and privacy threats to the data each organization exposes. This is mainly due to the weak trust relationships that may hold among the collaborating parties, which result in a potential lack of trust on how data/operations are managed. In this paper, we discuss, how blockchain, one of today hottest technology, can be used in support of secure inter-organizational processes. We further point out which additional security issues the use of blockchain can bring, illustrate the ongoing research projects in the area and discuss future research directions.

Bit coin, as the foundation for a secure electronic payment system, has drawn broad interests from researchers in recent years. In this paper, we analyze a comprehensive Bit coin transaction dataset and investigate the interrelationship between the flow of Bit coin transactions and its price movement. Using network theory, we examine a few complexity measures of the Bit coin transaction flow networks, and we model the joint dynamic relationship between these complexity measures and Bit coin market variables such as return and volatility. We find that a particular complexity measure of the Bit coin transaction network flow is significantly correlated with the Bit coin market return and volatility. More specifically we document that the residual diversity or freedom of Bit coin network flow scaled by the total system throughput can significantly improve the predictability of Bit coin market return and volatility.


Question: Interfering with the delivery of blocks and transactions within the Bitcoin network.

This study delves into the dissemination dynamics of transactions and blocks within the Bitcoin network, a decentralized digital currency system that operates without a central authority. Bitcoin's operational integrity hinges on a distributed network of volunteers who collaboratively maintain a replicated ledger and authenticate transactions. Our analysis centers on the multi-hop broadcast mechanism employed by Bitcoin for updating ledger replicas across the network. Using empirical data, we investigate the hypothesis that propagation latency is the primary catalyst for blockchain forks, which are detrimental to the network's consistency as they indicate discrepancies among ledger replicas. Subsequently, we explore the potential improvements achievable by optimizing the existing protocol through client-side modifications, focusing on the limits of such unilateral changes. This research not only underscores the significance of efficient information propagation in maintaining the network's integrity but also identifies areas for enhancing the Bitcoin protocol's robustness.

In response to increasingly sophisticated methods of blocking access to censorship circumvention schemes such as Tor, recently proposed systems such as Skypemorph, FreeWave, and CensorSpoofer have used voice and video conferencing protocols as "cover channels" to hide proxy connections. We demonstrate that even with perfect emulation of the cover channel, these systems can be vulnerable to attacks that detect or disrupt the covert communications while having no effect on legitimate cover traffic. Our attacks stem from differences in the channel requirements for the cover protocols, which are peer-to-peer and loss tolerant, and the covert traffic, which is client-proxy and loss intolerant. These differences represent significant limitations and suggest that such protocols are a poor choice of cover channel for general censorship circumvention schemes.

Mobile security has become more and more important due to the boom of mobile commerce (m-commerce). However, the development of m-commerce is facing many challenges regarding data security problems. Recently, blockchain has been introduced as an effective security solution deployed successfully in many applications in practice, such as, Bitcoin, cloud computing, and Internet-of-Things. In this paper, we introduce a new m-commerce application using blockchain technology, namely, MobiChain, to secure transactions in the m-commerce. Especially, in the MobiChain application, the mining processes can be executed efficiently on mobile devices using our proposed Android core module. Through real experiments, we evaluate the performance of the proposed model and show that blockchain will be an efficient security solution for future m-commerce.

A blockchain is a distributed and decentralized ledger that contains connected blocks of transactions. Unlike other ledger approaches, blockchain guarantees tamper proof storage of approved transactions. Due to its distributed and decentralized organization, blockchain is beeing used within IoT e.g. to manage device configuration, store sensor data and enable micro-payments. This paper presents the idea of using blockchain as a service for IoT and evaluates the performance of a cloud and edge hosted blockchain implementation.

Recent archaeological discoveries on the island of Paxos reveal that the parliament functioned despite the peripatetic propensity of its part-time legislators. The legislators maintained consistent copies of the parliamentary record, despite their frequent forays from the chamber and the forgetfulness of their messengers. The Paxon parliament's protocol provides a new way of implementing the state machine approach to the design of distributed systems.

Digital information has become a social infrastructure and with the expansion of the Internet, network infrastructure has become an indispensable part of social life and industrial activity for mankind. The idea of using existing electronics in smart home appliances and connecting them to the Internet is a new dimension along which technologies continue to grow, and in recent years mankind has witnessed an upsurge of usage of devices such as smart phone, smart television, home health-care device, smart LED light bulbs system, etc. Their build-in internet-controlled function has made them quite attractive to many segments of consumers and smart phone has become a common gadget for social networking. There are, however, serious challenges which need to be addressed as these tiny devices are designed for specific functions and lack processing capacity required for most security software. This research explores how these internet-enabled smart devices can be turned into very dangerous spots for distributed attacks purposes by cybercriminals for various ill intensions in a pinpointed manner. It then introduces a new approach to deal with such problems by taking advantage of the anonymous communication of the Onion Router (hereafter: TOR). It compares pros and cons of using anonymous communication scheme and justifies it be an efficient countermeasure to most attack scenarios.

We present a distributed efficiently amortizable CPU cost-function with no trap–door. The absence of a trap–door allows us to avoid needing to trust any party. Applications for such cost-functions are in distributed document popularity estimation, and metering of web advertising. None of the servers involved have any advantage over users in computing the cost function. The amortized token has a small fixed sized representation independent of the amortized value. The valuation function is efficient. Limits can be placed on the resources which can be expended in computing the cost-function to prevent the user inflating the value of his contribution by using more CPU time than the expected token value. The amortized part of the cost result can be blinded so that clients can not obtain service but avoid contributing to the amortization by expending more resources than the expected token value. Interactive and non-interactive variants of the cost-function can be constructed which can be used in situations where the server can issue a challenge (connection oriented interactive protocol), and where it can't (where the communication is store–and–forward, or packet oriented) respectively.

The concept of peer-to-peer applications is not new, nor is the concept of distributed hash tables. What emerged in 2008 with the publication of the Bitcoin white paper was an incentive structure that unified these two software paradigms with a set of economic stimuli to motivate the creation of a dedicated computing network orders of magnitude more powerful than the world’s fastest supercomputers. The purpose of which is the maintenance of a massive distributed database known as the Bitcoin “block chain”. Apart from the digital currency it enables, block chain technology is a fascinating new computing paradigm with broad implications for the future development of the World Wide Web, and by extension, the further growth of Linked Data and the Semantic Web. This work is divided into two main sections, we first demonstrate how block chain technologies can contribute toward the realization of a more robust Semantic Web, and subsequently we provide a framework wherein the Semantic Web is utilized to ameliorate block chain technology itself.

Embedded devices are going to be used extremely in Internet of Things (IoT) environments. The small and tiny IoT devices will operate and communicate each other without involvement of users, while their operations must be correct and protected against various attacks. In this paper, we focus on a secure firmware update issue, which is a fundamental security challenge for the embedded devices in an IoT environment. A new firmware update scheme that utilizes a blockchain technology is proposed to securely check a firmware version, validate the correctness of firmware, and download the latest firmware for the embedded devices. In the proposed scheme, an embedded device requests its firmware update to nodes in a blockchain network and gets a response to determine whether its firmware is up-to-date or not. If not latest, the embedded device downloads the latest firmware from a peer-to-peer firmware sharing network of the nodes. Even in the case that the version of the firmware is up-to-date, its integrity, i.e., correctness of firmware, is checked. The proposed scheme guarantees that the embedded device’s firmware is up-to-date while not tampered. Attacks targeting known vulnerabilities on firmware of embedded devices are thus mitigated.

Blockchains started as an enabling technology in the area of digital currencies with the introduction of Bitcoin. However, blockchains have emerged as a technology that goes beyond financial transactions by providing a platform supporting secure and robust distributed public ledgers. We think that the Internet of Things (IoT) can also benefit from blockchain technology, especially in the areas of security, privacy, fault tolerance, and autonomous behavior. Here we present a decision framework to help practitioners systematically evaluate the potential use of blockchains in an IoT context.

Distributed Infrastructure Initiatives (DIIs) are communities that collaboratively produce and consume infrastructure. To develop a healthy ecosystem, DIIs require an economic model that balances supply and demand, but there is currently a lack of tooling to support implementing these. In this research, we propose an architecture for a platform that enables DIIs to implement such models, focused around a digital currency based on blockchain technology. The currency is issued according to the amount participants contribute to the initiative, which is quantified based on operational metrics gathered from the infrastructure. Furthermore, the platform enables participants to deploy smart contracts which encode self-enforcing agreements about the infrastructure services they exchange. The architecture has been evaluated through a case study at The Things Network (TTN) a global distributed crowdsourced Internet of Things initiative. The case study revealed that the architecture is effective for the selected case at TTN. In addition, the results motivate future research lines to support scalability (i.e., to deploy the architecture on a larger scale) and security.