Question: Analog CMOS-designed resistive processing unit for training deep neural networks.

This paper presents an innovative Resistive Processing Unit (RPU) design strategy for significantly accelerating the training of Deep Neural Networks (DNNs), particularly in the realm of large-scale analysis and classification tasks such as speech recognition and visual object detection. The training of extensive DNNs is notorious for its computational intensity and time-consuming nature, often necessitating substantial datacenter resources for extended periods. The proposed RPU device aims to revolutionize this process by potentially speeding up training by several orders of magnitude while consuming considerably less power. The RPU's unique capability to store and update weight values locally minimizes data movement during training, thereby maximizing the exploitation of locality and parallelism inherent in the training algorithm. To ensure practical implementation, we scrutinize the impact of various RPU device characteristics and system parameters on performance, delineating device and system level requirements for an accelerator chip designed for DNN training within a CMOS-compatible technology. In the case of large DNNs with approximately 1 billion weights, this highly parallel RPU architecture can achieve acceleration factors of up to 30,000 times compared to current state-of-the-art microprocessors, with a power efficiency of 84,000 GigaOps/s/W. This breakthrough enables problems that once required days of training on a datacenter-scale cluster to be solved within hours on a single RPU accelerator. Moreover, a cluster of RPU accelerators will have the potential to address Big Data challenges with trillions of parameters, currently unattainable with existing technologies, such as natural speech recognition, translation between all world languages, real-time analytics on vast streams of business and scientific data, and the integration and analysis of multimodal sensory data from a multitude of Internet of Things (IoT) sensors.

In this paper, we propose a novel Spin Orbit Torque Magnetic Random Access Memory (SOT-MRAM) array design that could simultaneously work as non-volatile memory and implement a reconfigurable in-memory logic (AND, OR) without add-on logic circuits to memory chip as in traditional logic-in-memory designs. The computed logic output could be simply read out like a normal MRAM bit-cell using the shared memory peripheral circuits. Such intrinsic in-memory logic could be used to process data within memory to greatly reduce power-hungry and long distance data communication in conventional Von-Neumann computing systems. We further employ in-memory data encryption using Advanced Encryption Standard (AES) algorithm as a case study to demonstrate the efficiency of the proposed design. The device to architecture co-simulation results show that the proposed design can achieve 70.15% and 80.87% lower energy consumption compared to CMOS-ASIC and CMOL-AES implementations, respectively. It offers almost similar energy consumption as recent DW-AES implementation, but with 60.65% less area overhead.

We present a deep, fully convolutional neural network that learns to route a circuit layout ‘net’ with appropriate choice of metal tracks and wire class combinations. Inputs to the network are the encoded layouts containing spatial location of pins to be routed. After 15 fully convolutional stages followed by a score comparator, the network outputs 8 layout layers (corresponding to 4 route layers, 3 via layers and an identity-mapped pin layer) which are then decoded to obtain the routed layouts. We formulate this as a binary segmentation problem on a per-pixel per-layer basis, where the network is trained to correctly classify pixels in each layout layer to be ‘on’ or ‘off’. To demonstrate learnability of layout design rules, we train the network on a dataset of 50,000 train and 10,000 validation samples that we generate based on certain pre-defined layout constraints. Precision, recall and F1 score metrics are used to track the training progress. Our network achieves F1 ≈ 97% on the train set and F1 ≈ 92% on the validation set. We use PyTorch for implementing our model. Code is made publicly available1.

In recent years significant progress has been made in successfully training recurrent neural networks (RNNs) on sequence learning problems involving long range temporal dependencies. The progress has been made on three fronts: (a) Algorithmic improvements involving sophisticated optimization techniques, (b) network design involving complex hidden layer nodes and specialized recurrent layer connections and (c) weight initialization methods. In this paper, we focus on recently proposed weight initialization with identity matrix for the recurrent weights in a RNN. This initialization is specifically proposed for hidden nodes with Rectified Linear Unit (ReLU) non linearity. We offer a simple dynamical systems perspective on weight initialization process, which allows us to propose a modified weight initialization strategy. We show that this initialization technique leads to successfully training RNNs composed of ReLUs. We demonstrate that our proposal produces comparable or better solution for three toy problems involving long range temporal structure: the addition problem, the multiplication problem and the MNIST classification problem using sequence of pixels. In addition, we present results for a benchmark action recognition problem.

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce “deep compression”, a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35× to 49× without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9× to 13×; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35×, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49× from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3× to 4× layerwise speedup and 3× to 7× better energy efficiency.

Binary weight convolutional neural networks (BCNNs) can achieve near state-of-the-art classification accuracy and have far less computation complexity compared with traditional CNNs using high-precision weights. Due to their binary weights, BCNNs are well suited for vision-based Internet-of-Things systems being sensitive to power consumption. BCNNs make it possible to achieve very high throughput with moderate power dissipation. In this paper, an energy-efficient architecture for BCNNs is proposed. It fully exploits the binary weights and other hardware-friendly characteristics of BCNNs. A judicious processing schedule is proposed so that off-chip I/O access is minimized and activations are maximally reused. To significantly reduce the critical path delay, we introduce optimized compressor trees and approximate binary multipliers with two novel compensation schemes. The latter is able to save significant hardware resource, and almost no computation accuracy is compromised. Taking advantage of error resiliency of BCNNs, an innovative approximate adder is developed, which significantly reduces the silicon area and data path delay. Thorough error analysis and extensive experimental results on several data sets show that the approximate adders in the data path cause negligible accuracy loss. Moreover, algorithmic transformations for certain layers of BCNNs and a memory-efficient quantization scheme are incorporated to further reduce the energy cost and on-chip storage requirement. Finally, the proposed BCNN hardware architecture is implemented with the SMIC 130-nm technology. The postlayout results demonstrate that our design can achieve an energy efficiency over 2.0TOp/s/W when scaled to 65 nm, which is more than two times better than the prior art.

With the die area of modern processors growing larger and larger, the IR drop across the power supply rail due to its parasitic resistance becomes considerable. There is an urgent demand for local power regulation to reduce the IR drop and to enhance transient response. A distributed power-delivery grid (DPDG) is an attractive solution for large area power-supply applications. The dual-loop distributed micro-regulator in [1] achieves a tight regulation and fast response, but suffers from large ripple and high power consumption due to the comparator-based regulator. Digital low-dropout regulators (DLDOs) [2] can be used as local micro-regulators to implement a DPDG, due to their low-voltage operation and process scalability. Adaptive control [3], asynchronous 3D pipeline control [4], analog-assisted tri-loop control [5], and event-driven PI control [6] are proposed to enhance the transient response speed. However, digital LDOs suffer from the intrinsic limitations of large output ripple and narrow current range. This paper presents an on-chip DPDg with cooperative regulation based on an analog-assisted digital LDO (AADLDO), which inherits the merits of low output ripple and sub-LSB current supply ability from the analog control, and the advantage of low supply voltage operation and adaptive fast response from the digital control.

This paper involves designing a small scale battery charging system which is powered via a photovoltaic panel. This work aims at the usage of solar energy for charging the battery and optimizing the power of the system. Implementation is done using Artificial Neural Network (ANN) on FPGA. To develop this system an Artificial Neural Network is trained and its result is further used for the PWM technique. PWM pulse generation has been done using Papilio board which is based on XILINX Spartan 3E FPGA. The ANN with PWM technique is ported on FPGA which is programmed using VHDL. This able to automatically control the whole charging system operation without requirement of external sensory unit. The simulation results are achieved by using MATLAB and XILINX. These results allowed demonstrating the charging of the battery using proposed ANN and PWM technique. KeywordsPhotovoltaic battery charger, PWM, ANN, FPGA

Spin Transfer Torque Magnetoresistive RAM (STT-MRAM) has been recently deemed as one promising main memory alternative for high-end mobile processors. With process technology scaling, the amplitude of write current approaches that of read current in deep sub-micrometer STT-MRAM arrays. As a result, read disturbance errors (RDEs) emerge. Both high current restore required (HCRR) reads and low current long latency (LCLL) reads can guarantee read reliability and utterly remove RDEs. However, both of them degrade system performance, because of extra restores or a longer read latency. And neither of them always achieves the better performance when running a wide variety of applications. In this paper, we present two architectural techniques to boost read performance for STT-MRAM based main memories in the presence of RDEs. We first propose Smash Read (S-RD) to shorten the latency of HCRR reads by injecting a larger read current. We further introduce Flexible Read (F-RD) to dynamically adopt different types of read schemes, S-RD and LCLL, to maximize main memory system performance. On average, our techniques improve system performance by 9~13% and reduces total energy by 4~8% over all existing read schemes including HCRR and LCLL.

While hybrid capacitive-resistive D/A Converter (DAC) has been known for many years, its potential for energy-efficient operation is sometimes overlooked. This paper investigates the utilization of hybrid DACs in successive-approximation register A/D converters. To improve energy efficiency of SAR ADCs, a new hybrid DAC is introduced. In an exemplar 10-bit 100-MS/s ADC, simulation results show that the energy efficiency and chip area (of passive devices) can be improved by more than an order of magnitude.

Humans are able to form internal representations of the information they process—a capability which enables them to perform many different memory tasks. Therefore, the neural system has to learn somehow to represent aspects of the environmental situation; this process is assumed to be based on synaptic changes. The situations to be represented are various as for example different types of static patterns but also dynamic scenes. How are neural networks consisting of mutually connected neurons capable of performing such tasks? Here we propose a new neuronal structure for artificial neurons. This structure allows one to disentangle the dynamics of the recurrent connectivity from the dynamics induced by synaptic changes due to the learning processes. The error signal is computed locally within the individual neuron. Thus, online learning is possible without any additional structures. Recurrent neural networks equipped with these computational units cope with different memory tasks. Examples illustrate how information is extracted from environmental situations comprising fixed patterns to produce sustained activity and to deal with simple algebraic relations.


Question: Analog CMOS-designed resistive processing unit for training deep neural networks.

This paper introduces MapReduce, a programming framework tailored for the efficient processing and generation of massive datasets. The model's versatility is underscored by its applicability to a wide range of practical tasks. Users define the computational logic through a map function and a reduce function, while the underlying system autonomously manages the parallelization of these tasks across extensive clusters of machines. This framework ensures robustness by handling hardware failures and optimizing inter-machine communication, thereby maximizing the utilization of network and disk resources. Its ease of use is evident from its widespread adoption: over ten thousand unique MapReduce applications have been developed internally at Google over the past four years. On a daily basis, an average of one hundred thousand MapReduce tasks are executed across Google's clusters, processing in excess of twenty petabytes of data. This showcases the framework's capacity to handle and process colossal volumes of data with efficiency and reliability.

We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure1.

As new neural network and neuromorphic architectures are being developed, new training methods that operate within the constraints of the new architectures are required. Evolutionary optimization (EO) is a convenient training method for new architectures. In this work, we review a spiking neural network architecture and a neuromorphic architecture, and we describe an EO training framework for these architectures. We present the results of this training framework on four classification data sets and compare those results to other neural network and neuromorphic implementations. We also discuss how this EO framework may be extended to other architectures.

Crossbar arrays based on non-volatile resistive devices such as resistive RAM and phase change memory have become an important technology due to the applications to memory systems. The energy consumption of integrated circuits has become a primary issue due to thermal constraints in high performance systems and limited battery time in mobile and IoT applications. In this paper, the energy efficiency of a crossbar array of a one-selector-one-resistor (1S1R) configuration during a write operation is explored for the V/2 and V/3 bias schemes. The characteristics that affect the most energy efficient bias scheme are demystified. The write energy of a crossbar array is modeled in terms of the array size, number of selected cells, and the nonlinearity factor. For a specific array size and selector technology, the number of selected cells during a write operation can affect the choice of bias scheme. Moreover, the effect of leakage current due to partially biased unselected cells is explored.

In this paper we work on the balance between hardware and software implementation of a machine learning algorithm, which belongs to the area of statistical learning theory. We use system-on-chip technology to demonstrate the potential usefulness of moving the critical sections of an algorithm into HW: the so-called hardware/software balance. Our experiments show that the approach can achieve speedups using a complex machine learning algorithm called a support vector machine. The experiments are conducted on a real-time Java virtual machine named Java optimized processor

Deep Neural nets (NNs) with millions of parameters are at the heart of many stateof-the-art computer vision systems today. However, recent works have shown that much smaller models can achieve similar levels of performance. In this work, we address the problem of pruning parameters in a trained NN model. Instead of removing individual weights one at a time as done in previous works, we remove one neuron at a time. We show how similar neurons are redundant, and propose a systematic way to remove them. Our experiments in pruning the densely connected layers show that we can remove upto 85% of the total parameters in an MNIST-trained network, and about 35% for AlexNet without significantly affecting performance. Our method can be applied on top of most networks with a fully connected layer to give a smaller network.

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers - 8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning. One aspect of the field receiving considerable attention is efficiently executing deep models in resource-constrained environments, such as mobile or embedded devices. This paper focuses on this problem, and proposes two new compression methods, which jointly leverage weight quantization and distillation of larger networks, called “teachers,” into compressed “student” networks. The first method we propose is called quantized distillation and leverages distillation during the training process, by incorporating distillation loss, expressed with respect to the teacher network, into the training of a smaller student network whose weights are quantized to a limited set of levels. The second method, differentiable quantization, optimizes the location of quantization points through stochastic gradient descent, to better fit the behavior of the teacher model. We validate both methods through experiments on convolutional and recurrent architectures. We show that quantized shallow students can reach similar accuracy levels to state-of-the-art full-precision teacher models, while providing up to order of magnitude compression, and inference speedup that is almost linear in the depth reduction. In sum, our results enable DNNs for resource-constrained environments to leverage architecture and accuracy advances developed on more powerful devices.

A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.

Convolutional Neural Networks (CNNs) have emerged as a fundamental technology for machine learning. High performance and extreme energy efficiency are critical for deployments of CNNs, especially in mobile platforms such as autonomous vehicles, cameras, and electronic personal assistants. This paper introduces the Sparse CNN (SCNN) accelerator architecture, which improves performance and energy efficiency by exploiting the zero-valued weights that stem from network pruning during training and zero-valued activations that arise from the common ReLU operator. Specifically, SCNN employs a novel dataflow that enables maintaining the sparse weights and activations in a compressed encoding, which eliminates unnecessary data transfers and reduces storage requirements. Furthermore, the SCNN dataflow facilitates efficient delivery of those weights and activations to a multiplier array, where they are extensively reused; product accumulation is performed in a novel accumulator array. On contemporary neural networks, SCNN can improve both performance and energy by a factor of 2.7x and 2.3x, respectively, over a comparably provisioned dense CNN accelerator.

This paper shows the existence of a finite neural network, made up of sigmoidal nenrons, which simulates a nniversal Turing machine. It is composed of less then lo5 synchronously evolving processors, interconnected linearly. High-order connections are not required.


Question: nTorrent: A Peer-to-Peer File Sharing Mechanism in Named Data Networking

Innovative File Sharing in Named Data Networking: nTorrent

Named Data Networking (NDN) represents a paradigm shift in the internet's architecture, funded by the U.S. National Science Foundation's Future Internet Architecture Program. Originating from the Content-Centric Networking (CCN) project, first introduced by Van Jacobson in 2006, NDN aims to transition from the current host-centric model (IP) to a data-centric approach. This fundamental transformation holds significant potential for reimagining network design, development, deployment, and application usage. This document explores the driving forces and overarching vision behind this novel architecture, detailing its key components and operational mechanisms. Additionally, it offers an update on the current design and development progress, along with the research hurdles that need to be addressed. For a comprehensive overview, including software prototypes, academic papers, and annual assessments, visit named-data.net.

This reimagined document maintains the original's length and complexity while preserving its relationship to the question about "nTorrent: Peer-to-Peer File Sharing in Named Data Networking." It introduces the concept of NDN, its origins, and the implications of shifting from host-centric to data-centric networks, paralleling the original's focus on a new architecture or technology (in this case, nTorrent).

The applicability of network-based computing depends on the availability of the underlying network bandwidth. However, network resources are shared and the available network bandwidth varies with time. There is no satisfactory solution available for network performance predictions. In this research, we propose, design, and implement the NBP (network bandwidth predictor) for rapid network performance prediction. NBP is a new system that employs a neural network based approach for network bandwidth forecasting. This system is designed to integrate with most advanced technologies. It employs the NWS (network weather service) monitoring subsystem to measure the network traffic, and provides an improved, more accurate performance prediction than that of NWS, especially with applications with a network usage pattern. The NBP system has been tested on real time data collected by NWS monitoring subsystem and on trace files. Experimental results confirm that NBP has an improved prediction.

The optimal storage-computation tradeoff is characterized for a MapReduce-like distributed computing system with straggling nodes, where only a part of the nodes can be utilized to compute the desired output functions. The result holds for arbitrary output functions and thus generalizes previous results that restricted to linear functions. Specifically, in this work, we propose a new information-theoretical converse and a new matching coded computing scheme, that we call coded computing for straggling systems (CCS).

Networks are expanding very fast and the number of clients is increasing dramatically, this causes the router forwarding table to become very large and present more demand on faster router operations. In this paper, we address the problem of packet forwarding in the routers aiming to increase the speed of address lookup and minimize the memory required for storing the forwarding table. We propose a new algorithm that makes use of two-level indexing and Btrees. We test the approach and compare it to other famous IP lookup approaches. The preliminary simulations show 20% less memory requirements and the lookup speed scaling linearly with increasing table size.

This paper presents a novel multi-copy routing protocol for disruption-tolerant networks whose objective is to minimize energy expended on communication. The protocol is designed for disaster-response applications, where power and infrastructure resources are disrupted. Unlike other delay-tolerant networks, energy is a vital resource in post-disaster scenarios to ensure availability of (disruption-tolerant) communication until infrastructure is restored. Our approach exploits naturally recurrent mobility and contact patterns in the network, formed by rescue workers, volunteers, survivors, and their (possibly stranded) vehicles to reduce the number of message copies needed to attain an adequate delivery ratio in the face of disconnection and intermittent connectivity. A new notion of inter-contact routing is proposed that allows estimating route delays and delivery probabilities, identifying more reliable routes and controlling message replication and forwarding accordingly. We simulate the scheme using a mobility model that reflects recurrence inspired by disaster scenarios, and compare our results to previous DTN routing techniques. The evaluation shows that the new approach reduces the resource overhead per message over previous approaches while maintaining a comparable delivery ratio at the expense of a small (bounded) increase in latency.

We propose a novel scheme for selective distribution of content, encoded as documents, that preserves the privacy of the users to whom the documents are delivered and is based on an efficient and novel group key management scheme. Our document broadcasting approach is based on access control policies specifying which users can access which documents, or subdocuments. Based on such policies, a broadcast document is segmented into multiple subdocuments, each encrypted with a different key. In line with modern attribute-based access control, policies are specified against identity attributes of users. However our broadcasting approach is privacy-preserving in that users are granted access to a specific document, or subdocument, according to the policies without the need of providing in clear information about their identity attributes to the document publisher. Under our approach, not only does the document publisher not learn the values of the identity attributes of users, but it also does not learn which policy conditions are verified by which users, thus inferences about the values of identity attributes are prevented. Moreover, our key management scheme on which the proposed broadcasting approach is based is efficient in that it does not require to send the decryption keys to the users along with the encrypted document. Users are able to reconstruct the keys to decrypt the authorized portions of a document based on subscription information they have received from the document publisher. The scheme also efficiently handles new subscription of users and revocation of subscriptions.

We have designed and implemented a new operating system facility for I/O buffer management and data transferacross protection domain boundaries on shared memory machines. This facility, called fast buffers (fbufs), combines virtual page remapping with shared virtual memory, and exploits locality in I/O traffic to achieve high throughput without compromising protection, security, or modularity. goal is to help deliver the high bandwidth afforded by emerging high-speed networks to user-level processes, both in monolithic and microkernel-based operating systems.This paper outlines the requirements for a cross-domain transfer facility, describes the design of the fbuf mechanism that meets these requirements, and experimentally quantifies the impact of fbufs on network performance.

Dear editor, As data are one of the most important factors of today’s intelligent systems but are relatively scarce, people try exchanging data with other organizations or public marketplaces. However, data exchange is now lack of a secure execution mechanism to automatically and fairly protect the rights which are seriously concerned by data owners. For instance, data owners cannot limit the identity of a data transferee who is banned to take the data by any way. Moreover, data transferees are also afraid that their applications for exchange may be unfairly treated. For instance, data owners could dynamically change data price when multiple transferees submit their applications at the same time. A data marketplace can take over the role of a trusted third party during data exchange, but single point failure is always a serious problem in a large-scale distributed system. Furthermore, these marketplaces usually use contracts based on natural languages which may be maliciously misinterpreted and cannot be automatically executed. Numerous researches have been presented to provide digital rights management (DRM) [1]. For instance, Zhu et al. [2] presented a privacypreserving video subscription scheme with the limitation of expire date. However, rights of data owners are more complex than rights of digital content owners. Decentralized and trustworthy enforcement for access control policies is a long-term issue. Han et al. [3] proposed an optimized mechanism which is a trusted and decentralized access control framework for the client/server architecture. These studies still need third party authorities which are used to execute or supervise the rights control, leading to a few trust and security issues. Existing studies [4] on attribute-based access control have been applied to many fields, such as e-commerce and Internet of Things. This article introduces attribute-based access control and proposes a secure framework for data exchange based on smart contracts (DESC) to enable automatic and fair access control. Featuring with decentralized, fair, transparent, immutable and traceable advantages, blockchain 2.0 [5] provides a generalized framework for implementing decentralized compute resources named smart contracts which define rights and policies in mathematical and programming forms. Once the predefined smart contract is triggered by a transaction, it can automatically execute the specific contractual clauses. To find out which data rights are seriously concerned by data owners, we investigate the DRM mechanisms and several data exchange platforms: GBDEX, national engineering lab for big data distribution and exchange technologies. Then,

Author: Arash Golchubian Title: Utilizing a Game Theoretical Approach to Prevent Collusion and Incentivize Cooperation in Cybersecurity Contexts Institution: Florida Atlantic University Thesis Advisor: Dr. Mehrdad Nojoumian Degree: Master of Science Year: 2017 In this research, a new reputation-based model is utilized to disincentivize collusion of defenders and attackers in Software Defined Networks (SDN), and also, to disincentivize dishonest mining strategies in Blockchain. In the context of SDN, the model uses the reputation values assigned to each entity to disincentivize collusion with an attacker. Our analysis shows that not-colluding actions become Nash Equilibrium using the reputationbased model within a repeated game setting. In the context of Blockchain and mining, we illustrate that by using the same socio-rational model, miners not only are incentivized to conduct honest mining but also disincentivized to commit to any malicious activities against other mining pools. We therefore show that honest mining strategies become Nash Equilibrium in our setting. This thesis is laid out in the following manner. In chapter 2 an introduction to game theory is provided followed by a survey of previous works in game theoretic network security, in chapter 3 a new reputation-based model is introduced to be used within the context of a Software Defined Network (SDN), in chapter 4 a reputation-based solution concept is introduced to force cooperation by each mining entity in Blockchain, and finally, in chapter 5, the concluding remarks and future works are presented. vii To: My loving wife, Sanaz. You inspire me to better myself. and My mother, Mina. You showed me that nothing is impossible. UTILIZING A GAME THEORETICAL APPROACH TO PREVENT COLLUSION AND INCENTIVIZE COOPERATION IN CYBERSECURITY CONTEXTS List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

An advertisement and matching subscription are tightly coupled to activate a single publication routing path in content-based publish/subscribe systems. Due to this tight coupling, instantaneous updates in routing tables are required to generate alternative paths for dynamic routing. This poses serious challenges in offering scalable and robust dynamic routing in cyclic overlays when network conditions like link congestion is detected. We propose, OctopiA, a distributed publish/subscribe system for content-based dynamic routing in structured cyclic overlays. OctopiA uses a novel concept of subscription subgrouping to divide subscriptions into disjoint sets, called subscription subgroups, to eliminate tight coupling. While aiming at deployment in data center networks, OctopiA generates routing paths of minimum lengths. We use a homogeneous clustering approach with a bit-vector to realize subscription subgrouping and offer inter-cluster dynamic routing without requiring updates in routing tables. Experiments on a cluster testbed with real world data show that OctopiA reduces the number of saved advertisements in routing tables by 93%, subscription broadcast delay by 33%, static and dynamic publication delivery delays by 25% and 54%, respectively.

New scientific instruments that are being designed and deployed in the coming years will dramatically increase the need for large, real-time data transfers among scientists throughout the world. One such instrument is the Large Synoptic Survey Telescope being built in Chile that will produce 6.4 GB images every 17 seconds. This paper describes an ongoing effort to meet the demands of these large data scientific instruments through the development of an international software defined exchange point (SDX) that will meet the provisioning needs for the scientific users. The specific planned and ongoing work in SDX architecture is described with specific consideration for policy specification and security. Keywords—Software Defined Exchange; Science Data


Question: nTorrent: A Peer-to-Peer File Sharing Mechanism in Named Data Networking.

This paper presents an in-depth exploration of Information-Centric Networking (ICN), a pivotal concept that underpins numerous future Internet research initiatives. ICN is distinguished by its reliance on in-network caching, enabling multiparty communication through data replication, and by its decoupling of senders and receivers in interaction models. The overarching objective is to engineer a network infrastructure that is more adept at addressing contemporary requirements, particularly in the realms of content distribution and mobility, while simultaneously enhancing resilience against disruptions and failures. A multitude of research projects are currently delving into the ICN approach, and we undertake a comparative analysis of the design decisions and features that characterize these proposed ICN architectures. Our focus is primarily on five critical components: the treatment of named data objects, the mechanisms of naming and security, the design of the Application Programming Interface (API), the principles of routing and transport, and the strategies employed for caching. Additionally, we elucidate the myriad benefits that the ICN approach offers, underscoring its potential to revolutionize the future of networking.

We present Resilient Distributed Datasets (RDDs), a distributed memory abstraction that allows programmers to perform in-memory computations on large clusters while retaining the fault tolerance of data flow models like MapReduce. RDDs are motivated by two types of applications that current data flow systems handle inefficiently: iterative algorithms, which are common in graph applications and machine learning, and interactive data mining tools. In both cases, keeping data in memory can improve performance by an order of magnitude. To achieve fault tolerance efficiently, RDDs provide a highly restricted form of shared memory: they are read-only datasets that can only be constructed through bulk operations on other RDDs. However, we show that RDDs are expressive enough to capture a wide class of computations, including MapReduce and specialized programming models for iterative jobs such as Pregel. Our implementation of RDDs can outperform Hadoop by 20× for iterative jobs and can be used interactively to search a 1 TB dataset with latencies of 5–7 seconds.

This paper is a survey of the current state of the art in the design and implementation of distributed file systems. It consists of four major parts: an overview of background material, case studies of a number of contemporary file systems, identification of key design techniques, and an examination of current research issues. The systems surveyed are Sun NFS, Apollo Domain, Andrew, IBM AIX DS, AT&T RFS, and Sprite. The coverage of background material includes a taxonomy of file system issues, a brief history of distributed file systems, and a summary of empirical research on file properties. A comprehensive bibliography forms an important of the paper.

Structured peer-to-peer (P2P) lookup services—such as Chord, CAN, Pastry and Tapestry—organize peers into a flat overlay network and offer distributed hash table (DHT) functionality. In these systems, data is associated with keys and each peer is responsible for a subset of the keys. We study hierarchical DHTs, in which peers are organized into groups, and each group has its autonomous intra-group overlay network and lookup service. The groups themselves are organized in a top-level overlay network. To find a peer that is responsible for a key, the top-level overlay first determines the group responsible for the key; the responsible group then uses its intra-group overlay to determine the specific peer that is responsible for the key. After providing a general framework for hierarchical P2P lookup, we consider the specific case of a two-tier hierarchy that uses Chord for the top level. Our analysis shows that by designating the most reliable peers in the groups as superpeers, the hierarchical design can significantly reduce the expected number of hops in Chord. We also propose a scalable design for managing the groups and the superpeers.

In flash storage, an FTL is a complex piece of code that resides completely inside the storage device and is provided by the manufacturer. Its principal virtue is providing interoperability with conventional HDDs. However, this virtue is also its biggest impediment in reaching the full performance of the underlying flash storage. We propose to refactor the flash storage architecture so that it relies on a new block I/O interface which does not permit overwriting of data without intervening erasures.We demonstrate how high-level applications, in particular file systems, can deal with this restriction efficiently by employing append-only segments. This refactoring dramatically reduces flash management overhead and improves performance of applications, such as file systems and databases, by permitting them to directly manage flash storage. Our experiments on a machine with the new block I/O interface show that DRAM in the flash controller is reduced by 128X and the performance of the file system improves by 80% over conventional SSDs.

This paper describes the design, implementation, and evaluation of MiMaze, a distributed 3D multiplayer game on the Internet. The major contribution of this work is to provide the first experimental results for this new type of application. An original fully distributed communication architecture has been designed for MiMaze. It uses multicast communication system based on RTP/UDP/IP, and a distributed synchronization mechanisms to guarantee the consistency of the game, regardless of heterogeneous network delay. This paper provides evaluation results on the Mbone. The experimental evaluation shows that the major problem will not be with real-time interaction, but with scalability.

The proposed architecture is structured into a secure API, a backbone, and separate device networks with standard interface to the backbone. The API decouples innovation of services and service logic from protocols and network elements. It also enables service portability between systems, i.e. a service may be allocated to end-systems or servers, with possible relocation and replication throughout its lifecycle. Ubiquitous service provisioning depends on interoperability, not only for provisioning of a standard QoS controlled IP bearer, but also for cross domain naming, security, mobility, multicast, location, routing and management, including fair compensation for utility provisioning. The proposed architecture not only includes these critical elements but also caters for multi-homing, mobile networks with dynamic membership, and third party persistent storage based on indirection.

An advertisement and matching subscription are tightly coupled to activate a single publication routing path in content-based publish/subscribe systems. Due to this tight coupling, instantaneous updates in routing tables are required to generate alternative paths for dynamic routing. This poses serious challenges in offering scalable and robust dynamic routing in cyclic overlays when network conditions like link congestion is detected. We propose, OctopiA, a distributed publish/subscribe system for content-based dynamic routing in structured cyclic overlays. OctopiA uses a novel concept of subscription subgrouping to divide subscriptions into disjoint sets, called subscription subgroups, to eliminate tight coupling. While aiming at deployment in data center networks, OctopiA generates routing paths of minimum lengths. We use a homogeneous clustering approach with a bit-vector to realize subscription subgrouping and offer inter-cluster dynamic routing without requiring updates in routing tables. Experiments on a cluster testbed with real world data show that OctopiA reduces the number of saved advertisements in routing tables by 93%, subscription broadcast delay by 33%, static and dynamic publication delivery delays by 25% and 54%, respectively.

The blockchain-based digital content distribution system was developed. Decentralized and pear-to-pear authentication mechanism can be considered as the ideal rights management mechanism. The blockchain has the potential to realize this ideal content distribution system. This is the successful model of the Superdistribution concept which was announced almost 30 years ago. The proposed system was demonstrated and got a lot of feedback for the future practical system.

Disk-oriented approaches to online storage are becoming increasingly problematic: they do not scale gracefully to meet the needs of large-scale Web applications, and improvements in disk capacity have far outstripped improvements in access latency and bandwidth. This paper argues for a new approach to datacenter storage called RAMCloud, where information is kept entirely in DRAM and large-scale systems are created by aggregating the main memories of thousands of commodity servers. We believe that RAMClouds can provide durable and available storage with 100-1000x the throughput of disk-based systems and 100-1000x lower access latency. The combination of low latency and large scale will enable a new breed of dataintensive applications.

In this paper, we present Nyx, the first system to both effectively mitigate modern Distributed Denial of Service (DDoS) attacks regardless of the amount of traffic under adversarial control and function without outside cooperation or an Internet redesign. Nyx approaches the problem of DDoS mitigation as a routing problem rather than a filtering problem. This conceptual shift allows Nyx to avoid many of the common shortcomings of existing academic and commercial DDoS mitigation systems. By leveraging how Autonomous Systems (ASes) handle route advertisement in the existing Border Gateway Protocol (BGP), Nyx allows the deploying AS to achieve isolation of traffic from a critical upstream AS off of attacked links and onto alternative, uncongested, paths. This isolation removes the need for filtering or de-prioritizing attack traffic. Nyx controls outbound paths through normal BGP path selection, while return paths from critical ASes are controlled through the use of specific techniques we developed using existing traffic engineering principles and require no outside coordination. Using our own realistic Internet-scale simulator, we find that in more than 98% of cases our system can successfully route critical traf?c around network segments under transit-link DDoS attacks; a new form of DDoS attack where the attack traf?c never reaches the victim AS, thus invaliding defensive filtering, throttling, or prioritization strategies. More significantly, in over 95% of those cases, the alternate path provides complete congestion relief from transit-link DDoS. Nyx additionally provides complete congestion relief in over 75% of cases when the deployer is being directly attacked.


Question: nTorrent: A Peer-to-Peer File Sharing Mechanism in Named Data Networking.

This paper introduces Chronos, a groundbreaking serverless design for multi-user chat applications operating over Named Data Networking (NDN). Unlike conventional multi-user applications that rely on centralized servers, Chronos employs a distributed approach, enabling all participants to share their views through the exchange of cryptographic digests associated with the chat room data set. When a new message is generated, it triggers a change in the digest at the originator's end, prompting efficient retrieval of the updated data by all other participants and facilitating the synchronization of chat room views. By adopting a serverless architecture, Chronos ensures there is no single point of failure and effectively mitigates the traffic concentration issues associated with server-based implementations. To assess Chronos' performance and compare it with traditional server-based chat room designs, we conducted simulations. The results highlight Chronos' resilience and efficiency in data dissemination. The innovative concept of replacing centralized servers with distributed data synchronization, as exemplified by Chronos, holds promise for simplifying the design and facilitating the deployment of a wide range of distributed applications.

New scientific instruments that are being designed and deployed in the coming years will dramatically increase the need for large, real-time data transfers among scientists throughout the world. One such instrument is the Large Synoptic Survey Telescope being built in Chile that will produce 6.4 GB images every 17 seconds. This paper describes an ongoing effort to meet the demands of these large data scientific instruments through the development of an international software defined exchange point (SDX) that will meet the provisioning needs for the scientific users. The specific planned and ongoing work in SDX architecture is described with specific consideration for policy specification and security. Keywords—Software Defined Exchange; Science Data

While demands on video traffic over mobile networks have been souring, the wireless link capacity cannot keep up with the traffic demand. The gap between the traffic demand and the link capacity, along with time-varying link conditions, results in poor service quality of video streaming over mobile networks such as long buffering time and intermittent disruptions. Leveraging the cloud computing technology, we propose a new mobile video streaming framework, dubbed AMES-Cloud, which has two main parts: adaptive mobile video streaming (AMoV) and efficient social video sharing (ESoV). AMoV and ESoV construct a private agent to provide video streaming services efficiently for each mobile user. For a given user, AMoV lets her private agent adaptively adjust her streaming flow with a scalable video coding technique based on the feedback of link quality. Likewise, ESoV monitors the social network interactions among mobile users, and their private agents try to prefetch video content in advance. We implement a prototype of the AMES-Cloud framework to demonstrate its performance. It is shown that the private agents in the clouds can effectively provide the adaptive streaming, and perform video sharing (i.e., prefetching) based on the social network analysis.

Data retrieval systems process data from many sources, each subject to its own data use policy. Ensuring compliance with these policies despite bugs, misconfiguration, or operator error in a large, complex, and fast evolving system is a major challenge. Thoth provides an efficient, kernel-level compliance layer for data use policies. Declarative policies are attached to the systems’ input and output files, key-value tuples, and network connections, and specify the data’s integrity and confidentiality requirements. Thoth tracks the flow of data through the system, and enforces policy regardless of bugs, misconfigurations, compromises in application code, or actions by unprivileged operators. Thoth requires minimal changes to an existing system and has modest overhead, as we show using a prototype Thoth-enabled data retrieval system based on the popular Apache Lucene.

Peer prediction mechanisms allow the truthful elicitation of private signals (e.g., experiences, or opinions) in regard to a true world state when this ground truth is unobservable. The original peer prediction method is incentive compatible for any number of agents n ≥ 2, but relies on a common prior, shared by all agents and the mechanism. The Bayesian Truth Serum (BTS) relaxes this assumption. While BTS still assumes that agents share a common prior, this prior need not be known to the mechanism. However, BTS is only incentive compatible for a large enough number of agents, and the particular number of agents required is uncertain because it depends on this private prior. In this paper, we present a robust BTS for the elicitation of binary information which is incentive compatible for every n ≥ 3, taking advantage of a particularity of the quadratic scoring rule. The robust BTS is the first peer prediction mechanism to provide strict incentive compatibility for every n ≥ 3 without relying on knowledge of the common prior. Moreover, and in contrast to the original BTS, our mechanism is numerically robust and ex post individually rational. Introduction Web services that are built around user-generated content are ubiquitous. Examples include reputation systems, where users leave feedback about the quality of products or services, and crowdsourcing platforms, where users (workers) are paid small rewards to do human computation tasks, such as annotating an image. Whereas statistical estimation techniques (Raykar et al. 2010) can be used to resolve noisy inputs, for example in order to determine the image tags most likely to be correct, they are appropriate only when user inputs are informative in the first place. But what if providing accurate information is costly for users, or if users otherwise have an external incentive for submitting false inputs? The peer prediction method (Miller, Resnick, and Zeckhauser 2005) addresses the quality control problem by providing payments (in cash, points or otherwise) that align an agent’s own interest with providing inputs that are predictive of the inputs that will be provided by other agents. Formally, the peer prediction method provides strict incentives for providing truthful inputs (e.g., in regard to a user’s information about the quality of a product, or a user’s view on Copyright c © 2012, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. the correct label for a training example) for a system of two or more agents, and when there is a common prior amongst agents and, critically, known to the mechanism. The Bayesian Truth Serum (BTS) by Prelec (2004) still assumes that agents share a common prior, but does not require this to be known by the mechanism. In addition to an information report from an agent, BTS asks each agent for a prediction report, that reflects the agent’s belief about the distribution of information reports in the population. An agent’s payment depends on both reports, with an information component that rewards reports that are “surprisingly common,” i.e., more common than collectively predicted, and a prediction component that rewards accurate predictions of the reports made by others. A significant drawback of BTS is that it only aligns incentives for a large enough number of agents, where this number depends on the prior and is thus unknown to the mechanism. In addition, BTS may leave a participant with a negative payment, and is not numerically robust for all inputs. In this paper, we present the robust Bayesian Truth Serum (RBTS) mechanism, which, to the best of our knowledge, is the first peer prediction mechanism that does not rely on knowledge of the common prior to provide strict incentive compatibility for every number of agents n ≥ 3. RBTS is also ex post individually rational (so that no agent makes a negative payment in any outcome) and numerically robust, being well defined for all possible agent reports. Moreover, the mechanism seems conceptually simpler than BTS, and the incentive analysis is more straightforward. The main limitation of RBTS relative to earlier mechanisms, is that it applies only to the elicitation of binary information; e.g., good or bad experiences, or true or false classification labels.1 Extending RBTS to incorporate more than two signals is the most important direction for future research. RBTS takes the same reports as BTS, and an agent’s payment continues to consist of one component that depends on an agent’s information report and a second component that Many interesting applications involve binary information reports. This is supported by the fact that Prelec’s own experimental papers have adopted the binary signal case (Prelec and Seung 2006; John, Loewenstein, and Prelec 2011). Indeed, as the number of possible information reports increases, so does the difficulty imposed on users in providing the prediction report, which must include estimates for the additional possible information reports. depends on an agent’s prediction report. The main innovation is to induce a “shadow” posterior belief report for an agent i from her information report and the prediction report of another agent j, adjusting this prediction report in the direction suggested by agent i’s information report. We couple this with a particularity of the quadratic scoring rule, by which an agent prefers a shadow posterior belief that is as close as possible to her true posterior. In order to determine the agent’s payment, we then apply both the shadow posterior belief and the agent’s prediction report to the quadratic scoring rule, adopting the information report of a third agent k as the event to be predicted.

The blockchain paradigm when coupled with cryptographically-secured transactions has demonstrated its utility through a number of projects, with Bitcoin being one of the most notable ones. Each such project can be seen as a simple application on a decentralised, but singleton, compute resource. We can call this paradigm a transactional singleton machine with shared-state. Ethereum implements this paradigm in a generalised manner. Furthermore it provides a plurality of such resources, each with a distinct state and operating code but able to interact through a message-passing framework with others. We discuss its design, implementation issues, the opportunities it provides and the future hurdles we envisage.

This paper presents our field experience in data collection from remote sensors. By letting tractors, farmers, and sensors have short-range radio communication devices with delay-disruption tolerant networking (DTN), we can collect data from those sensors to our central database. Although, several implementations have been made with cellular phones or mesh networks in the past, DTN-based systems for such applications are still under explored. The main objective of this paper is to present our practical implementation and experiences in DTN-based data collection from remote sensors. The software, which we have developed for this research, has about 50 kbyte footprint, which is much smaller than any other DTN implementation. We carried out an experiment with 39 DTN nodes at the University of Tokyo assuming an agricultural scenario. They achieved 99.8% success rate for data gathering with moderate latency, showing sufficient usefulness in data granularity.

Caching introduces the overhead and complexity of ensuring consistency, reducing some of its performance benefits. In a distributed system, caching must deal with the additional complications of communication and host failures.
Leases are proposed as a time-based mechanism that provides efficient consistent access to cached data in distributed systems. Non-Byzantine failures affect performance, not correctness, with their effect minimized by short leases. An analytic model and an evaluation for file access in the V system show that leases of short duration provide good performance. The impact of leases on performance grows more significant in systems of larger scale and higher processor performance.

Network Function Virtualization has received a large amount of research and recent efforts have been made to further leverage the cloud to enhance NFV. However, since there are privacy and security issues with using cloud computing, work has been done to allow for operating on encrypted data, which introduces a large amount of overhead in both computation and data, while only providing a limited set of operations, since these encryption schemes are not fully homomorphic. We propose using trusted computing to circumvent these limitations by having hardware enforce data privacy and provide guaranteed computation. Prior work has shown that Intel's Software Guard Extensions can be used to protect the state of network functions, but there are still questions about the usability of SGX in arbitrary NFV applications and the performance of SGX in these applications. We extend prior work to show how SGX can be used in network deployments by extending the Click modular router to perform secure packet processing with SGX. We also present a performance evaluation of SGX on real hardware to show that processing inside of SGX has a negligible performance impact, compared to performing the same processing outside of SGX.

With data storage and sharing services in the cloud, users can easily modify and share data as a group. To ensure shared data integrity can be verified publicly, users in the group need to compute signatures on all the blocks in shared data. Different blocks in shared data are generally signed by different users due to data modifications performed by different users. For security reasons, once a user is revoked from the group, the blocks which were previously signed by this revoked user must be re-signed by an existing user. The straightforward method, which allows an existing user to download the corresponding part of shared data and re-sign it during user revocation, is inefficient due to the large size of shared data in the cloud. In this paper, we propose a novel public auditing mechanism for the integrity of shared data with efficient user revocation in mind. By utilizing the idea of proxy re-signatures, we allow the cloud to re-sign blocks on behalf of existing users during user revocation, so that existing users do not need to download and re-sign blocks by themselves. In addition, a public verifier is always able to audit the integrity of shared data without retrieving the entire data from the cloud, even if some part of shared data has been re-signed by the cloud. Moreover, our mechanism is able to support batch auditing by verifying multiple auditing tasks simultaneously. Experimental results show that our mechanism can significantly improve the efficiency of user revocation.

Nowadays, cloud-based storage services are rapidly growing and becoming an emerging trend in data storage field. There are many problems when designing an efficient storage engine for cloud-based systems with some requirements such as big-file processing, lightweight meta-data, low latency, parallel I/O, deduplication, distributed, high scalability. Key-value stores played an important role and showed many advantages when solving those problems. This paper presents about Big File Cloud (BFC) with its algorithms and architecture to handle most of problems in a big-file cloud storage system based on key-value store. It is done by proposing low-complicated, fixed-size meta-data design, which supports fast and highly-concurrent, distributed file I/O, several algorithms for resumable upload, download and simple data deduplication method for static data. This research applied the advantages of ZDB - an in-house key-value store which was optimized with auto-increment integer keys for solving big-file storage problems efficiently. The results can be used for building scalable distributed data cloud storage that support big-file with size up to several terabytes.


Question: nTorrent: A Peer-to-Peer File Sharing Mechanism in Named Data Networking

This paper introduces the Named-data Link State Routing (NLSR) protocol, a specialized routing mechanism designed for the Named Data Networking (NDN) paradigm. Unlike traditional IP-based routing protocols, NLSR focuses on propagating reachability to name prefixes rather than IP prefixes, capitalizing on NDN's naming convention for data identification and retrieval. NLSR diverges significantly from conventional link-state routing protocols in two key aspects. Firstly, it utilizes Interest/Data packets for routing update dissemination, leveraging NDN's inherent data authenticity feature. Secondly, NLSR generates a prioritized list of forwarding options for each name prefix, enhancing NDN's adaptive forwarding capabilities. The paper delves into the core design decisions of NLSR, encompassing: (1) a hierarchical naming structure for routers, keys, and routing updates; (2) a hierarchical trust model for secure routing within a single administrative domain; (3) a hop-by-hop synchronization protocol as an alternative to the widespread flooding method for routing update distribution; and (4) a straightforward mechanism to rank multiple forwarding alternatives. When contrasted with IP-based link state routing, NLSR offers more efficient update propagation, intrinsic update authentication, and inherent support for multipath forwarding, showcasing its superiority in the realm of NDN.

This paper presents a novel multi-copy routing protocol for disruption-tolerant networks whose objective is to minimize energy expended on communication. The protocol is designed for disaster-response applications, where power and infrastructure resources are disrupted. Unlike other delay-tolerant networks, energy is a vital resource in post-disaster scenarios to ensure availability of (disruption-tolerant) communication until infrastructure is restored. Our approach exploits naturally recurrent mobility and contact patterns in the network, formed by rescue workers, volunteers, survivors, and their (possibly stranded) vehicles to reduce the number of message copies needed to attain an adequate delivery ratio in the face of disconnection and intermittent connectivity. A new notion of inter-contact routing is proposed that allows estimating route delays and delivery probabilities, identifying more reliable routes and controlling message replication and forwarding accordingly. We simulate the scheme using a mobility model that reflects recurrence inspired by disaster scenarios, and compare our results to previous DTN routing techniques. The evaluation shows that the new approach reduces the resource overhead per message over previous approaches while maintaining a comparable delivery ratio at the expense of a small (bounded) increase in latency.

Many decades of work have been invested in the area of distributed transactions including protocols such as 2PC, Paxos, and various approaches to quorum. These protocols provide the application programmer a façade of global serializability. Personally, I have invested a nontrivial portion of my career as a strong advocate for the implementation and use of platforms providing guarantees of global serializability. My experience over the last decade has led me to liken these platforms to the Maginot Line. In general, application developers simply do not implement large scalable applications assuming distributed transactions. When they attempt to use distributed transactions, the projects founder because the performance costs and fragility make them impractical. Natural selection kicks in... 1 The Maginot Line was a huge fortress that ran the length of the Franco-German border and was constructed at great expense between World War I and World War II. It successfully kept the German army from directly crossing the border between France and Germany. It was quickly bypassed by the Germans in 1940 who invaded through Belgium. This article is published under a Creative Commons License Agreement (http://creativecommons.org/licenses/by/2.5/). You may copy, distribute, display, and perform the work, make derivative works and make commercial use of the work, but you must attribute the work to the author and CIDR 2007. 3 Biennial Conference on Innovative DataSystems Research (CIDR) January 7-10, Asilomar, California USA. Instead, applications are built using different techniques which do not provide the same transactional guarantees but still meet the needs of their businesses. This paper explores and names some of the practical approaches used in the implementations of large-scale mission-critical applications in a world which rejects distributed transactions. We discuss the management of fine-grained pieces of application data which may be repartitioned over time as the application grows. We also discuss the design patterns used in sending messages between these repartitionable pieces of data. The reason for starting this discussion is to raise awareness of new patterns for two reasons. First, it is my belief that this awareness can ease the challenges of people hand-crafting very large scalable applications. Second, by observing the patterns, hopefully the industry can work towards the creation of platforms that make it easier to build these very large applications.

We present the design, implementation, and evaluation of B4, a private WAN connecting Google's data centers across the planet. B4 has a number of unique characteristics: i) massive bandwidth requirements deployed to a modest number of sites, ii) elastic traffic demand that seeks to maximize average bandwidth, and iii) full control over the edge servers and network, which enables rate limiting and demand measurement at the edge.
 These characteristics led to a Software Defined Networking architecture using OpenFlow to control relatively simple switches built from merchant silicon. B4's centralized traffic engineering service drives links to near 100% utilization, while splitting application flows among multiple paths to balance capacity against application priority/demands. We describe experience with three years of B4 production deployment, lessons learned, and areas for future work.

This paper describes the design, implementation, and evaluation of MiMaze, a distributed 3D multiplayer game on the Internet. The major contribution of this work is to provide the first experimental results for this new type of application. An original fully distributed communication architecture has been designed for MiMaze. It uses multicast communication system based on RTP/UDP/IP, and a distributed synchronization mechanisms to guarantee the consistency of the game, regardless of heterogeneous network delay. This paper provides evaluation results on the Mbone. The experimental evaluation shows that the major problem will not be with real-time interaction, but with scalability.

Our growing reliance on online services accessible on the Internet demands highly-available systems that provide correct service without interruptions. Byzantine faults such as software bugs, operator mistakes, and malicious attacks are the major cause of service interruptions. This thesis describes a new replication algorithm, BFT, that can be used to build highly-available systems that tolerate Byzantine faults. It shows, for the first time, how to build Byzantine-fault-tolerant systems that can be used in practice to implement real services because they do not rely on unrealistic assumptions and they perform well. BFT works in asynchronous environments like the Internet, it incorporates mechanisms to defend against Byzantine-faulty clients, and it recovers replicas proactively. The recovery mechanism allows the algorithm to tolerate any number of faults over the lifetime of the system provided fewer than 1/3 of the replicas become faulty within a small window of vulnerability. The window may increase under a denial-of-service attack but the algorithm can detect and respond to such attacks and it can also detect when the state of a replica is corrupted by an attacker. BFT has been implemented as a generic program library with a simple interface. The BFT library provides a complete solution to the problem of building real services that tolerate Byzantine faults. We used the library to implement the first Byzantine-fault-tolerant NFS file system, BFS. The BFT library and BFS perform well because the library incorporates several important optimizations. The most important optimization is the use of symmetric cryptography to authenticate messages. Public-key cryptography, which was the major bottleneck in previous systems, is used only to exchange the symmetric keys. The performance results show that BFS performs 2% faster to 24% slower than production implementations of the NFS protocol that are not replicated. Therefore, we believe that the BFT library can be used to build practical systems that tolerate Byzantine faults. Thesis Supervisor: Barbara H. Liskov Title: Ford Professor of Engineering

Chain replication is a new approach to coordinating clusters of fail-stop storage servers. The approach is intended for supporting large-scale storage services that exhibit high throughput and availability without sacrificing strong consistency guarantees. Besides outlining the chain replication protocols themselves, simulation experiments explore the performance characteristics of a prototype implementation. Throughput, availability, and several objectplacement strategies (including schemes based on distributed hash table routing) are discussed.

Network-on-Chips (NoCs) are likely to become a critical shared resource in future many-core processors. The challenge is to develop policies and mechanisms that enable multiple applications to efficiently and fairly share the network, to improve system performance. Existing local packet scheduling policies in the routers fail to fully achieve this goal, because they treat every packet equally, regardless of which application issued the packet.
 This paper proposes prioritization policies and architectural extensions to NoC routers that improve the overall application-level throughput, while ensuring fairness in the network. Our prioritization policies are application-aware, distinguishing applications based on the stall-time criticality of their packets. The idea is to divide processor execution time into phases, rank applications within a phase based on stall-time criticality, and have all routers in the network prioritize packets based on their applications' ranks. Our scheme also includes techniques that ensure starvation freedom and enable the enforcement of system-level application priorities.
 We evaluate the proposed prioritization policies on a 64-core CMP with an 8x8 mesh NoC, using a suite of 35 diverse applications. For a representative set of case studies, our proposed policy increases average system throughput by 25.6% over age-based arbitration and 18.4% over round-robin arbitration. Averaged over 96 randomly-generated multiprogrammed workload mixes, the proposed policy improves system throughput by 9.1% over the best existing prioritization policy, while also reducing application-level unfairness.

Advances in processor, memory and radio technology will enable small and cheap nodes capable of sensing, communication and computation. Networks of such nodes can coordinate to perform distributed sensing of environmental phenomena. In this paper, we explore the directed diffusion paradigm for such coordination. Directed diffusion is datacentric in that all communication is for named data. All nodes in a directed diffusion-based network are application-aware. This enables diffusion to achieve energy savings by selecting empirically good paths and by caching and processing data in-network. We explore and evaluate the use of directed diffusion for a simple remote-surveillance sensor network.

Clustering as one of the main branches of data mining, has gained an important place in the different applied fields. On the other hand, Peer-to-Peer (P2P) networks with features such as simplicity, low cost communication, and high availability resources, have gained a worldwide popularity in the present days. In P2P network, high volumes of data are distributed between dispersed data sources. In such distributed systems, sending data to a central site is not possible, due to processing, storage and handling costs. This issue makes the need for specific methods for distributed data clustering in P2P networks. This paper makes a technical review of distributed data clustering approaches in P2P network, to understand the research trends in this area. Keywords—Peer to Peer Networks, Distributed Data Mining, Distributed Data Clustering

The sharing economy, the business of collectively using privately owned objects and services, has fuelled some of the fastest growing businesses of the past years. However, popular sharing platforms like Airbnb or Uber exhibit several drawbacks: a cumbersome sign up procedure, lack of participant privacy, overbearing terms and conditions, and significant fees for users. We demonstrate a Decentralised App (DAPP) for the sharing of everyday objects based on a smart contract on the Ethereum blockchain. This contract enables users to register and rent devices without involvement of a Trusted Third Party (TTP), disclosure of any personal information or prior sign up to the service. With increasing distribution of cryptocurrencies the use of smart contracts such as proposed in this paper has the potential to revolutionise the sharing economy.


Question: nTorrent: A Peer-to-Peer File Sharing Mechanism in Named Data Networking.

In the realm of Named Data Networking (NDN), an emerging future Internet architecture, the issue of Interest flooding attacks poses a significant threat, necessitating robust countermeasures. Unlike traditional Distributed Denial of Service (DDoS) attacks, which inundate victim sites with packets from compromised hosts, NDN's design, where end users send Interest packets to request data and receive Data packets in response, theoretically mitigates many DDoS vulnerabilities. However, NDN is susceptible to a unique DDoS variant, the Interest packet flooding attack. This paper delves into the development of efficient strategies to counteract Interest flooding in NDN networks. We leverage NDN's intrinsic features, such as per packet state storage on routers and flow balance maintenance (limiting one Interest packet to retrieve no more than one Data packet), to devise potent DDoS mitigation algorithms. Through simulation studies, we validate that our solution is capable of swiftly and effectively responding to and alleviating the impact of Interest flooding attacks in NDN environments.

The “blockchain” distributed ledger pioneered by Bitcoin is effective at preventing double-spending, but inherently attracts (1) “user cartels” and (2) incompressible delays, as a result of linear verification and a winner-takes-all incentive lottery. We propose to forgo the “blocks” and “chain” entirely, and build a truly distributed ledger system based on a lean graph of cross-verifying transactions, which now become the main and only objects in the system. A fully distributed consensus mechanism, based on progressive proofs of work with predictable incentives, ensures rapid convergence even across a large network of unequal participants, who all get rewards working at their own pace. Graph-based affirmation fosters snappy response through automatic scaling, while application-agnostic design supports all modern cryptocurrency features such as multiple denominations, swaps, securitisation, scripting, smart contracts, etc. We prove theoretically, and experimentally verify, our proposal to show it achieves a crucial “convergence” property — meaning that any valid transaction entering the system will quickly become enshrined into the ancestry upon which all future transactions will rest.

Digital information has become a social infrastructure and with the expansion of the Internet, network infrastructure has become an indispensable part of social life and industrial activity for mankind. The idea of using existing electronics in smart home appliances and connecting them to the Internet is a new dimension along which technologies continue to grow, and in recent years mankind has witnessed an upsurge of usage of devices such as smart phone, smart television, home health-care device, smart LED light bulbs system, etc. Their build-in internet-controlled function has made them quite attractive to many segments of consumers and smart phone has become a common gadget for social networking. There are, however, serious challenges which need to be addressed as these tiny devices are designed for specific functions and lack processing capacity required for most security software. This research explores how these internet-enabled smart devices can be turned into very dangerous spots for distributed attacks purposes by cybercriminals for various ill intensions in a pinpointed manner. It then introduces a new approach to deal with such problems by taking advantage of the anonymous communication of the Onion Router (hereafter: TOR). It compares pros and cons of using anonymous communication scheme and justifies it be an efficient countermeasure to most attack scenarios.

We present the design and implementation of an end-to-end architecture for Internet host mobility using dynamic updates to the Domain Name System (DNS) to track host location. Existing TCP connections are retained using secure and efficient connection migration, enabling established connections to seamlessly negotiate a change in endpoint IP addresses without the need for a third party. Our architecture is secure—name updates are effected via the secure DNS update protocol, while TCP connection migration uses a novel set of Migrate options—and provides a pure end-system alternative to routing-based approaches such as Mobile IP.
Mobile IP was designed under the principle that fixed Internet hosts and applications were to remain unmodified and only the underlying IP substrate should change. Our architecture requires no changes to the unicast IP substrate, instead modifying transport protocols and applications at the end hosts. We argue that this is not a hindrance to deployment; rather, in a significant number of cases, it allows for an easier deployment path than Mobile IP, while simultaneously giving better performance. We compare and contrast the strengths of end-to-end and network-layer mobility schemes, and argue that end-to-end schemes are better suited to many common mobile applications. Our performance experiments show that hand-off times are governed by TCP migrate latencies, and are on the order of a round-trip time of the communicating peers.

As one of the fundamental infrastructures for cloud computing, data center networks (DCN) have recently been studied extensively. We currently use pure software-based systems, FPGA based platforms, e.g., NetFPGA, or OpenFlow switches, to implement and evaluate various DCN designs including topology design, control plane and routing, and congestion control. However, software-based approaches suffer from high CPU overhead and processing latency; FPGA based platforms are difficult to program and incur high cost; and OpenFlow focuses on control plane functions at present. In this paper, we design a ServerSwitch to address the above problems. ServerSwitch is motivated by the observation that commodity Ethernet switching chips are becoming programmable and that the PCI-E interface provides high throughput and low latency between the server CPU and I/O subsystem. ServerSwitch uses a commodity switching chip for various customized packet forwarding, and leverages the server CPU for control and data plane packet processing, due to the low latency and high throughput between the switching chip and server CPU. We have built our ServerSwitch at low cost. Our experiments demonstrate that ServerSwitch is fully programmable and achieves high performance. Specifically, we have implemented various forwarding schemes including source routing in hardware. Our in-network caching experiment showed high throughput and flexible data processing. Our QCN (Quantized Congestion Notification) implementation further demonstrated that ServerSwitch can react to network congestions in 23us. ∗Correspondent author. †This work was performed when Zhiqiang Zhou was a visiting student at Microsoft Research Asia.

TCP and its variants have suffered from surprisingly poor performance for decades. We argue the TCP family has little hope to achieve consistent high performance due to a fundamental architectural deficiency: hardwiring packet-level events to control responses without understanding the real performance result of its actions. We propose Performance-oriented Congestion Control (PCC), a new congestion control architecture in which each sender continuously observes the connection between its actions and empirically experienced performance, enabling it to consistently adopt actions that result in high performance. We prove that PCC converges to a stable and fair equilibrium. Across many real-world and challenging environments, PCC shows consistent and often 10× performance improvement, with better fairness and stability than TCP. PCC requires no router hardware support or new packet format.

Chain replication is a new approach to coordinating clusters of fail-stop storage servers. The approach is intended for supporting large-scale storage services that exhibit high throughput and availability without sacrificing strong consistency guarantees. Besides outlining the chain replication protocols themselves, simulation experiments explore the performance characteristics of a prototype implementation. Throughput, availability, and several objectplacement strategies (including schemes based on distributed hash table routing) are discussed.

This paper is a survey of the current state of the art in the design and implementation of distributed file systems. It consists of four major parts: an overview of background material, case studies of a number of contemporary file systems, identification of key design techniques, and an examination of current research issues. The systems surveyed are Sun NFS, Apollo Domain, Andrew, IBM AIX DS, AT&T RFS, and Sprite. The coverage of background material includes a taxonomy of file system issues, a brief history of distributed file systems, and a summary of empirical research on file properties. A comprehensive bibliography forms an important of the paper.

This paper studies two design tasks faced by a geo-distributed cloud data market: which data to purchase data purchasing and where to place/replicate the data for delivery data placement. We show that the joint problem of data purchasing and data placement within a cloud data market can be viewed as a facility location problem and is thus NP-hard. However, we give a provably optimal algorithm for the case of a data market made up of a single data center and then generalize the structure from the single data center setting in order to develop a near-optimal, polynomial-time algorithm for a geo-distributed data market. The resulting design, $\mathsf {Datum}$ , decomposes the joint purchasing and placement problem into two subproblems, one for data purchasing and one for data placement, using a transformation of the underlying bandwidth costs. We show, via a case study, that $\mathsf {Datum}$ is near optimal within 1.6% in practical settings.

Internet evolution has been recently related with some aspect of user empowerment, mostly in terms of content distribution, and this has been ultimately accelerated by the fast-paced introduction and expansion of wireless technologies. Hence, the Internet should start to be seen as a communications infrastructure able to support the integration of a myriad of embedded and personal wireless objects. This way a future Internet will support the interaction between users’ social, physical and virtual sphere. This position paper aims to raise some discussion about the technology required to ensure an efficient interaction between the physical, social and virtual worlds by extending the Internet by means of interconnected objects. Namely, it is argued that an efficient interaction between the physical, social and virtual worlds requires the development of a data-centric architecture based on IP-driven opportunisitc networking able to make useful data available to people when and where they really need it, augmenting their social and environmental awareness.

Memory-intensive applications suffer large performance loss when their working sets do not fully fit in memory. Yet, they cannot leverage otherwise unused remote memory when paging out to disks even in the presence of large imbalance in memory utilizations across a cluster. Existing proposals for memory disaggregation call for new architectures, new hardware designs, and/or new programming models, making them infeasible. This paper describes the design and implementation of INFINISWAP, a remote memory paging system designed specifically for an RDMA network. INFINISWAP opportunistically harvests and transparently exposes unused memory to unmodified applications by dividing the swap space of each machine into many slabs and distributing them across many machines’ remote memory. Because one-sided RDMA operations bypass remote CPUs, INFINISWAP leverages the power of many choices to perform decentralized slab placements and evictions. We have implemented and deployed INFINISWAP on an RDMA cluster without any modifications to user applications or the OS and evaluated its effectiveness using multiple workloads running on unmodified VoltDB, Memcached, PowerGraph, GraphX, and Apache Spark. Using INFINISWAP, throughputs of these applications improve between 4× (0.94×) to 15.4× (7.8×) over disk (Mellanox nbdX), and median and tail latencies between 5.4× (2×) and 61× (2.3×). INFINISWAP achieves these with negligible remote CPU usage, whereas nbdX becomes CPU-bound. INFINISWAP increases the overall memory utilization of a cluster and works well at scale.