Question: Propagating uncertainty through the hyperbolic tangent function for application in reservoir computing systems.

This document, akin to a comprehensive guide in the field of matrix algebra, has been enriched by the insights and contributions of numerous experts and enthusiasts. Among those who have generously shared their knowledge are Bill Baxter, Brian Templeton, Christian Rishøj, Christian Schröppel, Douglas L. Theobald, Esben Hoegh-Rasmussen, Glynne Casteel, Jan Larsen, Jun Bin Gao, Jürgen Struckmeier, Kamil Dedecius, Korbinian Strimmer, Lars Christiansen, Lars Kai Hansen, Leland Wilkinson, Liguo He, Loic Thibaut, Miguel Barão, Ole Winther, Pavel Sakov, Stephan Hattinger, Vasile Sima, Vincent Rabaud, and Zhaoshui He. Their collective expertise has significantly enhanced the depth and breadth of this resource. Moreover, the authors extend their gratitude to The Oticon Foundation for financially supporting their PhD studies, which played a pivotal role in the creation of this valuable compendium of matrix algebra techniques and applications.

However, this acknowledgment section does not directly address the topic of propagating uncertainty through the tanh function in the context of reservoir computing. To align more closely with the question, a more suitable document might delve into the mathematical underpinnings of the tanh function, its role in reservoir computing, and how uncertainty is managed and propagated through such systems. This could include discussions on the statistical properties of the tanh function, its application in neural networks, and methodologies for handling stochastic inputs or parameters in computational models.

While the accuracy of feature measurements heavily depends on changing environmental conditions, studying the consequences of this fact in pattern recognition tasks has received relatively little attention to date. In this paper, we explicitly take feature measurement uncertainty into account and show how multimodal classification and learning rules should be adjusted to compensate for its effects. Our approach is particularly fruitful in multimodal fusion scenarios, such as audiovisual speech recognition, where multiple streams of complementary time-evolving features are integrated. For such applications, provided that the measurement noise uncertainty for each feature stream can be estimated, the proposed framework leads to highly adaptive multimodal fusion rules which are easy and efficient to implement. Our technique is widely applicable and can be transparently integrated with either synchronous or asynchronous multimodal sequence integration architectures. We further show that multimodal fusion methods relying on stream weights can naturally emerge from our scheme under certain assumptions; this connection provides valuable insights into the adaptivity properties of our multimodal uncertainty compensation approach. We show how these ideas can be practically applied for audiovisual speech recognition. In this context, we propose improved techniques for person-independent visual feature extraction and uncertainty estimation with active appearance models, and also discuss how enhanced audio features along with their uncertainty estimates can be effectively computed. We demonstrate the efficacy of our approach in audiovisual speech recognition experiments on the CUAVE database using either synchronous or asynchronous multimodal integration models.

We show that the output of a (residual) convolutional neural network (CNN) with an appropriate prior over the weights and biases is a Gaussian process (GP) in the limit of infinitely many convolutional filters, extending similar results for dense networks. For a CNN, the equivalent kernel can be computed exactly and, unlike “deep kernels”, has very few parameters: only the hyperparameters of the original CNN. Further, we show that this kernel has two properties that allow it to be computed efficiently; the cost of evaluating the kernel for a pair of images is similar to a single forward pass through the original CNN with only one filter per layer. The kernel equivalent to a 32-layer ResNet obtains 0.84% classification error on MNIST, a new record for GPs with a comparable number of parameters. 1

Unsupervised neural networks, such as restricted Boltzmann machines (RBMs) and deep belief networks (DBNs), are powerful tools for feature selection and pattern recognition tasks. We demonstrate that overfitting occurs in such models just as in deep feedforward neural networks, and discuss possible regularization methods to reduce overfitting. We also propose a “partial” approach to improve the efficiency of Dropout/DropConnect in this scenario, and discuss the theoretical justification of these methods from model convergence and likelihood bounds. Finally, we compare the performance of these methods based on their likelihood and classification error rates for various pattern recognition data sets.

A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.

In this paper the task of training subsymbolic systems is considered as a combinatorial optimization problem and solved with the heuristic scheme of the reactive tabu search (RTS). An iterative optimization process based on a "modified local search" component is complemented with a meta-strategy to realize a discrete dynamical system that discourages limit cycles and the confinement of the search trajectory in a limited portion of the search space. The possible cycles are discouraged by prohibiting (i.e., making tabu) the execution of moves that reverse the ones applied in the most recent part of the search. The prohibition period is adapted in an automated way. The confinement is avoided and a proper exploration is obtained by activating a diversification strategy when too many configurations are repeated excessively often. The RTS method is applicable to nondifferentiable functions, is robust with respect to the random initialization, and effective in continuing the search after local minima. Three tests of the technique on feedforward and feedback systems are presented.

This paper presents two models of complex-valued neurons (CVNs) for real-valued classification problems, incorporating two newly-proposed activation functions, and presents their abilities as well as differences between them on benchmark problems. In both models, each real-valued input is encoded into a phase between 0 and  of a complex number of unity magnitude, and multiplied by a complex-valued weight. The weighted sum of inputs is fed to an activation function. Activation functions of both models map complex values into real values, and their role is to divide the net-input (weighted sum) space into multiple regions representing the classes of input patterns. The gradient-based learning rule is derived for each of the activation functions. Ability of such CVNs are discussed and tested with two-class problems, such as two and three input Boolean problems, and symmetry detection in binary sequences. We exhibit here that both the models can form proper boundaries for these linear and nonlinear problems. For solving n-class problems, a complex-valued neural network (CVNN) consisting of n CVNs is also considered in this paper. We tested such single-layered CVNNs on several real world benchmark problems. The results show that the classification and generalization abilities of single-layered CVNNs are comparable to the conventional real-valued neural networks (RVNNs) having one hidden layer. Moreover, convergence of CVNNs is much faster than that of RVNNs in most of the cases.

The main aim of this dissertation is to study the topic of wavelet neural networks and see how they are useful for dynamical systems applications such as predicting chaotic time series and nonlinear noise reduction. To do this, the theory of wavelets has been studied in the first chapter, with the emphasis being on discrete wavelets. The theory of neural networks and its current applications in the modelling of dynamical systems has been shown in the second chapter. This provides sufficient background theory to be able to model and study wavelet neural networks. In the final chapter a wavelet neural network is implemented and shown to accurately estimate the dynamics of a chaotic system, enabling prediction and enhancing methods already available in nonlinear noise reduction.

This paper proposes a new method for training multi-layer spiking convolutional neural networks (CNNs). Training a multi-layer spiking network poses difficulties because the output spikes do not have derivatives and the commonly use backpropagation method for non-spiking networks is not easily applied. Our method uses a novel version of layered spike-timing- dependent plasticity (STDP) that incorporates supervised and unsupervised components. Our method starts with conventional learning methods and converts them to spatio-temporally local rules suited for spiking neural networks (SNNs). The training process uses two components for unsupervised feature extraction and supervised classification. The first component is a new STDP rule for spike-based representation learning which trains convolutional filters. The second introduces a new STDP-based supervised learning rule for spike pattern classification via an approximation to gradient descent. Stacking these components implements a novel spiking CNN of integrate-and-fire (IF) neurons with performances comparable with the state-of-the-art deep SNNs. The experimental results show the success of the proposed model for the MNIST handwritten digit classification. Our network architecture is the only high performance, spiking CNN which provides bio-inspired STDP rules in a hierarchy of feature extraction and classification in an entirely spike-based framework.

Abstract- In this paper, a new method for adaptive control of nonlinear systems using neural networks and Proportional-Integral-Derivative (PID) methodology is proposed. In this method, a PID control and adaptive linear neural network is used to control a inverted pendulum with uncertainities and external disturbances. The system consists of an inverted pole hinged on a cart which is free to move in the x direction. The performance of the proposed method is demonstrated via simulations.

Deep neural networks have emerged as a widely used and effective means for tackling complex, real-world problems. However, a major obstacle in applying them to safety-critical systems is the great difficulty in providing formal guarantees about their behavior. We present a novel, scalable, and efficient technique for verifying properties of deep neural networks (or providing counter-examples). The technique is based on the simplex method, extended to handle the non-convex Rectified Linear Unit (ReLU ) activation function, which is a crucial ingredient in many modern neural networks. The verification procedure tackles neural networks as a whole, without making any simplifying assumptions. We evaluated our technique on a prototype deep neural network implementation of the next-generation airborne collision avoidance system for unmanned aircraft (ACAS Xu). Results show that our technique can successfully prove properties of networks that are an order of magnitude larger than the largest networks verified using existing methods.


Question: Propagating uncertainty through the hyperbolic tangent function for reservoir computing applications.

This paper explores the intriguing characteristics of deep neural networks, which, despite their remarkable success in speech and visual recognition tasks, exhibit behaviors that are counter-intuitive and difficult to interpret. The study uncovers two notable properties of these models. Firstly, it reveals that there is no significant difference between individual high-level units and random linear combinations of these units when analyzed through various methods. This suggests that the semantic information is encoded in the space formed by the high-level units, rather than in the units themselves. Secondly, the research identifies that deep neural networks often learn input-output mappings that exhibit a degree of discontinuity. This means that small, barely perceptible perturbations to an input can cause the network to misclassify it. Moreover, the nature of these perturbations is not arbitrary or a result of overfitting; the same perturbation can lead a different network, trained on a distinct subset of the data, to misclassify the same input. These findings highlight the complex and sometimes unexpected behavior of deep neural networks, challenging our understanding of how they process and represent information.

For the efficient execution of deep convolutional neural networks (CNN) on edge devices, various approaches have been presented which reduce the bit width of the network parameters down to 1 bit. Binarization of the first layer was always excluded, as it leads to a significant error increase. Here, we present the novel concept of binary input layer (BIL), which allows the usage of binary input data by learning bit specific binary weights. The concept is evaluated on three datasets (PAMAP2, SVHN, CIFAR-10). Our results show that this approach is in particular beneficial for multimodal datasets (PAMAP2) where it outperforms networks using full precision weights in the first layer by 1.92 percentage points (pp) while consuming only 2% of the chip area.

The goal of this paper is to demonstrate a method for tensorizing neural networks based upon an efficient way of approximating scale invariant quantum states, the Multi-scale Entanglement Renormalization Ansatz (MERA). We employ MERA as a replacement for linear layers in a neural network and test this implementation on the CIFAR-10 dataset. The proposed method outperforms factorization using tensor trains, providing greater compression for the same level of accuracy and greater accuracy for the same level of compression. We demonstrate MERAlayers with 3900 times fewer parameters and a reduction in accuracy of less than 1% compared to the equivalent fully connected layers.

A slope formed when there is a height difference between two points of the ground surface. Limited or ambiguous data defined in the studying of the slip surface have always been posed as a challenge to engineers and geologists; however, previous studies presented the useful strategies for resolving this challenge. In general, Multi Layer Perceptrons (MLPs) (with three input, output and hidden layers) artificial neural networks (ANNs), as well as the transfer function and sufficient number of neurons in hidden layer are able to estimate any nonlinear function with any degree of complexity. In the present study, the Zonouz embankment dam at Iran was firstly modeled by using the classic limit equilibrium method (LEM) then, obtained results were used for training the ANN. Then the effect of various parameters such as logarithmic data and training epoch number on the network and sensitivity analysis on the input parameters were discussed. Finally the best network was selected in training step, and obtained results were verified. The results show that the ANN can be used as a suitable alternative technique instead of LEM for probing the slope stability in embankment dams.

Received () Revised () Recurrent nets are in principle capable to store past inputs to produce the currently desired output. Because of this property recurrent nets are used in time series prediction and process control. Practical applications involve temporal dependencies spanning many time steps, e.g. between relevant inputs and desired outputs. In this case, however, gradient based learning methods take too much time. The extremely increased learning time arises because the error vanishes as it gets propagated back. In this article the decaying error ow is theoretically analyzed. Then methods trying to overcome vanishing gradients are brieey discussed. Finally, experiments comparing conventional algorithms and alternative methods are presented. With advanced methods long time lag problems can be solved in reasonable time.

The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. Training is recast as a control problem and this allows us to formulate necessary optimality conditions in continuous time using the Pontryagin’s maximum principle (PMP). A modification of the method of successive approximations is then used to solve the PMP, giving rise to an alternative training algorithm for deep learning. This approach has the advantage that rigorous error estimates and convergence results can be established. We also show that it may avoid some pitfalls of gradient-based methods, such as slow convergence on flat landscapes near saddle points. Furthermore, we demonstrate that it obtains favorable initial convergence rate periteration, provided Hamiltonian maximization can be efficiently carried out a step which is still in need of improvement. Overall, the approach opens up new avenues to attack problems associated with deep learning, such as trapping in slow manifolds and inapplicability of gradient-based methods for discrete trainable variables.

This is a brief review of recent work on the prospective hybrid CMOS/memristor circuits. Such hybrids combine the flexibility, reliability and high functionality of the CMOS subsystem with very high density of nanoscale thin film resistance switching devices operating on different physical principles. Simulation and initial experimental results demonstrate that performance of CMOS/memristor circuits for several important applications is well beyond scaling limits of conventional VLSI paradigm.

Empirically, neural networks that attempt to learn programs from data have exhibited poor generalizability. Moreover, it has traditionally been difficult to reason about the behavior of these models beyond a certain level of input complexity. In order to address these issues, we propose augmenting neural architectures with a key abstraction: recursion. As an application, we implement recursion in the Neural Programmer-Interpreter framework on four tasks: grade-school addition, bubble sort, topological sort, and quicksort. We demonstrate superior generalizability and interpretability with small amounts of training data. Recursion divides the problem into smaller pieces and drastically reduces the domain of each neural network component, making it tractable to prove guarantees about the overall system’s behavior. Our experience suggests that in order for neural architectures to robustly learn program semantics, it is necessary to incorporate a concept like recursion.

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.

Regularized training of an autoencoder typically results in hidden unit biases that take on large negative values. We show that negative biases are a natural result of using a hidden layer whose responsibility is to both represent the input data and act as a selection mechanism that ensures sparsity of the representation. We then show that negative biases impede the learning of data distributions whose intrinsic dimensionality is high. We also propose a new activation function that decouples the two roles of the hidden layer and that allows us to learn representations on data with very high intrinsic dimensionality, where standard autoencoders typically fail. Since the decoupled activation function acts like an implicit regularizer, the model can be trained by minimizing the reconstruction error of training data, without requiring any additional regularization.

Recent developments in memory-augmented neural networks allowed sequential problems requiring long-term memory to be solved, which were intractable for traditional neural networks. However, current approaches still struggle to scale to large memory sizes and sequence lengths. In this paper we show how access to an external memory component can be encoded geometrically through a novel HyperNEAT-based Neural Turing Machine (HyperNTM). The indirect HyperNEAT encoding allows for training on small memory vectors in a bit vector copy task and then applying the knowledge gained from such training to speed up training on larger size memory vectors. Additionally, we demonstrate that in some instances, networks trained to copy nine bit vectors can be scaled to sizes of 1,000 without further training. While the task in this paper is simple, the HyperNTM approach could now allow memory-augmented neural networks to scale to problems requiring large memory vectors and sequence lengths.


Question: Propagating uncertainty through the tanh function for reservoir computing applications.

This report delineates a novel learning methodology for recurrent neural networks, specifically tailored to adjust solely the weights connected to the output neurons, thereby achieving the desired learning outcome. This approach, coined the "echo state" method, emphasizes a selective weight modification strategy, focusing exclusively on the output layer, to accomplish the learning objective. Keywords: recurrent neural networks, supervised learning, echo state approach.

In essence, the report outlines a distinctive learning algorithm designed for recurrent neural networks, characterized by its unique approach of altering only the weights that link to output neurons, in pursuit of fulfilling the learning task. This technique, referred to as the "echo state" strategy, advocates for a selective alteration of weights, targeting the output neurons exclusively, to achieve the intended learning goal. Key terms: recurrent neural networks, supervised learning, echo state technique.

The core innovation lies in its selective weight adjustment mechanism, aimed at the output neurons, enabling the recurrent neural network to learn effectively. This "echo state" paradigm represents a groundbreaking approach in the realm of recurrent neural networks, offering a refined method for supervised learning tasks. Essential concepts: recurrent neural networks, supervised learning, echo state principle.

We propose Logic Tensor Networks: a uniform framework for integrating automatic learning and reasoning. A logic formalism called Real Logic is defined on a first-order language whereby formulas have truth-value in the interval [0,1] and semantics defined concretely on the domain of real numbers. Logical constants are interpreted as feature vectors of real numbers. Real Logic promotes a well-founded integration of deductive reasoning on a knowledge-base and efficient data-driven relational machine learning. We show how Real Logic can be implemented in deep Tensor Neural Networks with the use of Google’s TENSORFLOW primitives. The paper concludes with experiments applying Logic Tensor Networks on a simple but representative example of knowledge comple-

The use of Recurrent Neural Networks is not as extensive as Feedforward Neural Networks. Training algorithms for Recurrent Neural Networks, based on the error gradient, are very unstable in their search for a minimum and require much computational time when the number of neurons is high. The problems surrounding the application of these methods have driven us to develop new training tools. In this paper, we present a Real-Coded Genetic Algorithm that uses the appropriate operators for this encoding type to train Recurrent Neural Networks. We describe the algorithm and we also experimentally compare our Genetic Algorithm with the Real-Time Recurrent Learning algorithm to perform the fuzzy grammatical inference.

In recent theoretical approaches addressing the problem of neural coding, tools from statistical estimation and information theory have been applied to quantify the ability of neurons to transmit information through their spike outputs. These techniques, though fairly general, ignore the specific nature of neuronal processing in terms of its known biophysical properties. However, a systematic study of processing at various stages in a biophysically faithful model of a single neuron can identify the role of each stage in information transfer. Toward this end, we carry out a theoretical analysis of the information loss of a synaptic signal propagating along a linear, one-dimensional, weakly active cable due to neuronal noise sources along the way, using both a signal reconstruction and a signal detection paradigm. Here we begin such an analysis by quantitatively characterizing three sources of membrane noise: (1) thermal noise due to the passive membrane resistance, (2) noise due to stochastic openings and closings of voltage-gated membrane channels (Na+ and K+), and (3) noise due to random, background synaptic activity. Using analytical expressions for the power spectral densities of these noise sources, we compare their magnitudes in the case of a patch of membrane from a cortical pyramidal cell and explore their dependence on different biophysical parameters.

State of the art Bayesian classification approaches typically maintain a posterior distribution over possible classes given available sensor observations (images). Yet, while these approaches fuse all classifier outputs thus far, they do not provide any indication regarding how reliable the posterior classification is, thus limiting its functionality in terms of autonomous systems and robotics. On the other hand, current deep learning based classifiers provide an uncertainty measure, thereby quantifying model uncertainty. However, they do so on a single frame basis and do not consider a sequential framework. In this letter, we develop a novel approach that infers a distribution over posterior class probabilities, while accounting for model uncertainty. This distribution enables reasoning about uncertainty in the posterior classification and, therefore, is of prime importance for robust classification, object-level perception in uncertain and ambiguous scenarios, and for safe autonomy in general. The distribution of the posterior class probability has no known analytical solution; thus, we propose to approximate this distribution via sampling. We evaluate our approach in simulation and using real images fed into a convolutional neural network classifier.

We show that gradient descent converges to a local minimizer , almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.

Deep learning as a means to inferencing has proliferated thanks to its versatility and ability to approach or exceed human-level accuracy. These computational models have seemingly insatiable appetites for computational resources not only while training, but also when deployed at scales ranging from data centers all the way down to embedded devices. As such, increasing consideration is being made to maximize the computational efficiency given limited hardware and energy resources and, as a result, inferencing with reduced precision has emerged as a viable alternative to the IEEE 754 Standard for Floating-Point Arithmetic. We propose a quantization scheme that allows inferencing to be carried out using arithmetic that is fundamentally more efficient when compared to even half-precision floating-point. Our quantization procedure is significant in that we determine our quantization scheme parameters by calibrating against its reference floating-point model using a single inference batch rather than (re)training and achieve end-to-end post quantization accuracies comparable to the reference model.

In this paper I try to describe both the role of mathematics in shaping our understanding of how neural networks operate, and the curious new mathematical concepts generated by our attempts to capture neural networks in equations. My target reader being the non-expert, I will present a biased selection of relatively simple examples of neural network tasks, models and calculations, rather than try to give a full encyclopedic review-like account of the many mathematical developments in this eld.

Deep residual networks [1] have emerged as a family of extremely deep architectures showing compelling accuracy and nice convergence behaviors. In this paper, we analyze the propagation formulations behind the residual building blocks, which suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation. A series of ablation experiments support the importance of these identity mappings. This motivates us to propose a new residual unit, which makes training easier and improves generalization. We report improved results using a 1001-layer ResNet on CIFAR-10 (4.62% error) and CIFAR-100, and a 200-layer ResNet on ImageNet. Code is available at: https://github.com/KaimingHe/ resnet-1k-layers.

The performance of neural network classifiers is determined by a number of hyperparameters, including learning rate, batch size, and depth. A number of attempts have been made to explore these parameters in the literature, and at times, to develop methods for optimizing them. However, exploration of parameter spaces has often been limited. In this note, I report the results of large scale experiments exploring these different parameters and their interactions. 1 Datasets and Libraries All experiments reported here were carried out using the Torch library [1] and CUDA (some of the experiments have been reproduced on a smaller scale with other libraries). The dataset for all the experiments is MNIST [3, 2]. Characters were deskewed prior to all experiments. Deskewing significantly reduces error rates in nearest neighbor classifiers. Skew corresponds to a simple one-parameter family of linear transformations in feature space and causes decision regions to become highly anisotropic. Without deskewing, differences in performance between different architectures might primarily reduce to their ability to “learn deskewing”. With deskewing, MNIST character classification become more of an instance of a typical classification problem. Prior results on classifying deskewed MNIST data both with neural networks and with other methods are shown in the table below. 2 Logistic vs Softmax Outputs Multi-Layer Perceptrons (MLPs) used for classification usually attempt to approximate posterior probabilities and use those as their discriminant function. Two common approaches to this are the use of least square regression with logistic output units trained with a least square error measure (“logistic outputs”) 1 ar X iv :1 50 8. 02 78 8v 1 [ cs .N E ] 1 2 A ug 2 01 5 Method Test Error Preprocessing Reference Reduced Set SVM deg 5 polynomial 1 deskewing LeCun et al. 1998 SVM deg 4 polynomial 1.1 deskewing LeCun et al. 1998 K-nearest-neighbors, L3 1.22 deskewing, noise removal, blurring, 2 pixel shift Kenneth Wilder, U. Chicago K-nearest-neighbors, L3 1.33 deskewing, noise removal, blurring, 1 pixel shift Kenneth Wilder, U. Chicago 2-layer NN, 300 HU 1.6 deskewing LeCun et al. 1998 Table 1: Other previously reported results on the MNIST database. and a softmax output layer (“softmax outputs”). In the limit of infinite amounts of training data, both approaches converge to true posterior probability estimates. Softmax output layers have the property that they are guaranteed to produce a normalized posterior probability distribution across all classes, while least square regression with logistic output units generates independent probability estimates for each class membership without any guarantees that these probabilities sum up to one. Softmax is often preferred, although there is no obvious theoretical reason why it should yield better discriminant functions or lower classification error for finite training sets. In OCR and speech recognition, some practitioners have observed that logistic outputs yield better posterior probability estimates and better results when combined with probabilistic language models. In addition, when the sum of the posterior probability estimates derived from logistic outputs differs significantly from unity, that is a strong indication that the input lies outside the training set and should be rejected. Figure 1 shows a scatterplot of test vs training error for a large number of MLPs with one hidden layer at different learning rates, different number of hidden units, and different batch sizes. Such scatterplots show what error rates are achievable by the different architectures, hyperparameter choices, initializations, and order of sample presentations. The lowest points in the vertical direction indicate the lowest test set error achievable by the architecture in this set of experiments. The scatterplot shows that logistic outputs achieve test set error rates of about 1.0% vs 1.1% for softmax outputs. At the same time, logistic outputs never achieve zero percent training set error, while softmax outputs frequently do. In order to ascertain that the difference in test set error between the two

Deep learning has led to significant advances in artificial intelligence, in part, by adopting strategies motivated by neurophysiology. However, it is unclear whether deep learning could occur in the real brain. Here, we show that a deep learning algorithm that utilizes multi-compartment neurons might help us to understand how the neocortex optimizes cost functions. Like neocortical pyramidal neurons, neurons in our model receive sensory information and higher-order feedback in electrotonically segregated compartments. Thanks to this segregation, neurons in different layers of the network can coordinate synaptic weight updates. As a result, the network learns to categorize images better than a single layer network. Furthermore, we show that our algorithm takes advantage of multilayer architectures to identify useful higher-order representations-the hallmark of deep learning. This work demonstrates that deep learning can be achieved using segregated dendritic compartments, which may help to explain the morphology of neocortical pyramidal neurons.


Question: Does massive MIMO possess infinite capacity?

This document explores the implications of pilot contamination and precoding in multi-cell Time-Division Duplex (TDD) systems, focusing on a scenario where precoding is employed at base stations for downlink data transmission. The acquisition of Channel State Information (CSI) is crucial for effective precoding. Time-division duplex (TDD) operation, which utilizes uplink training combined with channel reciprocity, is a viable method for obtaining both downlink and uplink channel estimates simultaneously at the base stations. The paper delves into the mathematical characterization of the impact that uplink training has on the performance of multi-cell multiple antenna systems. When non-orthogonal training sequences are utilized for uplink training, the paper reveals that the precoding matrix at one cell's base station becomes adversely affected by the channels between that base station and users in neighboring cells, leading to a phenomenon known as pilot contamination. The paper addresses this fundamental issue by introducing a novel Multi-Cell Minimum Mean Square Error (MMSE)-based precoding technique that effectively reduces the effects of pilot contamination. This method is linear and comes with a straightforward closed-form expression, derived through a logical optimization process. Simulation results demonstrate substantial performance enhancements compared to conventional single-cell precoding methods, highlighting the effectiveness of the proposed MMSE-based precoding strategy in mitigating pilot contamination in multi-cell TDD systems.

We consider the problem of recovering two unknown vectors, w and x, of length L from their circular convolution. We make the structural assumption that the two vectors are members of known subspaces, one with dimension N and the other with dimension K. Although the observed convolution is nonlinear in both w and x, it is linear in the rank-1 matrix formed by their outer product wx*. This observation allows us to recast the deconvolution problem as low-rank matrix recovery problem from linear measurements, whose natural convex relaxation is a nuclear norm minimization program. We prove the effectiveness of this relaxation by showing that, for “generic” signals, the program can deconvolve w and x exactly when the maximum of N and K is almost on the order of L. That is, we show that if x is drawn from a random subspace of dimension N, and w is a vector in a subspace of dimension K whose basis vectors are spread out in the frequency domain, then nuclear norm minimization recovers wx* without error. We discuss this result in the context of blind channel estimation in communications. If we have a message of length N, which we code using a random L x N coding matrix, and the encoded message travels through an unknown linear time-invariant channel of maximum length K, then the receiver can recover both the channel response and the message when L ≳ N + K, to within constant and log factors.

In this paper, we investigate the downlink transmission of a non-orthogonal multiple access (NOMA)-based integrated terrestrial-satellite network, in which the NOMA-based terrestrial networks and the satellite cooperatively provide coverage for ground users while reusing the entire bandwidth. For both terrestrial networks and the satellite network, multi-antennas are equipped and beamforming techniques are utilized to serve multiple users simultaneously. A channel quality-based scheme is proposed to select users for the satellite, and we then formulate the terrestrial user pairing as a max–min problem to maximize the minimum channel correlation between users in one NOMA group. Since the terrestrial networks and the satellite network will cause interference to each other, we first investigate the capacity performance of the terrestrial networks and the satellite networks separately, which can be decomposed into the designing of beamforming vectors and the power allocation schemes. Then, a joint iteration algorithm is proposed to maximize the total system capacity, where we introduce the interference temperature limit for the satellite since the satellite can cause interference to all base station users. Finally, numerical results are provided to evaluate the user paring scheme as well as the total system performance, in comparison with some other proposed algorithms and existing algorithms.

This paper thoroughly explores the fundamental interactions between cell association, cell load, and throughput in a green (energy-efficient) small cell network in which all base stations form a homogeneous Poisson point process (PPP) of intensity λB and all users form another independent PPP of intensity λ∪. Cell voidness, usually disregarded due to rarity in cellular network modeling, is first theoretically analyzed under generalized (channel-aware) cell association (GCA). We show that the void cell probability cannot be neglected any more since it is bounded above by exp(-λ∪/λB) that is typically not small in a small cell network. The accurate expression of the void cell probability for GCA is characterized and it is used to derive the average cell and user throughputs. We learn that cell association and cell load λ∪/λB significantly affect these two throughputs. According to the average cell and user throughputs, the green cell and user throughputs are defined respectively to reflect whether the energy of a base station is efficiently used to transmit information or not. In order to achieve satisfactory throughput with certain level of greenness, cell load should be properly determined. We present the theoretical solutions of the optimal cell loads that maximize the green cell and user throughputs, respectively, and verify their correctness by simulation.

Attractive features of time-hopping spread-spectrum multiple-access systems employing impulse signal technology are outlined, and emerging design issues are described. Performance of such communications systems in terms of achievable transmission rate and multiple-access capability are estimated for both analog and digital data modulation formats under ideal multiple-access channel conditions.

This paper considers a multiple-input-multiple-output (MIMO) system with low-resolution analog-to-digital converters (ADCs). In this system, we propose a novel detection framework that performs data symbol detection without explicitly knowing channel state information at a receiver. The underlying idea of the proposed framework is to exploit supervised learning. Specifically, during channel training, the proposed approach sends a sequence of data symbols as pilots so that the receiver learns a nonlinear function that is determined by both a channel matrix and a quantization function of the ADCs. During data transmission, the receiver uses the learned nonlinear function to detect which data symbols were transmitted. In this context, we propose two blind detection methods to determine the nonlinear function from the training-data set. We also provide an analytical expression for the symbol-vector-error probability of the MIMO systems with one-bit ADCs when employing the proposed framework. Simulations demonstrate the performance improvement of the proposed framework compared to existing detection techniques.

Multiple-Input and Multiple-Output (MIMO) arrays provide more degrees of freedom than conventional phased arrays. They require that every transmitting element of the array can be identified when received. One way to achieve this is to give every element its own unique frequency. Offset-transmitters may be used to introduce MIMO or Doppler Division Multiple Access (DDMA) into phased-arrays without an excessive increase in waveform-generating hardware. Our dual-channel demonstrator IC can obtain a phase accuracy better than 1 degree and an spurious level of better than -65dBc for a single on-chip channel. This work investigates at X-band the effects of the limited on-chip isolation of 35dB, when multiple offset outputs are generated on a single chip for both beam steering and DDMA. In case of beam steering, the requirements on channel-to-channel isolation are less strict, making it well within reach. In the case of DDMA, we recommend increasing the channel-to-channel isolation by implementing multiple chips, in which case independent signals can be generated.

4G is proving good speeds up to 1Gbps. Then why do we need anything more. The problem is that it is not able to provide real time applications. 5G is the name given to the next generation of mobile data connectivity. It will definitely provide great speeds between 10Gbps to 100Gbps and it will have enough capacity. But the thing that separated 5G from 4G is latency; the latency provided by 4G is between 40ms to 60ms, whereas in 5G it will provide ultra latency between 1ms to 10ms. The standards for 5G will be set till 2020 and it will be applicable by 2022/23. In this paper we have discussed about 5G and its advantages, a probable architecture for 5G and some challenges in 5G. The main technology that maybe used in 5G are massive MIMO, millimetre wave communication, device to device communication, beam division multiple access etc. In this paper we have discussed about massive MIMO, channel estimate in massive MIMO, beam division multiple access technique to be used in massive MIMO, antenna selection in massive MIMO, capacity and energy efficiency in massive MIMO. In future 5G is going to be a technology which will be invisible, I will be just there everywhere just like electricity. It is a very good area for research as standards and frequency band for 5G are yet to be standardised.

Next-generation mobile technology (5G) aims to provide an improved experience through higher data-rates, lower latency, and improved link robustness. Millimeter-wave phased arrays offer a path to support multiple users at high data-rates using high-bandwidth directional links between the base station and mobile devices. To realize this vision, a phased-array-based pico-cell must support a large number of precisely controlled beams, yet be compact and power efficient. These system goals have significant mm-wave radio interface implications, including scalability of the RFIC+antenna-array solution, increase in the number of concurrent beams by supporting dual polarization, precise beam steering, and high output power without sacrificing TX power efficiency. Packaged Si-based phased arrays [1–3] with nonconcurrent dual-polarized TX and RX operation [2,3], concurrent dual-polarized RX operation [3] and multi-IC scaling [3,4] have been demonstrated. However, support for concurrent dual-polarized operation in both RX and TX remains unaddressed, and high output power comes at the cost of power consumption, cooling complexity and increased size. The RFIC reported here addresses these challenges. It supports concurrent and independent dual-polarized operation in TX and RX modes, and is compatible with a volume-efficient, scaled, antenna-in-package array. A new TX/RX switch at the shared antenna interface enables high output power without sacrificing TX efficiency, and a t-line-based phase shifter achieves <1° RMS error and <5° phase steps for precise beam control.

Indoor localization has been an active research field for decades, where the received signal strength (RSS) fingerprinting based methodology is widely adopted and induces many important localization techniques such as the recently proposed one building the fingerprint database with crowd-sourcing. While efforts have been dedicated to improve the accuracy and efficiency of localization, the fundamental limits of RSS fingerprinting based methodology itself is still unknown in a theoretical perspective. In this paper, we present a general probabilistic model to shed light on a fundamental question: how good the RSS fingerprinting based indoor localization can achieve? Concretely, we present the probability that a user can be localized in a region with certain size, given the RSS fingerprints submitted to the system. We reveal the interaction among the localization accuracy, the reliability of location estimation and the number of measurements in the RSS fingerprinting based location determination. Moreover, we present the optimal fingerprints reporting strategy that can achieve the best accuracy for given reliability and the number of measurements, which provides a design guideline for the RSS fingerprinting based indoor localization facilitated by crowdsourcing paradigm.

Space-time codes (STC) are a class of signaling techniques, offering coding and diversity gains along with improved spectral efficiency. These codes exploit both the spatial and the temporal diversity of the wireless link by combining the design of the error correction code, modulation scheme and array processing. STC are well suited for improving the downlink performance, which is the bottleneck in asymmetric applications such as downstream Internet. Three original contributions to the area of STC are presented in this dissertation. First, the development of analytic tools that determine the fundamental limits on the performance of STC in a variety of channel conditions. For trellis-type STC, transfer function based techniques are applied to derive performance bounds over Rayleigh, Rician and correlated fading environments. For block-type STC, an analytic framework that supports various complex orthogonal designs with arbitrary signal cardinalities and array configurations is developed. In the second part of the dissertation, the Virginia Tech Space-Time Advanced Radio (VT-STAR) is designed, introducing a multi-antenna hardware laboratory test bed, which facilitates characterization of the multiple-input multiple-output (MIMO) channel and validation of various space-time approaches. In the third part of the dissertation, two novel space-time architectures paired with iterative processing principles are proposed. The first scheme extends the suitability of STC to outdoor wireless communications by employing iterative equalization/decoding for time dispersive channels and the second scheme employs iterative interference cancellation/decoding to solve the error propagation problem of Bell-Labs Layered Space-Time Architecture (BLAST). Results show that remarkable energy and spectral efficiencies are achievable by combining concepts drawn from space-time coding, multiuser detection, array processing and iterative decoding.


Question: Does Massive MIMO possess boundless capacity?

This tutorial aims to provide an overview of convex optimization, highlighting its significance in the fields of communication systems and signal processing. Convex optimization techniques are pivotal in the design and analysis of these systems, and this paper aims to summarize recent advancements in this area. The tutorial is divided into two sections. The initial segment delves into fundamental concepts and methodologies in convex optimization, with a particular focus on conic optimization problems, encompassing second-order cone programming and semidefinite programming. The latter half of the tutorial illustrates the application of conic programming to various communication challenges. It elucidates the interpretation of Lagrangian duality within the context of multiuser multi-antenna communication systems. Additionally, the role of semidefinite relaxation in resolving multiuser detection problems is demonstrated. The tutorial also revisits techniques for formulating robust optimization problems through the use of second-order cone programming. This comprehensive exploration underscores the versatility and power of convex optimization in addressing complex problems in communication and signal processing.

Millimetre-wave (mmWave) 5G communications is an emerging technology to enhance the capacity of existing systems by thousand-fold improvement. Heterogeneous networks employing densely distributed small cells can optimise the available coverage and throughput of 5G systems. Efficiently utilising the spectrum bands by small cells is one of the approaches that will considerably increase the available data rate and capacity of the heterogeneous networks. This challenging task can be achieved by spectrum sensing capability of cognitive radios and new modulation techniques for data transmission. In this study, a wavelet-based filter bank is proposed for spectrum sensing and modulation in 5G heterogeneous networks. The proposed technique can mitigate the spectral leakage and interference by adapting the subcarriers according to cognitive information provided by wavelet packet based spectrum sensing (WPSS) and lowering sidelobes using wavelet-based filter bank multicarrier modulation. The performance improvement of WPSS compared with Fourier-based spectrum sensing is verified in terms of power spectral density comparison and probabilities of detection and false alarm. Meanwhile, the bit error rate performance demonstrates the superiority of the proposed wavelet-based system compared with its Fourier-based counterpart over the 60 GHz mmWave channel.

This paper analyzes the performance of downlink non-orthogonal multiple-access (NOMA) networks over independent but not necessarily identically distributed Rayleigh fading channels for arbitrary number of network users. Specifically, closed-form expressions for the average SNR, average achievable rate, and outage probability are derived, where it has been shown that network users achieve diversity orders equivalent to their ordered channel gains. Numerical evaluations are presented to validate the derived closed-form expressions for the different performance metrics, which are found to be in agreement with the network simulation results.

This paper evaluates the performance of the IEEE 802.11 broadcast traffic under both saturation and nonsaturation conditions. The evaluation highlights some important characteristics of IEEE 802.11 broadcast traffic as compared to corresponding unicast traffic. Moreover, it underlines the inaccuracy of the broadcast saturation model proposed by Ma and Chen due to the absence of backoff counter freeze process when channel is busy. Computer simulations are used to validate the accuracy of the new model and demonstrate the importance of capturing the freezing of backoff counter in the analytical study of IEEE 802.11 broadcast.

This article provides an overview of power-domain non-orthogonal multiple access for 5G systems. The basic concepts and benefits are briefly presented, along with current solutions and standardization activities. In addition, limitations and research challenges are discussed.

This paper presents a new design concept of a beam steerable high gain phased array antenna based on WR28 waveguide at 29 GHz frequency for fifth generation (5G) full dimension multiple input multiple output (FD-MIMO) system. The 8×8 planar phased array is fed by a three dimensional beamformer to obtain volumetric beam scanning ranging from −60 to +60 degrees both in azimuth and elevation direction. Beamforming network (BFN) is designed using 16 set of 8×8 Butler matrix beamformer to get 64 beam states, which control the horizontal and vertical angle. This is a new concept to design waveguide based high power three-dimensional beamformer for volumetric multibeam in Ka band for 5G application. The maximum gain of phased array is 28.5 dBi that covers 28.9 GHz to 29.4 GHz frequency band.

In this paper the use of electronically steerable passive array radiator (ESPAR) antennas is introduced for achieving increased spectral efficiency characteristics in multiple-input, multiple-output (MIMO) systems using a single active element, and compact antennas. The proposed ESPAR antenna is capable of mapping phase-shift-keying (PSK) modulated symbols to be transmitted onto orthogonal basis functions on the wavevector domain of the multi-element antenna (MEA), instead of the traditional approach of sending different symbol streams in different locations on the antenna domain. In this way, different symbols are transmitted simultaneously towards different angles of departure at the transmitter side. We show that the proposed system, called beamspace-MIMO (BS-MIMO) can achieve performance characteristics comparable to traditional MIMO systems while using a single radio-frequency (RF) front-end and ESPAR antenna arrays of total length equal to lambda/8, for popular PSK modulation schemes such as binary-PSK (BPSK), quaternary-PSK (QPSK), as well as for their offset-PSK and differential-PSK variations.

Array processing involves manipulation of signals induced on various antenna elements. Its capabilities of steering nulls to reduce cochannel interferences and pointing independent beams toward various mobiles, as well as its ability to provide estimates of directions of radiating sources, make it attractive to a mobile communications system designer. Array processing is expected to play an important role in fulfilling the increased demands of various mobile communications services. Part I of this paper showed how an array could be utilized in different configurations to improve the performance of mobile communications systems, with references to various studies where feasibility of an array system for mobile communications is considered. This paper provides a comprehensive and detailed treatment of different beam-forming schemes, adaptive algorithms to adjust the required weighting on antennas, direction-of-arrival estimation methods—including their performance comparison—and effects of errors on the performance of an array system, as well as schemes to alleviate them. This paper brings together almost all aspects of array signal processing. It is presented at a level appropriate to nonexperts in the field and contains a large reference list to probe further.

Indoor localization has been an active research field for decades, where the received signal strength (RSS) fingerprinting based methodology is widely adopted and induces many important localization techniques such as the recently proposed one building the fingerprint database with crowd-sourcing. While efforts have been dedicated to improve the accuracy and efficiency of localization, the fundamental limits of RSS fingerprinting based methodology itself is still unknown in a theoretical perspective. In this paper, we present a general probabilistic model to shed light on a fundamental question: how good the RSS fingerprinting based indoor localization can achieve? Concretely, we present the probability that a user can be localized in a region with certain size, given the RSS fingerprints submitted to the system. We reveal the interaction among the localization accuracy, the reliability of location estimation and the number of measurements in the RSS fingerprinting based location determination. Moreover, we present the optimal fingerprints reporting strategy that can achieve the best accuracy for given reliability and the number of measurements, which provides a design guideline for the RSS fingerprinting based indoor localization facilitated by crowdsourcing paradigm.

In this paper, we study the resource allocation problem for a single-cell non-orthogonal multiple access (NOMA) relay network where an OFDM amplify-and-forward (AF) relay allocates the spectrum and power resources to the source-destination (SD) pairs. We aim to optimize the spectrum and power resource allocation to maximize the total sum-rate. This is a very complicated problem and the optimal approach requires an exhaustive search, leading to a NP hard problem. To solve this problem, we propose an efficient many-to-many two sided SD pair-subchannel matching algorithm in which the SD pairs and sub-channels are considered as two sets of rational and selfish players chasing their own interests. The algorithm converges to a pair-wise stable matching after a limited number of iterations with a low complexity compared with the optimal solution. Simulation results show that the sum-rate of the proposed algorithm approaches the performance of the optimal exhaustive search and significantly outperforms the conventional orthogonal multiple access scheme, in terms of the total sum-rate and number of accessed SD pairs.

In a conventional cellular system, devices are not allowed to directly communicate with each other in the licensed cellular bandwidth and all communications take place through the base stations. In this article, we envision a two-tier cellular network that involves a macrocell tier (i.e., BS-to-device communications) and a device tier (i.e., device-to-device communications). Device terminal relaying makes it possible for devices in a network to function as transmission relays for each other and realize a massive ad hoc mesh network. This is obviously a dramatic departure from the conventional cellular architecture and brings unique technical challenges. In such a two-tier cellular system, since the user data is routed through other users' devices, security must be maintained for privacy. To ensure minimal impact on the performance of existing macrocell BSs, the two-tier network needs to be designed with smart interference management strategies and appropriate resource allocation schemes. Furthermore, novel pricing models should be designed to tempt devices to participate in this type of communication. Our article provides an overview of these major challenges in two-tier networks and proposes some pricing schemes for different types of device relaying.


Question: The capacity of massive MIMO systems is effectively limitless.

This paper explores the influence of fading correlations on the capacity of multielement antenna (MEA) systems in communication. Previous research highlighted that, assuming independent and identically distributed (IID) fades between transmit and receive antenna elements, MEA systems can significantly enhance capacity compared to single-antenna setups. An MEA system can be decomposed into spatial eigenmodes, which represent individual single-input single-output (SISO) subchannels. The total capacity of an MEA system is the aggregate of the capacities of these subchannels. We demonstrate that fading correlations impact the MEA capacity by altering the distributions of the subchannel gains. These correlations are contingent on both the physical attributes of the MEA system and the scatterer properties in the environment. In this work, we employ a generalized model to encapsulate fading correlations, suitable for characterizing narrow-band Rayleigh fading in fixed wireless systems.

**Question**: The Use of Deep Learning for Network Intrusion Detection

**Documents**: A deep learning approach for intrusion detection in SDN
This paper presents a novel approach to intrusion detection in software-defined networks (SDNs) using deep learning. The proposed system is designed to analyze network traffic data in real-time and identify potential security threats. The system uses a deep neural network (DNN) to classify network traffic as either normal or malicious. The DNN is trained on a large dataset of labeled network traffic, which includes both normal and malicious traffic. The system is designed to be scalable and adaptable, allowing it to be deployed in a variety of SDN environments. The proposed system has been evaluated using a standard dataset of network traffic and has shown promising results in terms of accuracy and efficiency.

**Similar Documents**: In this work, we introduce an innovative intrusion detection mechanism for software-defined networks (SDNs) that harnesses the power of deep learning. The proposed system is engineered to scrutinize network traffic in real-time, aiming to pinpoint any security breaches. This system leverages a deep neural network (DNN) for traffic classification, distinguishing between normal and malicious activities. The DNN is meticulously trained on a vast dataset that includes both benign and malicious network traffic, ensuring a comprehensive understanding of traffic patterns. The system is crafted with scalability and adaptability in mind, making it suitable for deployment across diverse SDN environments. We have thoroughly evaluated the proposed system using a benchmark dataset of network traffic, and the outcomes indicate high accuracy and efficiency, highlighting its potential in enhancing SDN security.

This paper presents 28 GHz and 73 GHz millimeter-wave propagation measurements performed in a typical office environment using a 400 Megachip-per-second broadband sliding correlator channel sounder and highly directional steerable 15 dBi (30° beamwidth) and 20 dBi (15° beamwidth) horn antennas. Power delay profiles were acquired for 48 transmitter-receiver location combinations over distances ranging from 3.9 m to 45.9 m with maximum transmit powers of 24 dBm and 12.3 dBm at 28 GHz and 73 GHz, respectively. Directional and omnidirectional path loss models and RMS delay spread statistics are presented for line-of-sight and non-line-of-sight environments for both co- and cross-polarized antenna configurations. The LOS omnidirectional path loss exponents were 1.1 and 1.3 at 28 GHz and 73 GHz, and 2.7 and 3.2 in NLOS at 28 GHz and 73 GHz, respectively, for vertically-polarized antennas. The mean directional RMS delay spreads were 18.4 ns and 13.3 ns, with maximum values of 193 ns and 288 ns at 28 GHz and 73 GHz, respectively.

The smart antennas are widely used for wireless com munication, because it has a ability to increase th coverage and capacity of a communication system. Smart anten na performs two main functions such as direction of arrival estimation (DOA) and beam forming. Using beam formi ng algorithm smart antenna is able to form main bea m towards desired user and null in the direction of i nterfering signals. In this project Direction of ar rival (DOA) is estimated by using MUSIC algorithm. Receive Beam fo r ing is performed by using LMS and LLMS algorithm .In this Paper, in order to perform secure transmission of signal over wireless communication we have used chaotic sequences. This paper evaluates the performance of B am forming with and without LMS and LLMS algorith m for MIMO-OFDM wireless system. The simulations are carr ied out using MATLAB.

The millimeter wave (mmWave) frequencies offer the potential of orders of magnitude increases in capacity for next-generation cellular wireless systems. However, links in mmWave networks are highly susceptible to blocking and may suffer from rapid variations in quality. Connectivity to multiple cells - both in the mmWave and in the traditional lower frequencies - is thus considered essential for robust connectivity. However, one of the challenges in supporting multi-connectivity in the mmWave space is the requirement for the network to track the direction of each link in addition to its power and timing. With highly directional beams and fast varying channels, this directional tracking may be the main bottleneck in realizing robust mmWave networks. To address this challenge, this paper proposes a novel measurement system based on (i) the UE transmitting sounding signals in directions that sweep the angular space, (ii) the mmWave cells measuring the instantaneous received signal strength along with its variance to better capture the dynamics and, consequently, the reliability of a channel/direction and, finally, (iii) a centralized controller making handover and scheduling decisions based on the mmWave cell reports and transmitting the decisions either via a mmWave cell or conventional microwave cell (when control signaling paths are not available). We argue that the proposed scheme enables efficient and highly adaptive cell selection in the presence of the channel variability expected at mmWave frequencies.

In this paper, we propose a non-orthogonal multiple access-based multiuser beamforming (NOMA-BF) system designed to enhance the sum capacity. In the proposed NOMA-BF system, a single BF vector is shared by two users, so that the number of supportable users can be increased. However, sharing a BF vector leads to interference from other beams as well as from the other user sharing the BF vector. Therefore, to reduce interference and improve the sum capacity, we additionally propose a clustering and power allocation algorithm. This clustering algorithm, which selects two users with high correlation and a large gain-difference between their channels, can reduce the interference from other beams and from the other user as well. Furthermore, power allocation ensures that each user's transmit power is allocated so as to maximize the sum capacity. Numerical results verify that the proposed NOMA-BF system improves the sum capacity, compared to the conventional multiuser BF system.

Information theory has recently been employed to specify more precisely than has hitherto been possible man's capacity in certain sensory, perceptual, and perceptual-motor functions (5, 10, 13, 15, 17, 18). The experiments reported in the present paper extend the theory to the human motor system. The applicability of only the basic concepts, amount of information, noise, channel capacity, and rate of information transmission, will be examined at this time. General familiarity with these concepts as formulated by recent writers (4,11, 20, 22) is assumed. Strictly speaking, we cannot study man's motor system at the behavioral level in isolation from its associated sensory mechanisms. We can only analyze the behavior of the entire receptor-neural-effector system. How-

A new transmission approach, referred to as subcarrier-index modulation (SIM) is proposed to be integrated with the orthogonal frequency division multiplexing (OFDM) systems. More specifically, it relates to adding an additional dimension to the conventional two-dimensional (2-D) amplitude/phase modulation (APM) techniques, i.e. amplitude shift keying (ASK) and quadrature amplitude modulation (QAM). The key idea of SIM is to employ the subcarrier-index to convey information to the receiver. Furthermore, a closed-form analytical bit error ratio (BER) of SIM OFDM in Rayleigh channel is derived. Analytical and simulation results show error probability performance gain of 4 dB over 4-QAM OFDM systems for both coded and uncoded data without power saving policy. Alternatively, power saving policy retains an average gain of 1 dB while using 3 dB less transmit power per OFDM symbol.

A 2 × 2 (four-element) multiple-input multiple-output (MIMO) patch antenna system is designed and fabricated for a 2.45-GHz ISM band operation. It uses complementary split-ring resonator (CSRR) loading on its ground plane for antenna miniaturization. This reduces the single-element antenna size by 76%. The total board size of the proposed MIMO antenna system, including the GND plane is 100 × 50 × 0.8 mm3, while the single-patch antenna element has a size of 14 × 18 mm2. The antenna is fabricated and tested. Measured results are in good agreement with simulations. A minimum measured isolation of 10 dB is obtained given the close interelement spacing of 0.17λ. The maximum measured gain for a single operating element is -0.8 dBi.

Far-field wireless power transfer (WPT) is a promising technique to resolve the painstaking power-charging problem inherent in various wireless terminals. This paper investigates the power transfer efficiency of the WPT segment in future communication systems in support of simultaneous power and data transfer, by means of analytically computing the time-average output direct current (DC) power at user equipments (UEs). In order to investigate the effect of channel variety among UEs on the average output DC power, different policies for the scheduling of the power transfer among the users are implemented and compared in two scenarios: homogeneous, whereby users are symmetric and experience similar path loss, and heterogeneous, whereby users are asymmetric and exhibit different path losses. Specifically, if opportunistic scheduling is performed among N symmetric/asymmetric UEs, the power scaling laws are attained by using extreme value theory and reveal that the gain in power transfer efficiency is lnN if UEs are symmetric whereas the gain is N if UEs are asymmetric, compared with that of conventional round-robin scheduling. Thus, the channel variety among UEs inherent to the wireless environment can be exploited by opportunistic scheduling to significantly improve the power transfer efficiency when designing future wireless communication systems in support of simultaneous power and data transfer.

The wireless communications in complex environments, such as underground and underwater, can enable various applications in the environmental, industrial, homeland security, law enforcement, and military fields. However, conventional electromagnetic wave-based techniques do not work due to the lossy media and complicated structures. Magnetic induction (MI) has been proved to achieve reliable communication in such environments. However, due to the small antenna size, the communication range of MI is still very limited, especially for the portable mobile devices. To this end, Metamaterial-enhanced MI (M2I) communication has been proposed, where the theoretical results predict that it can significantly increase the data rate and range. Nevertheless, there exists a significant gap between the theoretical prediction and the practical realization of M2I; the theoretical model relies on an ideal spherical metamaterial, while it does not exist in nature. In this paper, a practical design is proposed by leveraging a spherical coil array to realize M2I communication. The full-wave simulation is conducted to validate the design objectives. By using the spherical coil array-based M2I communication, the communication range can be significantly extended, exactly as we predicted in the ideal M2I model. Finally, the proposed M2I communication is implemented and tested in various environments.

Multicarrier CDMA is a multiplexing approach in which modulated QAM symbols are spread over multiple OFDMA tones by using a generally complex spreading sequence. Effectively, a QAM symbol is repeated over multiple tones. Low density signature (LDS) is a version of CDMA with low density spreading sequence allowing us to take advantage of a near optimal ML receiver with practically feasible complexity. In this paper, we propose a new multiple access scheme so called sparse code multiple access (SCMA) which still enjoys the low complexity reception technique but with better performance compared to LDS. In SCMA, the procedure of bit to QAM symbol mapping and spreading are combined together and incoming bits are directly mapped to a multidimensional codeword of an SCMA codebook set. Each layer or user has its dedicated codebook. Shaping gain of a multidimensional constellation is the main source of the performance improvement in comparison to the simple repetition of QAM symbols in LDS. In general, SCMA codebook design is an optimization problem. A systematic sub-optimal approach is proposed here for SCMA codebook design.


Question: The concept that Massive MIMO systems possess boundless capacity.

Reinventing the MIMO Landscape
Multi-user MIMO (MU-MIMO) systems present unprecedented prospects by merging antenna combining strategies with resource allocation mechanisms. This amalgamation not only fortifies the system against multipath fading but also facilitates tighter antenna spacing at the base station (BS), crucially enabling diversity and multiplexing gains without necessitating multi-antenna user devices. Yet, to harness these benefits, the BS must possess knowledge of the channel coefficients from the users, which may confine practical deployments to time division duplexing (TDD) systems or scenarios with low user mobility. To address this limitation and alleviate feedback overhead, integrating MU-MIMO with opportunistic scheduling strategies appears to be a viable solution. However, the efficacy of such scheduling is significantly influenced by traffic patterns and quality of service (QoS) requirements.

In this paper we present a general method for determining the capacity of low-density paritycheck codes under message-passing decoding when used over any binary-input memoryless channel with discrete or continuous output alphabets. Transmitting at rates below this capacity, a randomly chosen element of the given ensemble will achieve an arbitrarily small target probability of error with a probability that approaches one exponentially fast in the length of the code. (By concatenating with an appropriate outer code one can achieve a probability of error that approaches zero exponentially fast in the length of the code with arbitrarily small loss in rate.) Conversely, transmitting at rates above this capacity the probability of error is bounded away from zero by a strictly positive constant which is independent of the length of the code and of the number of iterations performed. Our results are based on the observation that the concentration of the performance of the decoder around its average performance, as observed by Luby et. al. [1] in the case of a binary symmetric channel and a binary message passing algorithm, is a general phenomenon. For the particularly important case of belief-propagation decoders, we provide an effective algorithm to determine the corresponding capacity to any desired degree of accuracy. The ideas presented in this paper are broadly applicable and extensions of the general method to low-density parity-check codes over larger alphabets, turbo codes, and other concatenated coding schemes are outlined. Index Terms – low-density parity-check codes, turbo codes, message-passing decoders, iterative decoding, belief-propagation, turbo decoding

In systems that allow device-to-device (D2D) communications, user pairs in close proximity communicate directly without using an access point (AP) as an intermediary. D2D communications leads to improved throughput, reduced power consumption and interference, and more flexible resource allocation. We show that the D2D paradigm also provides significantly improved security at the physical layer, by reducing exposure of the information to eavesdroppers from two relatively high-power transmissions to a single low-power hop. We derive the secrecy outage probability (SOP) for the D2D and cellular systems, and compare performance for D2D scenarios in the presence of a multi-antenna eavesdropper. The cellular approach is only seen to have an advantage in certain cases when the AP has a large number of antennas and perfect channel state information.

For millimeter-wave (mmWave) massive multiple-input-multiple-output (MIMO) systems, codebook-based analog beamforming (including transmit precoding and receive combining) is usually used to compensate the severe attenuation of mmWave signals. However, conventional beamforming schemes involve complicated search among predefined codebooks to find out the optimal pair of analog precoder and analog combiner. To solve this problem, by exploring the idea of turbo equalizer together with the tabu search (TS) algorithm, we propose a Turbo-like beamforming scheme based on TS, which is called Turbo-TS beamforming in this paper, to achieve near-optimal performance with low complexity. Specifically, the proposed Turbo-TS beamforming scheme is composed of the following two key components: 1) Based on the iterative information exchange between the base station (BS) and the user, we design a Turbo-like joint search scheme to find out the near-optimal pair of analog precoder and analog combiner; and 2) inspired by the idea of the TS algorithm developed in artificial intelligence, we propose a TS-based precoding/combining scheme to intelligently search the best precoder/combiner in each iteration of Turbo-like joint search with low complexity. Analysis shows that the proposed Turbo-TS beamforming can considerably reduce the searching complexity, and simulation results verify that it can achieve near-optimal performance.

Vehicle-to-vehicle (V2V) communications aim to enhance driver safety and traffic efficiency by using the recently designated frequency bands in the 5.9 GHz range in Europe. Due to the time-frequency selective fading behavior of the vehicular communication channel, multi-antenna techniques can provide enhanced link conditions by means of diversity processing. This paper highlights the integration of a four-element (N =4) linear array antenna into the roof-top compartment of a vehicle to conduct Multiple-Input Multiple-Output (MIMO) high-resolution mobile-to-mobile channel measurements.

We introduce space–time block coding , a new paradigm for communication over Rayleigh fading channels using multiple transmit antennas. Data is encoded using a space–time block code and the encoded data is split inton streams which are simultaneously transmitted usingn transmit antennas. The received signal at each receive antenna is a linear superposition of the n transmitted signals perturbed by noise. Maximumlikelihood decoding is achieved in a simple way through decoupling of the signals transmitted from different antennas rather than joint detection. This uses the orthogonal structure of the space–time block code and gives a maximum-likelihood decoding algorithm which is based only on linear processing at the receiver. Space–time block codes are designed to achieve the maximum diversity order for a given number of transmit and receive antennas subject to the constraint of having a simple decoding algorithm. The classical mathematical framework of orthogonal designs is applied to construct space–time block codes. It is shown that space–time block codes constructed in this way only exist for few sporadic values ofn. Subsequently, a generalization of orthogonal designs is shown to provide space–time block codes for both real and complex constellations for any number of transmit antennas. These codes achieve the maximum possible transmission rate for any number of transmit antennas using any arbitrary real constellation such as PAM. For an arbitrary complex constellation such as PSK and QAM, space–time block codes are designed that achieve1=2 of the maximum possible transmission rate for any number of transmit antennas. For the specific cases of two, three, and four transmit antennas, space–time block codes are designed that achieve, respectively, all, 3=4, and 3=4 of maximum possible transmission rate using arbitrary complex constellations. The best tradeoff between the decoding delay and the number of transmit antennas is also computed and it is shown that many of the codes presented here are optimal in this sense as well.

As the number of wireless devices grows, the increasing demand for the shared radio frequency (RF) spectrum becomes a critical problem. Unlike wired communications in which, theoretically, more fibers can be used to accommodate the increasing bandwidth demand, wireless spectrum cannot be arbitrarily increased due to the fundamental limitations imposed by the physical laws. On the other hand, recent advances in free space optical (FSO) technologies promise a complementary approach to increase wireless capacity. However, high-speed FSO technologies are currently confined to short distance transmissions, resulting in limited mobility. In this paper, we briefly describe WiFO, a hybrid WiFi-FSO network for Gbps wireless local area network (WLAN) femtocells that can provide up to one Gbps per user while maintaining seamless mobility. While typical RF femtocells are non-overlapped to minimize inter-cell interference, there are advantages of using overlapped femtocells to increase mobility and throughput when the number of users is small. That said, the primary contribution of this paper will be a novel location assisted coding (LAC) technique used in the WiFO network that aims to increase throughput and reduce interference for multiple users in a dense array of femtocells. Both theoretical analysis and numerical experiments show orders of magnitude increase in throughput using LAC over basic codes.

Array processing involves manipulation of signals induced on various antenna elements. Its capabilities of steering nulls to reduce cochannel interferences and pointing independent beams toward various mobiles, as well as its ability to provide estimates of directions of radiating sources, make it attractive to a mobile communications system designer. Array processing is expected to play an important role in fulfilling the increased demands of various mobile communications services. Part I of this paper showed how an array could be utilized in different configurations to improve the performance of mobile communications systems, with references to various studies where feasibility of an array system for mobile communications is considered. This paper provides a comprehensive and detailed treatment of different beam-forming schemes, adaptive algorithms to adjust the required weighting on antennas, direction-of-arrival estimation methods—including their performance comparison—and effects of errors on the performance of an array system, as well as schemes to alleviate them. This paper brings together almost all aspects of array signal processing. It is presented at a level appropriate to nonexperts in the field and contains a large reference list to probe further.

The exponentially increasing number of internet of things (IoT) devices and the data generated by these devices introduces the spectrum crisis at the already crowded ISM 2.4 GHz band. To address this issue and enable more flexible and concurrent communications among IoT devices, we propose B2W2, a novel communication framework that enables N-way concurrent communication among WiFi and Bluetooth Low Energy (BLE) devices. Specifically, we demonstrate that it is possible to enable the BLE to WiFi cross-technology communication while supporting the concurrent BLE to BLE and WiFi to WiFi communications. We conducted extensive experiments under different real-world settings and results show that its throughput is more than 85X times higher than the most recently reported cross-technology communication system [22], which only supports one-way communication (i.e., broadcasting) at any specific time.

This paper considers the antenna selection (AS) problem for a multiple-input multiple-output nonorthogonal multiple access system. In particular, we develop new computationally efficient AS algorithms for two commonly used scenarios: NOMA with fixed power allocation (F-NOMA) and NOMA with cognitive radio-inspired power allocation (CR-NOMA). For the F-NOMA scenario, a new max–max–max AS (A<inline-formula><tex-math notation="LaTeX">$^3$</tex-math> </inline-formula>-AS) scheme is first proposed to maximize the system sum-rate. This is achieved by selecting one antenna at the base station (BS) and corresponding best receive antenna at each user that maximizes the channel gain of the resulting <italic>strong</italic> user. To improve the user fairness, a new max–min–max AS (AIA-AS) scheme is subsequently developed, in which we jointly select one transmit antenna at BS and corresponding best receive antennas at users to maximize the channel gain of the resulting <italic>weak</italic> user. For the CR-NOMA scenario, we propose another new AS algorithm, termed maximum-channel-gain-based antenna selection (MCG-AS), to maximize the achievable rate of the secondary user, under the condition that the primary user's quality-of-service requirement is satisfied. The asymptotic closed-form expressions of the average sum-rate for A<inline-formula> <tex-math notation="LaTeX">$^3$</tex-math></inline-formula>-AS and AIA-AS and that of the average rate of the secondary user for MCG-AS are derived. Numerical results demonstrate that the AIA-AS provides better user fairness, whereas the A<inline-formula><tex-math notation="LaTeX">$^3$</tex-math></inline-formula>-AS achieves a near-optimal sum-rate in F-NOMA systems. For the CR-NOMA scenario, MCG-AS achieves a near-optimal performance in a wide signal-to-noise-ratio regime. Furthermore, all the proposed AS algorithms yield a significant computational complexity reduction, compared to exhaustive search-based counterparts.

Carrier frequencies beyond 300 GHz, belonging to the so-called THz range, have received attention for considering this frequency band for future multi-gigabit short-range communication systems. This review paper gives an overview of current issues in the emerging field of THz communications targeting to deliver wireless 100 Gbps over short distances. The paper will start by introducing scenarios and applications requiring such high data rates followed by a discussion on the radio channel characteristics. In the 300 GHz frequency band, the path loss is even more significant than at 60 GHz and appropriate measures to mitigate effects in none-line-of-sight (NLOS) cases, caused e. g. by the influence of moving people, are required. Advanced antenna techniques like beam forming or beam switching are a pre-requisite to guarantee seamless service. In order to consider such techniques in the standards development, the propagation channel operating at these mmand sub-mm wave bands in realistic environments must be well understood. Therefore, intensive channel measurement and modeling activities have been done at 300 GHz, which started a couple of years ago at Terahertz Communications Lab (TCL). Due to the short wave length, the influence of surface roughness of typical building materials plays an important role. Hence, the modeling of rough surface scattering has been one of the main areas in these investigations. In this contribution, the main results of the propagation research activities at TCL are summarized. In the last part of the paper, an overview of the state-of-the-art in technology development and successful demonstrations of data transmission will be given together with a report on the status quo of ongoing activities in standardization at IEEE 802.15 IG THz and the regulation of the spectrum beyond 300 GHz.