Question: StochasticNet: Creating deep neural networks through stochastic connectivity mechanisms.

This paper describes Deep Speech, an advanced speech recognition system built using an end-to-end deep learning methodology. Our design stands in stark contrast to conventional speech recognition systems, which are often complex and rely on meticulously engineered processing pipelines. Such traditional systems are prone to degradation in performance when faced with noisy environments. Our innovative system, however, does not necessitate any hand-crafted components to mitigate the effects of background noise, reverberation, or speaker variation. Instead, it directly learns a robust function that effectively deals with these environmental challenges. Notably, our system operates without the need for a phoneme dictionary or even the traditional concept of a "phoneme."

A pivotal aspect of our approach is the utilization of a highly optimized recurrent neural network (RNN) training system that leverages multiple GPUs. Additionally, we employ innovative data synthesis techniques that enable us to efficiently generate a vast and diverse dataset for training purposes. The result is a system, dubbed Deep Speech, that surpasses previously published results on the well-established Switchboard Hub5’00 dataset, achieving a commendable 16.0% error rate on the full test set. Remarkably, Deep Speech demonstrates superior performance in challenging, noisy environments compared to leading commercial speech systems that are widely utilized.

The Achilles Heel of stochastic optimization algorithms is getting trapped on local optima. Novelty Search avoids this problem by encouraging a search in all interesting directions. That occurs by replacing a performance objective with a reward for novel behaviors, as defined by a human-crafted, and often simple, behavioral distance function. While Novelty Search is a major conceptual breakthrough and outperforms traditional stochastic optimization on certain problems, it is not clear how to apply it to challenging, high-dimensional problems where specifying a useful behavioral distance function is difficult. For example, in the space of images, how do you encourage novelty to produce hawks and heroes instead of endless pixel static? Here we propose a new algorithm, the Innovation Engine, that builds on Novelty Search by replacing the human-crafted behavioral distance with a Deep Neural Network (DNN) that can recognize interesting differences between phenotypes. The key insight is that DNNs can recognize similarities and differences between phenotypes at an abstract level, wherein novelty means interesting novelty. For example, a novelty pressure in image space does not explore in the low-level pixel space, but instead creates a pressure to create new types of images (e.g. churches, mosques, obelisks, etc.). Here we describe the long-term vision for the Innovation Engine algorithm, which involves many technical challenges that remain to be solved. We then implement a simplified version of the algorithm that enables us to explore some of the algorithm's key motivations. Our initial results, in the domain of images, suggest that Innovation Engines could ultimately automate the production of endless streams of interesting solutions in any domain: e.g. producing intelligent software, robot controllers, optimized physical components, and art.

Inspired by the information theoretic idea of minimum description length, we add a term to the back propagation cost function that penalizes network complexity. We give the details of the procedure, called weight-elimination, describe its dynamics, and clarify the meaning of the parameters involved. From a Bayesian perspective, the complexity term can be usefully interpreted as an assumption about prior distribution of the weights. We use this procedure to predict the sunspot time series and the notoriously noisy series of currency exchange rates.

We present a new method of designing neural networks using the genetic algorithm. Recently there have been several reports claiming attempts to design neural networks using genetic algorithms were successful. However, these methods have a problem in scalability, i.e., the convergence characteristic degrades significantly as the size of the network increases. This is because these methods employ direct mapp ing of chromosomes into network connectivities. As an alternative approach, we propose a graph grammatical encoding that will encode graph generation grammar to the chromosome so that it generates more regular connectivity patterns with shorter chromosome length. Experimental results support that our new scheme provides magnitude of speedup in convergence of neural network design and exhibits desirable scaling property.

Supervised deep learning has been successfully applied to many recognition problems. Although it can approximate a complex many-to-one function well when a large amount of training data is provided, it is still challenging to model complex structured output representations that effectively perform probabilistic inference and make diverse predictions. In this work, we develop a deep conditional generative model for structured output prediction using Gaussian latent variables. The model is trained efficiently in the framework of stochastic gradient variational Bayes, and allows for fast prediction using stochastic feed-forward inference. In addition, we provide novel strategies to build robust structured prediction algorithms, such as input noise-injection and multi-scale prediction objective at training. In experiments, we demonstrate the effectiveness of our proposed algorithm in comparison to the deterministic deep neural network counterparts in generating diverse but realistic structured output predictions using stochastic inference. Furthermore, the proposed training methods are complimentary, which leads to strong pixel-level object segmentation and semantic labeling performance on Caltech-UCSD Birds 200 and the subset of Labeled Faces in the Wild dataset.

We study the behavior of weight-tied multilayer vanilla autoencoders under the assumption of random weights. Via an exact characterization in the limit of large dimensions, our analysis reveals interesting phase transition phenomena when the depth becomes large. This, in particular, provides quantitative answers and insights to three questions that were yet fully understood in the literature. Firstly, we provide a precise answer on how the random deep weight-tied autoencoder model performs “approximate inference” as posed by Scellier et al. (2018), and its connection to reversibility considered by several theoretical studies. Secondly, we show that deep autoencoders display a higher degree of sensitivity to perturbations in the parameters, distinct from the shallow counterparts. Thirdly, we obtain insights on pitfalls in training initialization practice, and demonstrate experimentally that it is possible to train a deep autoencoder, even with the tanh activation and a depth as large as 200 layers, without resorting to techniques such as layer-wise pre-training or batch normalization. Our analysis is not specific to any depths or any Lipschitz activations, and our analytical techniques may have broader applicability.

We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Using stochastic variational inference, we analyze several large collections of documents: 300K articles from Nature, 1.8M articles from The New York Times, and 3.8M articles from Wikipedia. Stochastic inference can easily handle data sets of this size and outperforms traditional variational inference, which can only handle a smaller subset. (We also show that the Bayesian nonparametric topic model outperforms its parametric counterpart.) Stochastic variational inference lets us apply complex Bayesian models to massive data sets.

The CIFAR-10 dataset consists of color images of size 32x32. The training set contains 50,000 images, the testing set 10,000 images. Each image in CIFAR-10 is categorized into one of 10 possible classes. In Table 3, we report the performance of different models trained on CIFAR-10. From these results we can observe that our models using learned connectivity achieve consistently better performance over the equivalent models trained with the fixed connectivity (Xie et al., 2017).

Deep belief networks (DBNs) are currently the dominant technique for modeling the architectural depth of brain, and can be trained efficiently in a greedy layer-wise unsupervised learning manner. However, DBNs without a narrow hidden bottleneck typically produce redundant, continuous-valued codes and unstructured weight patterns. Taking inspiration from rate distortion (RD) theory, which encodes original data using as few bits as possible, we introduce in this paper a variant of DBN, referred to as sparse-response DBN (SR-DBN). In this approach, Kullback–Leibler divergence between the distribution of data and the equilibrium distribution defined by the building block of DBN is considered as a distortion function, and the sparse response regularization induced by L1-norm of codes is used to achieve a small code rate. Several experiments by extracting features from different scale image datasets show that our approach SR-DBN learns codes with small rate, extracts features at multiple levels of abstraction mimicking computations in the cortical hierarchy, and obtains more discriminative representation than PCA and several basic algorithms of DBNs. & 2014 Elsevier Ltd. All rights reserved.

We describe a neural network architecture designed to learn the musical structure of jazz melodies over chord progressions, then to create new melodies over arbitrary chord progressions from the resulting connectome (representation of neural network structure). Our architecture consists of two sub-networks, the interval expert and the chord expert, each being LSTM (long short-term memory) recurrent networks. These two sub-networks jointly learn to predict a probability distribution over future notes conditioned on past notes in the melody. We describe a training procedure for the network and an implementation as part of the opensource Impro-Visor (Improvisation Advisor) application, and demonstrate our method by providing improvised melodies based on a variety of training sets.

Modern deep neural networks have a large number of parameters, making them very hard to train. We propose DSD, a dense-sparse-dense training flow, for regularizing deep neural networks and achieving better optimization performance. In the first D (Dense) step, we train a dense network to learn connection weights and importance. In the S (Sparse) step, we regularize the network by pruning the unimportant connections with small weights and retraining the network given the sparsity constraint. In the final D (re-Dense) step, we increase the model capacity by removing the sparsity constraint, re-initialize the pruned parameters from zero and retrain the whole dense network. Experiments show that DSD training can improve the performance for a wide range of CNNs, RNNs and LSTMs on the tasks of image classification, caption generation and speech recognition. On ImageNet, DSD improved the Top1 accuracy of GoogLeNet by 1.1%, VGG-16 by 4.3%, ResNet-18 by 1.2% and ResNet-50 by 1.1%, respectively. On the WSJ’93 dataset, DSD improved DeepSpeech and DeepSpeech2 WER by 2.0% and 1.1%. On the Flickr-8K dataset, DSD improved the NeuralTalk BLEU score by over 1.7. DSD is easy to use in practice: at training time, DSD incurs only one extra hyper-parameter: the sparsity ratio in the S step. At testing time, DSD doesn’t change the network architecture or incur any inference overhead. The consistent and significant performance gain of DSD experiments shows the inadequacy of the current training methods for finding the best local optimum, while DSD effectively achieves superior optimization performance for finding a better solution. DSD models are available to download at https://songhan.github.io/DSD.


Question: StochasticNet: Creating deep neural networks through stochastic connectivity mechanisms.

This paper delves into the application of gradient-based learning methodologies to enhance document recognition systems. Unlike conventional approaches, this study explores how deep learning techniques, specifically those employing gradients, can significantly improve the accuracy and efficiency of document recognition tasks. By leveraging gradient descent algorithms, the system is able to iteratively adjust its parameters, thereby optimizing its performance in recognizing and interpreting written or printed text within images. This technique is particularly beneficial in scenarios where the variability in handwriting or font styles poses challenges to traditional recognition methods. The gradient-based learning model is designed to handle complex patterns and features, making it highly adaptable to diverse document structures and content. Through extensive experimentation and comparative analysis, the effectiveness of gradient-based learning in document recognition is substantiated, highlighting its potential to revolutionize the field with more accurate and reliable automated recognition capabilities.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.

**Similar Documents**: This work introduces an innovative evolutionary learning algorithm, termed Hybrid Genetic Algorithm-Particle Swarm Optimization (HGAPSO),

1 A neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects (Haykin 1998): 1. Knowledge is acquired by the network through a learning process. 2. Interconnection strengths known as synaptic weights are used to store the knowledge. Basically, learning is a process by which the free parameters (i.e., synaptic weights and bias levels) of a neural network are adapted through a continuing process of stimulation by the environment in which the network is embedded. The type of learning is determined by the manner in which the parameter changes take place. In a general sense, the learning process may be classified as follows:

Replicator neural networks self-organize by using their inputs as desired outputs; they internally form a compressed representation for the input data. A theorem shows that a class of replicator networks can, through the minimization of mean squared reconstruction error (for instance, by training on raw data examples), carry out optimal data compression for arbitrary data vector sources. Data manifolds, a new general model of data sources, are then introduced and a second theorem shows that, in a practically important limiting case, optimal-compression replicator networks operate by creating an essentially unique natural coordinate system for the manifold.

Training deep neural networks generally requires massive amounts of data and is very computation intensive. We show here that it may be possible to circumvent the expensive gradient descent procedure and derive the parameters of a neural network directly from properties of the training data. We show that, near convergence, the gradient descent equations for layers close to the input can be linearized and become stochastic equations with noise related to the covariance of data for each class. We derive the distribution of solutions to these equations and discover that it is related to a “supervised principal component analysis.” We implement these results on image datasets MNIST, CIFAR10 and CIFAR100 and find that, indeed, pretrained layers using our findings performs comparable or superior to neural networks of the same size and architecture trained with gradient descent. Moreover, our pretrained layers can often be calculated using a fraction of the training data, owing to the quick convergence of the covariance matrix. Thus, our findings indicate that we can cut the training time both by requiring only a fraction of the data used for gradient descent, and by eliminating layers in the costly backpropagation step of the training. Additionally, these findings partially elucidate the inner workings of deep neural networks and allow us to mathematically calculate optimal solutions for some stages of classification problems, thus significantly boosting our ability to solve such problems efficiently.

Despite the recent achievements in machine learning, we are still very far from achieving real artificial intelligence. In this paper, we discuss the limitations of standard deep learning approaches and show that some of these limitations can be overcome by learning how to grow the complexity of a model in a structured way. Specifically, we study the simplest sequence prediction problems that are beyond the scope of what is learnable with standard recurrent networks, algorithmically generated sequences which can only be learned by models which have the capacity to count and to memorize sequences. We show that some basic algorithms can be learned from sequential data using a recurrent network associated with a trainable memory.

In this paper we introduce deep Gaussian process (GP) models. Deep GPs are a deep belief network based on Gaussian process mappings. The data is modeled as the output of a multivariate GP. The inputs to that Gaussian process are then governed by another GP. A single layer model is equivalent to a standard GP or the GP latent variable model (GP-LVM). We perform inference in the model by approximate variational marginalization. This results in a strict lower bound on the marginal likelihood of the model which we use for model selection (number of layers and nodes per layer). Deep belief networks are typically applied to relatively large data sets using stochastic gradient descent for optimization. Our fully Bayesian treatment allows for the application of deep models even when data is scarce. Model selection by our variational bound shows that a five layer hierarchy is justified even when modelling a digit data set containing only 150 examples.

Stochastic gradient algorithms are the main focus of large-scale optimization problems and led to important successes in the recent advancement of the deep learning algorithms. The convergence of SGD depends on the careful choice of learning rate and the amount of the noise in stochastic estimates of the gradients. In this paper, we propose an adaptive learning rate algorithm, which utilizes stochastic curvature information of the loss function for automatically tuning the learning rates. The information about the element-wise curvature of the loss function is estimated from the local statistics of the stochastic first order gradients. We further propose a new variance reduction technique to speed up the convergence. In our experiments with deep neural networks, we obtained better performance compared to the popular stochastic gradient algorithms.1

Empirically, neural networks that attempt to learn programs from data have exhibited poor generalizability. Moreover, it has traditionally been difficult to reason about the behavior of these models beyond a certain level of input complexity. In order to address these issues, we propose augmenting neural architectures with a key abstraction: recursion. As an application, we implement recursion in the Neural Programmer-Interpreter framework on four tasks: grade-school addition, bubble sort, topological sort, and quicksort. We demonstrate superior generalizability and interpretability with small amounts of training data. Recursion divides the problem into smaller pieces and drastically reduces the domain of each neural network component, making it tractable to prove guarantees about the overall system’s behavior. Our experience suggests that in order for neural architectures to robustly learn program semantics, it is necessary to incorporate a concept like recursion.

We have used information-theoretic ideas to derive a class of practical and nearly optimal schemes for adapting the size of a neural network. By removing unimportant weights from a network, several improvements can be expected: better generalization, fewer training examples required, and improved speed of learning and/or classification. The basic idea is to use second-derivative information to make a tradeoff between network complexity and training set error. Experiments confirm the usefulness of the methods on a real-world application.

We study the behavior of weight-tied multilayer vanilla autoencoders under the assumption of random weights. Via an exact characterization in the limit of large dimensions, our analysis reveals interesting phase transition phenomena when the depth becomes large. This, in particular, provides quantitative answers and insights to three questions that were yet fully understood in the literature. Firstly, we provide a precise answer on how the random deep weight-tied autoencoder model performs “approximate inference” as posed by Scellier et al. (2018), and its connection to reversibility considered by several theoretical studies. Secondly, we show that deep autoencoders display a higher degree of sensitivity to perturbations in the parameters, distinct from the shallow counterparts. Thirdly, we obtain insights on pitfalls in training initialization practice, and demonstrate experimentally that it is possible to train a deep autoencoder, even with the tanh activation and a depth as large as 200 layers, without resorting to techniques such as layer-wise pre-training or batch normalization. Our analysis is not specific to any depths or any Lipschitz activations, and our analytical techniques may have broader applicability.

The ability to carry out signal processing, classification, recognition, and computation in artificial spiking neural networks (SNNs) is mediated by their synapses. In particular, through activity-dependent alteration of their efficacies, synapses play a fundamental role in learning. The mathematical prescriptions under which synapses modify their weights are termed synaptic plasticity rules. These learning rules can be based on abstract computational neuroscience models or on detailed biophysical ones. As these rules are being proposed and developed by experimental and computational neuroscientists, engineers strive to design and implement them in silicon and en masse in order to employ them in complex real-world applications. In this paper, we describe analog very large-scale integration (VLSI) circuit implementations of multiple synaptic plasticity rules, ranging from phenomenological ones (e.g., based on spike timing, mean firing rates, or both) to biophysically realistic ones (e.g., calcium-dependent models). We discuss the application domains, weaknesses, and strengths of various representative approaches proposed in the literature, and provide insight into the challenges that engineers face when designing and implementing synaptic plasticity rules in VLSI technology for utilizing them in real-world applications.


Question: StochasticNet: Creating deep neural networks through stochastic connectivity mechanisms.

This paper delves into the intricate task of identifying and deciphering text within natural images, a pivotal challenge in the evolving landscape of computer vision, with particular emphasis on the recognition of digits. Despite significant advancements in character recognition for structured documents, such as the near-perfect performance in reading handwritten digits, the capability to accurately discern characters within the complex and variable contexts of real-world photographs remains elusive. Existing methods, while sophisticated, still fall short of human-level accuracy when confronted with the task of reading digits in such images. To address this gap, we introduce an innovative benchmark dataset specifically designed for research in this domain, comprising over 600,000 annotated digits extracted from Street View imagery. We highlight the limitations of traditional approaches that rely on manually crafted features by showcasing the challenges they encounter in recognizing these digits. Subsequently, we explore and evaluate two cutting-edge unsupervised feature learning techniques that have been proposed recently, and we present compelling evidence of their superiority in tackling this benchmark dataset compared to conventional methods.

Stochastic computation graphs (SCGs) provide a formalism to represent structured optimization problems arising in artificial intelligence, including supervised, unsupervised, and reinforcement learning. Previous work has shown that an unbiased estimator of the gradient of the expected loss of SCGs can be derived from a single principle. However, this estimator often has high variance and requires a full model evaluation per data point, making this algorithm costly in large graphs. In this work, we address these problems by generalizing concepts from the reinforcement learning literature. We introduce the concepts of value functions, baselines and critics for arbitrary SCGs, and show how to use them to derive lower-variance gradient estimates from partial model evaluations, paving the way towards general and efficient credit assignment for gradient-based optimization. In doing so, we demonstrate how our results unify recent advances in the probabilistic inference and reinforcement learning literature.

We propose a new weight initialization suited for arbitrary nonlinearities by generalizing previous weight initializations. The initialization corrects for the influence of dropout rates and an arbitrary nonlinearity’s influence on variance through simple corrective scalars. Consequently, this initialization does not require computing mini-batch statistics nor weight pre-initialization. This simple method enables improved accuracy over previous initializations, and it allows for training highly regularized neural networks where previous initializations lead to poor convergence.

As new neural network and neuromorphic architectures are being developed, new training methods that operate within the constraints of the new architectures are required. Evolutionary optimization (EO) is a convenient training method for new architectures. In this work, we review a spiking neural network architecture and a neuromorphic architecture, and we describe an EO training framework for these architectures. We present the results of this training framework on four classification data sets and compare those results to other neural network and neuromorphic implementations. We also discuss how this EO framework may be extended to other architectures.

We describe a neural network architecture designed to learn the musical structure of jazz melodies over chord progressions, then to create new melodies over arbitrary chord progressions from the resulting connectome (representation of neural network structure). Our architecture consists of two sub-networks, the interval expert and the chord expert, each being LSTM (long short-term memory) recurrent networks. These two sub-networks jointly learn to predict a probability distribution over future notes conditioned on past notes in the melody. We describe a training procedure for the network and an implementation as part of the opensource Impro-Visor (Improvisation Advisor) application, and demonstrate our method by providing improvised melodies based on a variety of training sets.

Cascade-Correlation is a new architecture and supervised learning algorithm for artificial neural networks. Instead of just adjusting the weights in a network of fixed topology, Cascade-Correlation begins with a minimal network, then automatically trains and adds new hidden units one by one, creating a multi-layer structure. Once a new hidden unit has been added to the network, its input-side weights are frozen. This unit then becomes a permanent feature-detector in the network, available for producing outputs or for creating other, more complex feature detectors. The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the connections of the network. This research was sponsored in part by the National Science Foundation under Contract Number EET-8716324 and by the Defense Advanced Research Projects Agency (DOD), ARPA Order No. 4976 under Contract F33615-87-C-1499 and monitored by: Avionics Laboratory, Air Force Wright Aeronautical Laboratories, Aeronautical Systems Division (AFSC), Wright-Patterson AFB, OH 45433-6543. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the US Government. 1. Why is Back-Propagation Learning So Slow? The Cascade-Correlation learning algorithm was developed in an attempt to overcome certain problems and limitations of the popular back-propagation (or “backprop”) learning algorithm [Rumelhart, 1986]. The most important of these limitations is the slow pace at which backprop learns from examples. Even on simple benchmark problems, a back-propagation network may require many thousands of epochs to learn the desired behavior from examples. (An epoch is defined as one pass through the entire set of training examples.) We have attempted to analyze the reasons why backprop learning is so slow, and we have identified two major problems that contribute to the slowness. We call these the step-size problem and the moving target problem. There may, of course, be other contributing factors that we have not yet identified. 1.1. The Step-Size Problem The step-size problem occurs because the standard back-propagation method computes only E w, the partial first derivative of the overall error function with respect to each weight in the network. Given these derivatives, we can perform a gradient descent in weight space, reducing the error with each step. It is straightforward to show that if we take infinitesimal steps down the gradient vector, running a new training epoch to recompute the gradient after each step, we will eventually reach a local minimum of the error function. Experience has shown that in most situations this local minimum will be a global minimum as well, or at least a “good enough” solution to the problem at hand. In a practical learning system, however, we do not want to take infinitesimal steps; for fast learning, we want to take the largest steps that we can. Unfortunately, if we choose a step size that is too large, the network will not reliably converge to a good solution. In order to choose a reasonable step size, we need to know not just the slope of the error function, but something about its higher-order derivatives—its curvature—in the vicinity of the current point in weight space. This information is not available in the standard back-propagation algorithm. A number of schemes have been proposed for dealing with the step-size problem. Some form of “momentum” [Rumelhart, 1986] is often used as a crude way of summarizing the slope of the error surface at earlier points in the computation. Conjugate gradient methods have been explored in the context of artificial neural networks by a number of researchers [Watrous, 1988, Lapedes, 1987, Kramer, 1989] with generally good results. Several schemes, for example [Franzini, 1987] and jacobs:dynamic, have been proposed that adjust the step-size dynamically, based on the change in gradient from one step to another. Becker and LeCun [Becker, 1988] explicitly compute an approximation to the second derivative of the error function at each step and use that information to guide the speed of descent. Fahlman’s quickprop algorithm [Fahlman, 1988] is one of the more successful algorithms for handling the step-size problem in back-propagation systems. Quickprop computes the E w values just as in standard backprop, but instead of simple gradient descent, Quickprop uses a second-order method, related to Newton’s method, to update the weights. On the learning benchmarks we have collected, quickprop consistently out-performs other backprop-like algorithms, sometimes by a large factor. Quickprop’s weight-update procedure depends on two approximations: first, that small changes in one weight have relatively little effect on the error gradient observed at other weights; second, that the error function with respect to each weight is locally quadratic. For each weight, quickprop keeps a copy of

To what extent is the success of deep visualization due to the training? Could we do deep visualization using untrained, random weight networks? To address this issue, we explore new and powerful generative models for three popular deep visualization tasks using untrained, random weight convolutional neural networks. First we invert representations in feature spaces and reconstruct images from white noise inputs. The reconstruction quality is statistically higher than that of the same method applied on well trained networks with the same architecture. Next we synthesize textures using scaled correlations of representations in multiple layers and our results are almost indistinguishable with the original natural texture and the synthesized textures based on the trained network. Third, by recasting the content of an image in the style of various artworks, we create artistic images with high perceptual quality, highly competitive to the prior work of Gatys et al. on pretrained networks. To our knowledge this is the first demonstration of image representations using untrained deep neural networks. Our work provides a new and fascinating tool to study the representation of deep network architecture and sheds light on new understandings on deep visualization.

A proposed sparse recurrent neural network with flexible topology is used for trajectory prediction of the Atlantic hurricanes. For prediction of the future trajectories of a target hurricane, the most similar hurricanes to the target hurricane are found by comparing directions of the hurricanes. Then, the first and second differences of their positions over their life time are used for training the proposed network. Comparison of the obtained predictions with actual trajectories of Sandy and Humberto hurricanes show that our approach is quite promising for this aim.

Stochastic gradient algorithms are the main focus of large-scale optimization problems and led to important successes in the recent advancement of the deep learning algorithms. The convergence of SGD depends on the careful choice of learning rate and the amount of the noise in stochastic estimates of the gradients. In this paper, we propose an adaptive learning rate algorithm, which utilizes stochastic curvature information of the loss function for automatically tuning the learning rates. The information about the element-wise curvature of the loss function is estimated from the local statistics of the stochastic first order gradients. We further propose a new variance reduction technique to speed up the convergence. In our experiments with deep neural networks, we obtained better performance compared to the popular stochastic gradient algorithms.1

In this work, we characterize the outputs of individual neurons in a trained feed-forward neural network by entropy, mutual information with the class variable, and a class selectivity measure based on Kullback-Leibler divergence. By cumulatively ablating neurons in the network, we connect these information-theoretic measures to the impact their removal has on classification performance on the test set. We observe that, looking at the neural network as a whole, none of these measures is a good indicator for classification performance, thus confirming recent results by Morcos et al. However, looking at specific layers separately, both mutual information and class selectivity are positively correlated with classification performance. We thus conclude that it is ill-advised to compare these measures across layers, and that different layers may be most appropriately characterized by different measures. We then discuss pruning neurons from neural networks to reduce computational complexity of inference. Drawing from our results, we perform pruning based on information-theoretic measures on a fully connected feed-forward neural network with two hidden layers trained on MNIST dataset and compare the results to a recently proposed pruning method. We furthermore show that the common practice of re-training after pruning can partly be obviated by a surgery step called bias balancing, without incurring significant performance degradation.

Deep neural networks (DNNs) applications are now increasingly pervasive and powerful. However, fashion designers are lagging behind in leveraging this increasingly common technology. DNNs are not yet a standard part of fashion de sign practice, either clothes patterns or prototyping tools. In this paper, we present DeepWear, a method using deep convolutional generative adversarial networks for clothes design. The DNNs learn the feature of specific brand clothes and generate images then patterns instructed from the images are made, and an author creates clothes based on that. We evaluated this system by evaluating the credibility of the actual sold clothes on market with our clothes. As the result, we found it is possible to make clothes look like actual products from the generated images. Our findings have implications for collaborative design between machine and human intelligence.


Question: A dual-band stepped-impedance transformer for transition to full-height substrate-integrated waveguide.

This paper delves into the guided-wave and leakage properties of the substrate integrated waveguide (SIW), a technique that enables the planar fabrication of circuits, transitions, and rectangular waveguides using standard printed circuit board (PCB) or other planar processing methods. Here, we present an in-depth analysis of the guided wave and mode characteristics of the SIW periodic structure, which has not been comprehensively studied before. To accurately determine the complex propagation constants of the SIW structure, we have devised a novel multimode calibration procedure using a commercial software package that employs a full-wave finite-element method. Through numerical simulations of two distinct lengths of the SIW under multimode excitation, our technique enables the precise extraction of the complex propagation constant for each mode of the SIW. Additionally, we explore the electromagnetic bandstop phenomena associated with periodic structures. To substantiate the effectiveness of our proposed technique, we conduct experimental validations. The paper also includes simple design guidelines and discussions on the implications of our findings.

This paper presents a C-band rectifier employing a third harmonic termination which is realized by a λg/12 short-ended microstrip transmission line. The proposed harmonic termination balances the capacitive impedance of the diode at the fundamental frequency and blocks harmonic current of the diode. The input low-pass filter or matching circuit is removed and the bandwidth of rectifier is enhanced. Theoretical analysis and experiments are carried out. The results show that the proposed topology can realize high efficiency with a wide input frequency range. The fabricated rectifier has a compact dimension of 14mm by 22mm. A maximum RF-DC conversion efficiency of 75.6% at 13 dBm input power is observed. The measured efficiency remains above 70% with the operating frequency form 5.2 GHz to 6 GHz.

A wideband 57.7–84.2 GHz Phase Shifter is presented using a compact Lange coupler to generate in-phase and quadrature signal. The Lange coupler is followed by two balun transformers that provide the IQ vector modulation with differential I and Q signals. The implemented Phase Shifter demonstrates an average 6-dB insertion loss and 5-dB gain variation. The measured average rms phase and gain errors are 7 degrees and 1 dB, respectively. The phase shifter is implemented in GlobalFoundries 45-nm SOI CMOS technology using a trap-rich substrate. The chip area is 385 μm × 285 μm and the Phase Shifter consumes less than 17 mW. To the best of authors knowledge, this is the first phase shifter that covers both 60 GHz band and E-band frequencies with a fractional bandwidth of 37%.

CMOS transmit-receive (T/R) switches have been integrated in a 0.18-/spl mu/m standard CMOS technology for wireless applications at 2.4 and 5.2 GHz. This switch design achieves low loss and high linearity by increasing the substrate impedance of a MOSFET at the frequency of operation using a properly tuned LC tank. The switch design is asymmetric to accommodate the different linearity and isolation requirements in the transmit and receive modes. In the transmit mode, the switch exhibits 1.5-dB insertion loss, 28-dBm power, 1-dB compression point (P/sub 1dB/), and 30-dB isolation, at 2.4 and 5.2 GHz. In the receive mode, the switch achieves 1.6-dB insertion loss, 11.5-dBm P/sub 1dB/, and 15-dB isolation, at 2.4 and 5.2 GHz. The linearity obtained in the transmit mode is the highest reported to date in a standard CMOS process. The switch passes the 4-kV Human Body Model electrostatic discharge test. These results show that the switch design is suitable for narrow-band applications requiring a moderate-high transmitter power level (<1 W).

A new type of substrate integrated waveguide (SIW) multibeam antenna is proposed for mobile satellite communications using beam switching and diversity techniques. It employs an SIW Rotman lens as the beamforming network. The prototype of a single multibeam antenna is implemented at 28.5 GHz with seven input ports and an antenna array constructed by nine SIW linear slot arrays, which can generate a corresponding number of beams along one dimension. Several such antennas are grouped in two different ways to cover a 2-D solid angle with multiple beams. Experiment results show that the 2-D solid angle around (-40deg, 40deg) X (-35deg, 35deg) or (-25deg, 25deg) x (-35deg, 35deg) are covered with 20 or 25 beams with 5 -dB beamwidth, respectively. It is demonstrated that this type of printed multibeam antenna is a good choice for communication applications where mobility and high gain are simultaneously required.

A reduced-size wideband single-feed circularly polarized patch antenna is introduced for telemetry applications in S-band around 2300 MHz. The proposed structure consists of a slot-loaded patch antenna printed over an optimized metamaterial-inspired reactive impedance substrate (RIS). We demonstrate, step by step, the main role of each antenna element by comparing numerically and experimentally the performance of various antenna configurations: antenna over a single- or dual-layer substrate, standard patch or slot-loaded patch, antenna with or without RIS. The final optimized structure exhibits an axial-ratio bandwidth of about 15% and an impedance bandwidth better than 11%, which is much wider than the conventional printed antenna on the same materials.

In this paper, we describe the architecture and performance of a fine pitch multiple chip heterogeneous integration solution using ceramic interconnect bridge on organic substrate package. We present the increased IO density and improvement of electrical high-speed performance on signal integrity are achievable through this novel integration scheme, where dense copper routings on small ceramic elements are served as interconnect bridges. The cost and signal attenuation of using ceramic bridge is far better than that of silicon bridge or wafer interposer on substrate.

Abstract— This paper describes 2 x 2 triangular microstrip patch antenna using T-junction with quarter wave transformer. By regulating the distance in patch antenna and adjusting feed position, bandwidth can be obtained and by using an array, directivity is enhanced. The requirement of large bandwidth, high directivity, and minimum size leads to the design of 2 x 2 triangular microstrip patch antenna array feeding with T-junction network operate at 5.5 GHz. An antenna designed on an FR4 substrate that had a dielectric constant (r) 4.4, a loss tangent 0.02 and thickness of 1.6 mm. Simulated results showed that designed antenna has directivity 12.91 dB and bandwidth 173 MHz with VSWR 1.07 using T-junction feeding network. The proposed 2 x 2 triangular array has the benefit of light weight, simplicity of fabrication, single layer structure, and high directivity. Keyword Bandwidth, Corporate feeding, Return Loss, T-junction, VSWR.

A broadband eight-way spatial combiner using coaxial probes and radial waveguides has been proposed and designed. The simple electromagnetic modeling for the radial waveguide power divider/combiner has been developed using equivalent-circuit method. The measured 10-dB return loss and 1-dB insertion loss bandwidth of this waveguide spatial combiner are all demonstrated to be about 8 GHz.

In this paper a multiband fractal based rectangular microstrip patch antenna is designed. FR4 substrate having thickness of 1.58 mm is used as substrate material for the design of proposed antenna and microstrip feed line provides the excitation to the antenna. The antenna operating frequency range is from 1 to 10 GHz. The proposed antenna resonate at twelve different frequencies as 1.86, 2.33, 3.67, 4.57, 5.08, 6.06, 7.03, 7.75, 8.08, 8.84, 9.56 and 10 GHz and the return losses are -15.39, -16.48, -10.02, -17.29, -13.15, -23.41, -10.22, -11.28, -17.02, -10.94, -15.15 and -15.48 dB respectively. The proposed antenna is designed and simulated by using the Ansoft HFSS V13 (high frequency structure simulator) software.

A novel low cost feeding mechanism based upon new synthesized planar waveguide substrate integrated waveguide (SIW), is presented for microstrip antenna arrays (MPAs) at millimeter-wave (mmW) band to avoid feeding loss, specially the conduction loss. As an example, SIW-fed two dimensional (2D) 2×2 MPA array @ 60 GHz is introduced. The presented antenna is simulated by HFSS to validate its performance for mmW applications. The simulated results show a broadside gain of 11.71 dB within an operating impedance bandwidth of 500 MHz around a center frequency of 59.95 GHz. Also, the simulated radiation efficiency of the developed antenna is better than 85%, which is higher than that of conventional mmW microstrip antenna array.


Question: A dual-band stepped-impedance transformer for integration into full-height substrate-integrated waveguides.

This paper introduces a new and efficient transition design between conductor-backed coplanar waveguide (CBCPW) and substrate integrated waveguide (SIW) for applications in microwave and millimeter-wave integrated circuits. The proposed integrated transition, which ensures concurrent field and impedance matching, exhibits exceptional low-loss characteristics over an extensive ultra-wideband range, specifically covering the entire Ka-band in our investigation. A critical aspect of this design involves the utilization of a generalized impedance inverter, whose parameters are meticulously extracted through a precise numerical Thru-Reflection-Line (TRL) calibration technique. This ensures the transition's effectiveness and accuracy. Experimental results obtained from the fabricated transition, tested in a back-to-back configuration, reveal that the insertion loss remains below 0.4 dB, and the return loss is superior to 20 dB throughout the Ka-band, showcasing the transition's superior performance and reliability in ultra-wideband applications.

A circularly polarized slot radiator in millimetre band is presented. Low reflection and good polarization performance is obtained in comparison with state of art slot elements. A high gain array prototype has been implemented in substrate integrated waveguides (SIW) technology, in order to check the new element performance. A cosecant amplitude coverage in the elevation plane, as well as a 6° tilted monopulse pattern in the azimuth plane are obtained. A gain peak value of 28.6 dBi and 79% of efficiency at 36.7 GHz have been obtained in the specified operation band (36.7 – 37 GHz) for a manufactured antenna prototype. Likewise, the top measured axial ratio is 1.95 dB at 36.85 GHz.

In this work, a miniature ultra-wideband (UWB) power divider (PD) is proposed. By implementing the transmission lines of a two-stage Wilkinson PD using bridged T-coils, very compact size with no reduction in bandwidth can be achieved. Specifically, a proposed UWB two-way PD with a center frequency f0=5.5 GHz is implemented using the commercial GaAs pHEMT process. The circuit size without pads is only 1.45 mm × 0.84 mm, which is about 0.027λ0 × 0.016λ0 at f0. The fractional bandwidth for 15-dB input/output return loss and isolation is 110%, and the in-band insertion loss is within 1.3 ± 0.36 dB.

In this work, we present a 4-band dual-polarized antenna designed for 8-chanel applications. Based on LTCC technology, the antenna is a patch coupled aperture with a backed cavity. Each antenna element of a designated band contains two channels through two orthogonally polarized ports. Combining four dual-polarized antenna elements under different frequency, an 8-channel antenna for 60GHz applications can be achieved. The array antenna contains 8 feeding ports, which correspond to 8 independent channels. The isolation between each port can reach 20dB in most of the frequency band.

Novel compact tri-band bandpass filter (BPF) with controllable bandwidths is presented in this letter. The tri-band performance is realized by using three sets of resonators, i.e., two sets of short-stub loaded dual-mode resonators, which are designed for the first and second passband, respectively, and half-wavelength resonators for the third passband. The three passband frequencies and bandwidths could be independently tuned and designed. The half-wavelength resonators are embedded between the two dual-mode resonators and serve as part of feed lines for the first and second passband, thus compact size is realized. The newly designed tri-band BPF is verified by circuit implementation and good agreement between the simulated and measured results can be observed.

In this letter, a new type of wideband substrate integrated waveguide (SIW) cavity-backed patch antenna and array for millimeter wave (mmW) are investigated and implemented. The proposed antenna is composed of a rectangular patch with a backed SIW cavity. In order to enhance the bandwidth and radiation efficiency, the cavity is designed to resonate at its TE210 mode. Based on the proposed antenna, a 4 × 4 array is also designed. Both the proposed antenna and array are fabricated with standard printed circuit board (PCB) process, which possess the advantage of easy integration with planar circuits. The measured bandwidth (|S11| ≤ -10 dB) of the antenna element is larger than 15%, and that of the antenna array is about 8.7%. The measured peak gains are 6.5 dBi for the element and 17.8 dBi for the array, and the corresponding simulated radiation efficiencies are 83.9% and 74.9%, respectively. The proposed antenna and array are promising for millimeter-wave applications due to its merits of wide band, high efficiency, low cost, low profile, etc.

The design and realization of a low-profile wideband dual-polarized patch antenna with high isolation and low cross polarization has been presented. The aperture-coupled antenna element has four layers of substrates. Two microstrip lines terminated with circular patches are printed on the backside of the bottom substrate for feeding the radiating patches, while on the other side is a ground plane with two offset H-shaped slots. One port is excited for horizontally polarized wave and the other one is for vertical polarization. Two slots are arranged in a `T' configuration to achieve a high isolation of over 35 dB. A square radiating patch is printed on the top layer of a Duroid substrate which is placed above the ground plane. A piece of Rohacell foam with a thickness of 3mm and a permittivity of 1.06 is located above the Duroid substrate. A top radiating patch is printed on the bottom side of a top Duroid substrate. Thus, a wideband performance can be obtained due to the multilayer structure and the impedance matching can be improved by adjusting the dimensions of the circular patches and the slots. Further to reduce the cross polarization, two identical elements with one rotated by 180° are combined and each pair of single linearly polarized elements are fed out of phase. In addition, a four-element array with two mirrored two-element arrays has been investigated. The obtained results can confirm that, the proposed dual-polarized patch array can provide a wideband performance across 8.5-11 GHz with VSWR <; 2. The isolation between the two ports is over 35 dB and the cross polarization level is less than -35 dB over the operating frequency band. Therefore, the proposed design technique is promising for X-band SAR applications.

This proposed paper, a single band microstrip patch antenna for 5G wireless application is presented. This proposed antenna is suitable for the millimeter wave frequency. The single band antenna consist of new H slot and E slot loaded on the radiating patch with the 50 ohms microstrip line feeding used. This single band antenna is simulated on a Rogers RT5880 dielectric substrate have relative permittivity 2.2, loss tangent 0.0009, and height 1.6mm. The antenna is simulated by Electromagnetic simulation, computer software technology Microwave studio. The proposed single band antenna and simulated result on return loss, VSWR, surface current and 3D radiation pattern is presented. The simulated antenna shows the return loss −40.99dB at 60 GHz millimeter wave 5G wireless application presented.

In this paper, a coax-to-microstrip transition incorporating a miniature coaxial connector with glass bead is developed. The transition is firstly simulated with the aid of HFSS and then measured in the frequency range of 1-15 GHz. The simulated and measured results are found in good agreement.

This paper describes an integrated frequency multiplier, implemented as a Gilbert cell based frequency doubler in a 130 nm SiGe BiCMOS technology. The circuit demonstrates a 3 dB bandwidth of 97–134GHz with peak output power of 1 dBm for 1 dBm input power. The fundamental suppression, measured at the single-ended output, is better than 21 dBc while the frequency doubler consumes 69mW from a 3.3V supply. The doubler is preceded by a differential amplifier functioning as an active balun to generate a differential signal for the Gilbert cell.

A dual-band textile antenna for multiple-input-multiple-output (MIMO) applications, based on substrate-integrated waveguide (SIW) technology, is designed. The fundamental SIW cavity mode is designed to resonate at 2.4 GHz. Meanwhile, the second and third modes are modified and combined by careful placement of a via within the cavity to enable wideband coverage in the 5-GHz WLAN band. The simple antenna topology can be fabricated fully using textiles in a planar form, ensuring reliability and comfort. Numerical and experimental results indicate satisfactory antenna performance when worn on body in terms of impedance bandwidth, radiation efficiency, and specific absorption ratio (SAR). In order to validate its potential for MIMO applications, two elements of the proposed SIW antenna are arranged in six configurations to study the performance in terms of mutual coupling and envelope correlation. It is observed that the placement of the shorted edges of the two elements adjacent to each other produces the lowest mutual coupling and consequently the best envelope correlation.


Question: A dual-band stepped-impedance transformer for conversion to full-height substrate-integrated waveguide.

This paper presents an innovative planar platform for the seamless integration of coplanar waveguide (CPW) and rectangular waveguide components on a shared substrate, eliminating the need for complex 3-D mounting structures typically required for transitioning between planar circuits and rectangular waveguides. This new approach significantly reduces the complexity and cost associated with high-precision mechanical alignment, as it enables the construction of both waveguide and planar components using standard printed circuit board (PCB) fabrication processes. Demonstrated through experimental results at 28 GHz, the proposed CPW-to-waveguide transition achieves an impressive effective bandwidth of 7% with a return loss of 15 dB. This development paves the way for the comprehensive integration of waveguide components with active devices like monolithic microwave integrated circuits (MMICs) directly on the substrate, enhancing the efficiency and compactness of microwave systems.

A microstrip bandpass filter using stepped-impedance resonators is designed in low-temperature co-fired ceramic technology for dual-band applications at 2.4 and 5.2 GHz. New coupling schemes are proposed to replace the normal counterparts. It is found that the new coupling scheme for the interstages can enhance the layout compactness of the bandpass filter; while the new coupling scheme at the input and output can improve the performance of the bandpass filter. To validate the design and analysis, a prototype of the bandpass filter was fabricated and measured. It is shown that the measured and simulated performances are in good agreement. The prototype of the bandpass filter achieved insertion loss of 1.25 and 1.87 dB, S11 of -29 and -40 dB, and bandwidth of 21% and 12.7% at 2.4 and 5.2 GHz, respectively. The bandpass filter is further studied for a single-package solution of dual-band radio transceivers. The bandpass filter is, therefore, integrated into a ceramic ball grid array package. The integration is analyzed with an emphasis on the connection of the bandpass filter to the antenna and to the transceiver die

In this paper, an X-band 1×3 substrate integrated waveguide (SIW) power divider design is presented. The designed SIW power divider provides equal amplitude with uniform phase distribution at the each output port. It has also a satisfactory operating bandwidth and low insertion loss. Moreover, the return loss is approximately 25 dB at the design frequency as shown in the EM simulation results.

The work presented in the paper focuses on accuracy of models for broad-band ferrite based coaxial transmission-line transformers. Soft-ferrites are largely used in VHF/UHF components allowing band enlargement on the low-edge side. Degradation of frequency performance on the high-edge side are produced both by ferrite losses, and by parasitic capacitance due to connection to the thermal and electrical ground in high power applications. Both a circuital model for low-power applications and a scalable e.m. model for high-power applications are presented and discussed.

A dual-band textile antenna for multiple-input-multiple-output (MIMO) applications, based on substrate-integrated waveguide (SIW) technology, is designed. The fundamental SIW cavity mode is designed to resonate at 2.4 GHz. Meanwhile, the second and third modes are modified and combined by careful placement of a via within the cavity to enable wideband coverage in the 5-GHz WLAN band. The simple antenna topology can be fabricated fully using textiles in a planar form, ensuring reliability and comfort. Numerical and experimental results indicate satisfactory antenna performance when worn on body in terms of impedance bandwidth, radiation efficiency, and specific absorption ratio (SAR). In order to validate its potential for MIMO applications, two elements of the proposed SIW antenna are arranged in six configurations to study the performance in terms of mutual coupling and envelope correlation. It is observed that the placement of the shorted edges of the two elements adjacent to each other produces the lowest mutual coupling and consequently the best envelope correlation.

A novel leaky-wave antenna is demonstrated and developed at Ka-band in this work based on the newly proposed half-mode substrate integrated waveguide (HWSIW). This antenna is accurately simulated by using a full-wave electromagnetic simulator and then fabricated through a single-layer printed circuit board (PCB) process. Wide bandwidth and a quasi-omnidirectional radiation pattern are obtained. The proposed antenna is therefore a good candidate for millimeter-wave applications. Measured results are in good agreement with simulated results.

A new type of substrate integrated waveguide (SIW) multibeam antenna is proposed for mobile satellite communications using beam switching and diversity techniques. It employs an SIW Rotman lens as the beamforming network. The prototype of a single multibeam antenna is implemented at 28.5 GHz with seven input ports and an antenna array constructed by nine SIW linear slot arrays, which can generate a corresponding number of beams along one dimension. Several such antennas are grouped in two different ways to cover a 2-D solid angle with multiple beams. Experiment results show that the 2-D solid angle around (-40deg, 40deg) X (-35deg, 35deg) or (-25deg, 25deg) x (-35deg, 35deg) are covered with 20 or 25 beams with 5 -dB beamwidth, respectively. It is demonstrated that this type of printed multibeam antenna is a good choice for communication applications where mobility and high gain are simultaneously required.

The idea of using higher order modes is demonstrated by an SIW slot array antenna with a TE20 mode to realize boresight radiation. In order to excite the TE20 mode, an SIW hybrid coupler and a compact SIW 90° phase shifter are designed to offer the two out-of-phase outputs with identical amplitude. Besides, a CPW-SIW transition is adopted for measurement. The SIW slot array antenna with the TE20 mode is designed at X-band in a single-layer PCB. The measured bandwidth is 5.5% (9.72–10.27 GHz). The measured total efficiency is 80% at 10 GHz. Compared to a conventional SIW slot array just with a power divider, using the TE20 mode provides the merits of not only reducing the number of vias but also relaxing fabrication limit and tolerance which could be the fabrication bottleneck at high frequencies.

Introduction: Short-circuited microstrip patch antennas, or PIFAs, can be used as directive internal handset antennas in various communication systems. They provide advantages over traditional external whip and helix antennas in terms of increased total efficiency (when the handset is near the user’s head), decreased radiation towards the user, and increased mechanical reliability. The main disadvantage is their narrow impedance bandwidth compared to the volume occupied by the antenna structure. Typical requirements for directive internal antennas designed for cellular handsets (with a thickness ≤ 20mm) are antenna thickness ≤ 5mm and bandwidth ≥ 10%. It is difficult to design such a small, directive, low-profile handset antenna with high radiation efficiency and an impedance bandwidth greater than 10%. Small antenna design is always a compromise between size, bandwidth, and efficiency. Often, the price for improved performance is increased complexity. If the total dimensions of a patch antenna element are fixed, there are two effective ways to enhance its bandwidth: either dissipative loading [1], which reduces the radiation efficiency, or the addition of more resonators into the antenna structure (matching networks or parasitic elements). The latter method may increase the manufacturing complexity of the antenna. In this Letter, a thin dual-resonant stacked shorted patch antenna is presented. Previously, stacked short-circuited patch antenna elements have been reported in [2, 3]. More recently, similar elements have also been discussed in [4, 5]. The presented antenna is dual-resonant and has a very low profile. The thickness is only one fifth of that of the antenna reported in [5], whereas the surface area reserved for the patches is approximately equal. The small size of the antenna presented in this Letter makes it relatively easy to fit inside a cellular handset. The bandwidth is sufficient for many communication systems. With a common short circuit element which connects both patches to the ground plane, the presented antenna is the simplest example of a stacked shorted microstrip patch antenna. No extra tuning or shorting posts inside the patch area are needed, which makes its manufacturing easier.

In this paper, a modified coaxial probe waveguide-to-microstrip transition structure at K-band is proposed. The transition uses two kinds of coaxial probes and has hermetic characteristics. It can also be used in other band, such as Ka-band. The structure is demonstrated for designing the transition. HFSS simulation and measurement results for the transition structure have been compared in terms of |S/sub 21/|. The comparison shows that the measurement agrees well with the simulation result. The measurement has shown 0.5 dB insertion loss.

A fully-integrated antenna-in-package (AiP) solution for W-band scalable phased-array systems is demonstrated. We present a fully operational compact W-band transceiver package with 64 dual-polarization antennas embedded in a multilayer organic substrate. This package has 12 metal layers, a size of 16.2 mm × 16.2 mm, and 292 ball-grid-array (BGA) pins with 0.4 mm pitch. Four silicon-germanium (SiGe) transceiver ICs are flip-chip attached to the package. Extensive full-wave electromagnetic simulation and radiation pattern measurements have been performed to optimize the antenna performance in the package environment, with excellent model-to-hardware correlation achieved. Enabled by detailed circuit-package co-design, a half-wavelength spacing, i.e., 1.6 mm at 94 GHz, is maintained between adjacent antenna elements to support array scalability at both the package and board level. Effective isotropic radiated power (EIRP) and radiation patterns are also measured to demonstrate the 64-element spatial power combining.


Question: A dual-band stepped-impedance transformer for transition to a full-height substrate-integrated waveguide.

This paper introduces substrate integrated circuits (SICs), a cutting-edge approach to high-frequency circuit design that offers significant advancements over traditional methodologies. The paper comprehensively examines the current state of SICs, detailing their design and implementation in various platforms. It highlights the diverse applications and benefits of SICs in microwave, millimeter-wave, and optoelectronics fields. Through a series of practical examples, the paper demonstrates the effectiveness of SICs in substrate integrated waveguide (SIW), substrate integrated slab waveguide (SISW), and substrate integrated nonradiating dielectric (SINRD) guide circuits. Theoretical and experimental outcomes are presented to support these applications. Moreover, the paper discusses future research directions and innovation trends in the design of low-cost millimeter-wave and optoelectronic integrated circuits, showcasing the potential of SICs in these areas.

A class of the novel compact-size branch-line couplers using the quasi-lumped elements approach with symmetrical or nonsymmetrical T-shaped structures is proposed in this paper. The design equations have been derived, and two circuits using the quasi-lumped elements approach were realized for physical measurements. This novel design occupies only 29% of the area of the conventional approach at 2.4 GHz. In addition, a third circuit was designed by using the same formula implementing a symmetrical T-shaped structure and occupied both the internal and external area of the coupler. This coupler achieved 500-MHz bandwidth while the phase difference between S21 and S31 is within 90degplusmn1deg. Thus, the bandwidth is not only 25% wider than that of the conventional coupler, but occupies only 70% of the circuit area compared to the conventional design. All three proposed couplers can be implemented by using the standard printed-circuit-board etching processes without any implementation of lumped elements, bonding wires, and via-holes, making it very useful for wireless communication systems

We have developed a simple and efficient procedure to design H-plane rectangular waveguide T-junctions for equal and unequal two-way power splitters. This synthesis procedure is scalable, renders manufacturable structures, applicable to any arbitrary power split-ratio, and can offer wide band operation. In our implementation, we utilized wedges and inductive windows (being an integral part of the T-junctions), to provide more degrees of freedom, thus, excellent match at the input port, flat power-split ratio over the band with equal phase, where phase balance is essential for various antenna feeds.

The successful demonstration of a 1:4 power divider using microstrip probes and a WR-430 rectangular waveguide is presented. The 15-dB return loss bandwidth of the nonoptimized structure is demonstrated to be 22% and its 0.5-dB insertion loss bandwidth 26%. While realized through conventional machining, such a structure is assembled in a fashion consistent with proven millimeter and submillimeter-wave micromachining techniques. Thus, the structure presents a potential power dividing and power combining architecture, which through micromachining, may be used for applications well above 100GHz.

A circularly polarized slot radiator in millimetre band is presented. Low reflection and good polarization performance is obtained in comparison with state of art slot elements. A high gain array prototype has been implemented in substrate integrated waveguides (SIW) technology, in order to check the new element performance. A cosecant amplitude coverage in the elevation plane, as well as a 6° tilted monopulse pattern in the azimuth plane are obtained. A gain peak value of 28.6 dBi and 79% of efficiency at 36.7 GHz have been obtained in the specified operation band (36.7 – 37 GHz) for a manufactured antenna prototype. Likewise, the top measured axial ratio is 1.95 dB at 36.85 GHz.

In this paper, we demonstrate a waveguide slot array antenna by integration of T-shaped filters using the diffusion bonding of laminated thin metal plates. The excellent performance of the T-shaped filter has been experimentally verified by the in-band transmission and the out-of-band rejection of the fabricated antenna.

Dual-polarized antenna is widely used in communication systems such as ECM and DF systems. In this paper a novel doubleridged horn antenna with dual polarizations is introduced for frequency range of 8–18 GHz. Common double ridged horn antennas have single polarization over the operating frequency. We have used five layers polarizer to provide dual polarizations performance of the doubleridged horn antenna. In order to achieve dual polarizations the strips width, strips spacing and layers distances are optimized. It is worth mentioning that the corresponding VSWR of the antenna during the optimization process should be maintain below a certain value (VSWR< 2). Simulation results show that the proposed antenna yields dual polarizations performance and low VSWR over the operating frequency. We have used CST software for antenna simulation which is based on the finite integral technique.

A metamaterial balun that converts a single-ended input to a differential output over a large bandwidth is presented. The device also exhibits excellent return loss, isolation, and through characteristics over the same frequency band. The balun comprises a Wilkinson divider, followed by a +90/spl deg/ negative-refractive-index (NRI) metamaterial (MM) phase-shifting line along the top branch, and a -90/spl deg/ MM phase-shifting line along the bottom branch. Utilizing MM lines for both the +90/spl deg/ and -90/spl deg/ branches allows the slopes of their phase responses to be matched, resulting in a broadband differential output signal. The theoretical performance of the balun is verified through circuit simulations and measurements of a fabricated prototype at 1.5 GHz. The MM balun exhibits a measured differential output phase bandwidth (180/spl deg//spl plusmn/10/spl deg/) of 1.16 GHz (77%), from 1.17 to 2.33 GHz. The measured isolation and return loss for all three ports remain below -10 dB over a bandwidth in excess of 2 GHz, while the output quantities |S/sub 21/| and |S/sub 31/| remain above -4 dB from 0.5 to 2.5 GHz.

We present a technology for the manufacturing of silicon-filled integrated waveguides enabling the realization of low-loss high-performance millimeter-wave passive components and high gain array antennas, thus facilitating the realization of highly integrated millimeter-wave systems. The proposed technology employs deep reactive-ion-etching (DRIE) techniques with aluminum metallization steps to integrate rectangular waveguides with high geometrical accuracy and continuous metallic side walls. Measurement results of integrated rectangular waveguides are reported exhibiting losses of 0.15 dB/ λg at 105 GHz. Moreover, ultra-wideband coplanar to waveguide transitions with 0.6 dB insertion loss at 105 GHz and return loss better than 15 dB from 80 to 110 GHz are described and characterized. The design, integration and measured performance of a frequency scanning slotted-waveguide array antenna is reported, achieving a measured beam steering capability of 82 ° within a band of 23 GHz and a half-power beam-width (HPBW) of 8.5 ° at 96 GHz. Finally, to showcase the capability of this technology to facilitate low-cost mm-wave system level integration, a frequency modulated continuous wave (FMCW) transmit-receive IC for imaging radar applications is flip-chip mounted directly on the integrated array and experimentally characterized.

A simple and compact slot antenna with a very wide tuning range is proposed. A 25 mm (roughly equal to λH/8, where λH corresponds to the highest frequency of the tuning range) open slot is etched at the edge of the ground. To achieve the tunability, only two lumped elements, namely, a PIN diode and a varactor diode are used in the structure. By switching the PIN diode placed at the open end of the slot, the slot antenna can resonate as a standard slot (when the switch is on) or a half slot (when the switch is off). Continuous tuning over a wide frequency range in those two modes can be achieved by adjusting the reverse bias (giving different capacitances) of the varactor diode loaded in the slot. Through optimal design, the tuning bands of the two modes are stitched together to form a very wide tuning range. The fabricated prototype has a tuning frequency range from 0.42 GHz to 1.48 GHz with Sll better than -10 dB, giving a frequency ratio (fR = fu/fL) of 3.52:1. The measured full spherical radiation patterns show consistent radiation characteristics of the proposed antenna within the whole tuning range.

A single-chip 65GHz Doppler radar transceiver with on-chip patch antenna is reported. Implemented in a production 0.18mum SiGe BiCMOS process, it features a differential output transmit power of 4.3dBm, 16.5dB single-ended down-conversion gain and a double-sideband noise figure of 12.8dB. The radar includes a 65GHz 2-stage cascode LNA with S 11<-15dB at 50-94GHz and 14dB gain at 65GHz, a double-balanced down-convert mixer, a SiGe HBT IF amplifier with low 1/f noise, a VCO and a 65GHz output buffer. The LO is provided by an integrated varactor-tuned 61-67GHz VCO optimized for low phase noise. The patch antenna is designed to be impedance-matched for dual-band operation at 62.8 and 64.6GHz. The use of lumped inductors in all blocks and a vertically-stacked transformer for single-ended to differential conversion in the receive path help reduce the transceiver area to 1mm times 1mm


Question: A dual-band stepped-impedance transformer for conversion to full-height substrate-integrated waveguide.

This study presents a detailed investigation into the dispersion characteristics of substrate integrated rectangular waveguides (SIRWs) utilizing the BI-RME method in conjunction with Floquet's theorem. The analysis reveals that SIRWs exhibit guided-wave properties similar to those of conventional rectangular waveguides. To facilitate practical design, empirical equations are developed from the dispersion curves to accurately predict the cutoff frequencies of the two primary dominant modes in SIRWs. The validity of the analytical findings is confirmed through the design and measurement of an SIRW guide, with experimental outcomes closely aligning with theoretical predictions, thereby substantiating the effectiveness of the proposed method.

A compact substrate-integrated waveguide (SIW) filtering power divider with high isolation is presented in this paper based on SIW triangular cavities. An isolation resistance is exploited to achieve high isolation between two output ports. The design parameters can be realized by determining proper offset positions of the feeding ports and coupling windows, which can be fixed by the synthesized Qe and Mij. By using three triangular cavities as the basic elements, a filtering power divider is designed and fabricated on a single-layer substrate with the standard printed circuit boards (PCB) process. This filtering power divider operates at the center frequency of 11.4 GHz with the bandwidths of 0.75 GHz. There are good agreements in the simulated and measured results.

We evaluate the possibility of using a finline orthomode transducer (OMT) at millimeter wavelengths. A finline OMT has low loss, low cross-polarization, and good return loss over a full waveguide band. We propose a novel finline OMT structure for millimeter-wavelengths and present results at X-band.

A novel substrate integrated waveguide (SIW) multibeam antenna based on the parabolic reflector principle is proposed and implemented at 37.5 GHz, which takes the advantages of low loss, low profile, high gain, and mass producible. A prototype of the proposed antenna with seven input ports generating a corresponding number of output beams is fabricated in a standard PCB process. The measurements are in good agreement with the simulations, and the results demonstrate that this type of printed multibeam antenna is a good choice for communications applications where mobility and high gain are simultaneously required.

A waveguide divider with folded lateral arms is presented for separating dual orthogonal linear polarizations in broadband ortho-mode transducers. The structure is based on a well-known double symmetry junction, where the metallic pins have been eliminated and the lateral outputs have been folded to achieve a combined effect: matching for the vertical polarization and a very significant size reduction. In addition, since the path for the lateral branches has been reduced, the insertion losses for the different polarizations are balanced. The isolation between orthogonal polarizations is kept because of the double-symmetry of the junction. From the mechanical point of view, the proposed junction allows a more simple manufacture and assembly of the ortho-mode transducer parts, which has been shown with a Ku-band design, covering the full Ku-band from 12.6 to 18.25 GHz. The experimental prototype has demonstrated a measured return loss better than 28 dB in the design band and insertion loss smaller than 0.15 dB for both polarizations.

This paper focuses on the analysis and the design methodology of the step-impedance phase-inverter rat-race coupler on a silicon-based process. The issues of impedance limitation and bandwidth are discussed in detail. Our proposed concept utilizes a high silicon dielectric constant, phase-inverter structure, step-impedance technique, and Chebyshev response to make the rat-race coupler more compact (~ 64% reduction) and highly balanced over a wide operating bandwidth. Moreover, the inter-digital coplanar stripline used in the step-impedance section effectively reduces the characteristic impedance of the transmission line for large size shrinkage and insertion-loss reduction. The demonstrated step-impedance rat-race coupler directly on silicon substrate has 6- ~ 7-dB insertion loss from 5 to 15 GHz and small variations in amplitude/phase balance. Compared with our previous work, the proposed rat-race coupler achieves a 3-dB improvement in the insertion loss. Thus, a 0.13-μm CMOS Gilbert down-converter with a miniature phase-inverter rat-race coupler at the RF path for wideband single-to-differential signal conversion achieves a noise figure of 16 dB.

Novel balanced-to-unbalanced (balun) bandpass filters are presented and carefully examined. By properly converting a symmetric four-port balanced-to-balanced bandpass filter to a three-port device, with two ports forming a balanced output and one port remaining unbalanced, a novel three-port balun bandpass filter with both functions of balun and bandpass filter may be realized. Specifically, a balun bandpass filter is implemented with its center frequency at 0.99 GHz and a 3-dB fractional bandwidth (Bomega) of 10.1%. The implemented balun bandpass filter presents an excellent in-band balance performance with common-mode rejection ratio (Rcm) better than 41 dB (the corresponding amplitude and phase imbalances are within 0.003deg and 1deg) over the pass- band. In this study, a wideband balun bandpass filter is also implemented so as to improve the wideband performance of the quasi- Yagi antenna. Being integrated with the proposed wideband balun bandpass filter, the implemented quasi-Yagi antenna achieves a bandwidth of 48% for IS11 | < -10 dB, with front-to-back ratio better than 13 dB, cross-polarization smaller than -20 dB, and peak gain of 3-5 dBi within the wide operating bandwidth.

This paper presents the design and measured performance of an X-band to Ka-band SiGe HBT SPDT switch. The proposed SPDT switch was fabricated using a 200 nm, 150 GHz peak fT silicon-germanium (SiGe) heterojunction bipolar transistor (HBT) BiCMOS technology. The SPDT switch design uses diode-connected SiGe HBTs in a series-shunt configuration to improve the switch bandwidth and isolation. Between 8 and 40 GHz, this SPDT switch achieves an insertion loss of less than 4.3 dB, an isolation of more than 20.3 dB, and a return loss of more than 9 dB.

A fully integrated technique for wideband cancellation of transmitter (TX) self-interference (SI) in the RF domain is proposed for multiband frequency-division duplexing (FDD) and full-duplex (FD) wireless applications. Integrated wideband SI cancellation (SIC) in the RF domain is accomplished through: 1) a bank of tunable, reconfigurable second-order high-Q RF bandpass filters in the canceller that emulate the antenna interface's isolation (essentially frequency-domain equalization in the RF domain) and 2) a linear N-path Gm-C filter implementation with embedded variable attenuation and phase shifting. A 0.8-1.4 GHz receiver (RX) with the proposed wideband SIC circuits is implemented in a 65 nm CMOS process. In measurement, >20 MHz 20 dB cancellation bandwidth (BW) is achieved across frequency-selective antenna interfaces: 1) a custom-designed LTElike 0.780/0.895 GHz duplexer with TX/RX isolation peak magnitude of 30 dB, peak group delay of 11 ns, and 7 dB magnitude variation across the TX band for FDD and 2) a 1.4 GHz antenna pair for FD wireless with TX/RX isolation peak magnitude of 32 dB, peak group delay of 9 ns, and 3 dB magnitude variation over 1.36-1.38 GHz. For FDD, SIC enhances the effective outof-band (OOB) IIP3 and IIP2 to +25-27 dBm and +90 dBm, respectively (enhancements of 8-10 and 29 dB, respectively). For FD, SIC eliminates RX gain compression for as high as -8 dBm of peak in-band (IB) SI, and enhances effective IB IIP3 and IIP2 by 22 and 58 dB.

The substrate integrated waveguide (SIW) allows the construct of some types of planar antennas which cannot be conventionally integrated on the substrate. However, owing to some technology restrictions, designed SIW horn antennas normally work in the frequency above 10 GHz. This paper proposes a 6.8GHz low-profile H-plane horn antenna based on the ridged SIW, which allows the substrates thinner than λ0/10. Far field radiation patterns are reported to reveal the good performance of proposed antenna. Comparisons among the ridged SIW horn antennas with different number of ridges are also made to shown the matching improvements.

Novel compact tri-band bandpass filter (BPF) with controllable bandwidths is presented in this letter. The tri-band performance is realized by using three sets of resonators, i.e., two sets of short-stub loaded dual-mode resonators, which are designed for the first and second passband, respectively, and half-wavelength resonators for the third passband. The three passband frequencies and bandwidths could be independently tuned and designed. The half-wavelength resonators are embedded between the two dual-mode resonators and serve as part of feed lines for the first and second passband, thus compact size is realized. The newly designed tri-band BPF is verified by circuit implementation and good agreement between the simulated and measured results can be observed.