Question: Visual language modeling utilizing CNN image representations.

This paper introduces an innovative learning algorithm for Deep Boltzmann Machines (DBMs) featuring multiple layers of hidden variables. To estimate data-dependent expectations, a variational approximation method is employed, which tends to concentrate on a single mode. Conversely, data-independent expectations are approximated using persistent Markov chains. The distinct methodologies used for estimating the two types of expectations that constitute the gradient of the log-likelihood enable efficient learning in DBMs, even those with extensive hidden layers and millions of parameters. To further optimize the learning process, a layer-by-layer "pre-training" phase is utilized. This phase facilitates variational inference with just a single bottom-up pass, streamlining the computation. Demonstrating the efficacy of this approach, we present experimental outcomes on the MNIST and NORB datasets. The results highlight that DBMs are adept at learning robust generative models and perform exceptionally well in the tasks of handwritten digit and visual object recognition.

Cross-view retrieval, which focuses on searching images as response to text queries or vice versa, has received increasing attention recently. Crossview hashing is to efficiently solve the cross-view retrieval problem with binary hash codes. Most existing works on cross-view hashing exploit multiview embedding method to tackle this problem, which inevitably causes the information loss in both image and text domains. Inspired by the Generative Adversarial Nets (GANs), this paper presents a new model that is able to Turn Crossview Hashing into single-view hashing (TUCH), thus enabling the information of image to be preserved as much as possible. TUCH is a novel deep architecture that integrates a language model network T for text feature extraction, a generator network G to generate fake images from text feature and a hashing network H for learning hashing functions to generate compact binary codes. Our architecture effectively unifies joint generative adversarial learning and cross-view hashing. Extensive empirical evidence shows that our TUCH approach achieves state-of-the-art results, especially on text to image retrieval, based on imagesentences datasets, i.e. standard IAPRTC-12 and large-scale Microsoft COCO.

Visual attributes are great means of describing images or scenes, in a way both humans and computers understand. In order to establish a correspondence between images and to be able to compare the strength of each property between images, relative attributes were introduced. However, since their introduction, hand-crafted and engineered features were used to learn increasingly complex models for the problem of relative attributes. This limits the applicability of those methods for more realistic cases. We introduce a deep neural network architecture for the task of relative attribute prediction. A convolutional neural network (ConvNet) is adopted to learn the features by including an additional layer (ranking layer) that learns to rank the images based on these features. We adopt an appropriate ranking loss to train the whole network in an end-to-end fashion. Our proposed method outperforms the baseline and state-of-the-art methods in relative attribute prediction on various coarse and fine-grained datasets. Our qualitative results along with the visualization of the saliency maps show that the network is able to learn effective features for each specific attribute. Source code of the proposed method is available at https://github.com/yassersouri/ghiaseddin.

Obtaining effective mid-level representations has become an increasingly important task in computer vision. In this paper, we propose a fully automatic algorithm which harvests visual concepts from a large number of Internet images (more than a quarter of a million) using text-based queries. Existing approaches to visual concept learning from Internet images either rely on strong supervision with detailed manual annotations or learn image-level classifiers only. Here, we take the advantage of having massive well organized Google and Bing image data, visual concepts (around 14, 000) are automatically exploited from images using word-based queries. Using the learned visual concepts, we show state-of-the-art performances on a variety of benchmark datasets, which demonstrate the effectiveness of the learned mid-level representations: being able to generalize well to general natural images. Our method shows significant improvement over the competing systems in image classification, including those with strong supervision.

Recent advances in deep learning and distributed representations of images and text have resulted in the emergence of several neural architectures for cross-modal retrieval tasks, such as searching collections of images in response to textual queries and assigning textual descriptions to images. However, the multi-modal retrieval scenario, when a query can be either a text or an image and the goal is to retrieve both a textual fragment and an image, which should be considered as an atomic unit, has been significantly less studied. In this paper, we propose a gated neural architecture to project image and keyword queries as well as multi-modal retrieval units into the same low-dimensional embedding space and perform semantic matching in this space. The proposed architecture is trained to minimize structured hinge loss and can be applied to both cross- and multi-modal retrieval. Experimental results for six different cross- and multi-modal retrieval tasks obtained on publicly available datasets indicate superior retrieval accuracy of the proposed architecture in comparison to the state-of-art baselines.

One characteristic that sets humans apart from modern learning-based computer vision algorithms is the ability to acquire knowledge about the world and use that knowledge to reason about the visual world. Humans can learn about the characteristics of objects and the relationships that occur between them to learn a large variety of visual concepts, often with few examples. This paper investigates the use of structured prior knowledge in the form of knowledge graphs and shows that using this knowledge improves performance on image classification. We build on recent work on end-to-end learning on graphs, introducing the Graph Search Neural Network as a way of efficiently incorporating large knowledge graphs into a vision classification pipeline. We show in a number of experiments that our method outperforms standard neural network baselines for multi-label classification.

Deep learning has achieved impressive results in many areas of Computer Vision and Natural Language Processing. Among others, Visual Question Answering (VQA), also referred to a visual Turing test, is considered one of the most compelling problems, and recent deep learning models have reported significant progress in vision and language modeling. Although Artificial Intelligence (AI) is getting closer to passing the visual Turing test, at the same time the existence of adversarial examples to deep learning systems may hinder the practical application of such systems. In this work, we conduct the first extensive study on adversarial examples for VQA systems. In particular, we focus on generating targeted adversarial examples for a VQA system while the target is considered to be a question-answer pair. Our evaluation shows that the success rate of whether a targeted adversarial example can be generated is mostly dependent on the choice of the target question-answer pair, and less on the choice of images to which the question refers. We also report the language prior phenomenon of a VQA model, which can explain why targeted adversarial examples are hard to generate for some question-answer targets. We also demonstrate that a compositional VQA architecture is slightly more resilient to adversarial attacks than a non-compositional one. Our study sheds new light on how to build deep vision and language resilient models robust against adversarial examples.

To what extent is the success of deep visualization due to the training? Could we do deep visualization using untrained, random weight networks? To address this issue, we explore new and powerful generative models for three popular deep visualization tasks using untrained, random weight convolutional neural networks. First we invert representations in feature spaces and reconstruct images from white noise inputs. The reconstruction quality is statistically higher than that of the same method applied on well trained networks with the same architecture. Next we synthesize textures using scaled correlations of representations in multiple layers and our results are almost indistinguishable with the original natural texture and the synthesized textures based on the trained network. Third, by recasting the content of an image in the style of various artworks, we create artistic images with high perceptual quality, highly competitive to the prior work of Gatys et al. on pretrained networks. To our knowledge this is the first demonstration of image representations using untrained deep neural networks. Our work provides a new and fascinating tool to study the representation of deep network architecture and sheds light on new understandings on deep visualization.

Deep learning yields great results across many fields, from speech recognition, image classification, to translation. But for each problem, getting a deep model to work well involves research into the architecture and a long period of tuning. We present a single model that yields good results on a number of problems spanning multiple domains. In particular, this single model is trained concurrently on ImageNet, multiple translation tasks, image captioning (COCO dataset), a speech recognition corpus, and an English parsing task. Our model architecture incorporates building blocks from multiple domains. It contains convolutional layers, an attention mechanism, and sparsely-gated layers. Each of these computational blocks is crucial for a subset of the tasks we train on. Interestingly, even if a block is not crucial for a task, we observe that adding it never hurts performance and in most cases improves it on all tasks. We also show that tasks with less data benefit largely from joint training with other tasks, while performance on large tasks degrades only slightly if at all.

What if we could effectively read the mind and transfer human visual capabilities to computer vision methods? In this paper, we aim at addressing this question by developing the first visual object classifier driven by human brain signals. In particular, we employ EEG data evoked by visual object stimuli combined with Recurrent Neural Networks (RNN) to learn a discriminative brain activity manifold of visual categories in a reading the mind effort. Afterward, we transfer the learned capabilities to machines by training a Convolutional Neural Network (CNN)–based regressor to project images onto the learned manifold, thus allowing machines to employ human brain–based features for automated visual classification. We use a 128-channel EEG with active electrodes to record brain activity of several subjects while looking at images of 40 ImageNet object classes. The proposed RNN-based approach for discriminating object classes using brain signals reaches an average accuracy of about 83%, which greatly outperforms existing methods attempting to learn EEG visual object representations. As for automated object categorization, our human brain–driven approach obtains competitive performance, comparable to those achieved by powerful CNN models and it is also able to generalize over different visual datasets.

Extremely low-quality images, on the order of 20 pixels in width, appear with frustrating frequency in many forensic investigations. Even advanced de-noising and super-resolution technologies are unable to extract useful information from such lowquality images. We show, however, that useful information is present in such highly degraded images. We also show that convolutional neural networks can be trained to decipher the contents of highly degraded images of license plates, and that these networks significantly outperform human observers. Introduction Recognizing text in images is a well-studied problem [1]. Text recognition can either be done by recognizing individual characters or by recognizing the full word. For degraded images, however, it is difficult to localize and recognize individual characters in an image. Word recognition, therefore, has become central to text recognition in degraded images. Deep convolutional neural networks [2] have been used for text recognition in natural images. Goodfellow et al. [3] used a deep neural network to localize, segment and recognize multiple digits on street view images. Jaderberg et al. [4] also proposed an end to end text recognition system for natural images using a deep neural network. Jaderberg et al. used a large word dictionary and formulated the text recognition task as a large scale classification problem. Recently, Svoboda et al. [5] used CNN to remove motion blur from images of license plate blurred with a blur kernel of various directions and lengths. Although this approach is able to deblur highly blurred images, it does not contend with extremely low resolution and noisy images. Unlike much of this previous work we focus on extracting text from highly degraded images on the scale of only a few pixels per character. Hsieh et al. [6] were the first to show that information can be extracted from highly degraded license plates. In this work the authors assume a known font type, font size, and character layout, and assume that the degraded image is blurry and perspectively distorted, but does not necessarily contain additive noise. Although the authors only show results on a small set of images, they do show that information is present in license plates as small as 20 pixels in width. Building on these ideas, in this paper we propose to train a CNN for recognizing highly degraded license plates with an unknown background template, font type, size and character location. We also explicitly work in the presence of high amounts of additive noise – a common occurrence in real-world imagery. Recognition by Human Observers We begin by performing a perceptual study to determine how well human observers can decipher degraded license plates. This study provides a baseline against which to compare our computational approaches. Observers were shown images of synthetically Figure 1. An example of the type of degraded license plate that we seek to


Question: Visual language modeling using CNN image representations.

Introducing CAT2000: A Comprehensive Eye Tracking Dataset for Advancing Saliency Detection Research

Saliency detection has been a thriving field within computer vision for nearly two decades, with existing models demonstrating remarkable accuracy in predicting human gaze patterns in natural scenes. However, there is a concern that these models might have become overly specialized to small, potentially biased datasets, thereby hindering further advancements. To address this issue and gain a clearer understanding of the current limitations in saliency modeling, we have compiled a large-scale fixation dataset by recording the eye movements of 120 participants as they freely viewed a diverse collection of images. Our dataset, named CAT2000, encompasses 4000 images, with 200 images from each of 20 distinct categories. These categories include a wide range of scenes such as Cartoons, Art, Objects, Low resolution images, Indoor and Outdoor scenes, Jumbled images, Random images, and Line drawings.

We have conducted a preliminary analysis of the dataset to uncover some fundamental characteristics and have also evaluated the performance of several prominent saliency models. Our findings suggest that CAT2000 presents novel challenges for the development of the next generation of saliency models and provides a valuable resource for conducting behavioral studies on bottom-up visual attention. We anticipate that this dataset will play a crucial role in fostering innovation and progress in saliency detection research.

In this paper, we use the framework of neural machine translation to learn joint sentence representations across six very different languages. Our aim is that a representation which is independent of the language, is likely to capture the underlying semantics. We define a new crosslingual similarity measure, compare up to 1.4M sentence representations and study the characteristics of close sentences. We provide experimental evidence that sentences that are close in embedding space are indeed semantically highly related, but often have quite different structure and syntax. These relations also hold when comparing sentences in different languages.

Traditional feature encoding scheme (e.g., Fisher vector) with local descriptors (e.g., SIFT) and recent convolutional neural networks (CNNs) are two classes of successful methods for image recognition. In this paper, we propose a hybrid representation, which leverages the discriminative capacity of CNNs and the simplicity of descriptor encoding schema for image recognition, with a focus on scene recognition. To this end, we make three main contributions from the following aspects. First, we propose a patch-level and end-to-end architecture to model the appearance of local patches, called PatchNet. PatchNet is essentially a customized network trained in a weakly supervised manner, which uses the image-level supervision to guide the patch-level feature extraction. Second, we present a hybrid visual representation, called VSAD, by utilizing the robust feature representations of PatchNet to describe local patches and exploiting the semantic probabilities of PatchNet to aggregate these local patches into a global representation. Third, based on the proposed VSAD representation, we propose a new state-of-the-art scene recognition approach, which achieves an excellent performance on two standard benchmarks: MIT Indoor67 (86.2%) and SUN397 (73.0%).

Visual object tracking is challenging as target objects often undergo significant appearance changes caused by deformation, abrupt motion, background clutter and occlusion. In this paper, we exploit features extracted from deep convolutional neural networks trained on object recognition datasets to improve tracking accuracy and robustness. The outputs of the last convolutional layers encode the semantic information of targets and such representations are robust to significant appearance variations. However, their spatial resolution is too coarse to precisely localize targets. In contrast, earlier convolutional layers provide more precise localization but are less invariant to appearance changes. We interpret the hierarchies of convolutional layers as a nonlinear counterpart of an image pyramid representation and exploit these multiple levels of abstraction for visual tracking. Specifically, we adaptively learn correlation filters on each convolutional layer to encode the target appearance. We hierarchically infer the maximum response of each layer to locate targets. Extensive experimental results on a largescale benchmark dataset show that the proposed algorithm performs favorably against state-of-the-art methods.

Current methods for training convolutional neural networks depend on large amounts of labeled samples for supervised training. In this paper we present an approach for training a convolutional neural network using only unlabeled data. We train the network to discriminate between a set of surrogate classes. Each surrogate class is formed by applying a variety of transformations to a randomly sampled ’seed’ image patch. We find that this simple feature learning algorithm is surprisingly successful when applied to visual object recognition. The feature representation learned by our algorithm achieves classification results matching or outperforming the current state-of-the-art for unsupervised learning on several popular datasets (STL-10, CIFAR-10, Caltech-101).

Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in visual domain. This paper aims at generating plausible natural language sentences, whose attributes are dynamically controlled by learning disentangled latent representations with designated semantics. We propose a new neural generative model which combines variational auto-encoders and holistic attribute discriminators for effective imposition of semantic structures. With differentiable approximation to discrete text samples, explicit constraints on independent attribute controls, and efficient collaborative learning of generator and discriminators, our model learns highly interpretable representations from even only word annotations, and produces realistic sentences with desired attributes. Quantitative evaluation validates the accuracy of sentence and attribute generation.

We study probabilistic models of natural images and extend the autoregressive family of PixelCNN models by incorporating latent variables. Subsequently, we describe two new generative image models that exploit different image transformations as latent variables: a quantized grayscale view of the image or a multi-resolution image pyramid. The proposed models tackle two known shortcomings of existing PixelCNN models: 1) their tendency to focus on low-level image details, while largely ignoring high-level image information, such as object shapes, and 2) their computationally costly procedure for image sampling. We experimentally demonstrate benefits of our LatentPixelCNN models, in particular showing that they produce much more realistically looking image samples than previous stateof-the-art probabilistic models.

We present a system to automatically generate natural language descriptions from images. This system consists of two parts. The first part, content planning, smooths the output of computer vision-based detection and recognition algorithms with statistics mined from large pools of visually descriptive text to determine the best content words to use to describe an image. The second step, surface realization, chooses words to construct natural language sentences based on the predicted content and general statistics from natural language. We present multiple approaches for the surface realization step and evaluate each using automatic measures of similarity to human generated reference descriptions. We also collect forced choice human evaluations between descriptions from the proposed generation system and descriptions from competing approaches. The proposed system is very effective at producing relevant sentences for images. It also generates descriptions that are notably more true to the specific image content than previous work.

In this work we propose a new CNN+LSTM architecture for camera pose regression for indoor and outdoor scenes. CNNs allow us to learn suitable feature representations for localization that are robust against motion blur and illumination changes. We make use of LSTM units on the CNN output in spatial coordinates in order to capture contextual information. This substantially enlarges the receptive field of each pixel leading to drastic improvements in localization performance. We provide extensive quantitative comparison of CNN-based vs SIFT-based localization methods, showing the weaknesses and strengths of each. Furthermore, we present a new large-scale indoor dataset with accurate ground truth from a laser scanner. Experimental results on both indoor and outdoor public datasets show our method outperforms existing deep architectures, and can localize images in hard conditions, e.g., in the presence of mostly textureless surfaces.

Most attention-based image captioning models attend to the image once per word. However, attending once per word is rigid and is easy to miss some information. Attending more times can adjust the attention position, find the missing information back and avoid generating the wrong word. In this paper, we show that attending more times per word can gain improvements in the image captioning task, without increasing the number of parameters. We propose a flexible two-LSTM merge model to make it convenient to encode more attentions than words. Our captioning model uses two LSTMs to encode the word sequence and the attention sequence respectively. The information of the two LSTMs and the image feature are combined to predict the next word. Experiments on the MSCOCO caption dataset show that our method outperforms the state-of-the-art. Using bottom up features and self-critical training method, our method gets BLEU-4, METEOR, ROUGE-L, CIDEr and SPICE scores of 0.381, 0.283, 0.580, 1.261 and 0.220 on the Karpathy test split.

We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods, which have tackled this problem in a deterministic or non-parametric way, we propose to model future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. To synthesize realistic movement of objects, we propose a novel network structure, namely a Cross Convolutional Network; this network encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, as well as on real-world video frames. We also show that our model can be applied to visual analogy-making, and present an analysis of the learned network representations.


Question: A visual language modeling technique applied to CNN image representations.

Caffe: A Framework for Efficient Deep Learning on Visual Data

Caffe offers a streamlined and adaptable environment for multimedia researchers and practitioners to develop and deploy cutting-edge deep learning models. This BSD-licensed C++ library, with Python and MATLAB interfaces, is designed for the efficient training and deployment of convolutional neural networks and other complex models on standard computing architectures. Tailored to meet the demands of the industry and the internet-scale multimedia processing, Caffe harnesses the power of CUDA GPU computation, achieving processing rates of over 40 million images per day on a single K40 or Titan GPU, with an impressive speed of approximately 2 milliseconds per image. By decoupling model representation from implementation, Caffe facilitates experimentation and ensures seamless transition across platforms, making it ideal for both development and deployment, from initial prototyping to cloud-based environments.

Maintained and developed by the Berkeley Vision and Learning Center (BVLC) with active contributions from the GitHub community, Caffe is a cornerstone for ongoing research, large-scale industrial applications, and innovative start-up projects in the domains of computer vision, speech recognition, and multimedia analysis.

We propose an approach for exploiting contextual information in semantic image segmentation, and particularly investigate the use of patch-patch context and patch-background context in deep CNNs. We formulate deep structured models by combining CNNs and Conditional Random Fields (CRFs) for learning the patch-patch context between image regions. Specifically, we formulate CNN-based pairwise potential functions to capture semantic correlations between neighboring patches. Efficient piecewise training of the proposed deep structured model is then applied in order to avoid repeated expensive CRF inference during the course of back propagation. For capturing the patch-background context, we show that a network design with traditional multi-scale image inputs and sliding pyramid pooling is very effective for improving performance. We perform comprehensive evaluation of the proposed method. We achieve new state-of-the-art performance on a number of challenging semantic segmentation datasets.

We study probabilistic models of natural images and extend the autoregressive family of PixelCNN models by incorporating latent variables. Subsequently, we describe two new generative image models that exploit different image transformations as latent variables: a quantized grayscale view of the image or a multi-resolution image pyramid. The proposed models tackle two known shortcomings of existing PixelCNN models: 1) their tendency to focus on low-level image details, while largely ignoring high-level image information, such as object shapes, and 2) their computationally costly procedure for image sampling. We experimentally demonstrate benefits of our LatentPixelCNN models, in particular showing that they produce much more realistically looking image samples than previous stateof-the-art probabilistic models.

We present a system to automatically generate natural language descriptions from images. This system consists of two parts. The first part, content planning, smooths the output of computer vision-based detection and recognition algorithms with statistics mined from large pools of visually descriptive text to determine the best content words to use to describe an image. The second step, surface realization, chooses words to construct natural language sentences based on the predicted content and general statistics from natural language. We present multiple approaches for the surface realization step and evaluate each using automatic measures of similarity to human generated reference descriptions. We also collect forced choice human evaluations between descriptions from the proposed generation system and descriptions from competing approaches. The proposed system is very effective at producing relevant sentences for images. It also generates descriptions that are notably more true to the specific image content than previous work.

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

Text classification to a hierarchical taxonomy of topics is a common and practical problem. Traditional approaches simply use bag-of-words and have achieved good results. However, when there are a lot of labels with different topical granularities, bagof-words representation may not be enough. Deep learning models have been proven to be effective to automatically learn different levels of representations for image data. It is interesting to study what is the best way to represent texts. In this paper, we propose a graph-CNN based deep learning model to first convert texts to graph-of-words, and then use graph convolution operations to convolve the word graph. Graph-ofwords representation of texts has the advantage of capturing non-consecutive and long-distance semantics. CNN models have the advantage of learning different level of semantics. To further leverage the hierarchy of labels, we regularize the deep architecture with the dependency among labels. Our results on both RCV1 and NYTimes datasets show that we can significantly improve large-scale hierarchical text classification over traditional hierarchical text classification and existing deep models.

While natural languages are compositional, how state-of-the-art neural models achieve compositionality is still unclear. We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior. That is, while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words. In contrast, the higher layers compose meaningful phrases and clauses, whose lengths increase as the networks get deeper until fully composing the sentence.

In this paper, we propose a Layer-RNN (L-RNN) module that is able to learn contextual information adaptively using within-layer recurrence. Our contributions are three-fold: (i) we propose a hybrid neural network architecture that interleaves traditional convolutional layers with L-RNN module for learning longrange dependencies at multiple levels; (ii) we show that a L-RNN module can be seamlessly inserted into any convolutional layer of a pre-trained CNN, and the entire network then fine-tuned, leading to a boost in performance; (iii) we report experiments on the CIFAR-10 classification task, showing that a network with interleaved convolutional layers and L-RNN modules, achieves comparable results (5.39% top1 error) using only 15 layers and fewer parameters to ResNet-164 (5.46%); and on the PASCAL VOC2012 semantic segmentation task, we show that the performance of a pre-trained FCN network can be boosted by 5% (mean IOU) by simply inserting Layer-RNNs.

Automatically describing an image is an important capability for virtual assistants. Significant progress has been achieved in recent years on this task of image captioning. However, classical prediction techniques based on maximum likelihood trained LSTM nets don’t embrace the inherent ambiguity of image captioning. To address this concern, recent variational auto-encoder and generative adversarial network based methods produce a set of captions by sampling from an abstract latent space. But, this latent space has limited interpretability and therefore, a control mechanism for captioning remains an open problem. This paper proposes a captioning technique conditioned on part-of-speech. Our method provides human interpretable control in form of part-of-speech. Importantly, part-of-speech is a language prior, and conditioning on it provides: (i) more diversity as evaluated by counting n-grams and the novel sentences generated, (ii) achieves high accuracy for the diverse captions on standard captioning metrics.

We propose a new approach for general object tracking with fully convolutional neural network. Instead of treating convolutional neural network (CNN) as a black-box feature extractor, we conduct in-depth study on the properties of CNN features offline pre-trained on massive image data and classification task on ImageNet. The discoveries motivate the design of our tracking system. It is found that convolutional layers in different levels characterize the target from different perspectives. A top layer encodes more semantic features and serves as a category detector, while a lower layer carries more discriminative information and can better separate the target from distracters with similar appearance. Both layers are jointly used with a switch mechanism during tracking. It is also found that for a tracking target, only a subset of neurons are relevant. A feature map selection method is developed to remove noisy and irrelevant feature maps, which can reduce computation redundancy and improve tracking accuracy. Extensive evaluation on the widely used tracking benchmark [36] shows that the proposed tacker outperforms the state-of-the-art significantly.

The advent of social media has brought Internet memes, a unique social phenomenon, to the front stage of the Web. Embodied in the form of images with text descriptions, little do we know about the “language of memes”. In this paper, we statistically study the correlations among popular memes and their wordings, and generate meme descriptions from raw images. To do this, we take a multimodal approach—we propose a robust nonparanormal model to learn the stochastic dependencies among the image, the candidate descriptions, and the popular votes. In experiments, we show that combining text and vision helps identifying popular meme descriptions; that our nonparanormal model is able to learn dense and continuous vision features jointly with sparse and discrete text features in a principled manner, outperforming various competitive baselines; that our system can generate meme descriptions using a simple pipeline.


Question: Visual language modeling utilizing CNN image representations.

This paper delves into the pivotal roles of initialization and momentum in the training of deep and recurrent neural networks (DNNs and RNNs). Historically, these complex models were deemed nearly untrainable using stochastic gradient descent (SGD) with momentum. We present evidence that, when SGD with momentum is initialized with a well-thought-out random scheme and employs a specific, gradually ascending schedule for the momentum parameter, it can achieve comparable performance levels to those obtained using the more sophisticated Hessian-Free optimization technique. This is particularly notable in the context of RNNs dealing with long-term dependencies in data. Our findings underscore the critical nature of both initialization and momentum: poorly initialized networks prove resistant to momentum-based training, while well-initialized networks exhibit significant performance degradation in the absence of momentum or when it is not finely tuned. This suggests that previous failures in training deep and recurrent networks from random initializations may have been attributable to inadequate initialization strategies. Moreover, our results indicate that, when momentum is carefully calibrated, it can effectively mitigate curvature challenges in the training objectives of deep and recurrent networks, rendering the use of complex second-order methods unnecessary.

A major challenge in computer vision is scaling activity understanding to the long tail of complex activities without requiring collecting large quantities of data for new actions. The task of video retrieval using natural language descriptions seeks to address this through rich, unconstrained supervision about complex activities. However, while this formulation offers hope of leveraging underlying compositional structure in activity descriptions, existing approaches typically do not explicitly model compositional reasoning. In this work, we introduce an approach for explicitly and dynamically reasoning about compositional natural language descriptions of activity in videos. We take a modular neural network approach that, given a natural language query, extracts the semantic structure to assemble a compositional neural network layout and corresponding network modules. We show that this approach is able to achieve state-of-the-art results on the DiDeMo video retrieval dataset.

Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual architecture that operates on a saliency map and uses a novel attention mechanism to sequentially focus on salient regions and take additional glimpses within those regions. The architecture is motivated by human visual attention, and is used for multi-label image classification on a novel multiset task, demonstrating that it achieves high precision and recall while localizing objects with its attention. Unlike conventional multi-label image classification models, the model supports multiset prediction due to a reinforcement-learning based training process that allows for arbitrary label permutation and multiple instances per label.

Building bilateral semantic associations between images and texts is among the fundamental problems in computer vision. In this paper, we study two complementary cross-modal prediction tasks: (i) predicting text(s) given a query image (“Im2Text”), and (ii) predicting image(s) given a piece of text (“Text2Im”). We make no assumption on the specific form of text; i.e., it could be either a set of labels, phrases, or even captions. We pose both these tasks in a retrieval framework. For Im2Text, given a query image, our goal is to retrieve a ranked list of semantically relevant texts from an independent text-corpus (i.e., texts with no corresponding images). Similarly, for Text2Im, given a query text, we aim to retrieve a ranked list of semantically relevant images from a collection of unannotated images (i.e., images without any associated textual meta-data). We propose a novel Structural SVM based unified framework for these two tasks, and show how it can be efficiently trained and tested. Using a variety of loss functions, extensive experiments are conducted on three popular datasets (two medium-scale datasets containing few thousands of samples, and one web-scale dataset containing one million samples). Experiments demonstrate that our framework gives promising results compared to competing baseline cross-modal search techniques, thus confirming its efficacy.

Here we present DeepGaze II, a model that predicts where people look in images. The model uses the features from the VGG-19 deep neural network trained to identify objects in images. Contrary to other saliency models that use deep features, here we use the VGG features for saliency prediction with no additional fine-tuning (rather, a few readout layers are trained on top of the VGG features to predict saliency). The model is therefore a strong test of transfer learning. After conservative crossvalidation, DeepGaze II explains about 87% of the explainable information gain in the patterns of fixations and achieves top performance in area under the curve metrics on the MIT300 hold-out benchmark. These results corroborate the finding from DeepGaze I (which explained 56% of the explainable information gain), that deep features trained on object recognition provide a versatile feature space for performing related visual tasks. We explore the factors that contribute to this success and present several informative image examples. A web service is available to compute model predictions at http://deepgaze.bethgelab.org.

Text classification to a hierarchical taxonomy of topics is a common and practical problem. Traditional approaches simply use bag-of-words and have achieved good results. However, when there are a lot of labels with different topical granularities, bagof-words representation may not be enough. Deep learning models have been proven to be effective to automatically learn different levels of representations for image data. It is interesting to study what is the best way to represent texts. In this paper, we propose a graph-CNN based deep learning model to first convert texts to graph-of-words, and then use graph convolution operations to convolve the word graph. Graph-ofwords representation of texts has the advantage of capturing non-consecutive and long-distance semantics. CNN models have the advantage of learning different level of semantics. To further leverage the hierarchy of labels, we regularize the deep architecture with the dependency among labels. Our results on both RCV1 and NYTimes datasets show that we can significantly improve large-scale hierarchical text classification over traditional hierarchical text classification and existing deep models.

Visual attributes are great means of describing images or scenes, in a way both humans and computers understand. In order to establish a correspondence between images and to be able to compare the strength of each property between images, relative attributes were introduced. However, since their introduction, hand-crafted and engineered features were used to learn increasingly complex models for the problem of relative attributes. This limits the applicability of those methods for more realistic cases. We introduce a deep neural network architecture for the task of relative attribute prediction. A convolutional neural network (ConvNet) is adopted to learn the features by including an additional layer (ranking layer) that learns to rank the images based on these features. We adopt an appropriate ranking loss to train the whole network in an end-to-end fashion. Our proposed method outperforms the baseline and state-of-the-art methods in relative attribute prediction on various coarse and fine-grained datasets. Our qualitative results along with the visualization of the saliency maps show that the network is able to learn effective features for each specific attribute. Source code of the proposed method is available at https://github.com/yassersouri/ghiaseddin.

In this report, the results from my CS224D final project are given and explained. The project was based on the application of relatively new neural network architectures, namely convolutional neural networks over word embeddings, to the task of authorship identification. The problem was posed as a classification task and models were evaluated over two datasets, a baseline of my own collection and a competition dataset with pre-existing, published results. Despite robust results on the baseline dataset, the CNN architecture failed to outmatch the best competition submission, although it did outscore most of the competitors’ submissions.

This work introduces a model that can recognize objects in images even if no training data is available for the objects. The only necessary knowledge about the unseen categories comes from unsupervised large text corpora. In our zero-shot framework distributional information in language can be seen as spanning a semantic basis for understanding what objects look like. Most previous zero-shot learning models can only differentiate between unseen classes. In contrast, our model can both obtain state of the art performance on classes that have thousands of training images and obtain reasonable performance on unseen classes. This is achieved by first using outlier detection in the semantic space and then two separate recognition models. Furthermore, our model does not require any manually defined semantic features for either words or images.

We present an integrated framework for using Convolutional Networks for classification , localization and detection. We show how a multiscale and sliding window approach can be efficiently implemented within a ConvNet. We also introduce a novel deep learning approach to localization by learning to predict object boundaries. Bounding boxes are then accumulated rather than suppressed in order to increase detection confidence. We show that different tasks can be learned simultaneously using a single shared network. This integrated framework is the winner of the localization task of the ImageNet Large Scale Visual Recognition Challenge 2013 (ILSVRC2013) and obtained very competitive results for the detection and classifications tasks. In post-competition work, we establish a new state of the art for the detection task. Finally, we release a feature extractor from our best model called OverFeat.

The current trend in object detection and localization is to learn predictions with high capacity deep neural networks trained on a very large amount of annotated data and using a high amount of processing power. In this work, we propose a new neural model which directly predicts bounding box coordinates. The particularity of our contribution lies in the local computations of predictions with a new form of local parameter sharing which keeps the overall amount of trainable parameters low. Key components of the model are spatial 2D-LSTM recurrent layers which convey contextual information between the regions of the image. We show that this model is more powerful than the state of the art in applications where training data is not as abundant as in the classical configuration of natural images and Imagenet/Pascal VOC tasks. We particularly target the detection of text in document images, but our method is not limited to this setting. The proposed model also facilitates the detection of many objects in a single image and can deal with inputs of variable sizes without resizing.


Question: A 122 GHz radar sensor utilizing a monostatic SiGe-BiCMOS integrated circuit with an antenna integrated on-chip.

A 77 GHz Radar Transceiver Featuring Dual On-Chip Antenna Elements

This paper describes a comprehensive radar transceiver operating at 77 GHz, which integrates two antenna elements directly onto the chip. Building upon a previously documented design [1], the transceiver's transmit and receive channels have been enhanced with the addition of these on-chip antenna elements. The antennas demonstrate a clear and defined radiation pattern, achieving an efficiency exceeding 50%. This integration of antenna elements onto the transceiver chip not only streamlines the design but also ensures improved performance and compactness, making it suitable for various high-frequency radar applications.

This paper presents a 77-GHz orthogonal frequency-division multiplexing (OFDM) multiple-input multiple-output (MIMO) radar system with 4 GHz bandwidth. Due to the limited bandwidth of the analog-to-digital and digital-to-analog converters a stepped-carrier approach is used. OFDM radar measurements with a bandwidth of 200MHz at 20 different carrier frequencies are combined in signal processing to achieve a high range resolution OFDM MIMO radar. Spectrally-interleaved OFDM signals are used to allow for simultaneous transmission and thus MIMO operation. The proposed approach is verified in practical measurements of a static target scenario and a slowly moving target scenario. Therefore a self-developed software defined radar evaluation platform with four transmitters and four receivers is used. The expected high range resolution could be achieved and range/angle/velocity measurements show the suitability for detecting slowly moving targets like pedestrians.

In this study, a wireless bio-radar sensor was designed to detect a human heartbeat and respiration signals without direct skin contact. In order to design a wireless bio-radar sensor quantitatively, the signal-to-noise ratio (SNR) in the baseband output of a sensor should be calculated. Therefore, we analyzed the SNR of the wireless bio-radar sensor, considering the signal power attenuation in a human body and all kinds of noise sources. Especially, we measured a residual phase noise of a typical free-running oscillator and used its value for the SNR analysis. Based on these analysis and the measurement results, a compact, low-cost 2.4 GHz direct conversion bio-radar sensor was designed and implemented in a printed circuit board. The demonstrated sensor consists of two printed antennas, a

In this paper an innovative antenna system for car2car applications is described. The antenna system is based on an active antenna module containing the antennas as well as the complete car2car chip hardware and software. This module is very small, so that it can be integrated e.g. into a shark fin antenna on the cars rooftop. Moreover additional modules can be mounted somewhere else in the car and connected with the main module via Ethernet. This allows high flexibility and re-configurability of the system. Two test cars have been equipped with the new car2car demonstrator systems and measurements have been performed in the 3D measurement chamber as well as in dynamic outdoor scenarios. The new system shows excellent performance in measurements and field trials.

Millimeter-wave technology brings a new paradigm of wireless communication in various areas including mobile devices, automotive, IoT (Internet of Things), medical, military and many others. In this paper, we present our recently developed antenna in a package and system for 5G (Fifth-Generation) millimeter-wave applications. The antenna is configured to be an array to compensate path loss and maintain data link budget. The antenna is designed with a multi-layer laminate substrate and is integrated with a 5G mmW RFIC. 5G NR (New Radio) wireless connection is demonstrated with a RFIC transceiver at 28 GHz band. Interoperability test is conducted to ensure compatibility with the 3GPP standard. Finally, indoor and outdoor field test including beamforming and beamtracking are presented to show realistic system-level throughput performance.

Automotive radar systems utilizing the millimeterwave band have been developed 2) since they can detect targets even in bad weather. In the automotive radar systems, linear polarization inclined 45 degrees is utilized in order to avoid mutual radio interference between them. The antenna for the automotive radar systems is required to have high efficiency, which means high gain against for an aperture area determined by the size of a radar sensor, in order to detect the targets at a long distance. In addition, low profile is required not to spoil the appearance of a car. Ease of manufacturing is also one of the significant factors in order to install the automotive radar systems in popular cars. As candidate antennas for the automotive radar systems, a slotted waveguide array antenna and a triplate-type array antenna have been studied. These antennas have low profile and high efficiency 7

In this paper, a half mode SIW based semi-circular antenna with co-axial feed has been designed for dual band operation. The fundamental resonating frequency is found to operate at 4.66 GHz while the next higher mode is at 8.3 GHz. The antenna performance is optimized through parametric studies and the dual band antenna is designed using Arlon AD270 substrate. Simulation studies have been carried out to study the various antenna parameters.

We report a 39 μm × 27 μm on-chip temperature sensor which uses the temperature-dependent reverse-bias leakage current of a lateral silicon on insulator (SOI) CMOS p-n diode to monitor the thermal profile of a 32-nm microprocessor core. In this sensor, the diode junction capacitance is first charged to a fixed voltage. Subsequently, the diode capacitance is allowed to self-discharge through its temperature-dependent reverse-bias current. Next, by using a time-to-digital-converter circuit, the discharge voltage is converted to a temperature-dependent time pulse, and finally, its width is measured by using a digital counter. This compact temperature sensor demonstrates a 3σ measurement inaccuracy of ±1.95°C across the 5°C-100°C temperature range while consuming only 100 μW from a single 1.65-V supply.

In this letter, a novel dual-band and polarization-flexible substrate integrated waveguide (SIW) cavity antenna is proposed. The SIW cavity used for the antenna is excited by a conventional TE120 mode for its first resonance. With the intervention of the slot, a second resonance excited by a modified- TE120 mode is also generated, thereby providing a broadside radiation pattern at two resonant frequencies. In addition, the proposed antenna has two orthogonal feeding lines. Therefore, it is possible to provide any six major polarization states. In this letter, three major polarization cases are simulated and compared to measured results. Since modern communication systems require multifunctional antennas, the proposed antenna concept is a promising candidate.

This paper describes the design of a dual-channel S-band digital radar test bed. The test bed combines stretch processing with a novel and cost-effective hardware architecture that enables it to achieve an in-band dynamic range of 60 dB over 600 MHz of instantaneous bandwidth. The dual digital receiver channels allow for adaptive digital beamforming which can be used to mitigate a directional source of interference. Experimental test and verification results are presented to demonstrate system performance.

A new miniaturized antenna for 433MHz frequency band is introduced for sensor networks. The antenna is a magnetic monopole loaded with a capacitor, which can significantly decrease the resonant frequency of the structure. It is designed to operate in the ISM 433MHz band on a small PCB (103mm⋆55mm) and has at least 3MHz bandwidth. Moreover, its radiation efficiency is suitable for the possible applications and its pattern is omnidirectionnal.


Question: A 122 GHz radar sensor utilizing a monostatic SiGe-BiCMOS IC with an integrated on-chip antenna.

This document describes the development of a focal-plane array (FPA) designed for room-temperature detection of 0.65 terahertz (THz) radiation, fully integrated using a low-cost 0.25-micron CMOS process technology. The design principle is grounded in distributed resistive self-mixing, enabling broadband direct detection that transcends the technology's cutoff frequency. The FPA, composed of a 3x5 pixel array, features differential on-chip patch antennas, NMOS direct detectors, and integrated voltage amplifiers with 43 dB of gain. At 0.65 THz, the array attains a responsivity (Rv) of 80 kV/W and a noise equivalent power (NEP) of 300 pW/√Hz. The FPA's potential for cost-effective terahertz imaging solutions is showcased through active multi-pixel imaging of postal envelopes. This achievement highlights the array's capability for future terahertz imaging applications, emphasizing its potential for widespread adoption due to its cost-effectiveness and performance.

The radar capable to detect and perform diagnostics of a state of the human being behind obstacles and in conditions of bad visibility could be used for the manifold applications. The potential consumers of the radar are: rescue services, antiterrorist detachments and law-enforcement bodies. It is possible also to use this device in medicine. In this paper, the experiments with using of continuouswave subsurface radar are described. The recorded oscillograms and their frequency spectrums for heartbeat, respiration and articulation of a man, which is taking place behind an obstacle (wall), are presented.

Results for cellular antenna switches using highresistivity silicon-on-insulator (SOI) CMOS technology are presented. The performance of SOI RF switch FETs is presented and compared to a production GaAs pHEMT technology. Data from prototype high-resistivity SOI RF switch designs is presented and compared to pHEMT based designs. Index Terms — RF CMOS, SOI, silicon-on-insulator, RF SOI, cellular switch, signal isolation

A systematic approach to designing high frequency and high power oscillators using activity condition is introduced. This method finds the best topology to achieve frequencies close to the fmax of the transistors. It also determines the maximum frequency of oscillation for a fixed circuit topology, considering the quality factor of the passive components. Using this technique, in a 0.13 μm CMOS process, we design and implement 121 GHz and 104 GHz fundamental oscillators with the output power of -3.5 dBm and -2.7 dBm, respectively. Next, we introduce a novel triple-push structure to realize 256 GHz and 482 GHz oscillators. The 256 GHz oscillator was implemented in a 0.13 μm CMOS process and the output power of -17 dBm was measured. The 482 GHz oscillator generates -7.9 dBm (0.16 mW) in a 65 nm CMOS process.

In this paper, a noncontact proximity vital sign sensor, using a phase locked loop (PLL) incorporated with voltage controlled oscillator (VCO) built-in planar type circular resonator, is proposed to enhance sensitivity in severe environments. The planar type circular resonator acts as a series feedback element of the VCO as well as a near-field receiving antenna. The frequency deviation of the VCO related to the body proximity effect ranges from 0.07 MHz/mm to 1.8 MHz/mm (6.8 mV/mm to 205 mV/mm in sensitivity) up to a distance of 50 mm, while the amount of VCO drift is about 21 MHz in the condition of 60 °C temperature range and discrete component tolerance of ±5%. Total frequency variation occurs in the capture range of the PLL which is 60 MHz. Thus, its loop control voltage converts the amount of frequency deviation into a difference of direct current (DC) voltage, which is utilized to extract vital signs regardless of the ambient temperature. The experimental results reveal that the proposed sensor placed 50 mm away from a subject can reliably detect respiration and heartbeat signals without the ambiguity of harmonic signals caused by respiration signal at an operating frequency of 2.4 GHz.

In automotive radar applications, a realistic road surface can cause multipath reflection, degrade the received power, and reduce detection probability. This paper proposes a one-transmitter–two-receiver (1T2R) 24-GHz frequency-modulation continuous-wave (FMCW) radar architecture with spatial diversity and ground effect compensation for automotive applications. A ray-tracing technique was developed to evaluate the multipath effect of the road surface. In the proposed design, suitable spacing between the two receiving antennas compensated for the multipath effect of the road surface and improved the signal-to-ground clutter ratio of the receivers by a minimum of 20 dB at certain distances. The measurement and calculation results of the proposed radar are in close agreement.

The radar signatures of automotive targets were measured from 22 to 29 GHz and 76 to 81 GHz inside an anechoic chamber using a vector network analyzer. Radar cross section maps of a sedan, truck, van, pedestrian, motorcycle, and bicycle as a function of effective radar sensor bandwidth, center frequency, and polarization are presented.

Using radars for detecting and tracking pedestrians has important safety and security applications. Most existing human detection radar systems operate in UHF, X-band, and 24GHz bands. The newly allocated 76-77 GHz frequency band for automobile collision avoidance radar systems has once again raised great interest in pedestrian detection and tracking at these frequencies due to its longer detection range and better location accuracy. The electromagnetic scattering properties must be thoroughly understood and analyzed so a catalog of human scattering can be utilized for intensive automotive radar testing. Measurements of real human subjects at these frequencies is not a trivial task. This paper presents validation between the measured radar cross section (RCS) patterns of various human subjects and a full-wave EM simulation. The RCS of a human is shown to depend on a number of factors, such as posture, body shape, clothing type, etc.

CMOS transmit-receive (T/R) switches have been integrated in a 0.18-/spl mu/m standard CMOS technology for wireless applications at 2.4 and 5.2 GHz. This switch design achieves low loss and high linearity by increasing the substrate impedance of a MOSFET at the frequency of operation using a properly tuned LC tank. The switch design is asymmetric to accommodate the different linearity and isolation requirements in the transmit and receive modes. In the transmit mode, the switch exhibits 1.5-dB insertion loss, 28-dBm power, 1-dB compression point (P/sub 1dB/), and 30-dB isolation, at 2.4 and 5.2 GHz. In the receive mode, the switch achieves 1.6-dB insertion loss, 11.5-dBm P/sub 1dB/, and 15-dB isolation, at 2.4 and 5.2 GHz. The linearity obtained in the transmit mode is the highest reported to date in a standard CMOS process. The switch passes the 4-kV Human Body Model electrostatic discharge test. These results show that the switch design is suitable for narrow-band applications requiring a moderate-high transmitter power level (<1 W).

Commercial automotive radars used today are based on frequency modulated continuous wave signals due to the simple and robust detection method and good accuracy. However, the increase in both the number of radars deployed per vehicle and the number of such vehicles leads to mutual interference, cutting short future plans for autonomous driving and active safety functionality. We propose and analyze a radar communication (RadCom) approach to reduce this mutual interference while simultaneously offering communication functionality. We achieve this by frequency division multiplexing radar and communication, where communication is built on a decentralized carrier sense multiple access protocol and is used to adjust the timing of radar transmissions. Our simulation results indicate that radar interference can be significantly reduced, at no cost in radar accuracy.

For the successful design of low-cost and high-performance radar systems accurate and efficient system simulation is a key requirement. In this paper we present a new versatile simulation environment for frequency-modulated continuous-wave radar systems. Besides common hardware simulation it covers integrated system simulation and concept analysis from signal synthesis to baseband. It includes a flexible scenario generator, accurate noise modeling, and efficiently delivers simulation data for development and testing of signal processing algorithms. A comparison of simulations and measurement results for an integrated 77-GHz radar prototype shows the capabilities of the simulator on two different scenarios.


Question: A monostatic 122 GHz radar sensor utilizing a SiGe-BiCMOS IC with an integrated on-chip antenna.

This paper explores the evolution and future prospects of GHz UWB (Ultra-Wideband) automotive short range radar (SSR) systems, which are increasingly pivotal in modern vehicle comfort and safety enhancements. With a focus on SiGe-based 79 GHz UWB SSR, a promising successor to the 24 GHz UWB SSR, the paper summarizes the outcomes of the completed BMBF joint project KOKON and the ongoing RoCC project. Both initiatives are dedicated to advancing this technology and developing practical sensor demonstrators. Daimler AG's role in these projects involves defining application-specific sensor specifications, as well as testing and evaluating the prototypes. The paper also delves into the latest regulatory trends concerning UWB SRR frequencies. Additionally, it highlights preliminary findings from Daimler's work within RoCC, particularly addressing the challenges and solutions associated with packaging and operating these sensors in the intricate automotive environment.

In this paper an innovative antenna system for car2car applications is described. The antenna system is based on an active antenna module containing the antennas as well as the complete car2car chip hardware and software. This module is very small, so that it can be integrated e.g. into a shark fin antenna on the cars rooftop. Moreover additional modules can be mounted somewhere else in the car and connected with the main module via Ethernet. This allows high flexibility and re-configurability of the system. Two test cars have been equipped with the new car2car demonstrator systems and measurements have been performed in the 3D measurement chamber as well as in dynamic outdoor scenarios. The new system shows excellent performance in measurements and field trials.

A wideband 57.7–84.2 GHz Phase Shifter is presented using a compact Lange coupler to generate in-phase and quadrature signal. The Lange coupler is followed by two balun transformers that provide the IQ vector modulation with differential I and Q signals. The implemented Phase Shifter demonstrates an average 6-dB insertion loss and 5-dB gain variation. The measured average rms phase and gain errors are 7 degrees and 1 dB, respectively. The phase shifter is implemented in GlobalFoundries 45-nm SOI CMOS technology using a trap-rich substrate. The chip area is 385 μm × 285 μm and the Phase Shifter consumes less than 17 mW. To the best of authors knowledge, this is the first phase shifter that covers both 60 GHz band and E-band frequencies with a fractional bandwidth of 37%.

A small and lightweight frequency-modulated continuous-wave (FMCW) radar system is used for the determination of snow height by measuring the distance to the snow surface from a platform. The measurements have been performed at the Centre des Études de la Neige (Col de Porte), which is located near Grenoble in the French Alps. It is shown that the FMCW radar at millimeter-wave frequencies is an extremely promising approach for distance measurements to snow surfaces, e.g., in the mountains or in an Arctic environment. The characteristics of the radar sensor are described in detail. The relevant accuracy to measure the distance to a snow layer is shown at different heights and over an extended time duration. A dedicated laser snow telemeter is used as reference. In addition, the reflection from different types of snow is shown.

A first reported experimental study of a 60 GHz millimeter-wave life detection system (MLDS) for noncontact human vital-signal monitoring is presented. This detection system is constructed by using V-band millimeter-wave waveguide components. A clutter canceller is implemented in the system with an adjustable attenuator and phase shifter. It performs clutter cancellation for the transmitting power leakage from the circulator and background reflection to enhance the detecting sensitivity of weak vital signals. The noncontact vital signal measurements have been conducted on a human subject in four different physical orientations from distances of 1 and 2 m. The time-domain and spectrum waveforms of the measured breathing and heartbeat are presented. This prototype system will be useful for the development of the 60-GHz CMOS MLDS detector chip design.

This paper presents a noninvasive wireless sensor platform for continuous health monitoring. The sensor system integrates a loop antenna, wireless sensor interface chip, and glucose sensor on a polymer substrate. The IC consists of power management, readout circuitry, wireless communication interface, LED driver, and energy storage capacitors in a 0.36-mm2 CMOS chip with no external components. The sensitivity of our glucose sensor is 0.18 μA·mm-2·mM-1. The system is wirelessly powered and achieves a measured glucose range of 0.05-1 mM with a sensitivity of 400 Hz/mM while consuming 3 μW from a regulated 1.2-V supply.

A compact 24-GHz Yagi-Uda antenna has been developed using standard design tables and simple scaling to take into account the added capacitance due to the supporting dielectric substrate. The antenna results in a directivity of 9.3 dB, a front-to-back ratio of 11 dB, and a bandwidth of 2.5-3%. The Yagi-Uda antenna has been implemented in an 11-beam system using a planar array and a 2-inch Teflon spherical lens. The measured patterns show a 22 dB gain beam, a cross-polarization level of -24 dB, and a crossover level of -6 dB. The design method presented in this paper is quite straightforward, and can be used to develop low-, medium-, and even high-gain endfire Yagi-Uda antennas.

A novel frequency-reconfigurable antenna is presented based on a circular monopolar patch antenna. The antenna comprises of a center-fed circular patch surrounded by four sector-shaped patches. Eight varactor diodes are introduced to bridge the gaps between the circular patch and the sector-shaped patches. By changing the reverse bias voltage of the varactor diodes, the antenna can be switched in the operating frequency. A fully functional prototype is developed and tested, which exhibits a continuously tunable frequency band from 1.64 GHz to 2.12 GHz. The measured efficiency rises from about 45% to about 85% as the operating frequency increases from 1.64 GHz to 2.12 GHz. Stable monopole like radiation patterns are achieved at all operating frequencies. In addition, the antenna owns a low-profile structure (0.04 free-space wavelengths at 1.9 GHz). The frequency selective feature and stable radiation patterns make the antenna potentially suitable for future wireless communication systems, such as cognitive radio.

We present a stacked-FET monolithic millimeter-wave (mmW) integrated circuit Doherty power amplifier (DPA). The DPA employs a novel asymmetrical stack gate bias to achieve high power and high efficiency at 6-dB power back-off (PBO). The circuit is fabricated in a 0.15-µm enhancement mode (E-mode) Gallium Arsenide (GaAs) process. Experimental results demonstrate output power at 1-dB gain compression (P1dB) of 28.2 dBm, peak power added efficiency (PAE) of 37% and PAE at 6-dB PBO of 27% at 28 GHz. Measured small signal gain is 15 dB while the 3-dB bandwidth covers from 25.5 to 29.5 GHz. Using digital predistortion (DPD) with a 20 MHz 64 QAM modulated signal, an adjacent channel power ratio (ACPR) of −46 dBc has been observed.

In this work the realization of a 77-GHz frequency-division multiple-access-based frequency-modulated continuous-wave multiple-input multiple-output radar with four transceiver channels in conjunction with a non-uniform linear antenna array is presented. The radar system, consisting of an RF-frontend utilizes transceiver chips with integrated inphase/quadrature-modulators to generate the frequency shifted transmit signals and a field programmable gate array-based baseband board, is used for test measurements to verify the system performance and to demonstrate the beamforming capability as well as the accuracy of the digital-beamforming method.

A compact, low cost and high radiation efficiency antenna structure, planar waveguide, substrate integrated waveguide (SIW), dielectric resonator antennas (DRA) is presented in this paper. Since SIW is a high Q- waveguide and DRA is a low loss radiator, then SIW-DRA forms an excellent antenna system with high radiation efficiency at millimeter-waveband, where the conductor loss dominates. The impact of different antenna parameters on the antenna performance is studied. Experimental data for SIW-DRA, based on two different slot orientations, at millimeter-wave band are introduced and compared to the simulated HFSS results to validate our proposed antenna model. A good agreement is obtained. The measured gain for SIW-DRA single element showed a broadside gain of 5.51 dB,-19 dB maximum cross polarized radiation level, and overall calculated (simulated using HFSS) radiation efficiency of greater than 95%.


Question: Problems, Difficulties, and Techniques in Clustering Algorithm Applications

In "Discovering Patterns in Data: An Overview of Cluster Analysis," the authors provide a comprehensive guide tailored for individuals without a deep mathematical background, aiming to make the subject more accessible to a broader audience. This book is lauded for its clarity, readability, and engaging presentation of various clustering techniques. It introduces innovative applications that have added significant value to the clustering literature. The text is praised for its practicality and ease of understanding, making it a top recommendation for students and professionals across diverse fields. This guide to cluster analysis focuses on the practical application of a selection of robust and widely applicable methods. It covers different types of data, such as interval-scaled and binary variables, as well as similarity data, detailing how these can be preprocessed to enhance clustering effectiveness. By addressing the challenges and tools associated with clustering algorithms, this resource equips readers with the knowledge to effectively analyze and group data in their respective fields.

This paper studies cluster ensembles for high dimensional data clustering. We examine three different approaches to constructing cluster ensembles. To address high dimensionality, we focus on ensemble construction methods that build on two popular dimension reduction techniques, random projection and principal component analysis (PCA). We present evidence showing that ensembles generated by random projection perform better than those by PCA and further that this can be attributed to the capability of random projection to produce diverse base clusterings. We also examine four different consensus functions for combining the clusterings of the ensemble. We compare their performance using two types of ensembles, each with different properties. In both cases, we show that a recent consensus function based on bipartite graph partitioning achieves the best performance.

To cluster increasingly massive data sets that are common today in data and text mining, we propose a parallel implementation of the k-means clustering algorithm based on the message passing model. The proposed algorithm exploits the inherent data-parallelism in the kmeans algorithm. We analytically show that the speedup and the scaleup of our algorithm approach the optimal as the number of data points increases. We implemented our algorithm on an IBM POWERparallel SP2 with a maximum of 16 nodes. On typical test data sets, we observe nearly linear relative speedups, for example, 15.62 on 16 nodes, and essentially linear scaleup in the size of the data set and in the number of clusters desired. For a 2 gigabyte test data set, our implementation drives the 16 node SP2 at more than 1.8 gigaflops.

Temporal clustering refers to the partitioning of a time series into multiple nonoverlapping segments that belong to k temporal clusters, in such a way that segments in the same cluster are more similar to each other than to those in other clusters. Temporal clustering is a fundamental task in many fields, such as computer animation, computer vision, health care, and robotics. The applications of temporal clustering in those areas are diverse, and include human-motion imitation and recognition, emotion analysis, human activity segmentation, automated rehabilitation exercise analysis, and human-computer interaction. However, temporal clustering using a completely unsupervised method may not produce satisfactory results. Similar to regular clustering, temporal clustering also benefits from some expert knowledge that may be available. The type of approach that utilizes a small amount of knowledge to “guide” the clustering process is known as “semi-supervised clustering.” Semi-supervised temporal clustering is a strategy in which extra knowledge, in the form of pairwise constraints, is incorporated into the temporal data to help with the partitioning problem. This thesis proposes a process to adapt and transform two kernel-based methods into semi-supervised temporal clustering methods. The proposed process is exclusive to kernel-based clustering methods, and is based on two concepts. First, it uses the idea of instance-level constraints, in the form of must-link and cannot-link, to supervise the clustering methods. Second, it uses a dynamic-programming method to search for the optimal temporal clusters. The proposed process is applied to two algorithms, aligned cluster analysis (ACA) and spectral clustering. To validate the advantages of the proposed temporal semi-supervised clustering methods, a comparative analysis was performed, using the original versions of the algorithm and another semi-supervised temporal cluster. This evaluation was conducted with both synthetic data and two real-world applications. The first application includes two naturalistic audio-visual human emotion datasets, and the second application focuses on human-motion segmentation. Results show substantial improvements in accuracy, with minimal supervision, compared to unsupervised and other temporal semi-supervised approaches, without compromising time performance.

The development of data-mining applications such as classification and clustering has shown the need for machine learning algorithms to be applied to large scale data. In this paper we present the comparison of different classification techniques using Waikato Environment for Knowledge Analysis or in short, WEKA. WEKA is an open source software which consists of a collection of machine learning algorithms for data mining tasks. The aim of this paper is to investigate the performance of different classification or clustering methods for a set of large data. The algorithm or methods tested are Bayes Network, Radial Basis Function, Pruned Tree, Single Conjunctive Rule Learner and Nearest Neighbors Algorithm. A fundamental review on the selected technique is presented for introduction purposes. The data breast cancer data with a total data of 6291 and a dimension of 699 rows and 9 columns will be used to test and justify the differences between the classification methods or algorithms. Subsequently, the classification technique that has the potential to significantly improve the common or conventional methods will be suggested for use in large scale data, bioinformatics or other general applications. Keywords— Machine Learning, Data Mining, WEKA, Classification, Bioinformatics.

Quite a large number of data mining techniques employed in financial fraud detection (FFD) are seen to be classification techniques. In this paper, we developed an algorithm to find the features of classification techniques (or method) that so much place it (classification techniques) in the heart of researchers in their various efforts in the study of insurance frauds detection. We also got to know the characteristics of insurance frauds data that made data mining classification techniques so much attracted to it (insurance data).

Knowledge Discovery and Data Mining are powerful automated data analysis tools and they are predicted to become the most frequently used analytical tools in the near future. The rapid dissemination of these technologies calls for an urgent examination of their social impact. This paper identifies social issues arising from Knowledge Discovery (KD) and Data Mining (DM). An overview of these technologies is presented, followed by a detailed discussion of each issue. The paper's intention is to primarily illustrate the cultural context of each issue and, secondly, to describe the impact of KD and DM in each case. Existing solutions specific to each issue are identified and examined for feasibility and effectiveness, and a solution that provides a suitably contextually sensitive means for gathering and analysing sensitive data is proposed and briefly outlined. The paper concludes with a discussion of topics for further consideration.

Knowledge discovery in databases is a rapidly growing field, whose development is driven by strong research interests as well as urgent practical, social, and economical needs. While the last few years knowledge discovery tools have been used mainly in research environments, sophisticated software products are now rapidly emerging. In this paper, we provide an overview of common knowledge discovery tasks and approaches to solve these tasks. We propose a feature classification scheme that can be used to study knowledge and data mining software. This scheme is based on the software's general characteristics, database connectivity, and data mining characteristics. We then apply our feature classification scheme to investigate 43 software products, which are either research prototypes or commercially available. Finally, we specify features that we consider important for knowledge discovery software to possess in order to accommodate its users effectively, as well as issues that are either not addressed or insufficiently solved yet.

Most data mining algorithms require the setting of many input parameters. Two main dangers of working with parameter-laden algorithms are the following. First, incorrect settings may cause an algorithm to fail in finding the true patterns. Second, a perhaps more insidious problem is that the algorithm may report spurious patterns that do not really exist, or greatly overestimate the significance of the reported patterns. This is especially likely when the user fails to understand the role of parameters in the data mining process.Data mining algorithms should have as few parameters as possible, ideally none. A parameter-free algorithm would limit our ability to impose our prejudices, expectations, and presumptions on the problem at hand, and would let the data itself speak to us. In this work, we show that recent results in bioinformatics and computational theory hold great promise for a parameter-free data-mining paradigm. The results are motivated by observations in Kolmogorov complexity theory. However, as a practical matter, they can be implemented using any off-the-shelf compression algorithm with the addition of just a dozen or so lines of code. We will show that this approach is competitive or superior to the state-of-the-art approaches in anomaly/interestingness detection, classification, and clustering with empirical tests on time series/DNA/text/video datasets.

We propose in this correspondence a new method to perform two-class clustering of 2-D data in a quick and automatic way by preserving certain features of the input data. The method is analytical, deterministic, unsupervised, automatic, and noniterative. The computation time is of order n if the data size is n, and hence much faster than any other method which requires the computation of an n-by-n dissimilarity matrix. Furthermore, the proposed method does not have the trouble of guessing initial values. This new approach is thus more suitable for fast automatic hierarchical clustering or any other fields requiring fast automatic two-class clustering of 2-D data. The method can be extended to cluster data in higher dimensional space. A 3-D example is included.

Name disambiguation can occur when one is seeking a list of publications of an author who has used different name variations and when there are multiple other authors with the same name. We present an efficient integrative framework for solving the name disambiguation problem: a blocking method retrieves candidate classes of authors with similar names and a clustering method, DBSCAN, clusters papers by author. The distance metric between papers used in DBSCAN is calculated by an online active selection support vector machine algorithm (LASVM), yielding a simpler model, lower test errors and faster prediction time than a standard SVM. We prove that by recasting transitivity as density reachability in DBSCAN, transitivity is guaranteed for core points. For evaluation, we manually annotated 3,355 papers yielding 490 authors and achieved 90.6% pairwise-F1. For scalability, authors in the entire CiteSeer dataset, over 700,000 papers, were readily disambiguated.