Question: Optimization techniques for colorization processes.

A Unified Perspective on Segmentation via Eigenvector Analysis
The challenge of automatically grouping and segmenting images in computer vision persists, with recent advancements showcasing promising results through eigenvector-based methods applied to affinity matrices. These techniques are appealing due to their reliance on eigendecomposition algorithms, which are known for their stability. Despite this, the application of eigendecompositions in segmentation contexts is not fully understood. This paper aims to consolidate the understanding of these algorithms, elucidating their interconnections while emphasizing their distinct characteristics. We delve into the performance analysis of these techniques in straightforward grouping scenarios through the examination of eigenvectors of block matrices. Our findings inspire the development of a novel approach that amalgamates elements from various eigenvector segmentation methodologies. We validate our analytical insights through demonstrations on both real-world and synthetic images. The human capacity to effortlessly segment a scene into coherent parts has been a pursuit in computer vision, with extensive efforts directed towards achieving comparable performance. This is typically achieved by assigning feature vectors, such as color, motion, texture, or position, to each pixel and applying clustering or grouping algorithms to these vectors. A particularly refined method for segmenting points in feature space is grounded in mixture models, where the assumption is made that data originates from multiple sources, and the goal is to estimate the parameters and the number of components in the mixture. The classification of points into clusters is then a straightforward process, involving the calculation of the posterior probability of a point's cluster membership. However, the elegance of this approach is often overshadowed by the complexity of the estimation process, which can result in a challenging optimization problem. The widely employed EM algorithm [3] is prone to converging to a local maximum, the selection of which can be influenced by initial conditions.

Images features extraction is very important for the grading process of flue-cured tobacco leaves. In this paper, a machine vision techniques base system is proposed for the automatic inspection of flue-cured tobacco leaves. Machine vision techniques are used in this system to solve problems of features extraction and analysis of tobacco leaves, which include features of color, size, shape andsurface texture. The experimental results show that this system is a viable way for the features extraction of tobacco leaves, and can be used for the automatic classification of tobacco leaves.

Image processing has been proved to be an effective tool for analysis in various human activity areas, namely, agricultural applications. Interpreting a digital color image of fruit orchard captured in field environment is extremely challenging due to adverse weather conditions, luminance variability and the presence of dust, insects and other unavoidable image noises. The purpose of this survey is to categorize and briefly review the literature on computer analysis of fruit images in agricultural operations, which comprises more than 60 papers published in the last 10 years. With the aim to perform applied research in agricultural imaging, this paper intends to focus on advanced image processing and analysis techniques used in applications for detection and classifications of fruits, developed in the last decade. For the reviewed techniques, some performance evaluation metrics achieved in various experiments are emphasized to help the researchers when making choices and develop new computer vision applications in fruit images.

Color constancy is the skill by which it is possible to tell the color of an object even under a colored light. I interpret the color of an object as its color under a fixed canonical light, rather than as a surface reflectance function. This leads to an analysis that shows two distinct sets of circumstances under which color constancy is possible. In this framework, color constancy requires estimating the illuminant under which the image was taken. The estimate is then used to choose one of a set of linear maps, which is applied to the image to yield a color descriptor at each point. This set of maps is computed in advance. The illuminant can be estimated using image measurements alone, because, given a number of weak assumptions detailed in the text, the color of the illuminant is constrained by the colors observed in the image. This constraint arises from the fact that surfaces can reflect no more light than is cast on them. For example, if one observes a patch that excites the red receptor strongly, the illuminant cannot have been deep blue. Two algorithms are possible using this constraint, corresponding to different assumptions about the world. The first algorithm, Crule will work for any surface reflectance. Crule corresponds to a form of coefficient rule, but obtains the coefficients by using constraints on illuminant color. The set of illuminants for which Crule will be successful depends strongly on the choice of photoreceptors: for narrowband photoreceptors, Crule will work in an unrestricted world. The second algorithm, Mwext, requires that both surface reflectances and illuminants be chosen from finite dimensional spaces; but under these restrictive conditions it can recover a large number of parameters in the illuminant, and is not an attractive model of human color constancy. Crule has been tested on real images of Mondriaans, and works well. I show results for Crule and for the Retinex algorithm of Land (Land 1971; Land 1983; Land 1985) operating on a number of real images. The experimental work shows that for good constancy, a color constancy system will need to adjust the gain of the receptors it employs in a fashion analagous to adaptation in humans.

A novel subpixel rendering algorithm for diamond-shaped PenTile displays, which improves the apparent display resolution and reduces color fringing artifacts, is proposed in this work. We develop two types of filters: main filters and nonnegative filters. First, the main filters are derived to minimize the error between an original input image and a perceived image. Second, the nonnegative filters are modified from the main filters. While the main filters are optimized for improving the resolution, the nonnegative filters suppress color distortions effectively. We analyze local characteristics of an image and estimate the amount of color distortions in each region. Based on the color distortion analysis, we render the image by applying adaptive combinations of the main filters and the nonnegative filters. Experimental results demonstrate that the proposed algorithm outperforms conventional algorithms, by not only improving the apparent resolution but also suppressing color distortions effectively.

We present a new image segmentation algorithm based on graph cuts. Our main tool is separation of each pixel p from a special point outside the image by a cut of a minimum cost. Such a cut creates a group of pixels Cp around each pixel. We show that these groups Cp are either disjoint or nested in each other, and so they give a natural segmentation of the image. In addition this property allows an efficient implementation of the algorithm because for most pixelsp the computation of Cp is not performed on the whole graph. We inspect all Cp’s and discard those which are not interesting, for example if they are too small. This procedure automatically groups small components together or merges them into nearby large clusters. Effectively our segmentation is performed by extracting significant non-intersecting closed contours. We present interes ting segmentation results on real and artificial images.

Chinese painting is distinct from other art in that the painting elements are exhibited by complex water-and-ink diffusion and shows gray, white and black visual effect. Rendering such a water-and-ink painting with polychrome style is a challenging problem. In this paper, we propose a novel style transfer method for Chinese painting. We firstly decompose the Chinese painting with adaptive patches based on its structure, and locally colorize the painting. Then, the colorized image is used for guiding the process of texture transfer that is modeled in Markov Random Field (MRF). More precisely, we improve the classic texture transfer algorithm by modifying the compatibility functions for searching the optimal matching, with the chromatism information. The experiment results show that proposed adaptive patches can well preserve the original content while match the example style. Moreover, we present the transfer results with our method and recent style transfer algorithms, in order to make a comparison.

Visual tracking is a challenging problem in computer vision. Most state-of-the-art visual trackers either rely on luminance information or use simple color representations for image description. Contrary to visual tracking, for object recognition and detection, sophisticated color features when combined with luminance have shown to provide excellent performance. Due to the complexity of the tracking problem, the desired color feature should be computationally efficient, and possess a certain amount of photometric invariance while maintaining high discriminative power. This paper investigates the contribution of color in a tracking-by-detection framework. Our results suggest that color attributes provides superior performance for visual tracking. We further propose an adaptive low-dimensional variant of color attributes. Both quantitative and attribute-based evaluations are performed on 41 challenging benchmark color sequences. The proposed approach improves the baseline intensity-based tracker by 24 % in median distance precision. Furthermore, we show that our approach outperforms state-of-the-art tracking methods while running at more than 100 frames per second.

RGB-D (Red Green Blue and Depth) sensors are devices that can provide color and depth information from a scene at the same time. Recently, they have been widely used in many solutions due to their commercial growth from the entertainment market to many diverse areas (e.g., robotics, CAD, etc.). In the research community, these devices have had good uptake due to their acceptable levelofaccuracyformanyapplicationsandtheirlowcost,butinsomecases,theyworkatthelimitof their sensitivity, near to the minimum feature size that can be perceived. For this reason, calibration processes are critical in order to increase their accuracy and enable them to meet the requirements of such kinds of applications. To the best of our knowledge, there is not a comparative study of calibration algorithms evaluating its results in multiple RGB-D sensors. Speciﬁcally, in this paper, a comparison of the three most used calibration methods have been applied to three different RGB-D sensors based on structured light and time-of-ﬂight. The comparison of methods has been carried out by a set of experiments to evaluate the accuracy of depth measurements. Additionally, an object reconstruction application has been used as example of an application for which the sensor works at the limit of its sensitivity. The obtained results of reconstruction have been evaluated through visual inspection and quantitative measurements.

Nonlinear anisotropic diffusion algorithms provide significant improvement in image enhancement as compared to linear filters. However, the excessive computational cost of solving nonlinear PDEs precludes their use in real-time vision applications. In the present paper, we show that two orders of magnitude speed improvement is provided by a new image filtering paradigm in which an adaptively determined vector field specifies nonlocal application points for an image filter.

A classical problem of imaging—the matting problem—is separation of a non-rectangular foreground image from a (usually) rectangular background image—for example, in a film frame, extraction of an actor from a background scene to allow substitu tion of a different background. Of the several attacks on this diff cult and persistent problem, we discuss here only the special ca of separating a desired foreground image from a background of constant, or almost constant, backing color. This backing color has often been blue, so the problem, and its solution, have bee called blue screen matting . However, other backing colors, such as yellow or (increasingly) green, have also been used, so we o ten generalize to constant color matting . The mathematics of constant color matting is presented and proven to be unsolvable as generally practiced. This, of course, flies in the face of the fact that the technique is commonly used in film and video, so we demonstrate constraints on the general problem that lead to so tions, or at least significantly prune the search space of solution We shall also demonstrate that an algorithmic solution is possib by allowing the foreground object to be shot against two constant backing colors—in fact, against two completely arbitrary backin so long as they differ everywhere.


Question: Optimization-based method for colorization.

This paper introduces an innovative image registration method that significantly accelerates the matching process, making it highly applicable to computer vision tasks, particularly in stereo vision applications. Conventional image registration methods are known to be computationally intensive, often requiring extensive processing to align images. Our proposed approach, however, utilizes the spatial intensity gradient of images to rapidly identify suitable matches through an optimized Newton-Raphson iteration technique. By focusing on a reduced set of potential matches, our method achieves faster processing times compared to established methodologies. Additionally, the versatility of this registration technique extends beyond simple alignment; it can be adapted to accommodate transformations such as rotation, scaling, and shearing. We demonstrate the effectiveness of our method by integrating it into a stereo vision system, showcasing its adaptability and efficiency in real-world scenarios.

Color constancy is the skill by which it is possible to tell the color of an object even under a colored light. I interpret the color of an object as its color under a fixed canonical light, rather than as a surface reflectance function. This leads to an analysis that shows two distinct sets of circumstances under which color constancy is possible. In this framework, color constancy requires estimating the illuminant under which the image was taken. The estimate is then used to choose one of a set of linear maps, which is applied to the image to yield a color descriptor at each point. This set of maps is computed in advance. The illuminant can be estimated using image measurements alone, because, given a number of weak assumptions detailed in the text, the color of the illuminant is constrained by the colors observed in the image. This constraint arises from the fact that surfaces can reflect no more light than is cast on them. For example, if one observes a patch that excites the red receptor strongly, the illuminant cannot have been deep blue. Two algorithms are possible using this constraint, corresponding to different assumptions about the world. The first algorithm, Crule will work for any surface reflectance. Crule corresponds to a form of coefficient rule, but obtains the coefficients by using constraints on illuminant color. The set of illuminants for which Crule will be successful depends strongly on the choice of photoreceptors: for narrowband photoreceptors, Crule will work in an unrestricted world. The second algorithm, Mwext, requires that both surface reflectances and illuminants be chosen from finite dimensional spaces; but under these restrictive conditions it can recover a large number of parameters in the illuminant, and is not an attractive model of human color constancy. Crule has been tested on real images of Mondriaans, and works well. I show results for Crule and for the Retinex algorithm of Land (Land 1971; Land 1983; Land 1985) operating on a number of real images. The experimental work shows that for good constancy, a color constancy system will need to adjust the gain of the receptors it employs in a fashion analagous to adaptation in humans.

Image registration is the process of establishing a common geometric reference frame between two or more image data sets possibly taken at different times. In this paper we present a method for computing elastic registration and warping maps based on the Monge–Kantorovich theory of optimal mass transport. This mass transport method has a number of important characteristics. First, it is parameter free. Moreover, it utilizes all of the grayscale data in both images, places the two images on equal footing and is symmetrical: the optimal mapping from image A to image B being the inverse of the optimal mapping from B to A. The method does not require that landmarks be specified, and the minimizer of the distance functional involved is unique; there are no other local minimizers. Finally, optimal transport naturally takes into account changes in density that result from changes in area or volume. Although the optimal transport method is certainly not appropriate for all registration and warping problems, this mass preservation property makes the Monge–Kantorovich approach quite useful for an interesting class of warping problems, as we show in this paper. Our method for finding the registration mapping is based on a partial differential equation approach to the minimization of the L 2 Kantorovich–Wasserstein or “Earth Mover's Distance” under a mass preservation constraint. We show how this approach leads to practical algorithms, and demonstrate our method with a number of examples, including those from the medical field. We also extend this method to take into account changes in intensity, and show that it is well suited for applications such as image morphing.

We present a method to replace a user specified target region of a photograph by using other photographs of the same scene downloaded from the Internet via viewpoint invariant image search. Each of the retrieved images is first geometrically then photometrically registered with the query photograph. Geometric registration is achieved using multiple homographies and photometric registration is performed using a global affine transformation on image intensities. Each retrieved image proposes a possible solution for the target region. In the final step we combine these proposals into a single visually consistent result, using a Markov random field optimisation to choose seams between proposals, followed by gradient domain fusion. We demonstrate removal of objects and people in challenging photographs of Oxford landmarks containing complex image structures.

We present a novel area-preservation mapping/flattening method using the optimal mass transport technique, based on the Monge-Brenier theory. Our optimal transport map approach is rigorous and solid in theory, efficient and parallel in computation, yet general for various applications. By comparison with the conventional Monge-Kantorovich approach, our method reduces the number of variables from O(n2) to O(n), and converts the optimal mass transport problem to a convex optimization problem, which can now be efficiently carried out by Newton's method. Furthermore, our framework includes the area weighting strategy that enables users to completely control and adjust the size of areas everywhere in an accurate and quantitative way. Our method significantly reduces the complexity of the problem, and improves the efficiency, flexibility and scalability during visualization. Our framework, by combining conformal mapping and optimal mass transport mapping, serves as a powerful tool for a broad range of applications in visualization and graphics, especially for medical imaging. We provide a variety of experimental results to demonstrate the efficiency, robustness and efficacy of our novel framework.

We propose a new competitive-learning neural network model for colour image segmentation. The model, which is based on the adaptive resonance theory (ART) of Carpenter and Grossberg and on the self-organizing map (SOM) of Kohonen, overcomes the limitations of (i) the stability–plasticity trade-offs in neural architectures that employ ART; and (ii) the lack of on-line learning property in the SOM. In order to explore the generation of a growing feature map using ART and to motivate the main contribution, we first present a preliminary experimental model, SOMART, based on Fuzzy ART. Then we propose the new model, SmART, that utilizes a novel lateral control of plasticity to resolve the stability–plasticity problem. SmART has been experimentally found to perform well in RGB colour space, and is believed to be more coherent than Fuzzy ART. q 2005 Elsevier Ltd. All rights reserved.

To see is to sketch - free-hand sketching naturally builds ties between human and machine vision. In this paper, we present a novel approach for translating an object photo to a sketch, mimicking the human sketching process. This is an extremely challenging task because the photo and sketch domains differ significantly. Furthermore, human sketches exhibit various levels of sophistication and abstraction even when depicting the same object instance in a reference photo. This means that even if photo-sketch pairs are available, they only provide weak supervision signal to learn a translation model. Compared with existing supervised approaches that solve the problem of D(E(photo)) â†’ sketch), where E(Â·) and D(Â·) denote encoder and decoder respectively, we take advantage of the inverse problem (e.g., D(E(sketch) â†’ photo), and combine with the unsupervised learning tasks of within-domain reconstruction, all within a multi-task learning framework. Compared with existing unsupervised approaches based on cycle consistency (i.e., D(E(D(E(photo)))) â†’ photo), we introduce a shortcut consistency enforced at the encoder bottleneck (e.g., D(E(photo)) â†’ photo) to exploit the additional self-supervision. Both qualitative and quantitative results show that the proposed model is superior to a number of state-of-the-art alternatives. We also show that the synthetic sketches can be used to train a better fine-grained sketch-based image retrieval (FG-SBIR) model, effectively alleviating the problem of sketch data scarcity.

Template matching is a technique widely used for finding patterns in digital images. An efficient template matching algorithm should be able to detect template instances that have undergone geometric transformations. Similarly, a color template matching should be able to deal with color constancy problem. Recently we have proposed a new grayscale template matching algorithm named Ciratefi, invariant to rotation, scale, translation, brightness and contrast. In this paper we introduce the Color-Ciratefi that takes into account the color information. We use a new similarity metric in the CIELAB space to obtain invariance to brightness and contrast changes. Experiments show that Color-Ciratefi is more accurate than C-color-SIFT, the well-known SIFT algorithm that uses a set of color invariants. In conventional computers, Color-Ciratefi is slower than C-color-SIFT. However Color-Ciratefi is more suitable than C-color-SIFT to be implemented in highly parallel architectures like FPGA, because it repeats exactly the same set of operations for each pixel.

A new method fabricates custom surface reflectance and spatially varying bidirectional reflectance distribution functions (svBRDFs). Researchers optimize a microgeometry for a range of normal distribution functions and simulate the resulting surface's effective reflectance. Using the simulation's results, they reproduce an input svBRDF's appearance by distributing the microgeometry on the printed material's surface. This method lets people print svBRDFs on planar samples with current 3D printing technology, even with a limited set of printing materials. It extends naturally to printing svBRDFs on arbitrary shapes.

Tianhe Yang Summary of " A Method for Registration of 3-D Shapes "

The accuracy of optical flow estimation algorithms has been improving steadily as evidenced by results on the Middlebury optical flow benchmark. The typical formulation, however, has changed little since the work of Horn and Schunck. We attempt to uncover what has made recent advances possible through a thorough analysis of how the objective function, the optimization method, and modern implementation practices influence accuracy. We discover that “classical” flow formulations perform surprisingly well when combined with modern optimization and implementation techniques. One key implementation detail is the median filtering of intermediate flow fields during optimization. While this improves the robustness of classical methods it actually leads to higher energy solutions, meaning that these methods are not optimizing the original objective function. To understand the principles behind this phenomenon, we derive a new objective function that formalizes the median filtering heuristic. This objective function includes a non-local smoothness term that robustly integrates flow estimates over large spatial neighborhoods. By modifying this new term to include information about flow and image boundaries we develop a method that can better preserve motion details. To take advantage of the trend towards video in wide-screen format, we further introduce an asymmetric pyramid downsampling scheme that enables the estimation of longer range horizontal motions. The methods are evaluated on the Middlebury, MPI Sintel, and KITTI datasets using the same parameter settings.


Question: Optimization-based method for colorization.

This paper introduces a groundbreaking methodology for addressing the perceptual grouping challenge in computer vision. Instead of concentrating on local features and their coherence within the image data, our approach is centered on capturing the holistic impression of the image. We frame image segmentation as a graph partitioning task and introduce a novel global criterion, the normalized cut, for effectively dividing the graph. This criterion is unique in that it evaluates both the aggregate dissimilarity among different segments and the total similarity within each segment. We demonstrate that an efficient computational technique, based on solving a generalized eigenvalue problem, can be employed to optimize this criterion. This method has been applied to the segmentation of static images, yielding results that are highly promising and encouraging.

In this paper we present a novel method to apply photometric stereo on textured dynamic surfaces. We aim at exploiting the high accuracy of photometric stereo and reconstruct local surface orientation from illumination changes. The main difficulty derives from the fact that photometric stereo requires varying illumination while the object remains still, which makes it quite impractical to use for dynamic surfaces. Using coloured lights gives a clear solution to this problem; however, the system of equations is still ill-posed and it is ambiguous whether the change of an observed surface colour is due to the change of the surface gradient or of the surface reflectance. In order to separate surface orientation from reflectance, our method tracks texture changes over time and exploits surface reflectance’s temporal constancy. This additional constraint allows us to reformulate the problem as an energy functional minimisation, solved by a standard quasi-Newton method. Our method is tested both on real and synthetic data, quantitatively evaluated and compared to a state-of-the-art method.

Color constancy is the skill by which it is possible to tell the color of an object even under a colored light. I interpret the color of an object as its color under a fixed canonical light, rather than as a surface reflectance function. This leads to an analysis that shows two distinct sets of circumstances under which color constancy is possible. In this framework, color constancy requires estimating the illuminant under which the image was taken. The estimate is then used to choose one of a set of linear maps, which is applied to the image to yield a color descriptor at each point. This set of maps is computed in advance. The illuminant can be estimated using image measurements alone, because, given a number of weak assumptions detailed in the text, the color of the illuminant is constrained by the colors observed in the image. This constraint arises from the fact that surfaces can reflect no more light than is cast on them. For example, if one observes a patch that excites the red receptor strongly, the illuminant cannot have been deep blue. Two algorithms are possible using this constraint, corresponding to different assumptions about the world. The first algorithm, Crule will work for any surface reflectance. Crule corresponds to a form of coefficient rule, but obtains the coefficients by using constraints on illuminant color. The set of illuminants for which Crule will be successful depends strongly on the choice of photoreceptors: for narrowband photoreceptors, Crule will work in an unrestricted world. The second algorithm, Mwext, requires that both surface reflectances and illuminants be chosen from finite dimensional spaces; but under these restrictive conditions it can recover a large number of parameters in the illuminant, and is not an attractive model of human color constancy. Crule has been tested on real images of Mondriaans, and works well. I show results for Crule and for the Retinex algorithm of Land (Land 1971; Land 1983; Land 1985) operating on a number of real images. The experimental work shows that for good constancy, a color constancy system will need to adjust the gain of the receptors it employs in a fashion analagous to adaptation in humans.

We present a novel CNN-based image editing method that allows the user to change the semantic information of an image over a user-specified region. Our method makes this possible by combining the idea of manifold projection with spatial conditional batch normalization (sCBN), a version of conditional batch normalization with userspecifiable spatial weight maps. With sCBN and manifold projection, our method lets the user perform (1) spatial class translation that changes the class of an object over an arbitrary region of user’s choice, and (2) semantic transplantation that transplants semantic information contained in an arbitrary region of the reference image to an arbitrary region in the target image. These two transformations can be used simultaneously, and can realize a complex composite image-editing task like “change the nose of a beagle to that of a bulldog, and open her mouth”. The user can also use our method with intuitive copy-paste-style manipulations. We demonstrate the power of our method on various images. Code will be available at https://github. com/pfnet-research/neural-collage.

Precise object segmentation in image data is a fundamental problem with various applications, including 3D object reconstruction. We present an efficient algorithm to automatically segment a static foreground object from highly cluttered background in light fields. A key insight and contribution of our article is that a significant increase of the available input data can enable the design of novel, highly efficient approaches. In particular, the central idea of our method is to exploit high spatio-angular sampling on the order of thousands of input frames, for example, captured as a hand-held video, such that new structures are revealed due to the increased coherence in the data. We first show how purely local gradient information contained in slices of such a dense light field can be combined with information about the camera trajectory to make efficient estimates of the foreground and background. These estimates are then propagated to textureless regions using edge-aware filtering in the epipolar volume. Finally, we enforce global consistency in a gathering step to derive a precise object segmentation in both 2D and 3D space, which captures fine geometric details even in very cluttered scenes. The design of each of these steps is motivated by efficiency and scalability, allowing us to handle large, real-world video datasets on a standard desktop computer. We demonstrate how the results of our method can be used for considerably improving the speed and quality of image-based 3D reconstruction algorithms, and we compare our results to state-of-the-art segmentation and multiview stereo methods.

Conventional unsupervised image segmentation methods use color and geometric information and apply clustering algorithms over pixels. They preserve object boundaries well but often suffer from over-segmentation due to noise and artifacts in the images. In this paper, we contribute on a preprocessing step for image smoothing, which alleviates the burden of conventional unsupervised image segmentation and enhance their performance. Our approach relies on a convolutional autoencoder (CAE) with the total variation loss (TVL) for unsupervised learning. We show that, after our CAE-TVL preprocessing step, the over-segmentation effect is significantly reduced using the same unsupervised image segmentation methods. We evaluate our approach using the BSDS500 image segmentation benchmark dataset and show the performance enhancement introduced by our approach in terms of both increased segmentation accuracy and reduced computation time. We examine the robustness of the trained CAE and show that it is directly applicable to other natural scene images.

Image reconstruction under multiple light scattering is crucial for a number of important applications in cell microscopy and tissue imaging. The reconstruction problem is often formulated as a nonconvex optimization, where a nonlinear measurement model is used to account for multiple scattering and a regularizer is used to enforce the prior on the object. In this letter, We propose a powerful alternative to this optimization-based view of image reconstruction by designing and training a deep convolutional neural network (CNN) for inverting multiple scattering. Simulations show that the proposed formulation is substantially faster and achieves higher imaging quality compared to the state-of-the-art methods based on optimization.

In this paper we describe a new technique for general purpose interactive segmentation of N-dimensional images . The user marks certain pixels as “object” or “background” to provide hard constraints for segmentation. Additional soft constraints incorporate both boundary and region information. Graph cuts are used to find the globally optimal segmentation of the N-dimensional image. The obtained solution gives the best balance of boundary and region properties among all segmentations satisfying the constraints . The topology of our segmentation is unrestricted and both “object” and “background” segments may consist of several isolated parts. Some experimental results are present ed in the context of photo/video editing and medical image segmentation. We also demonstrate an interesting Gestalt example. A fast implementation of our segmentation method is possible via a new max-flow algorithm in [2].

We present a novel approach for model-based 6D pose refinement in color data. Building on the established idea of contour-based pose tracking, we teach a deep neural network to predict a translational and rotational update. At the core, we propose a new visual loss that drives the pose update by aligning object contours, thus avoiding the definition of any explicit appearance model. In contrast to previous work our method is correspondence-free, segmentation-free, can handle occlusion and is agnostic to geometrical symmetry as well as visual ambiguities. Additionally, we observe a strong robustness towards rough initialization. The approach can run in real-time and produces pose accuracies that come close to 3D ICP without the need for depth data. Furthermore, our networks are trained from purely synthetic data and will be published together with the refinement code at http://campar.in.tum. de/Main/FabianManhardt to ensure reproducibility.

The accuracy of optical flow estimation algorithms has been improving steadily as evidenced by results on the Middlebury optical flow benchmark. The typical formulation, however, has changed little since the work of Horn and Schunck. We attempt to uncover what has made recent advances possible through a thorough analysis of how the objective function, the optimization method, and modern implementation practices influence accuracy. We discover that “classical” flow formulations perform surprisingly well when combined with modern optimization and implementation techniques. Moreover, we find that while median filtering of intermediate flow fields during optimization is a key to recent performance gains, it leads to higher energy solutions. To understand the principles behind this phenomenon, we derive a new objective that formalizes the median filtering heuristic. This objective includes a nonlocal term that robustly integrates flow estimates over large spatial neighborhoods. By modifying this new term to include information about flow and image boundaries we develop a method that ranks at the top of the Middlebury benchmark.

The primary problem dealt with in this paper is the following. Given some description of a visual object, find that object in an actual photograph. Part of the solution to this problem is the specification of a descriptive scheme, and a metric on which to base the decision of "goodness" of matching or detection.


Question: Optimization techniques for colorization processes.

This paper presents a computational methodology for edge detection in images. The effectiveness of this approach hinges on the establishment of a well-defined set of objectives for the computational identification of edge points. These objectives must be articulated with sufficient precision to guide the behavior of the edge detector, while minimizing assumptions about the solution's form. We outline detection and localization criteria for a broad category of edges, and mathematically formalize these criteria as functionals operating on the impulse response of the edge detector. A third criterion is introduced to ensure that the detector yields a singular response to a single edge. These criteria are then applied in a numerical optimization framework to develop detectors for various common image features, notably including step edges. Upon focusing our analysis on step edges, we uncover a fundamental trade-off between detection and localization performance, two of our primary objectives. This trade-off is encapsulated in an uncertainty principle, from which we derive a single optimal operator shape that remains effective across all scales. The optimal edge detector is characterized by a straightforward approximate implementation, where edges are identified at the peaks of the gradient magnitude of an image smoothed by a Gaussian filter. We augment this basic detector by employing operators of varying widths to accommodate different signal-to-noise ratios within the image. Furthermore, we introduce a technique called feature synthesis, designed for the integration of information across different scales, progressing from fine to coarse details. Ultimately, we demonstrate that the performance of step edge detectors is significantly enhanced as the operator's point spread function is extended along the edge, thereby improving the detector's robustness and accuracy.

The task of optimizing a complex system presents at least two levels of problems for the system designer. First, a class of optimization algorithms must be chosen that is suitable for application to the system. Second, various parameters of the optimization algorithm need to be tuned for efficiency. A class of adaptive search procedures called genetic algorithms (GA) has been used to optimize a wide variety of complex systems. GA's are applied to the second level task of identifying efficient GA's for a set of numerical optimization problems. The results are validated on an image registration problem. GA's are shown to be effective for both levels of the systems optimization problem.

Image defects which could be caused by the digitization process or by faults in the imaging set-up (for example, bad lighting) can be corrected using Image Enhancement techniques. Once the image is in good condition, the Measurement Extraction operations can be used to obtain useful information from the image. Course Objectives “Introduction to Digital Image Processing” is two-day hands-on course that provide fundamental concepts and techniques for digital image processing and the software principles used in their practical implementation.

Normal human color perception is a product of three independent sensory systems. By mirroring this mechanism, full-color display devices create colors as mixtures of three primaries. Any displayable color can be described by the corresponding values of these primaries. Frequently it is more convenient to define various other color spaces, or coordinate systems, for color representation or manipulation. Several such color spaces are presented which are suitable for applications involving user specification of color, along with the defining equations and illustrations. The use of special color spaces for particular kinds of color computations is discussed.

We consider the problem of how to simultaneously and well correct the over- and under-exposure regions in a single low dynamic range (LDR) image. Recent methods typically focus on global visual quality but cannot well-correct much potential details in extremely wrong exposure areas, and some are also time consuming. In this paper, we propose a fast and detail-enhanced correction method based on automatic fusion which combines a pair of complementarily corrected images, i.e. backlight & highlight correction images (BCI &HCI). A BCI with higher visual quality in details is quickly produced based on a proposed faster multi-scale retinex algorithm; meanwhile, a HCI is generated through contrast enhancement method. Then, an automatic fusion algorithm is proposed to create a color-protected exposure mask for fusing BCI and HCI when avoiding potential artifacts on the boundary. The experiment results show that the proposed method can fast correct over/under-exposed regions with higher detail quality than existing methods.

This paper presents a novel variational image inpainting method to solve the problem of generating, from 3-D lidar measures, a dense depth map coherent with a given color image, tackling visibility issues. When projecting the lidar point cloud onto the image plane, we generally obtain a sparse depth map, due to undersampling. Moreover, lidar and image sensor positions generally differ during acquisition, such that depth values referring to objects that are hidden from the image view point might appear with a naive projection. The proposed algorithm estimates the complete depth map, while simultaneously detecting and excluding those hidden points. It consists in a primal-dual optimization method, where a coupled total variation regularization term is included to match the depth and image gradients and a visibility indicator handles the selection of visible points. Tests with real data prove the effectiveness of the proposed strategy.

We present a method for creating black-and-white illustrations from photographs of human faces. In addition an interactive technique is demonstrated for deforming these black-and-white facial illustrations to create caricatures which highlight and exaggerate representative facial features. We evaluate the effectiveness of the resulting images through psychophysical studies to assess accuracy and speed in both recognition and learning tasks. These studies show that the facial illustrations and caricatures generated using our techniques are as effective as photographs in recognition tasks. For the learning task we find that illustrations are learned two times faster than photographs and caricatures are learned one and a half times faster than photographs. Because our techniques produce images that are effective at communicating complex information, they are useful in a number of potential applications, ranging from entertainment and education to low bandwidth telecommunications and psychology research.

It is our aim in this research to optimize the numerical computation of the Haralick texture features [1] that consists of two steps. Haralick texture features are used as a primary component to discern between different protein structures in microscopic bio-images. The first of these two parts is the construction of the co-occurrence matrix. Upon completion of this implementation, we will attempt to optimize the code using a novel recursive blocking algorithm. We will also use standard practices of optimizing code such as scalar replacement and unrolling. The second part, feature calculation, will be optimized by putting to use information about the data that we will be working with and eliminating any redundancies in the code. We will also apply the practices of unrolling and scalar replacement. With these optimizations in place, we will show that we reduce the runtime by a20% in the co-occurrence construction phase and by a factor of20 in the feature calculation phase of computation.

Absnacr-In this paper we present an overview of colour spaces used In electrical engineering and image processing. We Streis the imparfance of the perceptual, historical and sppllcananal~hackgraund that led to a colour spare. The colour spaces presented are : RGB;opponentsolours rpsces, phenomenal colaur spscer, CMY, CMYK, TV colour s~aces (UW and YIQ), PhotoYCC, CIE XYZ, Lab and Luv colour spaces. Keywordr-colaur spaces, RGB, HSV, CIE

GrabCut is an innovative 2D image segmentation technique developed by Rother et al. [2004]. This paper provides implementation details omitted from the original paper. Details covered in background papers are summarized here so that future implementors can refer to a single paper. Our implementation of GrabCut is described and results are included. Our main contribution is correcting errors in equation (9) and in equation (11) of the original paper. We also discuss weaknesses of the algorithm that were not discussed in the original paper. We present possible research directions to address these problems. CR Categories: I.4.6 [Image Processing and Computer Vision]: Segmentation—Pixel classification

We propose a new fast algorithm for solving one of the standard formulations of image restoration and reconstruction which consists of an unconstrained optimization problem where the objective includes an l2 data-fidelity term and a nonsmooth regularizer. This formulation allows both wavelet-based (with orthogonal or frame-based representations) regularization or total-variation regularization. Our approach is based on a variable splitting to obtain an equivalent constrained optimization formulation, which is then addressed with an augmented Lagrangian method. The proposed algorithm is an instance of the so-called alternating direction method of multipliers, for which convergence has been proved. Experiments on a set of image restoration and reconstruction benchmark problems show that the proposed algorithm is faster than the current state of the art methods.


Question: Modeling and performance evaluation of millimeter wave cellular networks.

This paper presents an overview of millimeter-wave (mmWave) mobile broadband systems, highlighting the potential of the 3-300 GHz spectrum for future mobile communication applications. With the majority of current mobile systems operating within the 300 MHz-3 GHz range, the article explores the rationale for expanding into higher frequencies. It addresses the inherent challenges related to propagation and device technology in the mmWave band, while also emphasizing the unique benefits for mobile communication. The concept of a mmWave mobile broadband (MMB) system is introduced as a promising candidate for next-generation mobile networks, capable of delivering gigabit-per-second data rates over distances up to 1 km in urban settings. Essential architectural components of MMB systems, such as the MMB base station grid, inter-base station (BS) backhaul links, and a hybrid MMB + 4G system, are discussed. The paper also delves into beamforming strategies and the frame structure of the MMB air interface, providing a comprehensive view of the technical aspects involved in mmWave mobile broadband systems.

Millimeter wave (mmWave) signals experience orders-of-magnitude more pathloss than the microwave signals currently used in most wireless applications and all cellular systems. MmWave systems must therefore leverage large antenna arrays, made possible by the decrease in wavelength, to combat pathloss with beamforming gain. Beamforming with multiple data streams, known as precoding, can be used to further improve mmWave spectral efficiency. Both beamforming and precoding are done digitally at baseband in traditional multi-antenna systems. The high cost and power consumption of mixed-signal devices in mmWave systems, however, make analog processing in the RF domain more attractive. This hardware limitation restricts the feasible set of precoders and combiners that can be applied by practical mmWave transceivers. In this paper, we consider transmit precoding and receiver combining in mmWave systems with large antenna arrays. We exploit the spatial structure of mmWave channels to formulate the precoding/combining problem as a sparse reconstruction problem. Using the principle of basis pursuit, we develop algorithms that accurately approximate optimal unconstrained precoders and combiners such that they can be implemented in low-cost RF hardware. We present numerical results on the performance of the proposed algorithms and show that they allow mmWave systems to approach their unconstrained performance limits, even when transceiver hardware constraints are considered.

The next generation of mobile radio communication systems - so called 5G - will provide some major changes to those generations to date. The ability to cope with huge increases in data traffic at reduced latencies and improved quality of user experience together with major reduction in energy usage are big challenges. In addition future systems will need to embody connections to billions of objects - the so called Internet of Things (IoT) which raise new challenges. Visions of 5G are now available from regions across the World and research is ongoing towards new standards. The consensus is a flatter architecture that adds a dense network of small cells operating in the millimetre wave bands and which are adaptable and software controlled. But what place for satellites in such a vision? The paper examines several potential roles for satellite including coverage extension, content distribution, providing resilience, improved spectrum utilisation and integrated signalling systems.

In this article we discuss the design of a new fronthaul interface for future 5G networks. The major shortcomings of current fronthaul solutions are first analyzed, and then a new fronthaul interface called next-generation fronthaul interface (NGFI) is proposed. The design principles for NGFI are presented, including decoupling the fronthaul bandwidth from the number of antennas, decoupling cell and user equipment processing, and focusing on high-performancegain collaborative technologies. NGFI aims to better support key 5G technologies, in particular cloud RAN, network functions virtualization, and large-scale antenna systems. NGFI claims the advantages of reduced bandwidth as well as improved transmission efficiency by exploiting the tidal wave effect on mobile network traffic. The transmission of NGFI is based on Ethernet to enjoy the benefits of flexibility and reliability. The major impact, challenges, and potential solutions of Ethernet-based fronthaul networks are also analyzed. Jitter, latency, and time and frequency synchronization are the major issues to overcome.

Millimeter wave (mmWave) communication is a promising technology for future wireless systems, while one practical challenge is to achieve its large-antenna gains with only limited radio frequency (RF) chains for cost-effective implementation. To this end, we study in this paper a new lens antenna array enabled mmWave multiple-input multiple-output (MIMO) communication system. We first show that the array response of lens antenna arrays follows a “sinc” function, where the antenna element with the peak response is determined by the angle of arrival (AoA)/departure (AoD) of the received/transmitted signal. By exploiting this unique property along with the multi-path sparsity of mmWave channels, we propose a novel low-cost and capacity-achieving spatial multiplexing scheme for both narrow-band and wide-band mmWave communications, termed path division multiplexing (PDM), where parallel data streams are transmitted over different propagation paths with simple per-path processing. We further propose a simple path grouping technique with group-based small-scale MIMO processing to effectively mitigate the inter-stream interference due to similar AoAs/AoDs. Numerical results are provided to compare the performance of the proposed mmWave lens MIMO against the conventional MIMO with uniform planar arrays (UPAs) and hybrid analog/digital processing. It is shown that the proposed design achieves significant throughput gains as well as complexity and cost reductions, thus leading to a promising new paradigm for mmWave MIMO communications.

By employing the lens antenna array, beamspace MIMO can utilize beam selection to reduce the number of required RF chains in mmWave massive MIMO systems without obvious performance loss. However, to achieve the capacity-approaching performance, beam selection requires the accurate information of beamspace channel of large size, which is challenging, especially when the number of RF chains is limited. To solve this problem, in this paper we propose a reliable support detection (SD)-based channel estimation scheme. Specifically, we propose to decompose the total beamspace channel estimation problem into a series of sub-problems, each of which only considers one sparse channel component. For each channel component, we first reliably detect its support by utilizing the structural characteristics of mmWave beamspace channel. Then, the influence of this channel component is removed from the total beamspace channel estimation problem. After the supports of all channel components have been detected, the nonzero elements of the sparse beamspace channel can be estimated with low pilot overhead. Simulation results show that the proposed SD-based channel estimation outperforms conventional schemes and enjoys satisfying accuracy, even in the low SNR region.

In this letter, a new type of wideband substrate integrated waveguide (SIW) cavity-backed patch antenna and array for millimeter wave (mmW) are investigated and implemented. The proposed antenna is composed of a rectangular patch with a backed SIW cavity. In order to enhance the bandwidth and radiation efficiency, the cavity is designed to resonate at its TE210 mode. Based on the proposed antenna, a 4 × 4 array is also designed. Both the proposed antenna and array are fabricated with standard printed circuit board (PCB) process, which possess the advantage of easy integration with planar circuits. The measured bandwidth (|S11| ≤ -10 dB) of the antenna element is larger than 15%, and that of the antenna array is about 8.7%. The measured peak gains are 6.5 dBi for the element and 17.8 dBi for the array, and the corresponding simulated radiation efficiencies are 83.9% and 74.9%, respectively. The proposed antenna and array are promising for millimeter-wave applications due to its merits of wide band, high efficiency, low cost, low profile, etc.

The increase in WiFi physical layer transmission speeds from 1~Mbps to 1 Gbps has reduced transmission times for a 1500 byte packet from 12 ms to 12 us. However, WiFi MAC overheads such as channel access and acks have not seen similar reductions and cumulatively contribute about 150 us on average per packet. Thus, the efficiency of WiFi has deteriorated from over 80% at 1 Mbps to under 10% at 1 Gbps.
 In this paper, we propose WiFi-Nano, a system that uses 800 ns slots} to significantly improve WiFi efficiency. Reducing slot time from 9 us to 800 ns makes backoffs efficient, but clear channel assessment can no longer be completed in one slot since preamble detection can now take multiple slots. Instead of waiting for multiple slots for detecting preambles, nodes speculatively transmit preambles as their backoff counters expire, while continuing to detect premables using self-interference cancellation. Upon detection of preambles from other transmitters, nodes simply abort their own preamble transmissions, thereby allowing the earliest transmitter to succeed. Further, receivers speculatively transmit their ack preambles at the end of packet reception, thereby reducing ack overhead. We validate the effectiveness of WiFi-Nano through implementation on an FPGA-based software defined radio platform, and through extensive simulations, demonstrate efficiency gains of up to 100%.

UAV networks are envisioned to play a crucial role in the future generations of wireless networks. The mechanical degrees of freedom in the movement of UAVs provides various advantages for tactical and civilian applications. Due to the high cost of failures in system-based tests, initial analysis and refinement of designs and algorithms for UAV applications are performed through rigorous simulations. Current trend of UAV specific simulators is mainly biased towards the mechanical properties of flying. For network-centric simulations, the intended measurements on the performance of protocols in mobile scenarios are conventionally captured from general-purpose network simulators, which are not natively equipped with comprehensive models for 3D movements of UAVs. To facilitate such simulations for UAV systems, this paper presents different mobility models for emulation of the movement of a UAV. Detailed description of three mobility models (random walk, random direction, and Gauss-Markov) are presented, and their associated movement patterns are characterized. This characterization is further extended by considering the effect of large obstacles on movement patterns of nodes following the three models. The mobility models are prepared as open-source add-ons for ns-3 network simulator.

The demand for rich multimedia services over mobile networks has been soaring at a tremendous pace over recent years. However, due to the centralized architecture of current cellular networks, the wireless link capacity as well as the bandwidth of the radio access networks and the backhaul network cannot practically cope with the explosive growth in mobile traffic. Recently, we have observed the emergence of promising mobile content caching and delivery techniques, by which popular contents are cached in the intermediate servers (or middleboxes, gateways, or routers) so that demands from users for the same content can be accommodated easily without duplicate transmissions from remote servers; hence, redundant traffic can be significantly eliminated. In this article, we first study techniques related to caching in current mobile networks, and discuss potential techniques for caching in 5G mobile networks, including evolved packet core network caching and radio access network caching. A novel edge caching scheme based on the concept of content-centric networking or information-centric networking is proposed. Using trace-driven simulations, we evaluate the performance of the proposed scheme and validate the various advantages of the utilization of caching content in 5G mobile networks. Furthermore, we conclude the article by exploring new relevant opportunities and challenges.

In this work, we present a 4-band dual-polarized antenna designed for 8-chanel applications. Based on LTCC technology, the antenna is a patch coupled aperture with a backed cavity. Each antenna element of a designated band contains two channels through two orthogonally polarized ports. Combining four dual-polarized antenna elements under different frequency, an 8-channel antenna for 60GHz applications can be achieved. The array antenna contains 8 feeding ports, which correspond to 8 independent channels. The isolation between each port can reach 20dB in most of the frequency band.


Question: Modeling and evaluation of millimeter wave cellular networks.

Millimeter Wave Cellular Systems: A Promising Solution for 5G Networks

The escalating demand for wireless bandwidth has prompted the investigation of millimeter wave (mm-wave) frequencies for next-generation cellular communication networks. Despite the vast availability of mm-wave spectrum, there is a scarcity of knowledge regarding cellular propagation characteristics in both indoor and outdoor urban settings. This information is indispensable for the development and deployment of 5G networks operating in the mm-wave band. In this paper, we outline the rationale for mm-wave cellular systems, detail the methodology and hardware employed for measurements, and present an array of results demonstrating the feasibility of using 28 and 38 GHz frequencies with steerable directional antennas for base stations and mobile devices. The findings contribute to the development of channel models essential for 5G, cellular, mobile communications, and MIMO (multiple-input multiple-output) systems.

Keywords: 28GHz, 38GHz, millimeter wave propagation, directional antennas, channel models, 5G, cellular, mobile communications, MIMO.

Increasing customer demand coupled with limitations in current fourth-generation (4G) wireless mobile networks and advances toward the next-generation wireless mobile network have all together created a need for us to look at what customers today expect from the upcoming fifth-generation (5G) network. This study was conducted among existing 4G users in the United States, the United Kingdom, Singapore, and Australia. The research was completed in two parts: qualitative and quantitative. The quantitative part was accomplished by performing data collection and selection through surveys and data evaluation by structural equation modeling (SEM) using SMARTPLS.

In this work we present SimuLTE, an OMNeT++-based simulator for LTE and LTE-Advanced networks. Following well-established OMNeT++ programming practices, SimuLTE exhibits a fully modular structure, which makes it easy to be extended, verified and integrated. Moreover, it inherits all the benefits of such a widely-used and versatile simulation framework as OMNeT++, i.e., experiment support and seamless integration with the OMNeT++ network modules, such as INET. This allows SimuLTE users to build up mixed scenarios where LTE is only a part of a wider network. This paper describes the architecture of SimuLTE, with particular emphasis on the modeling choices at the MAC layer, where resource scheduling is located. Furthermore, we describe some of the verification and validation efforts and present an example of the performance analysis that can be carried out with SimuLTE.

We develop channel estimation agorithms for millimeter wave (mmWave) multiple input multiple output (MIMO) systems with one-bit analog-to-digital converters (ADCs). Since the mmWave MIMO channel is sparse due to the propagation characteristics, the estimation problem is formulated as a one-bit compressed sensing problem. We propose a modified EM algorithm that exploits sparsity and has better performance than the conventional EM algorithm. We also present a second solution using the generalized approximate message passing (GAMP) algorithm to solve this optimization problem. The simulation results show that GAMP can reduce mean squared error in the important low and medium SNR regions.

A dual-band textile antenna for multiple-input-multiple-output (MIMO) applications, based on substrate-integrated waveguide (SIW) technology, is designed. The fundamental SIW cavity mode is designed to resonate at 2.4 GHz. Meanwhile, the second and third modes are modified and combined by careful placement of a via within the cavity to enable wideband coverage in the 5-GHz WLAN band. The simple antenna topology can be fabricated fully using textiles in a planar form, ensuring reliability and comfort. Numerical and experimental results indicate satisfactory antenna performance when worn on body in terms of impedance bandwidth, radiation efficiency, and specific absorption ratio (SAR). In order to validate its potential for MIMO applications, two elements of the proposed SIW antenna are arranged in six configurations to study the performance in terms of mutual coupling and envelope correlation. It is observed that the placement of the shorted edges of the two elements adjacent to each other produces the lowest mutual coupling and consequently the best envelope correlation.

Using radars for detecting and tracking pedestrians has important safety and security applications. Most existing human detection radar systems operate in UHF, X-band, and 24GHz bands. The newly allocated 76-77 GHz frequency band for automobile collision avoidance radar systems has once again raised great interest in pedestrian detection and tracking at these frequencies due to its longer detection range and better location accuracy. The electromagnetic scattering properties must be thoroughly understood and analyzed so a catalog of human scattering can be utilized for intensive automotive radar testing. Measurements of real human subjects at these frequencies is not a trivial task. This paper presents validation between the measured radar cross section (RCS) patterns of various human subjects and a full-wave EM simulation. The RCS of a human is shown to depend on a number of factors, such as posture, body shape, clothing type, etc.

A low-cost high-gain and broadband substrate integrated waveguide (SIW)-fed patch antenna array is demonstrated at the 60-GHz band. A single-layered SIW feeding network with wideband T-junctions and wideband high-gain cavity-backed patch antennas are employed to achieve high gain and wideband performance simultaneously. Although the proposed antenna array has a multilayered structure, it can be fabricated by conventional low-cost single-layered printed circuit board (PCB) technology and then realized by stacking and fixing all of single layers together. The simulated and measured impedance bandwidths of a 4 × 4 antenna array are 27.5% and 22.6% for 10 dB. The discrepancy between simulation and measurement is analyzed. A gain up to 19.6 dBi, and symmetrical unidirectional radiation patterns with low cross polarization are also achieved. With advantages of low fabrication cost and good performances, the proposed antenna array is a promising candidate for millimeter-wave wireless communication systems.

Millimeter-wave anti-collision radars have been widely investigated in advanced CMOS technologies recently. This paper presents a fully integrated 77GHz FMCW radar system in 65nm CMOS. The FMCW radar transmits a continuous wave which is triangularly modulated in frequency and receives the wave reflected from objects. As can be illustrated in Fig. 11.2.1, for a moving target, the received frequency would be shifted (i.e. Doppler shift), resulting in two different offset frequencies f + and f− for the falling and rising ramps. Denoting the modulation range and period as B and Tm, respectively, we can derive the distance R and the relative velocity VR as listed in Fig. 11.2.1, where fc represents the center frequency and c the speed of light.

Mobility prediction is one of the most essential issues that need to be explored for mobility management in mobile computing systems. In this paper, we propose a new algorithm for predicting the next inter-cell movement of a mobile user in a Personal Communication Systems network. In the first phase of our threephase algorithm, user mobility patterns are mined from the history of mobile user trajectories. In the second phase, mobility rules are extracted from these patterns, and in the last phase, mobility predictions are accomplished by using these rules. The performance of the proposed algorithm is evaluated through simulation as compared to two other prediction methods. The performance results obtained in terms of Precision and Recall indicate that our method can make more accurate predictions than the other methods. 2004 Elsevier B.V. All rights reserved.

At millimeter-wave frequencies, corrugated horns can be difficult and expensive to manufacture. As an alternative we present here the results of a theoretical and measurement study of a smooth-walled spline-profile horn for specific application in the 80-120 GHz band. While about 50% longer than its corrugated counterpart, the smooth-walled horn is shown to give improved performance across the band as well as being much easier to manufacture.

Recent advancements in self-interference (SI) cancellation capability of low-power wireless devices motivate in-band full-duplex (FD) wireless backhauling in small cell networks (SCNs). In-band FD wireless backhauling concurrently allows the use of the same frequency spectrum for the backhaul as well as access links of the small cells. In this paper, using tools from stochastic geometry, we develop a framework to model the downlink rate coverage probability of a user in a given SCN with massive multiple-input-multiple-output (MIMO)-enabled wireless backhauls. The considered SCN is composed of a mixture of small cells that are configured in either in-band or out-of-band backhaul modes with a certain probability. The performance of the user in the considered hierarchical network is limited by several sources of interference, such as the backhaul interference, small cell base station (SBS)-to-SBS interference, and the SI. Moreover, due to the channel hardening effect in massive MIMO, the backhaul links only experience long term channel effects, whereas the access links experience both the long term and the short term channel effects. Consequently, the developed framework is flexible to characterize different sources of interference while capturing the heterogeneity of the access and backhaul channels. In specific scenarios, the framework enables deriving closed-form coverage probability expressions. Under perfect backhaul coverage, the simplified expressions are utilized to optimize the proportion of in-band and out-of-band small cells in the SCN in the closed form. Finally, a few remedial solutions are proposed that can potentially mitigate the backhaul interference and in turn improve the performance of in-band FD wireless backhauling. Numerical results investigate the scenarios in which in-band wireless backhauling is useful and demonstrate that maintaining a correct proportion of in-band and out-of-band FD small cells is crucial in wireless backhauled SCNs.


Question: Modeling and examining millimeter wave cellular networks.

This paper presents Blind Beam Steering (BBS), an innovative architecture and algorithm designed to eliminate in-band overhead for establishing directional millimeter-wave (mm-Wave) links. Millimeter-wave communication is characterized by its ability to deliver multi-gigabit per second data rates through highly directional beamforming, which compensates for high pathloss and ensures the required signal-to-noise ratio (SNR). However, the process of setting up communication with narrow beamwidths can be resource-intensive, with the search space expanding based on device mobility and the product of the sender-receiver beam resolution. BBS addresses this challenge by integrating mm-Wave and legacy 2.4/5 GHz bands using out-of-band direction inference, enabling overhead-free multi-Gbps mm-Wave communication. The system evaluates direction estimates derived from passively overheard 2.4/5 GHz frames to ensure optimal mm-Wave link quality on unobstructed direct paths. By removing in-band overhead, BBS capitalizes on mm-Wave's high throughput, scalable beam-width, and mobility resilience. Experimental results show that BBS achieves 97.8% accuracy in estimating direction between pairing nodes, utilizing at least 5 detection band antennas. Moreover, BBS accurately identifies unobstructed direct path conditions with 96.5% precision and significantly reduces IEEE 802.11ad beamforming training overhead by 81%, demonstrating its effectiveness in optimizing mm-Wave cellular systems.

We design and fabricate double-layer slotted waveguide array antennas with wide bandwidths and high efficiencies for 120 GHz and 350 GHz bands. To achieve high gain and high efficiency in high frequency bands, the diffusion bonding technique of plate laminated waveguide is used as a fabrication method. The antennas with 16×16 slots show about 70% efficiency with 32 dBi gain and about 50% efficiency with 31 dBi gain in the 120 GHz and 350 GHz bands, respectively.

We present a technology for the manufacturing of silicon-filled integrated waveguides enabling the realization of low-loss high-performance millimeter-wave passive components and high gain array antennas, thus facilitating the realization of highly integrated millimeter-wave systems. The proposed technology employs deep reactive-ion-etching (DRIE) techniques with aluminum metallization steps to integrate rectangular waveguides with high geometrical accuracy and continuous metallic side walls. Measurement results of integrated rectangular waveguides are reported exhibiting losses of 0.15 dB/ λg at 105 GHz. Moreover, ultra-wideband coplanar to waveguide transitions with 0.6 dB insertion loss at 105 GHz and return loss better than 15 dB from 80 to 110 GHz are described and characterized. The design, integration and measured performance of a frequency scanning slotted-waveguide array antenna is reported, achieving a measured beam steering capability of 82 ° within a band of 23 GHz and a half-power beam-width (HPBW) of 8.5 ° at 96 GHz. Finally, to showcase the capability of this technology to facilitate low-cost mm-wave system level integration, a frequency modulated continuous wave (FMCW) transmit-receive IC for imaging radar applications is flip-chip mounted directly on the integrated array and experimentally characterized.

The paper presents several feasible millimeter wave on-chip antenna designs suitable to be fabricated in CMOS technology without any additional process. The results are listed and compared with state-of-the-art designs in the literature. The difficulties in designing high efficiency antenna on CMOS chip are discussed.

In this paper, we propose a hybridly connected structure for hybrid beamforming in millimeter-wave (mmWave) massive MIMO systems, where the antenna arrays at the transmitter and receiver consist of multiple sub-arrays, each of which connects to multiple radio frequency (RF) chains, and each RF chain connects to all the antennas corresponding to the sub-array. In this structure, through successive interference cancelation, we decompose the precoding matrix optimization problem into multiple precoding sub-matrix optimization problems. Then, near-optimal hybrid digital and analog precoders are designed through factorizing the precoding sub-matrix for each sub-array. Furthermore, we compare the performance of the proposed hybridly connected structure with the existing fully and partially connected structures in terms of spectral efficiency, the required number of phase shifters, and energy efficiency. Finally, simulation results are presented to demonstrate that the spectral efficiency of the hybridly connected structure is better than that of the partially connected structure and that its spectral efficiency can approach that of the fully connected structure with the increase in the number of RF chains. Moreover, the proposed algorithm for the hybridly connected structure is capable of achieving higher energy efficiency than existing algorithms for the fully and partially connected structures.

With the rapid deployment of digital TV, there is an increasing need for accurate point-to-area prediction tools. There is a great deal of propagation models for coverage prediction of DTV. Some of them are pure empirical models, and others are mixed, empirical-analytical models, based on measurement campaigns and electromagnetic theory. The aim of this paper is to compare accurate measurements taken by a Rohde & Schwarz FSH-3 portable spectrum analyzer and precision antennas (biconical and log-periodic), with simulation results derived from coverage prediction models, like the NTIA-ITS Longley-Rice model, the ITM (Irregular Terrain Model) using the 3-arc-second SRTM (Satellite Radar Topography Mission) data that is available freely, and the newer ITWOM (Irregular Terrain with Obstructions Model) model which combines equations from ITU-R P.1546 model with Beer's law and Snell's law. Furthermore, measurements for analog FM broadcasting are compared to predictions from the above mentioned models.

Device-to-Device (D2D) communication is a technology component for LTE-A. The existing researches allow D2D as an underlay to the cellular network to increase the spectral efficiency. In this book, D2D communication underlaying cellular networks is studied. Some physical-layer techniques and cross-layer optimization methods on resource management and interference avoidance are proposed and discussed. WINNER II channel models is applied to be the signal and interference model and simulation results show that the performance of D2D link is closely related to the distance between D2D transmitter and receiver and that between interference source and the receiver. Besides, by power control, D2D SINR degrades, which will naturally contribute to low interference to cellular communication. A simple mode selection method of D2D communication is introduced. Based on path-loss (PL) mode selection criterion, D2D gives better performance than traditional cellular system. When D2D pair is farther away from the BS, a better results can be obtained. Game theory, which offers a wide variety of analytical tools to study the complex interactions of players and predict their choices, can be used for power and radio resource management in D2D communication. A reverse iterative combinatorial auction is formulated as a mechanism to allocate the spectrum resources for D2D communications with multiple user pairs sharing the same channel. In addition, a game theoretic approach is developed to implement joint scheduling, power control and channel allocation for D2D communication. Finally, joint power and spectrum resource allocation method is studied under consideration of battery lifetime, which is an important application of D2D communication on increasing user's energy efficiency. The simulation results show that all these methods have beneficial effects on improving the system performance.

Millimeter-wave anti-collision radars have been widely investigated in advanced CMOS technologies recently. This paper presents a fully integrated 77GHz FMCW radar system in 65nm CMOS. The FMCW radar transmits a continuous wave which is triangularly modulated in frequency and receives the wave reflected from objects. As can be illustrated in Fig. 11.2.1, for a moving target, the received frequency would be shifted (i.e. Doppler shift), resulting in two different offset frequencies f + and f− for the falling and rising ramps. Denoting the modulation range and period as B and Tm, respectively, we can derive the distance R and the relative velocity VR as listed in Fig. 11.2.1, where fc represents the center frequency and c the speed of light.

This article explores network densification as the key mechanism for wireless evolution over the next decade. Network densification includes densification over space (e.g, dense deployment of small cells) and frequency (utilizing larger portions of radio spectrum in diverse bands). Large-scale cost-effective spatial densification is facilitated by self-organizing networks and intercell interference management. Full benefits of network densification can be realized only if it is complemented by backhaul densification, and advanced receivers capable of interference cancellation.

Massive multiple input and multiple output (MIMO) has attracted significant interests in both academia and industry. It has been considered as one of most promising technologies for 5G wireless systems. The large-scale antenna array for base stations naturally becomes the key to deploy the Massive MIMO technologies. In this communication, we present a dual-polarized antenna array with 144 ports for Massive MIMO operating at 3.7 GHz. The proposed array consists of 18 low profile subarrays. Each subarray consists of four single units. Each single antenna unit consists of one vertically polarized port and one horizontally polarized port connected to power splitters, which serve as a feeding network. A stacked patch design is used to construct the single unit with the feeding network, which gives higher gain and lower mutual coupling within the size of a conversional dual-port patch antenna. Simulation results of the proposed single antenna unit, sub-array, and Massive MIMO array are verified by measurement.

In this article device-to-device (D2D) communication underlaying a 3GPP LTE-Advanced cellular network is studied as an enabler of local services with limited interference impact on the primary cellular network. The approach of the study is a tight integration of D2D communication into an LTE-Advanced network. In particular, we propose mechanisms for D2D communication session setup and management involving procedures in the LTE System Architecture Evolution. Moreover, we present numerical results based on system simulations in an interference limited local area scenario. Our results show that D2D communication can increase the total throughput observed in the cell area.


Question: Modeling and evaluation of millimeter wave cellular networks.

This paper presents a comprehensive and analytically feasible framework for addressing joint resource partitioning and offloading in heterogeneous cellular networks (HCNs). HCNs, characterized by a mix of macrocells and small cells, offer the potential for load balancing by offloading users from the often congested macrocells to the less crowded small cells. However, this offloading process can lead to a reduction in signal-to-interference-plus-noise ratio (SINR) for the offloaded users. To mitigate this issue, interference management techniques, such as time or frequency resource partitioning, can be employed, where the macrocell is selectively deactivated during certain resource intervals. The effectiveness of offloading is closely tied to the extent of resource partitioning, which in turn should be adjusted based on the number of users being offloaded. We propose a systematic approach for analyzing and optimizing the joint resource partitioning and offloading strategy in a two-tier cellular network. Using this framework, we derive the distribution of downlink rates across the entire network and determine an optimal strategy for the joint management of resources. Our findings highlight that merely balancing the load is not enough; resource partitioning must be combined with offloading to enhance the data rates for users at the cell edge in co-channel heterogeneous networks.

In this paper, we discuss the application of machine learning techniques for performance prediction problems in wireless networks. These problems often involve using existing measurement data to predict network performance where direct measurements are not available. We explore the performance of existing machine learning algorithms for these problems and propose a simple taxonomy of main problem categories. As an example, we use an extensive real-world drive test data set to show that classical machine learning methods such as Gaussian process regression, exponential smoothing of time series, and random forests can yield excellent prediction results. Applying these methods to the management of wireless mobile networks has the potential to significantly reduce operational costs while simultaneously improving user experience. We also discuss key challenges for future work, especially with the focus on practical deployment of machine learning techniques for performance prediction in mobile wireless networks.

Frequency modulated Continuous Wave (FMCW) is continuous wave RADAR widely used for Intelligent Adaptive Cruise Control (ACC) and Collision Warning System (CWS). The paper presents a design and simulation of millimeter wave FMCW RADAR. Paper introduces a mathematical modeling of a FMCW RADAR including its target model. Design of a 76.5 GHz FMCW RADAR & its target model by using Advanced Design System (ADS) software and target range & velocity at various conditions are employed in the simulated FMCW RADAR system to verify the operation of the system.

Cellular networks today are interference-limited and only becomes increasingly so in the future due to the many users that need to share the spectrum to achieve high-rate multimedia communication. Despite the enormous amount of academic and industrial research in the past 20 years on interference-aware receivers and the large performance improvements promised by these multi-user techniques, today's receivers still generally treat interference as background noise. In this article, we enumerate the reasons for this widespread scepticism, and discuss how current and future trends increases the need for and viability of multi-user receivers for both the uplink, where many asynchronous users are simultaneously detected, and the downlink, where users are scheduled and largely orthogonalized; but the mobile handset still needs to cope with a few dominant interfering base stations. New results for interference cancelling receivers that use conventional front-ends are shown to alleviate many of the shortcomings of prior techniques, particularly for the challenging uplink. This article gives an overview of key recent research breakthroughs on interference cancellation and highlights system-level considerations for future multi-user receivers.

Cellular radios consume more power and suffer reduced data rate when the signal is weak. According to our measurements, the communication energy per bit can be as much as 6x higher when the signal is weak than when it is strong. To realize energy savings, applications must preferentially communicate when the signal is strong, either by deferring non-urgent communication or by advancing anticipated communication to coincide with periods of strong signal. Allowing applications to perform such scheduling requires predicting signal strength, so that opportunities for energy-efficient communication can be anticipated. Furthermore, such prediction must be performed at little energy cost.
 In this paper, we make several contributions towards a practical system for energy-aware cellular data scheduling called Bartendr. First, we establish, via measurements, the relationship between signal strength and power consumption. Second, we show that location alone is not sufficient to predict signal strength and motivate the use of tracks to enable effective prediction. Finally, we develop energy-aware scheduling algorithms for different workloads - syncing and streaming - and evaluate these via simulation driven by traces obtained during actual drives, demonstrating energy savings of up to 60%. Our experiments have been performed on four cellular networks across two large metropolitan areas, one in India and the other in the U.S.

IEEE 802.16 WiMAX is a promising new wireless technology for providing broadband ubiquitous network access. As more and more researchers and industrials are interested in simulating such networks, a number of WiMAX simulators have been emerged in the networking community. One of the most recent WiMAX simulator is the one developed for ns-3. This module provides a standard compliant and well designed implementation of the standard and benefits from the major enhancements provided by ns-3 (compared to other network simulators) which has all the capabilities of becoming the leading network simulator in near future. However, this WiMAX module still lacks some important features which motivated this work. In this paper, we first provide a snapshot of existing WiMAX simulators available in the public domain, while highlighting their limitations. Then, we describe the new features and enhancements we have integrated within the ns-3 WiMAX module, and in particular: a realistic and scalable physical model, an IP packet classifier for the convergence sub-layer, efficient uplink and downlink schedulers, support for multicast traffic and pcap packet tracing functionality. The new design of the physical layer has improved the simulation time by several magnitude orders while still providing a realistic implementation of the standard. Furthermore, the IP classifier has enabled the simulation of an unlimited number of service flows per subscriber station, while the proposed schedulers improve the management of the QoS requirements for the different service flows.

A millimeter wave Wilkinson power divider is presented on a thin and durable organic substrate, along with an analysis of bend topologies to reduce coupling between the output paths. The divider provides excellent performance across the entire Ka-band from 27 to 40 GHz. Insertion loss is approximately 0.3 dB and isolation between the output ports is around 30 dB at the design frequency.

Millimeter wave (mmWave) massive MIMO can achieve orders of magnitude increase in spectral and energy efficiency, and it usually exploits the hybrid analog and digital precoding to overcome the serious signal attenuation induced by mmWave frequencies. However, most of hybrid precoding schemes focus on the full-array structure, which involves a high complexity. In this paper, we propose a near-optimal iterative hybrid precoding scheme based on the more realistic subarray structure with low complexity. We first decompose the complicated capacity optimization problem into a series of ones easier to be handled by considering each antenna array one by one. Then we optimize the achievable capacity of each antenna array from the first one to the last one by utilizing the idea of successive interference cancelation (SIC), which is realized in an iterative procedure that is easy to be parallelized. It is shown that the proposed hybrid precoding scheme can achieve better performance than other recently proposed hybrid precoding schemes, while it also enjoys an acceptable computational complexity.

Device-to-device (D2D) communications was initially proposed in cellular networks as a new paradigm for enhancing network performance. The emergence of new applications such as content distribution and location-aware advertisement introduced new user cases for D2D communications in cellular networks. The initial studies showed that D2D communications has advantages such as increased spectral efficiency and reduced communication delay. However, this communication mode introduces complications in terms of interference control overhead and protocols that are still open research problems. The feasibility of D2D communications in Long-Term Evolution Advanced is being studied by academia, industry, and standardization bodies. To date, there are more than 100 papers available on D2D communications in cellular networks, but there is no survey on this field. In this paper, we provide a taxonomy based on the D2D communicating spectrum and review the available literature extensively under the proposed taxonomy. Moreover, we provide new insights into the over-explored and under-explored areas that lead us to identify open research problems of D2D communications in cellular networks.

Operating unmanned aerial vehicle (UAV) over cellular networks would open the barriers of remote navigation and far-flung flying by combining the benefits of UAVs and the ubiquitous availability of cellular networks. In this letter, we provide an initial insight on the radio propagation characteristics of cellular-to-UAV (CtU) channel. In particular, we model the statistical behavior of the path-loss from a cellular base station toward a flying UAV. Where we report the value of the path-loss as a function of the depression angle and the terrestrial coverage beneath the UAV. The provided model is derived based on extensive experimental data measurements conducted in a typical suburban environment for both terrestrial (by drive test) and aerial coverage (using a UAV). The model provides simple and accurate prediction of CtU path-loss that can be useful for both researchers and network operators alike.

In this paper, a new single element broad band cavity backed multi-layer microstrip patch antenna is presented. This antenna is designed to be used in 79 GHz MIMO radar applications, and is fabricated with advanced high resolution multi-layer PCB technology. It has a wide bandwidth of 11.43 GHz, which is 14.5% of the center frequency. The beam widths in both E and H-planes are very wide, and are 144 degrees and 80 degrees respectively. Also, the antenna gain and efficiency are respectively 4.7 dB and 87%. Furthermore, measurements have been done, and show a good agreement with simulations.