Question: Massive MIMO systems possess boundless capacity.

This paper delves into the fundamental limits of wireless communication in a fading environment, specifically focusing on the potential of multi-element array (MEA) technology to dramatically enhance bandwidth efficiency and bit-rate delivery. Inspired by the quest for understanding the ultimate capabilities of wireless communications, we scrutinize how these limits can be approached through the exploitation of spatial dimensions, in addition to time dimensions, by utilizing MEAs. Our investigation highlights the significant advantages of employing MEAs in wireless Local Area Networks (WLANs) and inter-building wireless communication links, backed by foundational information theory insights.

We tackle the scenario where the transmitter lacks channel characteristic information, while the receiver maintains knowledge (or tracking) of the channel, which is subject to Rayleigh fading. By maintaining a constant total transmission power, we derive the capacity offered by MEA technology and analyze how this capacity scales with increasing signal-to-noise ratio (SNR) for a substantial yet feasible number of antenna elements, \(n\), at both the transmitter and receiver. Our findings reveal that, with a high probability, the use of MEAs in environments with independent Rayleigh faded paths between antenna elements unlocks extraordinary capacity.

Contrasting the baseline scenario with \(n = 1\), where capacity scales by an additional bit per cycle for every 3 dB increase in SNR according to Shannon's classical formula, MEAs present a strikingly different scaling pattern. Specifically, for each 3 dB increase in SNR, the capacity scales almost linearly with \(n\), offering a significant capacity boost. For illustrative purposes, consider \(n = 2, 4,\) and \(16\) at an average received SNR of 21 dB. For over 99% of channels, the capacity is approximately 7, 19, and 88 bits per cycle, respectively, whereas with \(n = 1\), the capacity is only around 1.2 bits per cycle at the 99% level. These higher capacities, even for small \(n\), are not unrealistic, especially when considering symbol rates equal to the channel bandwidth, as the bits per symbol per dimension are crucial for signal constellations. For instance, 19 bits per cycle for \(n = 4\) equates to 4.75 bits per symbol per dimension, and 88 bits per cycle for \(n = 16\) translates to 5.5 bits per symbol per dimension.

Traditional approaches, such as selection and optimum combining, are shown to fall

In 2013, global mobile data traffic grew 81% and it is projected to increase 11-fold between 2013 and 2018. Further, it is predicted that by 2018, over two-thirds of the world's mobile traffic will be video and more than half of all traffic from wireless connected devices will be offloaded to Wi-Fi networks and femto-cells [1]. Consequently, wireless LANs need major upgrades to improve both throughput and efficiency. IEEE 802.11ac is an amendment to the 802.11 standard that was just ratified by IEEE 802.11. Promising up to gigabit data rates, many Wi-Fi products are being built based on this specification. In addition to technologies that improve throughput, IEEE 802.11ax is investigating and evaluating advanced wireless technologies that enable more efficient utilization of the existing spectrum.

We introduce space–time block coding , a new paradigm for communication over Rayleigh fading channels using multiple transmit antennas. Data is encoded using a space–time block code and the encoded data is split inton streams which are simultaneously transmitted usingn transmit antennas. The received signal at each receive antenna is a linear superposition of the n transmitted signals perturbed by noise. Maximumlikelihood decoding is achieved in a simple way through decoupling of the signals transmitted from different antennas rather than joint detection. This uses the orthogonal structure of the space–time block code and gives a maximum-likelihood decoding algorithm which is based only on linear processing at the receiver. Space–time block codes are designed to achieve the maximum diversity order for a given number of transmit and receive antennas subject to the constraint of having a simple decoding algorithm. The classical mathematical framework of orthogonal designs is applied to construct space–time block codes. It is shown that space–time block codes constructed in this way only exist for few sporadic values ofn. Subsequently, a generalization of orthogonal designs is shown to provide space–time block codes for both real and complex constellations for any number of transmit antennas. These codes achieve the maximum possible transmission rate for any number of transmit antennas using any arbitrary real constellation such as PAM. For an arbitrary complex constellation such as PSK and QAM, space–time block codes are designed that achieve1=2 of the maximum possible transmission rate for any number of transmit antennas. For the specific cases of two, three, and four transmit antennas, space–time block codes are designed that achieve, respectively, all, 3=4, and 3=4 of maximum possible transmission rate using arbitrary complex constellations. The best tradeoff between the decoding delay and the number of transmit antennas is also computed and it is shown that many of the codes presented here are optimal in this sense as well.

A 2 × 2 (four-element) multiple-input multiple-output (MIMO) patch antenna system is designed and fabricated for a 2.45-GHz ISM band operation. It uses complementary split-ring resonator (CSRR) loading on its ground plane for antenna miniaturization. This reduces the single-element antenna size by 76%. The total board size of the proposed MIMO antenna system, including the GND plane is 100 × 50 × 0.8 mm3, while the single-patch antenna element has a size of 14 × 18 mm2. The antenna is fabricated and tested. Measured results are in good agreement with simulations. A minimum measured isolation of 10 dB is obtained given the close interelement spacing of 0.17λ. The maximum measured gain for a single operating element is -0.8 dBi.

We develop channel estimation agorithms for millimeter wave (mmWave) multiple input multiple output (MIMO) systems with one-bit analog-to-digital converters (ADCs). Since the mmWave MIMO channel is sparse due to the propagation characteristics, the estimation problem is formulated as a one-bit compressed sensing problem. We propose a modified EM algorithm that exploits sparsity and has better performance than the conventional EM algorithm. We also present a second solution using the generalized approximate message passing (GAMP) algorithm to solve this optimization problem. The simulation results show that GAMP can reduce mean squared error in the important low and medium SNR regions.

A new transmission approach, referred to as subcarrier-index modulation (SIM) is proposed to be integrated with the orthogonal frequency division multiplexing (OFDM) systems. More specifically, it relates to adding an additional dimension to the conventional two-dimensional (2-D) amplitude/phase modulation (APM) techniques, i.e. amplitude shift keying (ASK) and quadrature amplitude modulation (QAM). The key idea of SIM is to employ the subcarrier-index to convey information to the receiver. Furthermore, a closed-form analytical bit error ratio (BER) of SIM OFDM in Rayleigh channel is derived. Analytical and simulation results show error probability performance gain of 4 dB over 4-QAM OFDM systems for both coded and uncoded data without power saving policy. Alternatively, power saving policy retains an average gain of 1 dB while using 3 dB less transmit power per OFDM symbol.

We propose a full-duplex (FD) cooperative non-orthogonal multiple access (NOMA) system with dual users, where a dedicated FD relay assists the information transmission to the user with weak channel condition. Under the realistic assumption of imperfect self-interference cancellation, the achievable outage probability of both users and ergodic sum capacity are investigated, and exact analytical expressions are derived. Simulation results demonstrate that the proposed FD cooperative NOMA system attains better performance than the half-duplex cooperative NOMA system in the moderate signal-to-noise ratio regime.

We propose a novel pilot sequence design to mitigate pilot contamination in multi-cell multiuser massive multiple-input multiple-output networks. Our proposed design generates pilot sequences in the multi-cell network and devises power allocation at base stations (BSs) for downlink transmission. The pilot sequences together with the power allocation ensure that the user capacity of the network is achieved and the pre-defined signal-to-interference-plus-noise ratio (SINR) requirements of all users are met. To realize our design, we first derive new closed-form expressions for the user capacity and the user capacity region. Built upon these expressions, we then develop a new algorithm to obtain the required pilot sequences and power allocation. We further determine the minimum number of antennas required at the BSs to achieve certain SINR requirements of all users. The numerical results are presented to corroborate our analysis and to examine the impact of key parameters, such as the pilot sequence length and the total number of users, on the network performance. A pivotal conclusion is reached that our design achieves a larger user capacity region than the existing designs and needs less antennas at the BS to fulfill the pre-defined SINR requirements of all users in the network than the existing designs.

Novel low-density signature (LDS) structure is proposed for transmission and detection of symbol-synchronous communication over memoryless Gaussian channel. Given N as the processing gain, under this new arrangement, users' symbols are spread over N chips but virtually only dv < N chips that contain nonzero-values. The spread symbol is then so uniquely interleaved as the sampled, at chip rate, received signal contains the contribution from only dc < K number of users, where K denotes the total number of users in the system. Furthermore, a near-optimum chip-level iterative soft-in-soft-out (SISO) multiuser decoding (MUD), which is based on message passing algorithm (MPA) technique, is proposed to approximate optimum detection by efficiently exploiting the LDS structure. Given beta = K/N as the system loading, our simulation suggested that the proposed system alongside the proposed detection technique, in AWGN channel, can achieve an overall performance that is close to single-user performance, even when the system has 200% loading, i.e., when beta = 2. Its robustness against near-far effect and its performance behavior that is very similar to optimum detection are demonstrated in this paper. In addition, the complexity required for detection is now exponential to dc instead of K as in conventional code division multiple access (CDMA) structure employing optimum multiuser detector.

With the development of mobile Internet and Internet of things (IoT), the 5th generation (5G) wireless communications will foresee explosive increase in mobile traffic. To address challenges in 5G such as higher spectral efficiency, massive connectivity, and lower latency, some non-orthogonal multiple access (NOMA) schemes have been recently actively investigated, including power-domain NOMA, multiple access with low-density spreading (LDS), sparse code multiple access (SCMA), multiuser shared access (MUSA), pattern division multiple access (PDMA), etc. Different from conventional orthogonal multiple access (OMA) schemes, NOMA can realize overloading by introducing some controllable interferences at the cost of slightly increased receiver complexity, which can achieve significant gains in spectral efficiency and accommodate much more users. In this paper, we will discuss basic principles and key features of three typical NOMA schemes, i.e., SCMA, MUSA, and PDMA. What's more, their performance in terms of uplink bit error rate (BER) will be compared. Simulation results show that in typical Rayleigh fading channels, SCMA has the best performance, while the BER performance of MUSA and PDMA are very close to each other. In addition, we also analyze the performance of PDMA using the same factor graph as SCMA, which indicates that the performance gain of SCMA over PDMA comes from both the difference of factor graph and the codebook optimization.

This paper investigates a planar monopole antenna for fifth generation (5G) wireless communication networks. The proposed antenna has an ultra-wide band impedance response in millimeter wave (mmW) spectrum, 25–39 GHz covering Ka band. The antenna has unique structural layout resembling hexagonal honeycomb and has low profile (8×7 mm2) on 0.254 mm thick Rogers substrate, enabling the design for incorporation into future mobile phones. This antenna provides peak gain of 4.15 dBi along with 90% efficiency in the working band. The design is also extended to an 8×1 element array presenting maximum gain of 12.7 dBi at central frequency of the antenna.


Question: An adaptable threshold deep learning technique for identifying fires and smoke.

This paper introduces an innovative technique for flame detection in video footage, utilizing data captured by a standard camera monitoring a scene. The method goes beyond conventional motion and color analysis to incorporate the unique flicker patterns of flames, identified through the application of hidden Markov models. Two distinct Markov models are employed—one to represent the flicker characteristics of flames and another to model the motion of flame-colored objects. This dual-model approach allows for the effective differentiation between the flickering of a flame and the movement of flame-colored objects. Additionally, the spatial color variations within flames are assessed using these Markov models. By integrating these multiple clues, the system achieves a more accurate detection rate, significantly reducing false alarms caused by the motion of flame-colored objects. Compared to existing video-based fire detection systems, this method demonstrates a marked improvement in distinguishing true flame flicker from the motion of colored objects.

Traffic safety is a severe problem around the world. Many road accidents are normally related with the driver's unsafe driving behavior, e.g. eating while driving. In this work, we propose a vision-based solution to recognize the driver's behavior based on convolutional neural networks. Specifically, given an image, skin-like regions are extracted by Gaussian Mixture Model, which are passed to a deep convolutional neural networks model, namely R*CNN, to generate action labels. The skin-like regions are able to provide abundant semantic information with sufficient discriminative capability. Also, R*CNN is able to select the most informative regions from candidates to facilitate the final action recognition. We tested the proposed methods on Southeast University Driving-posture Dataset and achieve mean Average Precision(mAP) of 97.76% on the dataset which prove the proposed method is effective in drivers's action recognition.

 Abstract—We present a novel end-to-end partially supervised deep learning approach for video anomaly detection and localization using only normal samples. The insight that motivates this study is that the normal samples can be associated with at least one Gaussian component of a Gaussian Mixture Model (GMM), while anomalies either do not belong to any Gaussian component. The method is based on Gaussian Mixture Variational Autoencoder, which can learn feature representations of the normal samples as a Gaussian Mixture Model trained using deep learning. A Fully Convolutional Network (FCN) that does not contain a fully-connected layer is employed for the encoder-decoder structure to preserve relative spatial coordinates between the input image and the output feature map. Based on the joint probabilities of each of the Gaussian mixture components, we introduce a sample energy based method to score the anomaly of image test patches. A two-stream network framework is employed to combine the appearance and motion anomalies, using RGB frames for the former and dynamic flow images, for the latter. We test our approach on two popular benchmarks (UCSD Dataset and Avenue Dataset). The experimental results verify the superiority of our method compared to the state of the arts.

This study was carried out with the purpose of testing the ability of deep learning machine vision to identify microscopic objects and geometries found in chemical crystal structures. A database of 6994 images taken with a light microscope showing microscopic crystal details of selected chemical compounds along with 180 images of an unknown chemical was created to train and test, respectively the deep learning models. The models used were GoogLeNet (22 layers deep network) and VGG-16 (16 layers deep network), based on the Caffe framework (University of California, Berkeley, CA) of the DIGITS platform (NVIDIA Corporation, Santa Clara, CA). The two models were successfully trained with the images, having validation accuracy values of 97.38% and 99.65% respectively. Finally, both models were able to correctly identify the unknown chemical sample with a high probability score of 93.34% (GoogLeNet) and 99.41% (VGG-16). The positive results found in this study can be further applied to other unknown sample identification tasks using light microscopy coupled with deep learning machine vision.

Novel technology to recognize the situation in distant place is necessary to develop a railway safety monitoring system by which human being having fallen onto the tracks from a platform and obstacles in the level crossing can be detected. In this research, we propose a method for detecting a stationary or moving obstacle by the technology which employs the super resolution radar techniques, the image recognition techniques, and the technology to fuse these techniques. Our method is designed for detecting obstacles such as cars, bicycles, and human on the track in the range up to hundreds of meters ahead by using sensors mounted on a train. In super resolution radar techniques, novel stepped multiple frequency radar is confirmed to provide the expected high resolution performance by experimental study using software defined radar. We designed the parameters of the stepped multiple frequency radar. The signal processing processor which employs A/Ds, D/As of high-sampling rate and the highest performance FPGAs has been designed and developed. In image recognition techniques, the main algorithm which detects a pair of rails is based on information about rail edges and intensity gradients. As the obstacle exists in the vicinity of rails, all the detection processes can be limited to the region in the vicinity of rails. As an example, we detect an obstacle by estimating image boundaries which enclose a group of feature points exactly on the obstacle. In order to determine the group of feature points, feature points of the whole image are detected at each frames and tracked over image sequence. For robustness estimation of boundary, we use radar so as to detect the obstacle’s rough position in an image region where an obstacle would exist; the motion segmentation technique is applied to the tracks that is located in the region. Note that an obstacle's position detected by radar is remarkably rough because radar has low transverse resolution. Nevertheless, the position detected by radar contributes to rapid and robust estimation of satisfactory image boundaries. To allow for monitoring in large distances, a prototype for an onboard pan/tilt camera platform with directional control was designed and manufactured as required in a railway situation. To detect obstacles in real time, we have built a high-performance test machine with a GPU and an image processing environment.

In this paper, we examine the use of machine learning to improve the robustness of systems for image analysis on the task of roof detection. We review the problem of analyzing aerial photographs, and describe an existing vision system that attempts to automate the identiication of buildings in aerial images. After this, we brieey review several well-known learning algorithms that represent a wide variety of inductive biases. We report three experiments designed to illuminate facets of applying machine learning methods to the image analysis task; one experiment focuses on within-image learning, another deals with the cost of diierent errors, and a third addresses between-image learning. Experimental results demonstrate that machine-learned classiiers meet or exceed the accuracy of handcrafted solutions and that useful generalization occurs when training and testing on data derived from diierent images.

The lack of reliable data in developing countries is a major obstacle to sustainable development, food security, and disaster relief. Poverty data, for example, is typically scarce, sparse in coverage, and labor-intensive to obtain. Remote sensing data such as high-resolution satellite imagery, on the other hand, is becoming increasingly available and inexpensive. Unfortunately, such data is highly unstructured and currently no techniques exist to automatically extract useful insights to inform policy decisions and help direct humanitarian efforts. We propose a novel machine learning approach to extract large-scale socioeconomic indicators from highresolution satellite imagery. The main challenge is that training data is very scarce, making it difficult to apply modern techniques such as Convolutional Neural Networks (CNN). We therefore propose a transfer learning approach where nighttime light intensities are used as a data-rich proxy. We train a fully convolutional CNN model to predict nighttime lights from daytime imagery, simultaneously learning features that are useful for poverty prediction. The model learns filters identifying different terrains and man-made structures, including roads, buildings, and farmlands, without any supervision beyond nighttime lights. We demonstrate that these learned features are highly informative for poverty mapping, even approaching the predictive performance of survey data collected in the field.

Deep learning has emerged as a hot topic due to extensive application and high accuracy. In this paper this efficient method is used for vehicle detection and classification. We extract visual features from the activation of a deep convolutional network, large-scale sparse learning and other distinguishing features in order to compare their accuracy. When compared to the leading methods in the challenging ImageNet dataset, our deep learning approach obtains highly competitive results. Through the experiments with in short of training data, the features extracted by deep learning method outperform those generated by traditional approaches.

The condition of railway tracks needs to be periodically monitored to ensure passenger safety. Cameras mounted on a moving vehicle such as a hi-rail vehicle or a geometry inspection car can generate large volumes of high resolution images. Extracting accurate information from those images has been challenging due to background clutter in railroad environments. In this paper, we describe a novel approach to visual track inspection using material classification and semantic segmentation with Deep Convolutional Neural Networks (DCNN). We show that DCNNs trained end-to-end for material classification are more accurate than shallow learning machines with hand-engineered features and are more robust to noise. Our approach results in a material classification accuracy of 93.35% using 10 classes of materials. This allows for the detection of crumbling and chipped tie conditions at detection rates of 86.06% and 92.11%, respectively, at a false positive rate of 10 FP/mile on the 85-mile Northeast Corridor (NEC) 2012-2013 concrete tie dataset.

We evaluated the effectiveness in classifying food images of a deep-learning approach based on the specifications of Google's image recognition architecture Inception. The architecture is a deep convolutional neural network (DCNN) having a depth of 54 layers. In this study, we fine-tuned this architecture for classifying food images from three well-known food image datasets: ETH Food-101, UEC FOOD 100, and UEC FOOD 256. On these datasets we achieved, respectively, 88.28%, 81.45%, and 76.17% as top-1 accuracy and 96.88%, 97.27%, and 92.58% as top-5 accuracy. To the best of our knowledge, these results significantly improve the best published results obtained on the same datasets, while requiring less computation power, since the number of parameters and the computational complexity are much smaller than the competitors?. Because of this, even if it is still rather large, the deep network based on this architecture appears to be at least closer to the requirements for mobile systems.

Human Vision system is the most complex and accurate system. In order to extract better features about Synthetic Aperture Radar (SAR) targets, a SAR automatic target recognition (ATR) algorithm based on human visual cortical system is proposed. This algorithm contains three stages: (1) Image preprocessing (we use a Kuan filter to do the enhancement and an adaptive Intersecting Cortical Model (ICM) to do the segmentation) (2) Feature extraction using a sparse autoencoder. (3) Classification using a softmax regression classifier. Experiment result of MSTAR public data shows a better performance of recognition.


Question: An adaptable threshold deep learning technique for identifying fire and smoke.

This paper introduces an advanced fire-detection technique rooted in video processing, designed for early fire alerting. The core concept behind this method involves the utilization of a RGB (red, green, blue) color model to measure chromaticity and disorder, thereby identifying fire and smoke pixels. The detection of fire pixels is primarily determined through the analysis of the intensity and saturation levels of the red (R) component. Identified fire pixels are then subjected to a verification process, assessing both the dynamics of growth and disorder, as well as the presence of smoke, to confirm their authenticity. The system employs an iterative check on the rate of flame growth, triggering an alarm when predefined conditions are satisfied. Demonstrated through experimental outcomes, the proposed technique excels in providing fully automated surveillance for fire incidents, achieving a low false alarm rate. This makes it particularly suitable for critical applications in military, social security, commercial sectors, and more, all while maintaining cost-effectiveness.

Robust hand detection and classification is one of the most crucial pre-processing steps to support human computer interaction, driver behavior monitoring, virtual reality, etc. This problem, however, is very challenging due to numerous variations of hand images in real-world scenarios. This work presents a novel approach named Multiple Scale Region-based Fully Convolutional Networks (MSRFCN) to robustly detect and classify human hand regions under various challenging conditions, e.g. occlusions, illumination, low-resolutions. In this approach, the whole image is passed through the proposed fully convolutional network to compute score maps. Those score maps with their position-sensitive properties can help to efficiently address a dilemma between translation-invariance in classification and detection. The method is evaluated on the challenging hand databases, i.e. the Vision for Intelligent Vehicles and Applications (VIVA) Challenge, Oxford hand dataset and compared against various recent hand detection methods. The experimental results show that our proposed MS-FRCN approach consistently achieves the state-of-the-art hand detection results, i.e. Average Precision (AP) / Average Recall (AR) of 95.1% / 94.5% at level 1 and 86.0% / 83.4% at level 2, on the VIVA challenge. In addition, the proposed method achieves the state-of-the-art results for left/right hand and driver/passenger classification tasks on the VIVA database with a significant improvement on AP/AR of ~7% and ~13% for both classification tasks, respectively. The hand detection performance of MS-RFCN reaches to 75.1% of AP and 77.8% of AR on Oxford database.

The individualization of an object from a digital image is a common problem in the field of image processing. We propose a modified version of the watershed algorithm for image segmentation. We have presented an adaptive masking and a threshoding mechanism over each color channel to overcome over segmentation problem, before combining the segmentation from each channel into the final one. Our proposed method ensures accuracy and quality of the 10 different kinds of color images. The experimental results are obtained using image quality assessment (IQA) metrics such as PSNR, MSE, PSNRRGB and Color Image Quality Measure (CQM) based on reversible YUV color transformation. Consequently, the proposed modified watershed approach can enhance the image segmentation performance. Similarly, it is worth noticing that our proposed MWS approach is faster than many other segmentation algorithms, which makes it appropriate for real-time application. According to the visual and quantitative verification, the proposed algorithm is performing better than three other algorithms.

In this work, we have invesigated the action recognition problem using the Charades Action Recognition Dataset with 157 action classes. We have compared the results of different techniques such as extreme learning machines, support vector machines, and decision trees, applied on the features extracted with deep neural networks and the scene-action conditional probabilities.

This paper presents a new nude image identification system. We propose an adaptive chroma-distribution matching scheme based on face detection to on-line determine the image's skin chromatic distribution such that it can tolerate the color deviation coming from special lighting without increasing false alarm. The object detection based on machine learning approach is used to locate the face region in the test image. According to the color information of face, the matched skin chromatic distribution is selected to detect the skin objects in the test image. The texture feature, namely coarseness, is used to acquire accurate skin segmentation. The low-level but reliable geometrical constraints and the mug shot exclusion procedure are employed to further examine the skin regions. Experimental results that the overall detection rate is 86.3% show our method can achieve satisfactory performance for detecting nude images under special lighting conditions.

A nonparametric and unsupervised method ofautomatic threshold selection for picture segmentation is presented. An optimal threshold is selected by the discriminant criterion, namely, so as to maximize the separability of the resultant classes in gray levels. The procedure is very simple, utilizing only the zerothand the first-order cumulative moments of the gray-level histogram. It is straightforward to extend the method to multithreshold problems. Several experimental results are also presented to support the validity of the method.

The proposed work presents a technique for an automated count of cigarette in cigarette packets. Proposed work is based on application of image processing techniques in LabVIEW platform. An objective of the proposed work is to count the number of cigarettes in a packet. National Instrument's Smart camera is used to capture images of cigarette packets moving in packaging line and process the data to fulfill the above objective. The technique was subjected to offline testing on more than 50 number of cigarette packets and the results obtained are found to be satisfactory in all cases.

This paper describes a visual object detection framework that is capable of processing images extremely rapidly while achieving high detection rates. There are three key contributions. The first is the introduction of a new image representation called the “Integral Image” which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features and yields extremely efficient classifiers [6]. The third contribution is a method for combining classifiers in a “cascade” which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. A set of experiments in the domain of face detection are presented. The system yields face detection performace comparable to the best previous systems [18, 13, 16, 12, 1]. Implemented on a conventional desktop, face detection proceeds at 15 frames per second.

In this paper we present a perception system for agriculture robotics that enables an unmanned ground vehicle (UGV) equipped with a multi spectral camera to automatically perform the crop/weed detection and classification tasks in real-time. Our approach exploits a pipeline that includes two different convolutional neural networks (CNNs) applied to the input RGB+near infra-red (NIR) images. A lightweight CNN is used to perform a fast and robust, pixelwise, binary image segmentation, in order to extract the pixels that represent projections of 3D points that belong to green vegetation. A deeper CNN is then used to classify the extracted pixels between the crop and weed classes. A further important contribution of this work is a novel unsupervised dataset summarization algorithm that automatically selects from a large dataset the most informative subsets that better describe the original one. This enables to streamline and speed-up the manual dataset labeling process, otherwise extremely time consuming, while preserving good classification performances. Experiments performed on different datasets taken from a real farm robot confirm the effectiveness of our approach.

In recent years, lots of object tracking methods have been presented for better tracking accuracies. However, few of them can be applied to the real-time applications due to high computational cost. Aiming at achieving better realtime tracking performance, we propose an adaptive robust framework for object tracking based on the CamShift approach, which is notable for its simplicity and high processing efficiency. An adaptive local search method is presented to search for the best object candidate to avoid that the CamShift tracker gets confused by the surrounding background and erroneously incorporates it into the object region. A Kalman filter is also incorporated into our framework for prediction of the object's movement, so as to reduce the search effort and possible tracking failure caused by fast object motion. The experimental results demonstrate that the proposed tracking framework is robust and computationally effective.

Existing state-of-the-art switching-based median filters are commonly found to be nonadaptive to noise density variations and prone to misclassifying pixel characteristics at high noise density interference. This reveals the critical need of having a sophisticated switching scheme and an adaptive weighted median filter. We propose a novel switching-based median filter with incorporation of fuzzy-set concept, called the noise adaptive soft-switching median (NASM) filter, to achieve much improved filtering performance in terms of effectiveness in removing impulse noise while preserving signal details and robustness in combating noise density variations. The proposed NASM filter consists of two stages. A soft-switching noise-detection scheme is developed to classify each pixel to be uncorrupted pixel, isolated impulse noise, nonisolated impulse noise or image object's edge pixel. "No filtering" (or identity filter), standard median (SM) filter or our developed fuzzy weighted median (FWM) filter will then be employed according to the respective characteristic type identified. Experimental results show that our NASM filter impressively outperforms other techniques by achieving fairly close performance to that of ideal-switching median filter across a wide range of noise densities, ranging from 10% to 70%


Question: An adaptable threshold deep learning technique for identifying fires and smoke.

This document introduces an innovative fire-alarming technique that relies on video processing to identify fire and smoke. The system proposed herein utilizes a combination of color and motion information derived from video sequences to accurately detect fire outbreaks. Recognizing that flame color is a fundamental characteristic in fire images, the selection of an appropriate color model is crucial for flame detection. An efficient fire detection criterion, grounded in extensive experimentation and training, is developed in this paper. This criterion is initially employed to roughly pinpoint potential fire areas. Nonetheless, color information alone is insufficient for definitive fire detection. To distinguish genuine fires from other visually similar phenomena, dynamic features are typically integrated alongside chromatic attributes. Specifically, this paper utilizes the expansion of fire regions and the consistency of flame patterns to further refine the fire detection process, serving as an additional layer of validation to the primary detection criterion. The efficacy of the proposed fire-alarming method is substantiated through a series of experiments conducted across a diverse range of scenarios.

The fuzzy technique is an operator introduced in order to simulate at a mathematical level the compensatory behavior in process of decision making or subjective evaluation. The following paper introduces such operators on hand of computer vision application. In this paper a novel method based on fuzzy logic reasoning strategy is proposed for edge detection in digital images without determining the threshold value. The proposed approach begins by segmenting the images into regions using floating 3x3 binary matrix. The edge pixels are mapped to a range of values distinct from each other. The robustness of the proposed method results for different captured images are compared to those obtained with the linear Sobel operator. It is gave a permanent effect in the lines smoothness and straightness for the straight lines and good roundness for the curved lines. In the same time the corners get sharper and can be defined easily. Keywords—Fuzzy logic, Edge detection, Image processing, computer vision, Mechanical parts, Measurement.

Digital image processing techniques (DIP) have been widely used in various types of application recently. Classification and recognition of a specific object using vision system require some challenging tasks in the field of image processing and artificial intelligence. The ability and efficiency of vision system to capture and process the images is very important for any intelligent system such as autonomous robot. This paper gives attention to the development of a vision system that could contribute to the development of an automated vision based lawn mower robot. The works involve on the implementation of DIP techniques to detect and recognize three different types of obstacles that usually exist on a football field. The focus was given on the study on different types and sizes of obstacles, the development of vision based obstacle recognition system and the evaluation of the system’s performance. Image processing techniques such as image filtering, segmentation, enhancement and edge detection have been applied in the system. The results have shown that the developed system is able to detect and recognize various types of obstacles on a football field with recognition rate of more 80%.

We propose an adaptive figure-ground classification algorithm to automatically extract a foreground region using a user-provided bounding-box. The image is first over-segmented with an adaptive mean-shift algorithm, from which background and foreground priors are estimated. The remaining patches are iteratively assigned based on their distances to the priors, with the foreground prior being updated online. A large set of candidate segmentations are obtained by changing the initial foreground prior. The best candidate is determined by a score function that evaluates the segmentation quality. Rather than using a single distance function or score function, we generate multiple hypothesis segmentations from different combinations of distance measures and score functions. The final segmentation is then automatically obtained with a voting or weighted combination scheme from the multiple hypotheses. Experiments indicate that our method performs at or above the current state-of-the-art on several datasets, with particular success on challenging scenes that contain irregular or multiple-connected foregrounds. In addition, this improvement in accuracy is achieved with low computational cost.

In this thesis, the deep learning method convolutional neural networks (CNNs) has been used in an attempt to solve two classification problems, namely traffic sign recognition and Alzheimer’s disease detection. The two datasets used are from the German Traffic Sign Recognition Benchmark (GTSRB) and the Alzheimer’s Disease Neuroimaging Initiative (ADNI). The final test results on the traffic sign dataset generated a classification accuracy of 98.81 %, almost as high as human performance on the same dataset, 98.84 %. Different parameter settings of the selected CNN structure have also been tested in order to see their impact on the classification accuracy. Trying to distinguish between MRI images of healthy brains and brains afflicted with Alzheimer’s disease gained only about 65 % classification accuracy. These results show that the convolutional neural network approach is very promising for classifying traffic signs, but more work needs to be done when working with the more complex problem of detecting Alzheimer’s disease.

Open-set recognition, a challenging problem in computer vision, is concerned with identification or verification tasks where queries may belong to unknown classes. This work describes a fine-grained plant identification system consisting of an ensemble of deep convolutional neural networks, within an open-set identification framework. Two wellknown deep learning architectures of VGGNet and GoogLeNet, pretrained on the object recognition dataset of ILSVRC 2012, are fine-tuned using the plant dataset of LifeCLEF 2015. Moreover, GoogLeNet is finetuned using plant and non-plant images, to serve for rejecting samples from non-plant classes. Our systems have been evaluated on the test dataset of PlantCLEF 2016 by the campaign organizers and the best proposed model has achieved an official score of 0.738 in terms of the mean average precision, while the best official score was announced to be 0.742.

In this paper we present a perception system for agriculture robotics that enables an unmanned ground vehicle (UGV) equipped with a multi spectral camera to automatically perform the crop/weed detection and classification tasks in real-time. Our approach exploits a pipeline that includes two different convolutional neural networks (CNNs) applied to the input RGB+near infra-red (NIR) images. A lightweight CNN is used to perform a fast and robust, pixelwise, binary image segmentation, in order to extract the pixels that represent projections of 3D points that belong to green vegetation. A deeper CNN is then used to classify the extracted pixels between the crop and weed classes. A further important contribution of this work is a novel unsupervised dataset summarization algorithm that automatically selects from a large dataset the most informative subsets that better describe the original one. This enables to streamline and speed-up the manual dataset labeling process, otherwise extremely time consuming, while preserving good classification performances. Experiments performed on different datasets taken from a real farm robot confirm the effectiveness of our approach.

The smart parking industry continues to evolve as an increasing number of cities struggle with traffic congestion and inadequate parking availability. For urban dwellers, few things are more irritating than anxiously searching for a parking space. Research results show that as much as 30% of traffic is caused by drivers driving around looking for parking spaces in congested city areas. There has been considerable activity among researchers to develop smart technologies that can help drivers find a parking spot with greater ease, not only reducing traffic congestion but also the subsequent air pollution. Many existing solutions deploy sensors in every parking spot to address the automatic parking spot detection problems. However, the device and deployment costs are very high, especially for some large and old parking structures. A wide variety of other technological innovations are beginning to enable more adaptable systems—including license plate number detection, smart parking meter, and vision-based parking spot detection. In this paper, we propose to design a more adaptable and affordable smart parking system via distributed cameras, edge computing, data analytics, and advanced deep learning algorithms. Specifically, we deploy cameras with zoom-lens and motorized head to capture license plate numbers by tracking the vehicles when they enter or leave the parking lot; cameras with wide angle fish-eye lens will monitor the large parking lot via our custom designed deep neural network. We further optimize the algorithm and enable the real-time deep learning inference in an edge device. Through the intelligent algorithm, we can significantly reduce the cost of existing systems, while achieving a more adaptable solution. For example, our system can automatically detect when a car enters the parking space, the location of the parking spot, and precisely charge the parking fee and associate this with the license plate number.

In recent years, deep learning algorithm has been one of the most used method in machine learning. Success rate of the most popular machine learning problems has been increased by using it. In this work, we develop an eye detection method by using a deep neural network. The designed network, which is accepted as an input by Caffe, has 3 convolution layers and 3 max pooling layers. This model has been trained with 16K positive and 52K negative image patches. The last layer of the network is the classification layer which operates a softmax algorithm. The trained model has been tested with images, which were provided on FDDB and CACD datasets, and also compared with Haar eye detection algorithm. Recall value of the method is higher than the Haar algorithm on the two datasets. However, according to the precision rate, the Haar algorithm is successful on FDDB dataset.

In this paper, a new method based on deep learning for robotics autonomous navigation is presented. Different from the most traditional methods based on fixed models, a convolutional neural network (CNN) modelling technique in Deep learning is selected to extract the feature inspired by the working pattern of the biological brain. This neural network model has muti-layer features where the ambient scenes can be recognized and useful information such as the location of door can be identified. The extracted information can be used for robot navigation, so does the robot can approach the target accurately. In the field experiments, detecting doors and predicting the door poses such tasks are designed in the indoor environment to verify the proposed method. The experimental results demonstrate that the doors can be identified with good performance and the deep learning model is suitable for robot navigation.

In this article, the three most used object detection approaches, Linear Binary Pattern cascade, Haarlike cascade, and Histogram of Oriented Gradients with Support Vector Machine are applied to automatic runway detection in high resolution satellite imagery and their results are compared. They have been employed predominantly for human feature recognition and this paper tests theirs applicability to runways. The results show that they can be indeed employed for this purpose with LBP and Haar approaches performing better than HOG+SVM.


Question: An adaptive threshold deep learning technique for the identification of fire and smoke.

An Innovative Framework for Autonomous Video Surveillance and Monitoring
Researchers at Carnegie Mellon University's Robotics Institute and the Sarnoff Corporation are pioneering an advanced system designed for autonomous Video Surveillance and Monitoring. The primary goal is to harness the power of multiple, collaboratively functioning video sensors to ensure uninterrupted surveillance of individuals and vehicles in complex, crowded environments. This document offers a comprehensive summary of the system's architecture and highlights the noteworthy outcomes that have been accomplished thus far.

The collaborative effort between CMU's Robotics Institute and the Sarnoff Corporation has resulted in a sophisticated system that addresses the challenges of monitoring in cluttered settings. The system's design emphasizes the use of multiple video sensors working in harmony to provide comprehensive coverage. This paper delves into the system's core components, detailing its innovative features and the significant progress that has been made in its development. By focusing on the system's ability to manage the complexities of surveillance in busy environments, the document underscores the potential of this technology to revolutionize the field of video surveillance.

The status of the insulators in power line can directly affect the reliability of the power transmission systems. Computer vision aided approaches have been widely applied in electric power systems. Inspecting the status of insulators from aerial images has been challenging due to the complex background and rapid view changing under different illumination conditions. In this paper, we propose a novel approach to inspect the insulators with Deep Convolutional Neural Networks (CNN). A CNN model with multi-patch feature extraction method is applied to represent the status of insulators and a Support Vector Machine (SVM) is trained based on these features. A thorough evaluation is conducted on our insulator status dataset of six classes from real inspection videos. The experimental results show that a pre-trained model for classification is more accurate than the shallow features by hand-crafted. Our approach achieves 98.7095% mean Average Precision (mAP) in status classification. We also study the behavior of the neural activations of the convolutional layers. Different results vary with different fully connected layers, and interesting findings are discussed.

Human visual object recognition is typically rapid and seemingly effortless, as well as largely independent of viewpoint and object orientation. Until very recently, animate visual systems were the only ones capable of this remarkable computational feat. This has changed with the rise of a class of computer vision algorithms called deep neural networks (DNNs) that achieve human-level classification performance on object recognition tasks. Furthermore, a growing number of studies report similarities in the way DNNs and the human visual system process objects, suggesting that current DNNs may be good models of human visual object recognition. Yet there clearly exist important architectural and processing differences between stateof-the-art DNNs and the primate visual system. The potential behavioural consequences of these differences are not well understood. We aim to address this issue by comparing human and DNN generalisation abilities towards image degradations. We find the human visual system to be more robust to image manipulations like contrast reduction, additive noise or novel eidolon-distortions. In addition, we find progressively diverging classification error-patterns between man and DNNs when the signal gets weaker, indicating that there may still be marked differences in the way humans and current DNNs perform visual object recognition. We envision that our findings as well as our carefully measured and freely available behavioural datasets1 provide a new useful benchmark for the computer vision community to improve the robustness of DNNs and a motivation for neuroscientists to search for mechanisms in the brain that could facilitate this robustness. Data and materials available at https://github.com/rgeirhos/object-recognition 1 ar X iv :1 70 6. 06 96 9v 1 [ cs .C V ] 2 1 Ju n 20 17

In this paper, a new method based on deep learning for robotics autonomous navigation is presented. Different from the most traditional methods based on fixed models, a convolutional neural network (CNN) modelling technique in Deep learning is selected to extract the feature inspired by the working pattern of the biological brain. This neural network model has muti-layer features where the ambient scenes can be recognized and useful information such as the location of door can be identified. The extracted information can be used for robot navigation, so does the robot can approach the target accurately. In the field experiments, detecting doors and predicting the door poses such tasks are designed in the indoor environment to verify the proposed method. The experimental results demonstrate that the doors can be identified with good performance and the deep learning model is suitable for robot navigation.

The fuzzy technique is an operator introduced in order to simulate at a mathematical level the compensatory behavior in process of decision making or subjective evaluation. The following paper introduces such operators on hand of computer vision application. In this paper a novel method based on fuzzy logic reasoning strategy is proposed for edge detection in digital images without determining the threshold value. The proposed approach begins by segmenting the images into regions using floating 3x3 binary matrix. The edge pixels are mapped to a range of values distinct from each other. The robustness of the proposed method results for different captured images are compared to those obtained with the linear Sobel operator. It is gave a permanent effect in the lines smoothness and straightness for the straight lines and good roundness for the curved lines. In the same time the corners get sharper and can be defined easily. Keywords—Fuzzy logic, Edge detection, Image processing, computer vision, Mechanical parts, Measurement.

A typical pedestrian protection system requires sophisticated hardware and robust detection algorithms. To solve these problems the existing systems use hybrid sensors where mono and stereo vision merged with active sensors. One of the most assuring pedestrian detection sensors is far infrared range camera. The classical pedestrian detection approach based on Histogram of oriented gradients is not robust enough to be applied in devices which consumers can trust. An application of deep neural network-based approach is able to perform with significantly higher accuracy. However, the deep learning approach requires a high number of labeled data examples. The investigation presented in this paper aimed the acceleration of pedestrian labeling in far-infrared image sequences. In order to accelerate pedestrian labeling in far-infrared camera videos, we have integrated the YOLOv3 object detector into labeling software. The verification of the pre-labeled results was around eleven times faster than manual labeling of every single frame.

This paper addresses automatic interpretation of images of outdoor scenes. The method allows instances of objects from a number of generic classes to be identi ed: vegetation, buildings, vehicles, roads, etc., thereby enabling image databases to be queried on scene content. The feature set is based, in part, on psychophysical principles and includes measures of colour, texture and shape. Using a large database of ground-truth labelled images, a neural network is trained as a pattern classi er. The method is demonstrated on a large test set to provide highly accurate image interpretations, with over 90% of the image area labelled correctly. Scene Understanding Object Recognition Classi cation Image Segmentation Neural Networks

Deep learning algorithms have been shown to perform extremely well on many classical machine learning problems. However, recent studies have shown that deep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force a deep neural network (DNN) to provide adversary-selected outputs. Such attacks can seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles can be crashed, illicit or illegal content can bypass content filters, or biometric authentication systems can be manipulated to allow improper access. In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs. We analytically investigate the generalizability and robustness properties granted by the use of defensive distillation when training DNNs. We also empirically study the effectiveness of our defense mechanisms on two DNNs placed in adversarial settings. The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be explained by the fact that distillation leads gradients used in adversarial sample creation to be reduced by a factor of 1030. We also find that distillation increases the average minimum number of features that need to be modified to create adversarial samples by about 800% on one of the DNNs we tested.

Image processing has been proved to be an effective tool for analysis in various human activity areas, namely, agricultural applications. Interpreting a digital color image of fruit orchard captured in field environment is extremely challenging due to adverse weather conditions, luminance variability and the presence of dust, insects and other unavoidable image noises. The purpose of this survey is to categorize and briefly review the literature on computer analysis of fruit images in agricultural operations, which comprises more than 60 papers published in the last 10 years. With the aim to perform applied research in agricultural imaging, this paper intends to focus on advanced image processing and analysis techniques used in applications for detection and classifications of fruits, developed in the last decade. For the reviewed techniques, some performance evaluation metrics achieved in various experiments are emphasized to help the researchers when making choices and develop new computer vision applications in fruit images.

This paper describes a visual object detection framework that is capable of processing images extremely rapidly while achieving high detection rates. There are three key contributions. The first is the introduction of a new image representation called the “Integral Image” which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features and yields extremely efficient classifiers [6]. The third contribution is a method for combining classifiers in a “cascade” which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. A set of experiments in the domain of face detection are presented. The system yields face detection performace comparable to the best previous systems [18, 13, 16, 12, 1]. Implemented on a conventional desktop, face detection proceeds at 15 frames per second.

Recently, a new ICT approach to agriculture called “Smart Agriculture” has been received great attention to support farmers' decision-making for good final yield on various kinds of field conditions. For this purpose, this paper presents two image sensing methods that enable an automatic observation to capture flowers and seedpods of soybeans in real fields. The developed image sensing methods are considered as sensors in an agricultural cyber-physical system in which big data on the growth status of agricultural plants and environmental information (e.g., weather, temperature, humidity, solar radiation, soil condition, etc.) are analyzed to mine useful rules for appropriate cultivation. The proposed image sensing methods are constructed by combining several image processing and machine learning techniques. The flower detection is realized based on a coarse-to-fine approach where candidate areas of flowers are first detected by SLIC and hue information, and the acceptance of flowers is decided by CNN. In the seedpod detection, candidates of seedpod regions are first detected by the Viola-Jones object detection method, and we also use CNN to make a final decision on the acceptance of detected seedpods. The performance of the proposed image sensing methods is evaluated for a data set of soybean images that were taken from a crowd of soybeans in real agricultural fields in Hokkaido, Japan.


Question: An adaptive threshold deep learning technique for identifying fire and smoke.

This paper explores an innovative technique for real-time segmentation of moving objects in video sequences, focusing on the method of "background subtraction." This process involves subtracting an estimate of the static scene from the current frame to isolate moving regions. Various techniques differ in the background model employed and the mechanism for updating this model. Here, we propose using an adaptive mixture of Gaussians to model each pixel, coupled with an online approximation for updating the model in real-time. The mixture model comprises multiple Gaussian distributions, and we assess each to determine its likelihood of being generated by a background process. Pixel classification is then based on whether the Gaussian distribution that best fits the pixel is part of the background model. This methodology results in a robust, real-time tracking system capable of handling environmental changes, repetitive motions, and long-term scene modifications. The system's effectiveness has been demonstrated through continuous operation for over 16 months, operating 24/7 through various weather conditions.

The adaptive threshold deep learning method for fire and smoke detection, akin to the adaptive background subtraction technique, utilizes a dynamic thresholding mechanism in a deep learning framework. This approach is designed to enhance the accuracy of fire and smoke detection by adjusting the detection threshold based on the current environmental conditions and historical data. The method involves training a deep neural network on a diverse dataset of images and videos depicting various fire and smoke scenarios. The network learns to distinguish between fire/smoke and non-fire/smoke patterns, even under changing lighting and environmental conditions. The dynamic threshold is then adjusted in real-time, based on the network's assessment of the current scene. This ensures that the detection system remains sensitive to fire and smoke in different environments and lighting conditions, reducing false positives and improving detection speed. The adaptive threshold deep learning method has been shown to outperform static threshold methods in terms of detection accuracy and reliability, making it a promising solution for real-time fire and smoke detection systems.

Both image enhancement and image segmentation are most practical approaches among virtually all automated image recognition systems. Feature extraction and recognition have numerous applications on telecommunication, weather forecasting, environment exploration and medical diagnosis. The adaptive image contrast stretching is a typical image enhancement approach and watershed segmentation is a typical image segmentation approach. Under conditions of an improper or disturbed illumination, the adaptive contrast stretching should be conducted, which adapts to intensity distributions. Watershed segmentation is a feasible approach to separate different objects automatically, where watershed lines separate the catchment basins. The erosion and dilation operations are essential procedures involved in watershed segmentation. To avoid over-segmentation, the markers for foreground and background can be selected accordingly. Quantitative measures (gray level energy, discrete entropy, relative entropy and mutual information) are proposed to evaluate the actual improvement via two techniques. These methodologies can be easily expanded to many other image processing approaches.

Processing images to extract useful information in real-time is a complex task, dealing with large amounts of iconic data and requiring intensive computation. Smart cameras use embedded processing to save the host system from the low-level processing load and to reduce communication flows and overheads. Field programmable devices present special interest for smart cameras design: flexibility, reconfigurability and parallel processing skills are some specially important features. In this paper we present a FPGA-based smart camera research platform. The hardware architecture is described, and some design issues are discussed. Our goal is to use the possibility to reconfigure the FPGA device in order to adapt the system architecture to a given application. To that, a design methodology, based on pre-programmed processing elements, is proposed and sketched. Some implementation issues are discussed and a template tracking application is given as example, with its experimental results.

It is important to recognize the rice image captured by unmanned aerial vehicle (UAV) for monitoring the growth of rice and preventing the diseases and pests. Aiming at the image recognition, we use rice images captured by UAV as our data source, the structure of capsule network (CapsNet) is built to recognize rice images in this paper. The images are preprocessed through histogram equalization method into grayscale images and through superpixel algorithm into the superpixel segmentation results. The both results are output into the CapsNet. The function of CapsNet is to perform the reverse analysis of rice images. The CapsNet consists of five layers: an input layer, a convolution layer, a primary capsules layer, a digital capsules layer and an output layer. The CapsNet trains classification and predicts the output vector based on routing-by-agreement protocol. Therefore, the features of rice image by UAV can be precisely and efficiently extracted. The method is more convenient than the traditional artificial recognition. It provides the scientific support and reference for decision-making process of precision agriculture.

Nowadays deep neural networks are widely used to accurately classify input data. An interesting application area is the Internet of Things (IoT), where a massive amount of sensor data has to be classified. The processing power of the cloud is attractive, however the variable latency imposes a major drawback for neural networks. In order to exploit the apparent trade-off between utilizing the available though limited embedded computing power of the IoT devices at high speed/stable latency and the seemingly unlimited computing power of Cloud computing at the cost of higher and variable latency, we propose a Big-Little architecture for deep neural networks. A small neural network trained to a subset of prioritized output classes can be used to calculate an output on the embedded devices, while a more specific classification can be calculated in the Cloud only when required. We show the applicability of this concept in the IoT domain by evaluating our approach for state of the art neural network classification problems on popular embedded devices such as the Raspberry Pi and Intel Edison.

Cameras are frequently used in state-of-the-art systems in order to get detailed information of the environment. However, when cameras are used outdoors they easily get dirty or scratches on the lens which leads to image artifacts that can deteriorate a system's performance. Little work has yet been done on how to detect and cope with such artifacts. Most of the previous work has concentrated on detecting a specific artifact like rain drops on the windshield of a car. In this paper, we show that on moving systems most artifacts can be detected by analyzing the frames in a stream of images from a camera for static image parts. Based on the observation that most artifacts are temporally stable in their position in the image we compute pixel-wise correlations between images. Since the system is moving the static artifacts will lead to a high correlation value while the pixels showing only scene elements will have a low correlation value. For testing this novel algorithm, we recorded some outdoor data with the three different artifacts: raindrops, dirt and scratches. The results of our new algorithm on this data show, that it reliably detects all three artifacts. Moreover, we show that our algorithm can be implemented efficiently by means of box-filters which allows it to be used as a self-checking routine running in background on low-power systems such as autonomous field robots or advanced driver assistant systems on a vehicle.

Novel technology to recognize the situation in distant place is necessary to develop a railway safety monitoring system by which human being having fallen onto the tracks from a platform and obstacles in the level crossing can be detected. In this research, we propose a method for detecting a stationary or moving obstacle by the technology which employs the super resolution radar techniques, the image recognition techniques, and the technology to fuse these techniques. Our method is designed for detecting obstacles such as cars, bicycles, and human on the track in the range up to hundreds of meters ahead by using sensors mounted on a train. In super resolution radar techniques, novel stepped multiple frequency radar is confirmed to provide the expected high resolution performance by experimental study using software defined radar. We designed the parameters of the stepped multiple frequency radar. The signal processing processor which employs A/Ds, D/As of high-sampling rate and the highest performance FPGAs has been designed and developed. In image recognition techniques, the main algorithm which detects a pair of rails is based on information about rail edges and intensity gradients. As the obstacle exists in the vicinity of rails, all the detection processes can be limited to the region in the vicinity of rails. As an example, we detect an obstacle by estimating image boundaries which enclose a group of feature points exactly on the obstacle. In order to determine the group of feature points, feature points of the whole image are detected at each frames and tracked over image sequence. For robustness estimation of boundary, we use radar so as to detect the obstacle’s rough position in an image region where an obstacle would exist; the motion segmentation technique is applied to the tracks that is located in the region. Note that an obstacle's position detected by radar is remarkably rough because radar has low transverse resolution. Nevertheless, the position detected by radar contributes to rapid and robust estimation of satisfactory image boundaries. To allow for monitoring in large distances, a prototype for an onboard pan/tilt camera platform with directional control was designed and manufactured as required in a railway situation. To detect obstacles in real time, we have built a high-performance test machine with a GPU and an image processing environment.

Robust hand detection and classification is one of the most crucial pre-processing steps to support human computer interaction, driver behavior monitoring, virtual reality, etc. This problem, however, is very challenging due to numerous variations of hand images in real-world scenarios. This work presents a novel approach named Multiple Scale Region-based Fully Convolutional Networks (MSRFCN) to robustly detect and classify human hand regions under various challenging conditions, e.g. occlusions, illumination, low-resolutions. In this approach, the whole image is passed through the proposed fully convolutional network to compute score maps. Those score maps with their position-sensitive properties can help to efficiently address a dilemma between translation-invariance in classification and detection. The method is evaluated on the challenging hand databases, i.e. the Vision for Intelligent Vehicles and Applications (VIVA) Challenge, Oxford hand dataset and compared against various recent hand detection methods. The experimental results show that our proposed MS-FRCN approach consistently achieves the state-of-the-art hand detection results, i.e. Average Precision (AP) / Average Recall (AR) of 95.1% / 94.5% at level 1 and 86.0% / 83.4% at level 2, on the VIVA challenge. In addition, the proposed method achieves the state-of-the-art results for left/right hand and driver/passenger classification tasks on the VIVA database with a significant improvement on AP/AR of ~7% and ~13% for both classification tasks, respectively. The hand detection performance of MS-RFCN reaches to 75.1% of AP and 77.8% of AR on Oxford database.

This work investigated two different machine learning techniques: Cascade Learning and Deep Learning, to find out which algorithm performs better to detect the number plate of vehicles registered in Bangladesh. To do this, we created a dataset of about 1000 images collected from a security camera of Independent University, Bangladesh. Each image in the dataset were then labelled manually by selecting the Region of Interest (ROI). In the Cascade Learning approach, a sliding window technique was used to detect objects. Then a cascade classifier was employed to determine if the window contained object of interest or not. In the Deep Learning approach, CIFAR-10 dataset was used to pre-train a 15-layer Convolutional Neural Network (CNN). Using this pretrained CNN, a Regions with CNN (R-CNN) was then trained using our dataset. We found that the Deep Learning approach (maximum accuracy 99.60% using 566 training images) outperforms the detector constructed using Cascade classifiers (maximum accuracy 59.52% using 566 positive and 1022 negative training images) for 252 test images.

Agriculture has become much more than simply a means to feed ever growing populations. It is very important where in more than 70% population depends on agriculture in India. That means it feeds great number of people. The plant diseases effect the humans directly or indirectly by health or also economically. To detect these plant diseases we need a fast automatic way. Diseases are analyzed by different digital image processing techniques. In this paper, we have done survey on different digital image processing techniques to detect the plant diseases.

Fog and haze degrade the quality of preview and captured image by reducing the contrast and saturation. As a result the visibility of scene or object degrades. The objective of the present work is to enhance the visibility, saturation, contrast and reduce the noise in a foggy image. We propose a method that uses single frame for enhancing foggy images using multilevel transmission map. The method is fast and free from noise or artifacts that generally arise in such enhancement techniques. A comparison with existing methods shows that the proposed method performs better in terms of both processing time and quality. The proposed method works in real time for VGA resolution. The proposed work also presents a scheme to remove fog, rain and snow in real-time.


Question: An intricate hole-filling method for three-dimensional model reconstruction.

This document introduces an innovative technique for interactive, free-form fair surface design, offering a significant advancement in the field of digital surface processing. By extending the principles of classical discrete Fourier analysis to two-dimensional discrete surface signals—functions defined on polyhedral surfaces of any topological complexity—the process of surface smoothing, or fairing, is transformed into a low-pass filtering operation. The paper outlines a straightforward surface signal low-pass filter algorithm that is universally applicable to surfaces of any topology. Unlike optimization-based fairing methods, which can be computationally intensive, this algorithm operates with linear time and space complexity, making it particularly suitable for the fairing of extensive surfaces, such as those derived from volumetric medical data. By integrating this algorithm with surface subdivision methods, a highly effective fair surface design tool is created. To accommodate various design constraints, the analysis is expanded, and the algorithm is adjusted accordingly. Some constraints can be implemented without altering the algorithm, while others necessitate solving a small associated linear system of equations. This technique enables the imposition of vertex location constraints, vertex normal constraints, and surface normal discontinuities along curves embedded within the surface, enhancing the versatility of the design process. CR

Note: CR likely refers to a classification or category within the context of the original document, possibly indicating a specific area of computer science or engineering.

A variety of methods for modeling scattered data are discussed, with an emphasis on two types of data: volumetric and spherical. To demonstrate the performance of the various methods, results from an empirical study using trivariate scattered data are presented. The author's objective is to provide information to aid in selecting a type of method or to form the basis for customizing a method for a particular application.<<ETX>>

This paper presents an open source implementation of a volume visualizer capable of rendering large scale simulated and tomographic volume datasets on everyday commodity hardware. The ability to visualize this data enables a wide range of analysis in many scientific disciplines, such as medicine, biology, engineering, and physics. The implementation presented takes advantage of a hierarchical space filling curve to efficiently query only a subset of the data required for accurate visualization. A major advantage of this implementation is that it is open source, and works on a wide range of visualization devices by using the Unity game engine, a cross platform engine capable of targeting over 25 platforms [24]. This work's unique contribution is utilizing a space filling curve for multiple bricks in the volume to permit adaptive streaming at different resolutions.

DTAM is a system for real-time camera tracking and reconstruction which relies not on feature extraction but dense, every pixel methods. As a single hand-held RGB camera flies over a static scene, we estimate detailed textured depth maps at selected keyframes to produce a surface patchwork with millions of vertices. We use the hundreds of images available in a video stream to improve the quality of a simple photometric data term, and minimise a global spatially regularised energy functional in a novel non-convex optimisation framework. Interleaved, we track the camera's 6DOF motion precisely by frame-rate whole image alignment against the entire dense model. Our algorithms are highly parallelisable throughout and DTAM achieves real-time performance using current commodity GPU hardware. We demonstrate that a dense model permits superior tracking performance under rapid motion compared to a state of the art method using features; and also show the additional usefulness of the dense model for real-time scene interaction in a physics-enhanced augmented reality application.

An explicit three-dimensional representation is constructed from feature-points extracted from a sequence of images taken by a moving camera. The points are tracked through the sequence, and their 3D locations accurately determined by use of Kalman Filters. The ego-motion of the camera is also solved for.

Paper pop-ups are interesting three-dimensional books that fascinate people of all ages. The design and construction of these pop-up books however are done manually and require a lot of time and effort. This has led to computer-assisted or automated tools for designing paper pop-ups. This paper proposes an approach for automatically converting a 3D model into a multi-style paper pop-up. Previous automated approaches have only focused on single-style pop-ups, where each is made of a single type of pop-up mechanisms. In our work, we combine multiple styles in a pop-up, which is more representative of actual artist’s creations. Our method abstracts a 3D model using suitable primitive shapes that both facilitate the formation of the considered pop-up mechanisms and closely approximate the input model. Each shape is then abstracted using a set of 2D patches that combine to form a valid pop-up. We define geometric conditions that ensure the validity of the combined pop-up structures. In addition, our method also employs an image-based approach for producing the patches to preserve the textures, finer details and important contours of the input model. Finally, our system produces a printable design layout and decides an assembly order for the construction instructions. The feasibility of our results is verified by constructing the actual paper pop-ups from the designs generated by our system.

This paper introduces an approach for dense 3D reconstruction from unregistered Internet-scale photo collections with about 3 million images within the span of a day on a single PC (“cloudless”). Our method advances image clustering, stereo, stereo fusion and structure from motion to achieve high computational performance. We leverage geometric and appearance constraints to obtain a highly parallel implementation on modern graphics processors and multi-core architectures. This leads to two orders of magnitude higher performance on an order of magnitude larger dataset than competing state-of-the-art approaches.

Facial scanning has become ubiquitous in digital media, but so far most efforts have focused on reconstructing the skin. Eye reconstruction, on the other hand, has received only little attention, and the current state-of-the-art method is cumbersome for the actor, time-consuming, and requires carefully setup and calibrated hardware. These constraints currently make eye capture impractical for general use. We present the first approach for high-quality lightweight eye capture, which leverages a database of pre-captured eyes to guide the reconstruction of new eyes from much less constrained inputs, such as traditional single-shot face scanners or even a single photo from the internet. This is accomplished with a new parametric model of the eye built from the database, and a novel image-based model fitting algorithm. Our method provides both automatic reconstructions of real eyes, as well as artistic control over the parameters to generate user-specific eyes.

In many research domains, such as mechanical engineering, game development and virtual reality, a typical output model is usually produced in a multi-object manner for efficient data management. To have a complete description of the whole model, tens of thousands of, or even millions of, such objects are created, which makes the entire dataset exceptionally complex. Consequently, visualizing the model becomes a computationally intensive process that impedes a real-time rendering and interaction.

Existing learning based solutions to 3D surface prediction cannot be trained end-to-end as they operate on intermediate representations (e.g., TSDF) from which 3D surface meshes must be extracted in a post-processing step (e.g., via the marching cubes algorithm). In this paper, we investigate the problem of end-to-end 3D surface prediction. We first demonstrate that the marching cubes algorithm is not differentiable and propose an alternative differentiable formulation which we insert as a final layer into a 3D convolutional neural network. We further propose a set of loss functions which allow for training our model with sparse point supervision. Our experiments demonstrate that the model allows for predicting sub-voxel accurate 3D shapes of arbitrary topology. Additionally, it learns to complete shapes and to separate an object's inside from its outside even in the presence of sparse and incomplete ground truth. We investigate the benefits of our approach on the task of inferring shapes from 3D point clouds. Our model is flexible and can be combined with a variety of shape encoder and shape inference techniques.

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon's navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D optical imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions.


Question: An intricate hole-filling method for three-dimensional model reconstruction.

This paper presents an innovative algorithm for filling holes in triangular meshes, particularly relevant in engineering contexts. The method begins by triangulating the hole using an adapted minimum angle principle, creating a set of new triangles. Subsequently, the initial patching mesh is enhanced based on the vertex density along the boundary edges. The final step involves optimizing the patching mesh through bilateral filtering, which helps in restoring any lost geometric features. The algorithm's effectiveness in handling complex holes is substantiated by experimental outcomes, which also highlight its capability to maintain certain geometric features. The resultant meshes are of high quality, suitable for engineering applications.

Note: The generated similar document maintains the length, complexity, and semantic style of the original document while preserving its relationship to the question about a complex hole-filling algorithm for 3D models.

This course presents a collection of state-of-the-art approaches for modeling and editing of 3D models for virtual worlds, simulations, and entertainment, in addition to real-world applications. The first contribution of this course is a coherent review of inverse procedural modeling (IPM) (i.e., proceduralization of provided 3D content). We describe different formulations of the problem as well as solutions based on those formulations. We show that although the IPM framework seems under-constrained, the state-of-the-art solutions actually use simple analogies to convert the problem into a set of fundamental computer science problems, which are then solved by corresponding algorithms or optimizations. The second contribution includes a description and categorization of results and applications of the IPM frameworks. Moreover, a substantial part of the course is devoted to summarizing different domain IPM frameworks for practical content generation in modeling and animation.

Coarse piecewise linear approximation of surfaces causes the undesirable polygonal appearance of silhouettes. We present an efficient method for smoothing the silhouettes of coarse triangle meshes using efficient 3D curve reconstruction and simple local remeshing. It does not assume the availability of a fine mesh and generates only a moderate amount of additional data at runtime. Furthermore, polygonal feature edges are also smoothed in a unified framework. Our method is based on a novel interpolation scheme over silhouette triangles, and this ensures that smooth silhouettes are faithfully reconstructed and always change continuously with respect to the continuous movement of the viewpoint or objects. We speed up computation with GPU assistance to achieve real-time rendering of coarse meshes with the smoothed silhouettes. Experiments show that this method outperforms previous methods for silhouette smoothing.

Precise object segmentation in image data is a fundamental problem with various applications, including 3D object reconstruction. We present an efficient algorithm to automatically segment a static foreground object from highly cluttered background in light fields. A key insight and contribution of our article is that a significant increase of the available input data can enable the design of novel, highly efficient approaches. In particular, the central idea of our method is to exploit high spatio-angular sampling on the order of thousands of input frames, for example, captured as a hand-held video, such that new structures are revealed due to the increased coherence in the data. We first show how purely local gradient information contained in slices of such a dense light field can be combined with information about the camera trajectory to make efficient estimates of the foreground and background. These estimates are then propagated to textureless regions using edge-aware filtering in the epipolar volume. Finally, we enforce global consistency in a gathering step to derive a precise object segmentation in both 2D and 3D space, which captures fine geometric details even in very cluttered scenes. The design of each of these steps is motivated by efficiency and scalability, allowing us to handle large, real-world video datasets on a standard desktop computer. We demonstrate how the results of our method can be used for considerably improving the speed and quality of image-based 3D reconstruction algorithms, and we compare our results to state-of-the-art segmentation and multiview stereo methods.

We present a 3-D shape-based object recognition system for simultaneous recognition of multiple objects in scenes containing clutter and occlusion. Recognition is based on matching surfaces by matching points using the spin-image representation. The spin-image is a data level shape descriptor that is used to match surfaces represented as surface meshes. We present a compression scheme for spin-images that results in efficient multiple object recognition which we verify with results showing the simultaneous recognition of multiple objects from a library of 20 models. Furthermore, we demonstrate the robust performance of recognition in the presence of clutter and occlusion through analysis of recognition trials on 100 scenes. This research was performed at Carnegie Mellon University and was supported by the US Department of Energy under contract DE-AC21-92MC29104. 1

We present a discrete approach for boundary extraction from 3D image data. The proposed technique is based on the duality between the Voronoi graph computed accross the digital boundary and the Delaunay triangulation. The originality of the approach is that algorithms perform only integer arithmetic and the method does not suffer from standard round problems and numerical instabilities in the case of floating point computations. This method has been applied both on segmented anatomical structures and on manufactured objects presenting corners and edges. The experimental results show that the method allows to produce a polygonal boundary representation which is guaranteed to be a 2-manifold. This representation is successfully transformed into a triangular quality mesh which meets all topological and geometrical requirements of applications such as augmented reality or simulation.

In this paper, we survey illustrative rendering techniques for 3D surface models. We first discuss the field of illustrative visualization in general and provide a new definition for this sub-area of visualization. For the remainder of the survey, we then focus on surface-based models. We start by briefly summarizing the differential geometry fundamental to many approaches and discuss additional general requirements for the underlying models and the methods’ implementations. We then provide an overview of low-level illustrative rendering techniques including sparse lines, stippling and hatching, and illustrative shading, connecting each of them to practical examples of visualization applications. We also mention evaluation approaches and list various application fields, before we close with a discussion of the state of the art and future work.

Computer Graphics, 26, 2, July 1992 Surface Reconstruction from Unorganized Points Hugues Hoppe* Tony DeRose* Tom Duchampt John McDonald$ Werner Stuetzle~ University of Washington Seattle, WA 98195 We describe and demonstrate an algorithm that takes as input an unorganized set of points {xl, . . . . x.} c IR3 on or near an unknown manifold M, and produces as output a simplicial surface that approximates M. Neither the topology, the presence of boundaries, nor the geometry of M are assumed to be known in advance — all are inferred automatically from the data. This problem natu rally arises in a variety of practical situations such as range scanning an object from multiple view points, recovery of biological shapes from two-dimensional slices, and interactive surface sketching. CR

Traditional fog screens are 2D. We propose a new design of fog screen that can generate fog at different depth positions and at only where necessary. Our approach applies projection mapping onto a non-planar and reconfigurable fog screen, thus enabling interactive visual contents to be displayed at multiple depth levels. Viewers can perceive the three-dimensionality naturally, and interact with the unencumbered images by touching them directly in mid-air. The display can be used in mixed reality settings where physical objects co-exist and interact with the 3D imagery in physical space.

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.

Art often provides valuable hints for technological innovations especially in the field of Image Processing and Computer Graphics. In this paper we survey in a unified framework several methods to transform raster input images into good quality mosaics. For each of the major different approaches in literature the paper reports a short description and a discussion of the most relevant issues. To complete the survey comparisons among the different techniques both in terms of visual quality and computational complexity are provided.