Question: 4D generic video object suggestions.

This paper introduces a novel framework for deep learning, termed Residual Learning, which significantly enhances the training of deep neural networks. We address the challenge of training deeper networks by reformulating the layers to learn residual functions relative to their inputs, rather than learning functions independently. This approach simplifies the optimization process, enabling the effective training of significantly deeper networks than previously possible. Our experiments on the ImageNet dataset demonstrate the efficacy of residual networks, achieving remarkable results with a depth of 152 layers, which is 8 times deeper than the VGG nets, yet with lower complexity. An ensemble of these residual networks achieved an error rate of 3.57% on the ImageNet test set, securing the first place in the ILSVRC 2015 classification task. We also conducted analyses on CIFAR-10 with 100 and 1000 layers, further validating the benefits of our approach. The depth of representations plays a crucial role in various visual recognition tasks. By leveraging our extremely deep representations, we observed a 28% relative improvement in performance on the COCO object detection dataset. Residual networks form the core of our submissions to the ILSVRC & COCO 2015 competitions, where we claimed the top positions in multiple tasks including ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

Recent technological advances in connected-speech recognition and position sensing in space have encouraged the notion that voice and gesture inputs at the graphics interface can converge to provide a concerted, natural user modality.
 The work described herein involves the user commanding simple shapes about a large-screen graphics display surface. Because voice can be augmented with simultaneous pointing, the free usage of pronouns becomes possible, with a corresponding gain in naturalness and economy of expression. Conversely, gesture aided by voice gains precision in its power to reference.

With Internet delivery of video content surging to an unprecedented level, video recommendation, which suggests relevant videos to targeted users according to their historical and current viewings or preferences, has become one of most pervasive online video services. This article presents a novel contextual video recommendation system, called VideoReach, based on multimodal content relevance and user feedback. We consider an online video usually consists of different modalities (i.e., visual and audio track, as well as associated texts such as query, keywords, and surrounding text). Therefore, the recommended videos should be relevant to current viewing in terms of multimodal relevance. We also consider that different parts of videos are with different degrees of interest to a user, as well as different features and modalities have different contributions to the overall relevance. As a result, the recommended videos should also be relevant to current users in terms of user feedback (i.e., user click-through). We then design a unified framework for VideoReach which can seamlessly integrate both multimodal relevance and user feedback by relevance feedback and attention fusion. VideoReach represents one of the first attempts toward contextual recommendation driven by video content and user click-through, without assuming a sufficient collection of user profiles available. We conducted experiments over a large-scale real-world video data and reported the effectiveness of VideoReach.

This paper presents a color structured light technique for recovering object shape from one or more images. The technique works by projecting a pattern of stripes of alternating colors and matching the projected color transitions with observed edges in the image. The correspondence problem is solved using a novel, multi-pass dynamic programming algorithm that eliminates global smoothness assumptions and strict ordering constraints present in previous formulations. The resulting approach is suitable for generating both highspeed scans of moving objects when projecting a single stripe pattern and high-resolution scans of static scenes using a short sequence of time-shifted stripe patterns. In the latter case, spacetime analysis is used at each sensor pixel to obtain inter-frame depth localization. Results are demonstrated for a variety of complex scenes.

Motion detection is very important in image processing. One way of detecting motion is using optical flow. Optical flow cannot be computed locally, since only one independent measurement is available from the image sequence at a point, while the flow velocity has two components. A second constraint is needed. The method used for finding the optical flow in this project is assuming that the apparent velocity of the brightness pattern varies smoothly almost everywhere in the image. This technique is later used in developing software for motion detection which has the capability to carry out four types of motion detection. The motion detection software presented in this project also can highlight motion region, count motion level as well as counting object numbers. Many objects such as vehicles and human from video streams can be recognized by applying optical flow technique. Keywords—Background modeling, Motion detection, Optical flow, Velocity smoothness constant, motion trajectories.

A computer vision system has been implemented that can recognize threedimensional objects from unknown viewpoints in single gray-scale images. Unlike most other approaches, the recognition is accomplished without any attempt to reconstruct depth information bottom-up from the visual input. Instead, three other mechanisms are used that can bridge the gap between the two-dimensional image and knowledge of three-dimensional objects. First, a process of perceptual organization is used to form groupings and structures in the image that are likely to be invariant over a wide range of viewpoints. Second, a probabilistic ranking method is used to reduce the size of the search space during model based matching. Finally, a process of spatial correspondence brings the projections of three-dimensional models into direct correspondence with the image by solving for unknown viewpoint and model parameters. A high level of robustness in the presence of occlusion and missing data can be achieved through full application of a viewpoint consistency constraint. It is argued that similar mechanisms and constraints form the basis for recognition in human vision. This paper has been published in Artificial Intelligence, 31, 3 (March 1987), pp. 355–395.

We introduce the challenge problem for generic video indexing to gain insight in intermediate steps that affect performance of multimedia analysis methods, while at the same time fostering repeatability of experiments. To arrive at a challenge problem, we provide a general scheme for the systematic examination of automated concept detection methods, by decomposing the generic video indexing problem into 2 unimodal analysis experiments, 2 multimodal analysis experiments, and 1 combined analysis experiment. For each experiment, we evaluate generic video indexing performance on 85 hours of international broadcast news data, from the TRECVID 2005/2006 benchmark, using a lexicon of 101 semantic concepts. By establishing a minimum performance on each experiment, the challenge problem allows for component-based optimization of the generic indexing issue, while simultaneously offering other researchers a reference for comparison during indexing methodology development. To stimulate further investigations in intermediate analysis steps that inuence video indexing performance, the challenge offers to the research community a manually annotated concept lexicon, pre-computed low-level multimedia features, trained classifier models, and five experiments together with baseline performance, which are all available at http://www.mediamill.nl/challenge/.

An Augmented Virtual Environment (AVE) fuses dynamic imagery with 3D models. The AVE provides a unique approach to visualize and comprehend multiple streams of temporal data or images. Models are used as a 3D substrate for the visualization of temporal imagery, providing improved comprehension of scene activities. The core elements of AVE systems include model construction, sensor tracking, real-time video/image acquisition, and dynamic texture projection for 3D visualization. This paper focuses on the integration of these components and the results that illustrate the utility and benefits of the resulting augmented virtual environment.

This paper proposes a new framework for semantic segmentation of objects in videos. We address the label inconsistency problem of deep convolutional neural networks (DCNNs) by exploiting the fact that videos have multiple frames; in a few frames the object is confidently-estimated (CE) and we use the information in them to improve labels of the other frames. Given the semantic segmentation results of each frame obtained from DCNN, we sample several CE frames to adapt the DCNN model to the input video by focusing on specific instances in the video rather than general objects in various circumstances. We propose offline and online approaches under different supervision levels. In experiments our method achieved great improvement over the original model and previous state-of-the-art methods. c © 2016 Elsevier Ltd. All rights reserved.

Volumetric 3D displays are frequently purported to lack the ability to reconstruct scenes with viewer-position-dependent effects such as occlusion. To counter these claims, a swept-screen 198-view horizontal-parallax-only 3D display is reported here that is capable of viewer-position-dependent effects. A digital projector illuminates a rotating vertical diffuser with a series of multiperspective 768 x 768 pixel renderings of a 3D scene. Evidence of near-far object occlusion is reported. The aggregate virtual screen surface for a stationary observer is described, as are guidelines to construct a full-parallax system and the theoretical ability of the present system to project imagery outside of the volume swept by the screen.

We present a method to replace a user specified target region of a photograph by using other photographs of the same scene downloaded from the Internet via viewpoint invariant image search. Each of the retrieved images is first geometrically then photometrically registered with the query photograph. Geometric registration is achieved using multiple homographies and photometric registration is performed using a global affine transformation on image intensities. Each retrieved image proposes a possible solution for the target region. In the final step we combine these proposals into a single visually consistent result, using a Markov random field optimisation to choose seams between proposals, followed by gradient domain fusion. We demonstrate removal of objects and people in challenging photographs of Oxford landmarks containing complex image structures.


Question: 4D Generic Video Object Suggestions

This paper introduces an innovative technique for unsupervised learning from raw spatiotemporal data in videos, focusing on the development of a powerful visual representation without the need for semantic labels. The method is grounded in an unsupervised sequential verification task, specifically designed to ascertain the correct temporal sequence of video frames. Through this straightforward yet effective approach, a Convolutional Neural Network (CNN) is trained to learn a visual representation that is rich in information distinct from that obtained from supervised image datasets such as ImageNet. Qualitative outcomes reveal that the learned representation is adept at capturing dynamic elements like human poses, demonstrating its sensitivity to temporal variations. When applied as pre-training for action recognition tasks, the technique significantly outperforms models trained without external data on established benchmarks like UCF101 and HMDB51. Furthermore, its proficiency in discerning human pose is showcased through competitive, if not superior, results for pose estimation on datasets such as FLIC and MPII, even when compared to methods utilizing more extensive supervision. The effectiveness of combining the learned representation with supervised ones to further enhance accuracy is also highlighted.

The accuracy of optical flow estimation algorithms has been improving steadily as evidenced by results on the Middlebury optical flow benchmark. The typical formulation, however, has changed little since the work of Horn and Schunck. We attempt to uncover what has made recent advances possible through a thorough analysis of how the objective function, the optimization method, and modern implementation practices influence accuracy. We discover that “classical” flow formulations perform surprisingly well when combined with modern optimization and implementation techniques. One key implementation detail is the median filtering of intermediate flow fields during optimization. While this improves the robustness of classical methods it actually leads to higher energy solutions, meaning that these methods are not optimizing the original objective function. To understand the principles behind this phenomenon, we derive a new objective function that formalizes the median filtering heuristic. This objective function includes a non-local smoothness term that robustly integrates flow estimates over large spatial neighborhoods. By modifying this new term to include information about flow and image boundaries we develop a method that can better preserve motion details. To take advantage of the trend towards video in wide-screen format, we further introduce an asymmetric pyramid downsampling scheme that enables the estimation of longer range horizontal motions. The methods are evaluated on the Middlebury, MPI Sintel, and KITTI datasets using the same parameter settings.

With the advent of perceptual loss functions, new possibilities in super-resolution have emerged, and we currently have models that successfully generate near-photorealistic high-resolution images from their low-resolution observations. Up to now, however, such approaches have been exclusively limited to single image super-resolution. The application of perceptual loss functions on video processing still entails several challenges, mostly related to the lack of temporal consistency of the generated images, i.e., flickering artifacts. In this work, we present a novel adversarial recurrent network for video upscaling that is able to produce realistic textures in a temporally consistent way. The proposed architecture naturally leverages information from previous frames due to its recurrent architecture, i.e. the input to the generator is composed of the low-resolution image and, additionally, the warped output of the network at the previous step. We also propose an additional loss function to further reinforce temporal consistency in the generated sequences. The experimental validation of our algorithm shows the effectiveness of our approach which obtains competitive samples in terms of perceptual quality with improved temporal consistency.

We describe how to create with machine learning techniques a generative, speech animation module. A human subject is first recorded using a videocamera as he/she utters a predetermined speech corpus. After processing the corpus automatically, a visual speech module is learned from the data that is capable of synthesizing the human subject's mouth uttering entirely novel utterances that were not recorded in the original video. The synthesized utterance is re-composited onto a background sequence which contains natural head and eye movement. The final output is videorealistic in the sense that it looks like a video camera recording of the subject. At run time, the input to the system can be either real audio sequences or synthetic audio produced by a text-to-speech system, as long as they have been phonetically aligned.The two key contributions of this paper are 1) a variant of the multidimensional morphable model (MMM) to synthesize new, previously unseen mouth configurations from a small set of mouth image prototypes; and 2) a trajectory synthesis technique based on regularization, which is automatically trained from the recorded video corpus, and which is capable of synthesizing trajectories in MMM space corresponding to any desired utterance.

Many algorithms for both identifying and reconstructing a 3-D object are based on the 2-D silhouettes of the object. In general, identifying a nonconvex object using a silhouettebased approach implies neglecting some features of its surface as identification clues. The same features cannot be reconstructed by volume intersection techniques using multiple silhouettes of the object. This paper addresses the problem of finding which parts of a nonconvex object are relevant for silhouette-based image understanding. For this purpose, the geometric concept of visual hull of a 3-D object is introduced. The visual hull of a 3-D object S is the closest approximation of S that can be obtained with the volume intersection approach. An equivalent statement, relative to object identification, is that the visual hull of S is the maximal object silhouette-equivalent to S, i.e., which can be substituted for S without affecting any silhouette. Only the parts of the surface of S that also lie on the surface of the visual hull can be reconstructed or identified using silhouette-based algorithms. The visual hull of an object depends not only on the object itself but also on the region allowed to the viewpoint. Two main viewing regions can be considered, resulting in the external and internal visual hull. In the former case the viewing region is related to the convex hull of S, in the latter it is bounded by S itself. The internal visual hull also admits an interpretation not related to silhouettes: the features of the surface of S that is not coincident with the surface of the internal visual hull cannot be observed from any viewpoint lying outside the convex hull. After a general discussion of the visual hull and its properties, algorithms for computing the internal and external visual hulls of 2-D objects and 3-D planar face objects are presented and their complexity analyzed. In general, the visual hull of a 3-D planar face object turns out to be bounded by planar and curved patches. A precise statement of the concept of visual hull appears to be novel, as is the problem of its computation.

DTAM is a system for real-time camera tracking and reconstruction which relies not on feature extraction but dense, every pixel methods. As a single hand-held RGB camera flies over a static scene, we estimate detailed textured depth maps at selected keyframes to produce a surface patchwork with millions of vertices. We use the hundreds of images available in a video stream to improve the quality of a simple photometric data term, and minimise a global spatially regularised energy functional in a novel non-convex optimisation framework. Interleaved, we track the camera's 6DOF motion precisely by frame-rate whole image alignment against the entire dense model. Our algorithms are highly parallelisable throughout and DTAM achieves real-time performance using current commodity GPU hardware. We demonstrate that a dense model permits superior tracking performance under rapid motion compared to a state of the art method using features; and also show the additional usefulness of the dense model for real-time scene interaction in a physics-enhanced augmented reality application.

3-D convolutional neural networks (3-D-convNets) have been very recently proposed for action recognition in videos, and promising results are achieved. However, existing 3-D-convNets has two “artificial” requirements that may reduce the quality of video analysis: 1) It requires a fixed-sized (e.g., 112  $\times$ 112) input video; and 2) most of the 3-D-convNets require a fixed-length input (i.e., video shots with fixed number of frames). To tackle these issues, we propose an end-to-end pipeline named Two-stream 3-D-convNet Fusion, which can recognize human actions in videos of arbitrary size and length using multiple features. Specifically, we decompose a video into spatial and temporal shots. By taking a sequence of shots as input, each stream is implemented using a spatial temporal pyramid pooling (STPP) convNet with a long short-term memory (LSTM) or CNN-E model, softmax scores of which are combined by a late fusion. We devise the STPP convNet to extract equal-dimensional descriptions for each variable-size shot, and we adopt the LSTM/CNN-E model to learn a global description for the input video using these time-varying descriptions. With these advantages, our method should improve all 3-D CNN-based video analysis methods. We empirically evaluate our method for action recognition in videos and the experimental results show that our method outperforms the state-of-the-art methods (both 2-D and 3-D based) on three standard benchmark datasets (UCF101, HMDB51 and ACT datasets).

This paper proposes a quasi-dense reconstruction from uncalibrated sequence. The main innovation is that all geometry is computed based on re-sampled quasi-dense correspondences rather than the standard sparse points of interest. It not only produces more accurate and robust reconstruction due to highly redundant and well spread input data, but also fills the gap of insufficiency of sparse reconstruction for visualization application. The computational engine is the quasi-dense 2-view and the quasi-dense 3-view algorithms developed in this paper. Experiments on real sequences demonstrate the superior performance of quasi-dense w.r.t. sparse reconstruction both in accuracy and robustness.

The topics of computer animation and object collision detection and resolution have become increasingly desirable in a wide range of applications including autonomous vehicle navigation, robotics, and the movie industry. However, the techniques currently available are typically difficult to use or are restricted to a particular domain. This paper presents a technique for providing automatic animation and collision avoidance of arbitrary objects in a computer graphics system. The underlying construct used in this process is a surrounding volume octree vector field. The system automatically generates these structures around objects in the scene. By judicious creation and use of these vector fields, objects in the scene move and interact, but do not collide. The manner in which these vector fields are created is given. Two applications cloud movement over terrain and autonomous aircraft navigation are presented which show typical usage of this technique.

Recognizing how objects interact with each other is a crucial task in visual recognition. If we define the context of the interaction to be the objects involved, then most current methods can be categorized as either: (i) training a single classifier on the combination of the interaction and its context; or (ii) aiming to recognize the interaction independently of its explicit context. Both methods suffer limitations: the former scales poorly with the number of combinations and fails to generalize to unseen combinations, while the latter often leads to poor interaction recognition performance due to the difficulty of designing a contextindependent interaction classifier.,,To mitigate those drawbacks, this paper proposes an alternative, context-aware interaction recognition framework. The key to our method is to explicitly construct an interaction classifier which combines the context, and the interaction. The context is encoded via word2vec into a semantic space, and is used to derive a classification result for the interaction. The proposed method still builds one classifier for one interaction (as per type (ii) above), but the classifier built is adaptive to context via weights which are context dependent. The benefit of using the semantic space is that it naturally leads to zero-shot generalizations in which semantically similar contexts (subject-object pairs) can be recognized as suitable contexts for an interaction, even if they were not observed in the training set. Our method also scales with the number of interaction-context pairs since our model parameters do not increase with the number of interactions. Thus our method avoids the limitation of both approaches. We demonstrate experimentally that the proposed framework leads to improved performance for all investigated interaction representations and datasets.

With the changes of eras, traditional movies may suffer continuous frame damage after digits due to improper preservation. In order to solve this problem, we proposed a new inpainting technique for continuous video sequences based on deep neural networks. We introduced the latest image restoration techniques for inpainting key frames of new scenes. Then we use the deep neural network interpolation algorithm to interpolate the intermediate frames, so that the video can achieve a coherent effect in time. In order to preserve the original information as much as possible, we only replace the damaged areas and preserve most of undamaged areas, and finally perform image blending. We test different types of videos and compare them with other methods. Our method has higher quality video inpainting than existing methods.


Question: 4D generic video object suggestions.

This paper presents an enhanced version of the Mask R-CNN framework, specifically tailored for the generation of 4D video object proposals. The proposed method extends the capabilities of Mask R-CNN, which is renowned for its effectiveness in object detection and segmentation in images, to the temporal domain, making it suitable for video analysis. By incorporating a temporal component, the adapted Mask R-CNN is able to track objects across frames, generating more accurate and contextually relevant object proposals. This is achieved through the integration of a recurrent neural network (RNN) module, which allows the model to maintain a memory of previous frames, thereby improving its ability to predict object locations in subsequent frames. The enhanced Mask R-CNN is evaluated on a variety of video datasets, demonstrating its superiority in generating object proposals that are consistent over time, making it a powerful tool for video object detection and tracking applications.

The ability to replace missing data in images and video is of key importance to many application fields. The general-purpose algorithm presented here is inspired by texture synthesis techniques but is suited to any complex natural scene and not restricted to stationary patterns. It has the property to be adapted to both still images and image sequences and to incorporate temporal information when available while preserving the simplicity of the algorithm. This method gives very good results in various situations without user intervention. The resulting computational cost is relatively low and corrections are usually produced within seconds.

We present a viewpoint-based approach for the quick fusion of multiple stereo depth maps. Our method selects depth estimates for each pixel that minimize violations of visibility constraints and thus remove errors and inconsistencies from the depth maps to produce a consistent surface. We advocate a two-stage process in which the first stage generates potentially noisy, overlapping depth maps from a set of calibrated images and the second stage fuses these depth maps to obtain an integrated surface with higher accuracy, suppressed noise, and reduced redundancy. We show that by dividing the processing into two stages we are able to achieve a very high throughput because we are able to use a computationally cheap stereo algorithm and because this architecture is amenable to hardware-accelerated (GPU) implementations. A rigorous formulation based on the notion of stability of a depth estimate is presented first. It aims to determine the validity of a depth estimate by rendering multiple depth maps into the reference view as well as rendering the reference depth map into the other views in order to detect occlusions and free- space violations. We also present an approximate alternative formulation that selects and validates only one hypothesis based on confidence. Both formulations enable us to perform video-based reconstruction at up to 25 frames per second. We show results on the multi-view stereo evaluation benchmark datasets and several outdoors video sequences. Extensive quantitative analysis is performed using an accurately surveyed model of a real building as ground truth.

This work proposes a learning method for deep architectures that takes advantage of sequential data, in particular from the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. This coherence is used as a supervisory signal over the unlabeled data, and is used to improve the performance on a supervised task of interest. We demonstrate the effectiveness of this method on some pose invariant object and face recognition tasks.

This paper presents a system which yields complete 3D models in a fast and inexpensive way. The major building blocks are a high-speed structured light range scanner and a registration module. The former generates ’raw’ range data whereas the latter performs a fast registration (ICP) and renders the partially integrated model as a preview of the final result. As the scanner uses only a single image to make a reconstruction, it is possible to scan while the object is moved. The use of an adaptive projection pattern gives a more robust behaviour. This allows the system to deal more easily with complicated geometry and texture than most other systems. During the scanning process the model is built up incrementally and rendered on the screen. This real-time visual feedback allows the user to check the current state of the 3D model. Holes or other flaws can be detected during the acquisition process itself. This speeds up the model building process, and solves indirectly the problem of view planning. The whole real-time pipeline comprising acquisition, merging and visualization only uses off-the-shel f hardware. A regular desktop PC connected to a camera and LCD projector is turned into a high-speed scanner and modeler. The current implementation has a throughput of approx. 5 fps.

The majority of methods for the automatic surface reconstruction of an environment from an image sequence have two steps: Structure-from-Motion and dense stereo. From the computational standpoint, it would be interesting to avoid dense stereo and to generate a surface directly from the sparse cloud of 3D points and their visibility information provided by Structure-fromMotion. The previous attempts to solve this problem are currently very limited: the surface is non-manifold or has zero genus, the experiments are done on small scenes or objects using a few dozens of images. Our solution does not have these limitations. Furthermore, we experiment with hand-held or helmet-held catadioptric cameras moving in a city and generate 3D models such that the camera trajectory can be longer than one kilometer.

We present a new method for generating a dynamic, concatenative, unit of visual speech that can generate realistic visual speech animation. We redefine visemes as temporal units that describe distinctive speech movements of the visual speech articulators. Traditionally visemes have been surmized as the set of static mouth shapes representing clusters of contrastive phonemes (e.g. /p, b, m/, and /f, v/). In this work, the motion of the visual speech articulators are used to generate discrete, dynamic visual speech gestures. These gestures are clustered, providing a finite set of movements that describe visual speech, the visemes. Dynamic visemes are applied to speech animation by simply concatenating viseme units. We compare to static visemes using subjective evaluation. We find that dynamic visemes are able to produce more accurate and visually pleasing speech animation given phonetically annotated audio, reducing the amount of time that an animator needs to spend manually refining the animation.

An unsupervised video object segmentation algorithm, which discovers a primary object in a video sequence automatically, is proposed in this work. We introduce three energies in terms of foreground and background probability distributions: Markov, spatiotemporal, and antagonistic energies. Then, we minimize a hybrid of the three energies to separate a primary object from its background. However, the hybrid energy is nonconvex. Therefore, we develop the alternate convex optimization (ACO) scheme, which decomposes the nonconvex optimization into two quadratic programs. Moreover, we propose the forward-backward strategy, which performs the segmentation sequentially from the first to the last frames and then vice versa, to exploit temporal correlations. Experimental results on extensive datasets demonstrate that the proposed ACO algorithm outperforms the state-of-the-art techniques significantly.

We present the first marker-less approach for temporally coherent 3D performance capture of a human with general clothing from monocular video. Our approach reconstructs articulated human skeleton motion as well as medium-scale non-rigid surface deformations in general scenes. Human performance capture is a challenging problem due to the large range of articulation, potentially fast motion, and considerable non-rigid deformations, even from multi-view data. Reconstruction from monocular video alone is drastically more challenging, since strong occlusions and the inherent depth ambiguity lead to a highly ill-posed reconstruction problem. We tackle these challenges by a novel approach that employs sparse 2D and 3D human pose detections from a convolutional neural network using a batch-based pose estimation strategy. Joint recovery of per-batch motion allows to resolve the ambiguities of the monocular reconstruction problem based on a low dimensional trajectory subspace. In addition, we propose refinement of the surface geometry based on fully automatically extracted silhouettes to enable medium-scale non-rigid alignment. We demonstrate state-of-the-art performance capture results that enable exciting applications such as video editing and free viewpoint video, previously infeasible from monocular video. Our qualitative and quantitative evaluation demonstrates that our approach significantly outperforms previous monocular methods in terms of accuracy, robustness and scene complexity that can be handled.

Recent research on the human visual system shows that our perception of object shape relies in part on compression and stretching of the reflected lighting environment onto its surface. We use this property to enhance the shape depiction of 3D objects by locally warping the environment lighting around main surface features. Contrary to previous work, which require specific illumination, material characteristics and/or stylization choices, our approach enhances surface shape without impairing the desired appearance.
 Thanks to our novel local shape descriptor, salient surface features are explicitly extracted in a view-dependent fashion at various scales without the need of any pre-process. We demonstrate our system on a variety of rendering settings, using object materials ranging from diffuse to glossy, to mirror or refractive, with direct or global illumination, and providing styles that range from photorealistic to non-photorealistic. The warping itself is very fast to compute on modern graphics hardware, enabling real-time performance in direct illumination scenarios.
 Note: Third-Party Material Attribution  Third-party material used in ACM Transactions on Graphics 28(3), Article 25 - "Light Warping for Enhanced Surface Depiction," by Vergne, Pacanowski, Barla, Granier, and Schlick - was used without proper attribution.
 The 3D model used in Figures 1, 3, and 5, as well as in the cover image of this volume of the journal, was downloaded from the Shape Repository of AIM@SHAPE Project (http://shapes.aimatshape.net) and is the property of CNR-IMATI.
 We regret this oversight.

We propose a new algorithm for the reliable detection and localization of video copy–move forgeries. Discovering well-crafted video copy–moves may be very difficult, especially when some uniform background is copied to occlude foreground objects. To reliably detect both additive and occlusive copy–moves, we use a dense-field approach, with invariant features that guarantee robustness to several postprocessing operations. To limit complexity, a suitable video-oriented version of PatchMatch is used, with a multiresolution search strategy, and a focus on volumes of interest. Performance assessment relies on a new dataset, designed ad hoc, with realistic copy–moves and a wide variety of challenging situations. Experimental results show the proposed method to detect and localize video copy–moves with good accuracy even in adverse conditions.


Question: A framework for environmental energy harvesting in sensor networks.

This paper addresses the challenge of topology control in multihop wireless networks by adjusting the transmit powers of nodes to achieve a desired network structure. The problem is approached through a constrained optimization framework, targeting two key constraints: connectivity and biconnectivity, while minimizing the maximum power used. Two centralized algorithms are proposed for static network scenarios, and their optimality is rigorously proven. For dynamic or mobile networks, two distributed heuristics are introduced. These heuristics dynamically adjust node transmit powers in response to changes in the network topology, aiming to maintain a connected and power-efficient topology. The performance of these algorithms is evaluated through software implementation, emulating a power-controllable radio and utilizing a detailed channel model. Results demonstrate that the effective use of topology control can significantly enhance the throughput, reduce delay, and optimize power consumption in multihop wireless networks, thereby improving overall network performance.

This paper proposes a new method for power telemetry and wireless data transfer to biomedical implants. Confinement of the transferred energy within a predefined area, hence not influencing the neighboring sensitive circuitry on the microsystem, and also allowing for more than one telemetry link on the same platform and even operating at the same carrier frequency are among the advantages of the proposed method over the traditional approaches for power and data telemetry.

In smart cities, wireless sensor networks (WSNs) act as a type of core infrastructure that collects data from the city to implement smart services. The security of WSNs is one of the key issues of smart cities. In resource-restrained WSNs, dynamic ongoing or unknown attacks usually steer clear of isolated defense components. Therefore, to resolve this problem, we propose a hierarchical framework based on chance discovery and usage control (UCON) technologies to improve the security of WSNs while still taking the low-complexity and high security requirements of WSNs into account. The features of continuous decision and dynamic attributes in UCON can address ongoing attacks using advanced persistent threat detection. In addition, we use a dynamic adaptive chance discovery mechanism to detect unknown attacks. To design and implement a system using the mechanism described above, a unified framework is proposed in which low-level attack detection with simple rules is performed in sensors, and high-level attack detection with complex rules is performed in sinks and at the base station. Moreover, software-defined networking and network function virtualization technologies are used to perform attack mitigation when either low-level or high-level attacks are detected. An experiment was performed to acquire an attack data set for evaluation. Then, a simulation was created to evaluate the resource consumption and attack detection rate. The results demonstrate the feasibility and efficiency of the proposed scheme.

As wireless sensor networks (WSN) continue to grow, so does the need for effective security mechanisms. Since sensor networks can interact with sensitive data and/or operate in hostile unattended environments, it is imperative that these security concerns be addressed from the beginning of the system design. This paper aims at describing security solutions for collecting and processing data in WSNs. Adequate security capabilities for medium and large scale WSNs are a hard but necessary goal to achieve to prepare these networks for the market. The paper includes an overview on WSNs space security solutioins and reliability challenges for

This paper presents a multi-hop localization technique for WSNs exploiting acquired received signal strength indications. The proposed system aims at providing an effective solution for the self localization of nodes in static/semi-static wireless sensor networks without requiring previous deployment information

Wireless Body Area Network (WBAN) has been a key element in e-health to monitor bodies. This technology enables new applications under the umbrella of different domains, including the medical field, the entertainment and ambient intelligence areas. This survey paper places substantial emphasis on the concept and key features of the WBAN technology. First, the WBAN concept is introduced and a review of key applications facilitated by this networking technology is provided. The study then explores a wide variety of communication standards and methods deployed in this technology. Due to the sensitivity and criticality of the data carried and handled by WBAN, fault tolerance is a critical issue and widely discussed in this paper. Hence, this survey investigates thoroughly the reliability and fault tolerance paradigms suggested for WBANs. Open research and challenging issues pertaining to fault tolerance, coexistence and interference management and power consumption are also discussed along with some suggested trends in these aspects.

Wireless sensor network (WSN) has emerged as one of the most promising technologies for the future. This has been enabled by advances in technology and availability of small, inexpensive, and smart sensors resulting in cost effective and easily deployable WSNs. However, researchers must address a variety of challenges to facilitate the widespread deployment of WSN technology in real-world domains. In this survey, we give an overview of wireless sensor networks and their application domains including the challenges that should be addressed in order to push the technology further. Then we review the recent technologies and testbeds for WSNs. Finally, we identify several open research issues that need to be investigated in future. Our survey is different from existing surveys in that we focus on recent developments in wireless sensor network technologies. We review the leading research projects, standards and technologies, and platforms. Moreover, we highlight a recent phenomenon in WSN research that is to explore synergy between sensor networks and other technologies and explain how this can help sensor networks achieve their full potential. This paper intends to help new researchers entering the domain of WSNs by providing a comprehensive survey on recent developments.

Recent advances in microelectronics and integrated circuits, system-on-chip design, wireless communication and intelligent low-power sensors have allowed the realization of a Wireless Body Area Network (WBAN). A WBAN is a collection of low-power, miniaturized, invasive/non-invasive lightweight wireless sensor nodes that monitor the human body functions and the surrounding environment. In addition, it supports a number of innovative and interesting applications such as ubiquitous healthcare, entertainment, interactive gaming, and military applications. In this paper, the fundamental mechanisms of WBAN including architecture and topology, wireless implant communication, low-power Medium Access Control (MAC) and routing protocols are reviewed. A comprehensive study of the proposed technologies for WBAN at Physical (PHY), MAC, and Network layers is presented and many useful solutions are discussed for each layer. Finally, numerous WBAN applications are highlighted.

The quest for sustainable energy models is the main factor driving research on smart grid technology. SGs represent the bridging paradigm to enable highly efficient energy production, transport, and consumption along the whole chain, from the source to the user. Although this concept promises to be very fruitful, the research on how to deploy it in the real world has just begun. A discussion on the enabling technologies for SGs and a possible roadmap for the profitable evolution thereof is the focus of this article. After introducing the recent trends that are pushing the SG paradigm, we will discuss various key scenarios for the SG, and briefly introduce some of its key requirements. We will then provide an analysis of how current and future standard solutions in the areas of communications and networking can be engineered into a system that fulfills the needs of the SG vision. We advocate the use of small, cheap, and resource-constrained devices with pervasive computing capabilities as the key component to deploy a ubiquitous energy control system. To this end, the recent efforts carried out by Internet standardization bodies such as the IETF and W3C toward the vision of the Internet of Things (IoT) are especially relevant. The various components of the proposed solution have been successfully showcased in real-world implementations, and relevant actors such as ETSI, ZigBee, and IPSO are already evaluating their potential for future IoT applications, making the Internet-based smart grid vision considered in this article practically achievable in the not too distant future.

Many sensor networks have been deployed to monitor Earth's environment, and more will follow in the future. Environmental sensors have improved continuously by becoming smaller, cheaper, and more intelligent. Due to the large number of sensor manufacturers and differing accompanying protocols, integrating diverse sensors into observation systems is not straightforward. A coherent infrastructure is needed to treat sensors in an interoperable, platform-independent and uniform way. The concept of the Sensor Web reflects such a kind of infrastructure for sharing, finding, and accessing sensors and their data across different applications. It hides the heterogeneous sensor hardware and communication protocols from the applications built on top of it. The Sensor Web Enablement initiative of the Open Geospatial Consortium standardizes web service interfaces and data encodings which can be used as building blocks for a Sensor Web. This article illustrates and analyzes the recent developments of the new generation of the Sensor Web Enablement specification framework. Further, we relate the Sensor Web to other emerging concepts such as the Web of Things and point out challenges and resulting future work topics for research on Sensor Web Enablement.

This paper gives a state-of-art of wireless sensor network (WSN) technologies and solutions applied to precision agriculture (PA). The paper first considers applications and existing experiences that show how WSN technologies have been introduced in to agricultural applications. Then, a survey in hardware and software solutions is related with special emphasis on technological aspects. Finally, the paper shows how five networking and technological solutions may impact the next generation of sensors. These are (i) scalar wireless sensor networks, (ii) wireless multimedia sensor networks, (iii) Mobility of nodes, (iv) tag-based systems, and (v) smart-phone applications.


Question: A framework for environmental energy harvesting in sensor networks.

This paper delves into the directed diffusion paradigm as a viable and efficient communication strategy for sensor networks, particularly in the context of environmental monitoring. With the rapid advancements in processor, memory, and radio technologies, the deployment of compact and cost-effective sensor nodes has become feasible. These nodes, capable of sensing, communication, and computation, can collaboratively monitor environmental conditions. Directed diffusion stands out in this context by being data-centric, meaning all network communication revolves around the acquisition and dissemination of specific data. Every node in a directed diffusion network is application-aware, enabling the network to optimize energy consumption through the selection of optimal data paths, in-network caching, and local data processing. The paper demonstrates the effectiveness of directed diffusion through its application in a basic remote-surveillance sensor network, highlighting its scalability and robustness.

The quest for sustainable energy models is the main factor driving research on smart grid technology. SGs represent the bridging paradigm to enable highly efficient energy production, transport, and consumption along the whole chain, from the source to the user. Although this concept promises to be very fruitful, the research on how to deploy it in the real world has just begun. A discussion on the enabling technologies for SGs and a possible roadmap for the profitable evolution thereof is the focus of this article. After introducing the recent trends that are pushing the SG paradigm, we will discuss various key scenarios for the SG, and briefly introduce some of its key requirements. We will then provide an analysis of how current and future standard solutions in the areas of communications and networking can be engineered into a system that fulfills the needs of the SG vision. We advocate the use of small, cheap, and resource-constrained devices with pervasive computing capabilities as the key component to deploy a ubiquitous energy control system. To this end, the recent efforts carried out by Internet standardization bodies such as the IETF and W3C toward the vision of the Internet of Things (IoT) are especially relevant. The various components of the proposed solution have been successfully showcased in real-world implementations, and relevant actors such as ETSI, ZigBee, and IPSO are already evaluating their potential for future IoT applications, making the Internet-based smart grid vision considered in this article practically achievable in the not too distant future.

This paper presents a novel ultra-low power rectifier for RF energy harvester, designed and implemented in standard 130 nm CMOS technology. The proposed 915 MHz ISM band RF energy harvester is designed for wearable medical devices and internet of things (IoT) applications. An off-chip differential matching network passively boosts the low-level incoming AC signal generated by the antenna. Then, a novel self-compensated cross-coupled rectifier is designed to convert the AC signal into a DC output voltage. The rectifier is comprised of 10 stages and it uses both dynamic and static bias compensation to decrease the transistors forward voltage drop. The post-layout simulation results demonstrate a sensitivity of −30.5 dBm for 1 V output at a capacitive load which is lower than the current state-of-the-art. The peak end-to-end efficiency is 42.8 % at −16 dBm input power, delivering 2.32 V at 0.5 MΩ resistor load.

This letter presents a compact fractal loop rectenna for RF energy harvesting at GSM1800 bands. First, a fractal loop antenna with novel in-loop ground-plane impedance matching is proposed for the rectenna design. Also, a high-efficiency rectifier is designed in the loop antenna to form a compact rectenna. Measured results show that an efficiency of 61% and an output dc voltage of 1.8 V have been achieved over 12-kΩ resistor for 10 μW/cm2 power density at 1.8 GHz. The rectenna is able to power up a battery-less LCD watch at a distance of 10 m from the cell tower. The proposed rectenna is compact, easy to fabricate, and useful for various energy harvesting applications.

Intrusion detection is a surveillance problem of practical import that is well suited to wireless sensor networks. In this paper, we study the application of sensor networks to the intrusion detection problem and the related problems of classifying and tracking targets. Our approach is based on a dense, distributed, wireless network of multi-modal resource-poor sensors combined into loosely coherent sensor arrays that perform in situ detection, estimation, compression, and exfiltration. We ground our study in the context of a security scenario called “A Line in the Sand” and accordingly define the target, system, environment, and fault models. Based on the performance requirements of the scenario and the sensing, communication, energy, and computation ability of the sensor network, we explore the design space of sensors, signal processing algorithms, communications, networking, and middleware services. We introduce the influence field, which can be estimated from a network of binary sensors, as the basis for a novel classifier. A contribution of our work is that we do not assume a reliable network; on the contrary, we quantitatively analyze the effects of network unreliability on application performance. Our work includes multiple experimental deployments of over 90 sensors nodes at MacDill Air Force Base in Tampa, Florida, as well as other field experiments of comparable scale. Based on these experiences, we identify a set of key lessons and articulate a few of the challenges facing extreme scaling to tens or hundreds of thousands of sensor nodes.

In smart cities, wireless sensor networks (WSNs) act as a type of core infrastructure that collects data from the city to implement smart services. The security of WSNs is one of the key issues of smart cities. In resource-restrained WSNs, dynamic ongoing or unknown attacks usually steer clear of isolated defense components. Therefore, to resolve this problem, we propose a hierarchical framework based on chance discovery and usage control (UCON) technologies to improve the security of WSNs while still taking the low-complexity and high security requirements of WSNs into account. The features of continuous decision and dynamic attributes in UCON can address ongoing attacks using advanced persistent threat detection. In addition, we use a dynamic adaptive chance discovery mechanism to detect unknown attacks. To design and implement a system using the mechanism described above, a unified framework is proposed in which low-level attack detection with simple rules is performed in sensors, and high-level attack detection with complex rules is performed in sinks and at the base station. Moreover, software-defined networking and network function virtualization technologies are used to perform attack mitigation when either low-level or high-level attacks are detected. An experiment was performed to acquire an attack data set for evaluation. Then, a simulation was created to evaluate the resource consumption and attack detection rate. The results demonstrate the feasibility and efficiency of the proposed scheme.

With recent advances in technology, various wearable sensors have been developed for the monitoring of human physiological parameters. A Body Sensor Network (BSN) consisting of such physiological and biomedical sensor nodes placed on, near or within a human body can be used for real-time health monitoring. In this paper, we describe an on-going effort to develop a system consisting of interconnected BSNs for real-time health monitoring of soldiers. We discuss the background and an application scenario for this project. We describe the preliminary prototype of the system and present a blast source localization application.

The satellite-based remote sensing technique has been widely used in monitoring wildfire spread. There are two prominent drawbacks with this approach of using satellites located in space: (1) very low sampling rate (temporal resolution problem) and (2) lack of accuracy (spatial and spectral resolution problem). To address these challenges, a wireless sensor network deployed at ground level with high-fidelity and low-altitude atmospheric sensing for wind speed of local wildfire spread has been used. An indirect approach in sensing wind speed has been proposed in this paper as an alternative to the bulky conventional wind anemometer to save cost and space. The wind speed is sensed by measuring the equivalent electrical output voltage of the wind turbine generator (WTG). The percentage error in the wind speed measurement using the proposed indirect method is measured to be well within the ±4% limit with respect to wind anemometer accuracy. The same WTG also functions as a wind energy harvesting (WEH) system to convert the available wind energy into electrical energy to sustain the operation of the wireless sensor node. The experimental results show that the designed WEH system is able to harvest an average electrical power of 7.7 mW at an average wind speed of 3.62 m/s for powering the operation of the wireless sensor node that consumes 3.5 mW for predicting the wildfire spread. Based on the sensed wind speed information, the fire control management system determines the spreading condition of the wildfire, and an adequate fire suppression action can be performed by the fire-fighting experts.

In wireless environments, transmission and reception costs dominate system power consumption, motivating research effort on new technologies capable of reducing the footprint of the radio, paving the way for the Internet of Things. The most important challenge is to reduce power consumption when receivers are idle, the so called idle-listening cost. One approach proposes switching off the main receiver, then introduces new wake-up circuitry capable of detecting an incoming transmission, optionally discriminating the packet destination using addressing, then switching on the main radio only when required. This wake-up receiver technology represents the ultimate frontier in low power radio communication. In this paper, we present a comprehensive literature review of the research progress in wake-up radio (WuR) hardware and relevant networking software. First, we present an overview of the WuR system architecture, including challenges to hardware design and a comparison of solutions presented throughout the last decade. Next, we present various medium access control and routing protocols as well as diverse ways to exploit WuRs, both as an extension of pre-existing systems and as a new concept to manage low-power networking.

Far-field wireless power transfer (WPT) is a promising technique to resolve the painstaking power-charging problem inherent in various wireless terminals. This paper investigates the power transfer efficiency of the WPT segment in future communication systems in support of simultaneous power and data transfer, by means of analytically computing the time-average output direct current (DC) power at user equipments (UEs). In order to investigate the effect of channel variety among UEs on the average output DC power, different policies for the scheduling of the power transfer among the users are implemented and compared in two scenarios: homogeneous, whereby users are symmetric and experience similar path loss, and heterogeneous, whereby users are asymmetric and exhibit different path losses. Specifically, if opportunistic scheduling is performed among N symmetric/asymmetric UEs, the power scaling laws are attained by using extreme value theory and reveal that the gain in power transfer efficiency is lnN if UEs are symmetric whereas the gain is N if UEs are asymmetric, compared with that of conventional round-robin scheduling. Thus, the channel variety among UEs inherent to the wireless environment can be exploited by opportunistic scheduling to significantly improve the power transfer efficiency when designing future wireless communication systems in support of simultaneous power and data transfer.

As wireless sensor networks continue to grow, so does the need for effective security mechanisms. Because sensor networks may interact with sensitive data and/or operate in hostile unattended environments, it is imperative that these security concerns be addressed from the beginning of the system design. However, due to inherent resource and computing constraints, security in sensor networks poses different challenges than traditional network/computer security. There is currently enormous research potential in the field of wireless sensor network security. Thus, familiarity with the current research in this field will benefit researchers greatly. With this in mind, we survey the major topics in wireless sensor network security, and present the obstacles and the requirements in the sensor security, classify many of the current attacks, and finally list their corresponding defensive measures.


Question: A framework for environmental energy harvesting in sensor networks.

This paper presents an innovative communication protocol designed to optimize energy efficiency in wireless microsensor networks, crucial for the continuous and dependable surveillance of various environments in both civilian and military contexts. Recognizing the limitations of traditional communication protocols such as direct transmission, minimum-transmission-energy, multihop routing, and static clustering, which may not be ideally suited for microsensor networks, we introduce LEACH (Low-Energy Adaptive Clustering Hierarchy). LEACH is a clustering-based protocol that employs a randomized rotation mechanism for local cluster base stations, or cluster-heads, to ensure an equitable distribution of energy consumption among the network's sensors. This approach not only enhances scalability and resilience in dynamic networks through localized coordination but also integrates data fusion into the routing protocol, thereby minimizing the volume of data transmitted to the central base station. Simulation results demonstrate that LEACH can achieve up to an eightfold reduction in energy consumption compared to conventional routing protocols. Moreover, LEACH effectively spreads energy consumption evenly across the sensor network, doubling the operational lifespan of the simulated networks.

Recent years have witnessed several new promising technologies to power wireless sensor networks, which motivate some key topics to be revisited. By integrating sensing and computation capabilities to the traditional radio-frequency identification (RFID) tags, the Wireless Identification and Sensing Platform (WISP) is an open-source platform acting as a pioneering experimental platform of wireless rechargeable sensor networks. Different from traditional tags, an RFID-based wireless rechargeable sensor node needs to charge its onboard energy storage above a threshold to power its sensing, computation, and communication components. Consequently, such charging delay imposes a unique design challenge for deploying wireless rechargeable sensor networks. In this paper, we tackle this problem by planning the optimal movement strategy of the mobile RFID reader, such that the time to charge all nodes in the network above their energy threshold is minimized. We first propose an optimal solution using the linear programming (LP) method. To further reduce the computational complexity, we then introduce a heuristic solution with a provable approximation ratio of (1 + θ)/(1 - ε) by discretizing the charging power on a 2-D space. Through extensive evaluations, we demonstrate that our design outperforms the set-cover-based design by an average of 24.7%, whereas the computational complexity is O((N/ε)2). Finally, we consider two practical issues in system implementation and provide guidelines for parameter setting.

Backscatter presents an emerging ultralow-power wireless communication paradigm. The ability to offer submilliwatt power consumption makes it a competitive core technology for Internet of Things (IoT) applications. In this article, we provide a tutorial of backscatter communication from the signal processing perspective as well as a survey of the recent research activities in this domain, primarily focusing on bistatic backscatter systems. We also discuss the unique real-world applications empowered by backscatter communication and identify open questions in this domain. We believe this article will shed light on the low-power wireless connectivity design toward building and deploying IoT services in the wild.

The current paper presents low-power analog integrated circuits (ICs) for wireless electrocardiogram (ECG) detection in personal health monitoring. Considering the power-efficient communication in the body sensor network (BSN), the required low-power analog ICs are developed for a healthcare system through miniaturization and system integration. The proposed system comprises the design and implementation with three subsystems, namely, (1) the ECG acquisition node, (2) the protocol for standard IEEE 802.15.4 ZigBee system, and (3) the radio frequency (RF) transmitter circuits. A preamplifier, a low-pass filter, and a successive-approximation analog-to-digital converter (SA-ADC) are integrated to detect an ECG signal. For high integration, the ZigBee protocol is adopted for wireless communication. To transmit the ECG signal through wireless communication, a quadrature voltage-controlled oscillator and a 2.4 GHz low-IF transmitter with a power amplifier and up-conversion mixer are also developed. In the receiver, a 2.4 GHz fully integrated CMOS radio-frequency front-end with a low-noise amplifier, and a quadrature mixer is proposed. The low-power wireless bio-signal acquisition SoC (WBSA-SoC) has been implemented in TSMC 0.18-μm standard CMOS process. The measurement results on the human body reveal that the ECG signals can be acquired effectively by the proposed SoC.

Real-world monitoring and surveillance applications are rapidly increasing with the growth in the demand for awareness of the environment and surrounding. Sensor-to-web integration models are envisaged to improve the utilisation of sensory infrastructures and data via advanced coordination and collaboration of sensor nodes and web applications. This paper proposes to employ micro-blogging to publish and share sensory data and resources. S-Sensors provides a framework to globally share locally measured sensory readings. Short messages depicting the status of the environment are used to convey sensory data of the physical world. Moreover, data accessibility and resource manageability are facilitated by using social networks paradigms to form communities of sensor networks.

Wired networks are increasing in size and their power consumption is becoming a matter of concern. Evaluating the end-to-end electrical cost of new network architectures and protocols is difficult due to the lack of monitored realistic infrastructures. We propose an End-to-End energy Cost mOdel and simulator For Evaluating power consumption in large-scale Networks (ECOFEN) whose user's entries are the network topology and traffic. Based on configurable measurement of different network components (routers, switches, NICs, etc.), it provides the power consumption of the overall network including the end-hosts as well as the power consumption of each equipment over time.

Declarative queries are proving to be an attractive paradigm for interacting with networks of wireless sensors. The metaphor that “the sensornet is a database” is problematic, however, because sensors do not exhaustively represent the data in the real world. In order to map the raw sensor readings onto physical reality, a modelof that reality is required to complement the readings. In this paper, we enrich interactive sensor querying with statistical modeling techniques. We demonstrate that such models can help provide answers that are both more meaningful, and, by introducing approximations with probabilistic confidences, significantly more efficient to compute in both time and energy. Utilizing the combination of a model and live data acquisition raises the challenging optimization problem of selecting the best sensor readings to acquire, balancing the increase in the confidence of our answer against the communication and data acquisition costs in the network. We describe an exponential time algorithm for finding the optimal solution to this optimization problem, and a polynomial-time heuristic for identifying solutions that perform well in practice. We evaluate our approach on several real-world sensor-network data sets, taking into account the real measured data and communication quality, demonstrating that our model-based approach provides a high-fidelity representation of the real phenomena and leads to significant performance gains versus traditional data acquisition techniques.

The full degree of freedom in mobile systems heavily depends on the energy provided by the mobile phone's batteries. Their capacity is in general limited and for sure not keeping pace as the mobile devices are crammed up with new functionalities. The discrepancy of Moore's law, offering twice the processing power at least each second year, and the development in batteries, which did not even double over the last decade, makes a shift in researchers' way of designing networks, protocols, and the mobile device itself. The bottleneck to take care of in the design process of mobile systems is not only the wireless data rate, but even more the energy limitation as the customers ask for new energy-hungry services, e.g., requiring faster connections or even multiple air interfaces, and longer standby or operational times of their mobile devices at the same time. In this survey, the energy consuming entities of a mobile device such as wireless air interfaces, display, mp3 player and others are measured and compared. The presented measurement results allow the reader to understand what the energy hungry parts of a mobile device are and use those findings for the design of future mobile protocols and applications. All results presented in this work and further results are made public on our web page [2].

Falls can cause serious traumas such as brain injuries and bone fractures, especially among elderly people. Fear of falling might reduce physical activities resulting in declining social interactions and eventually causing depression. To lessen the effects of a fall, timely delivery of medical treatment can play a vital role. In a similar scenario, an IoT-based wearable system can pave the most promising way to mitigate serious consequences of a fall while providing the convenience of usage. However, to deliver sufficient degree of monitoring and reliability, wearable devices working at the core of fall detection systems are required to work for a prolonged period of time. In this work, we focus on energy efficiency of a wearable sensor node in an Internet-of-Things (IoT) based fall detection system. We propose the design of a tiny, lightweight, flexible and energy efficient wearable device. We investigate different parameters (e.g. sampling rate, communication bus interface, transmission protocol, and transmission rate) impacting on energy consumption of the wearable device. In addition, we provide a comprehensive analysis of energy consumption of the wearable in different configurations and operating conditions. Furthermore, we provide hints (hardware and software) for system designers implementing the optimal wearable device for IoT-based fall detection systems in terms of energy efficiency and high quality of service. The results clearly indicate that the proposed sensor node is novel and energy efficient. In a critical condition, the wearable device can be used continuously for 76 h with a 1000 mAh li-ion battery.

This paper introduces the sensor-networked IoT model as a prototype to support the design of Body Area Network (BAN) applications for healthcare. Using the model, we analyze the synergistic effect of the functional requirements (data collection from the human body and transferring it to the top level) and non-functional requirements (trade-offs between energy-security-environmental factors, treated as Quality-of-Service (QoS)). We use feature models to represent the requirements at the earliest stage for the analysis and describe a model-driven methodology to design the possible BAN applications. Firstly, we specify the requirements as the problem domain (PD) variability model for the BAN applications. Next, we introduce the generative technology (meta-programming as the solution domain (SD)) and the mapping procedure to map the PD feature-based variability model onto the SD feature model. Finally, we create an executable meta-specification that represents the BAN functionality to describe the variability of the problem domain though transformations. The meta-specification (along with the meta-language processor) is a software generator for multiple BAN-oriented applications. We validate the methodology with experiments and a case study to generate a family of programs for the BAN sensor controllers. This enables to obtain the adequate measure of QoS efficiently through the interactive adjustment of the meta-parameter values and re-generation process for the concrete BAN application.

Long-term wildfire monitoring using distributed in situ temperature sensors is an accurate, yet demanding environmental monitoring application, which requires long-life, low-maintenance, low-cost sensors and a simple, fast, error-proof deployment procedure. We present in this paper the most important design considerations and optimizations of all elements of a low-cost WSN platform prototype for long-term, low-maintenance pervasive wildfire monitoring, its preparation for a nearly three-month field test, the analysis of the causes of failure during the test and the lessons learned for platform improvement. The main components of the total cost of the platform (nodes, deployment and maintenance) are carefully analyzed and optimized for this application. The gateways are designed to operate with resources that are generally used for sensor nodes, while the requirements and cost of the sensor nodes are significantly lower. We define and test in simulation and in the field experiment a simple, but effective communication protocol for this application. It helps to lower the cost of the nodes and field deployment procedure, while extending the theoretical lifetime of the sensor nodes to over 16 years on a single 1 Ah lithium battery.


Question: A framework for environmental energy harvesting in sensor networks.

This paper introduces Parssec, a Parallel Simulation Environment for Complex Systems, designed to facilitate the evaluation of large-scale systems with thousands of components, such as parallel architectures, wireless networks, and parallel database systems. The increasing complexity and scale of these systems have outpaced the capabilities of traditional design and performance prediction techniques, making simulation a critical tool. However, sequential simulation models often require excessively long execution times, hindering the evaluation process. For instance, simulating wireless networks with thousands of mobile nodes in a realistic environment can take weeks, while simulating parallel programs on multi-processor systems can take from a few weeks to a few months even on high-end workstations.

Moreover, as system size grows, the memory requirements of simulation models can exceed the capacity of workstations. The limitations of sequential simulation have spurred interest in parallel simulation, which can significantly reduce execution times. However, the adoption of parallel simulation has been limited by the lack of tools that integrate parallel execution into the broader context of system simulation.

Parssec aims to address these issues by offering a comprehensive environment for parallel simulation. It supports easy migration of simulation models to operational software prototypes, is compatible with both distributed and shared-memory platforms, and offers a range of parallel simulation protocols. Additionally, Parssec provides tools for visual and hierarchical model design, simplifying the model creation and maintenance process. This environment is particularly valuable for performance analysts who lack expertise in parallel simulation, as it allows them to experiment with various algorithmic and architectural alternatives for model execution, reducing the costs associated with detailed simulation model design and development.

In summary, Parssec is a robust solution for simulating complex systems, offering a suite of features that facilitate the creation, execution, and evaluation of simulation models for large-scale systems, thereby reducing design and development costs and improving the overall performance prediction process.

Energy-efficient communication is one of the main concerns of wireless sensor networks nowadays. A commonly employed approach for achieving energy efficiency has been the use of duty-cycled operation of the radio, where the node's transceiver is turned off and on regularly, listening to the radio channel for possible incoming communication during its on-state. Nonetheless, such a paradigm performs poorly for scenarios of low or bursty traffic because of unnecessary activations of the radio transceiver. As an alternative technology, Wake-up Radio (WuR) systems present a promising energy-efficient network operation, where target devices are only activated in an on-demand fashion by means of a special radio signal and a WuR receiver. In this paper, we analyze a novel wake-up radio approach that integrates both data communication and wake-up functionalities into one platform, providing a reconfigurable radio operation. Through physical experiments, we characterize the delay, current consumption and overall operational range performance of this approach under different transmit power levels. We also present an actual single-hop WuR application scenario, as well as demonstrate the first true multi-hop capabilities of a WuR platform and simulate its performance in a multi-hop scenario. Finally, by thorough qualitative comparisons to the most relevant WuR proposals in the literature, we state that the proposed WuR system stands out as a strong candidate for any application requiring energy-efficient wireless sensor node communications.

This paper presents a sentry-based approach to power management in wireless sensor networks for applications such as intruder detection and tracking. To minimize average power consumption while maintaining sufficient node density for coarse sensing, nodes are partitioned dynamically into two sets: sentries and non-sentries. Sentry nodes provide sufficient coverage for continuous monitoring and basic communication services. Non-sentry nodes sleep for designated periods of time to conserve power, and switch to full power only when needed to provide more refined sensing for tracking. Non-sentry nodes check for beacons from sentry nodes to determine when they should remain on. Experimental results are presented demonstrating trade-offs between power savings and tracking performance for a network of seventeen nodes using the first implementation of a basic sentry-based power management scheme. The paper concludes with a brief description of a full set of power-management services being implemented as middle-ware for general wireless sensor applications.

The increasing adoption of wireless sensor network technology in a variety of applications, from agricultural to volcanic monitoring, has demonstrated their ability to gather data with unprecedented sensing capabilities and deliver it to a remote user. However, a key issue remains how to maintain these sensor network deployments over increasingly prolonged deployments. In this article, we present the challenges that were faced in maintaining continual operation of an automated wildlife monitoring system over a one-year period. This system analyzed the social colocation patterns of European badgers (Meles meles) residing in a dense woodland environment using a hybrid RFID-WSN approach. We describe the stages of the evolutionary development, from implementation, deployment, and testing, to various iterations of software optimization, followed by hardware enhancements, which in turn triggered the need for further software optimization. We highlight the main lessons learned: the need to factor in the maintenance costs while designing the system; to consider carefully software and hardware interactions; the importance of rapid prototyping for initial deployment (this was key to our success); and the need for continuous interaction with domain scientists which allows for unexpected optimizations.

The aim of the present paper is to review the technical and scientific state of the art of wireless sensor technologies and standards for wireless communications in the Agri-Food sector. These technologies are very promising in several fields such as environmental monitoring, precision agriculture, cold chain control or traceability. The paper focuses on WSN (Wireless Sensor Networks) and RFID (Radio Frequency Identification), presenting the different systems available, recent developments and examples of applications, including ZigBee based WSN and passive, semi-passive and active RFID. Future trends of wireless communications in agriculture and food industry are also discussed.

Battery lifetime is critical for wireless video sensors. To enable battery-powered wireless video sensors, low-power design is required. In this paper, we consider applying multi-view summarization to wireless video sensors to remove redundant contents such that the compression and transmission power can be reduced. A low-complexity online multi-view video summarization scheme is proposed. Experiments show that the proposed summarization method successfully reduces the video content while keeping important events. A power analysis of the system also shows that a significant amount of energy can be saved.

Recent advances in microelectronics and integrated circuits, system-on-chip design, wireless communication and intelligent low-power sensors have allowed the realization of a Wireless Body Area Network (WBAN). A WBAN is a collection of low-power, miniaturized, invasive/non-invasive lightweight wireless sensor nodes that monitor the human body functions and the surrounding environment. In addition, it supports a number of innovative and interesting applications such as ubiquitous healthcare, entertainment, interactive gaming, and military applications. In this paper, the fundamental mechanisms of WBAN including architecture and topology, wireless implant communication, low-power Medium Access Control (MAC) and routing protocols are reviewed. A comprehensive study of the proposed technologies for WBAN at Physical (PHY), MAC, and Network layers is presented and many useful solutions are discussed for each layer. Finally, numerous WBAN applications are highlighted.

Advances in wireless technology and supporting infrastructure provide unprecedented opportunity for ubiquitous real-time healthcare and fitness monitoring without constraining the activities of the user. Wirelessly connected miniaturized sensors and actuators placed in, on, and around the body form a body area network for continuous, automated, and unobtrusive monitoring of physiological signs to support medical, lifestyle and entertainment applications. BAN technology is in the early stage of development, and several research challenges have to be overcome for it to be widely accepted. In this article we study the core set of application, functional, and technical requirements of the BAN. We also discuss fundamental research challenges such as scalability (in terms of data rate, power consumption, and duty cycle), antenna design, interference mitigation, coexistence, QoS, reliability, security, privacy, and energy efficiency. Several candidate technologies poised to address the emerging BAN market are evaluated, and their merits and demerits are highlighted. A brief overview of standardization activities relevant to BANs is also presented.

Emerging technologies have made low-power and low-cost wireless sensor networks feasible. This paper presents a hierarchical wireless sensor network for measuring soil parameters such as temperature and humidity. Specifically, we designed sensor nodes that are placed completely underground and are used to collect soil measurements. These nodes use their radios to deliver the collected measurements to one of multiple relay nodes located above ground. In turn, relay nodes that are capable of long-range communications forward the data collected from the network's sensor nodes to a base node, which is connected to a workstation. The proposed hierarchical wireless sensor network uses a probabilistic communication protocol to achieve a very low duty cycle and hence a long life-time for soil monitoring applications.

This paper presents a novel ultra-low power rectifier for RF energy harvester, designed and implemented in standard 130 nm CMOS technology. The proposed 915 MHz ISM band RF energy harvester is designed for wearable medical devices and internet of things (IoT) applications. An off-chip differential matching network passively boosts the low-level incoming AC signal generated by the antenna. Then, a novel self-compensated cross-coupled rectifier is designed to convert the AC signal into a DC output voltage. The rectifier is comprised of 10 stages and it uses both dynamic and static bias compensation to decrease the transistors forward voltage drop. The post-layout simulation results demonstrate a sensitivity of −30.5 dBm for 1 V output at a capacitive load which is lower than the current state-of-the-art. The peak end-to-end efficiency is 42.8 % at −16 dBm input power, delivering 2.32 V at 0.5 MΩ resistor load.

The notion of IoT has ignited umpteen possibilities of different heterogeneous devices to blend within a network. In IoT, especially sensor devices are mostly deployed in an extremely resource constrained environment and thus pose the necessity to extend the capability and life expectancy of these kind of devices in terms of energy consumption. Since, the devices in IoT have limited energy sources they often run on battery with a certain energy capability, the deployment of green communication and system model in IoT has been a core challenging issue. This paper addresses the energy efficiency issues across diverse IoT driven networks by proposing a system model for G-IoT and energy efficient scheme for the IoT devices to extend the life expectancy of the whole IoT network.