Question: Real-Time Multi-human Tracking Employing a Probability Hypothesis Density Filter and Numerous Detectors

This paper presents a rigorous metric for assessing the performance of multi-target tracking algorithms, a critical aspect in the development, tuning, and comparative analysis of tracking systems. The primary objective is to quantitatively measure the disparity between two sets of tracks: the ground truth and the estimated tracks. Inspired by the OSPA (Optimal Sub-pattern Assignment) metric, a recently established consistent measure for evaluating multi-target filters, the proposed metric extends this framework to the tracking domain. Multi-target filters are designed to sequentially estimate the quantity of targets and their locations in the state space. However, tracking algorithms yield tracks, which are labeled temporal sequences of state estimates tied to individual targets. Consequently, the metric introduced here is tailored to operate on the space of finite sets of tracks, uniquely accounting for labeling discrepancies. Illustrative numerical examples confirm that the metric aligns with anticipated behavior, effectively capturing the nuances of multi-target tracking performance.

In this paper, a robust, real-time object tracking approach capable of dealing with multiple symmetric and non-symmetric objects in a real-time requirement setting is proposed. The approach relies only on depth data to track multiple objects in a dynamic environment and uses random-forest based learning to deal with problems like object occlusion, motion-blur due to camera motion and clutter. We show that the relation between object motion and the corresponding change in its 3D point cloud data can be learned using only 6 random forests. A framework that unites object pose estimation and object pose tracking to efficiently track multiple objects in 3D space is presented. The approach is robust in tracking objects even in presence of motion blur that causes noisy depth data and is capable of real-time performance with 1.8ms per frame. The experimental evaluations demonstrate the performance of the approach against robustness, accuracy and speed and compare the approach with the state of the art. A publicly available dataset with real-world data is also provided for future benchmarking.

Online multi-object tracking aims at estimating the tracks of multiple objects instantly with each incoming frame and the information provided up to the moment. It still remains a difficult problem in complex scenes, because of the large ambiguity in associating multiple objects in consecutive frames and the low discriminability between objects appearances. In this paper, we propose a robust online multi-object tracking method that can handle these difficulties effectively. We first define the tracklet confidence using the detectability and continuity of a tracklet, and decompose a multi-object tracking problem into small subproblems based on the tracklet confidence. We then solve the online multi-object tracking problem by associating tracklets and detections in different ways according to their confidence values. Based on this strategy, tracklets sequentially grow with online-provided detections, and fragmented tracklets are linked up with others without any iterative and expensive association steps. For more reliable association between tracklets and detections, we also propose a deep appearance learning method to learn a discriminative appearance model from large training datasets, since the conventional appearance learning methods do not provide rich representation that can distinguish multiple objects with large appearance variations. In addition, we combine online transfer learning for improving appearance discriminability by adapting the pre-trained deep model during online tracking. Experiments with challenging public datasets show distinct performance improvement over other state-of-the-arts batch and online tracking methods, and prove the effect and usefulness of the proposed methods for online multi-object tracking.

This paper present a particle filter also known as Monte Carlo Localization (MCL) to solve the localization problem presented before. A new resampling mechanism is proposed. This new resampling mechanism enables the particle filter to converge quicker and more robust to kidnaping problem. This particle filter is simulated in MATLAB and also experimented physically using a simple autonomous mobile robot built with Lego Mindstorms NXT with 3 ultrasonic sonar and RWTH Mindstorms NXT Toolbox for MATLAB to connect the robot to MATLAB. The particle filter with the new resampling algorithm can perform very well in the physical experiments.

We have implemented a GUI-based tracking testbed system in C and Intel OpenCV, running within the Microsoft Windows environment. The motivation for the testbed is to run and log tracking experiments in near real-time. The testbed allows a tracking experiment to be repeated from the same starting state using different tracking algorithms and parameter settings, thereby facilitating comparison of algorithms. We have also developed a tracking evaluation web site to encourage third-party self-evaluation of state-of-the-art tracking algorithms. The site hosts source code for the tracking testbed, a set of ground-truth datasets, and a method for on-line evaluation of uploaded tracking results.

We propose a method for human pose estimation based on Deep Neural Networks (DNNs). The pose estimation is formulated as a DNN-based regression problem towards body joints. We present a cascade of such DNN regres- sors which results in high precision pose estimates. The approach has the advantage of reasoning about pose in a holistic fashion and has a simple but yet powerful formula- tion which capitalizes on recent advances in Deep Learn- ing. We present a detailed empirical analysis with state-of- art or better performance on four academic benchmarks of diverse real-world images.

The 3D tracking of the articulation of human hands is a theoretically interesting and challenging problem with numerous and diverse applications [1]. Several attempts have been made to address the problem by considering markerless visual data. Existing approaches can be categorized into modeland appearance-based [1]. In this work, we propose a model-based approach to the problem (Fig. 1). We formulate it as an optimization problem that minimizes the discrepancy between the 3D structure and appearance of hypothesized 3D hand model instances, and actual visual observations. Optimization is performed with a variant of an existing stochastic optimization method (Particle Swarm Optimization PSO) [2]. The most computationally demanding parts of the process have been implemented to run efficiently on a GPU [3]. The resulting system performs hand articulations tracking at a rate of 15Hz.

We describea markerlesscamera tracking systemfor augmentedreality thatoperatesin environmentswhich contain oneor more planes. This is a commonspecialcase, which we showsignificantlysimplifiestracking. The result is a practical, reliable, vision-basedtracker. Furthermore, thetrackedplaneimposesa natural referenceframe, sothat thealignmentof thereal andvirtual coordinatesystemsis rather simplerthan would be the casewith a general structure-and-motionsystem.Multiple planescan be tracked, and additional data such as 2D point tracks are easilyincorporated.

We present an unsupervised technique for visual learning, which is based on density estimation in high-dimensional spaces using an eigenspace decomposition. Two types of density estimates are derived for modeling the training data: a multivariate Gaussian (for unimodal distributions) and a Mixture-of-Gaussians model (for multimodal distributions). These probability densities are then used to formulate a maximum-likelihood estimation framework for visual search and target detection for automatic object recognition and coding. Our learning technique is applied to the probabilistic visual modeling, detection, recognition, and coding of human faces and nonrigid objects, such as hands.

A Random Forest consists of several independent decision trees arranged in a forest. A majority vote over all trees leads to the final decision. In this paper we propose a Random Forest framework which incorporates a cascade structure consisting of several stages together with a bootstrap approach. By introducing the cascade, 99% of the test images can be rejected by the first and second stage with minimal computational effort leading to a massively speeded-up detection framework. Three different cascade voting strategies are implemented and evaluated. Additionally, the training and classification speed-up is analyzed. Several experiments on public available datasets for pedestrian detection, lateral car detection and unconstrained face detection demonstrate the benefit of our contribution.

Visual tracking, in essence, deals with non-stationary image streams that change over time. While most existing algorithms are able to track objects well in controlled environments, they usually fail in the presence of significant variation of the object’s appearance or surrounding illumination. One reason for such failures is that many algorithms employ fixed appearance models of the target. Such models are trained using only appearance data available before tracking begins, which in practice limits the range of appearances that are modeled, and ignores the large volume of information (such as shape changes or specific lighting conditions) that becomes available during tracking. In this paper, we present a tracking method that incrementally learns a low-dimensional subspace representation, efficiently adapting online to changes in the appearance of the target. The model update, based on incremental algorithms for principal component analysis, includes two important features: a method for correctly updating the sample mean, and a forgetting factor to ensure less modeling power is expended fitting older observations. Both of these features contribute measurably to improving overall tracking performance. Numerous experiments demonstrate the effectiveness of the proposed tracking algorithm in indoor and outdoor environments where the target objects undergo large changes in pose, scale, and illumination.


Question: Real-time multi-human tracking employing a probability hypothesis density filter and multiple detectors.

This paper introduces a novel and consistent metric for evaluating the performance of multi-object filters, a critical aspect in the field of multi-object tracking and control. The concept of miss-distance, or error, is fundamental in filtering and control problems, serving as a measure of discrepancy between a reference value and its estimate or controlled output. However, in multi-object filtering, a well-established but complex field, a universally accepted definition of miss-distance has been lacking. We highlight the inconsistencies in current metrics used for assessing multi-object miss-distances and their implications on performance evaluation. To rectify these issues, we propose a new metric that is both mathematically rigorous and intuitively clear, designed to overcome the limitations of existing multi-object performance evaluation measures. This advancement is crucial for achieving more accurate and reliable multi-object tracking and control systems.

We present a novel lane detection and tracking system using a fusion of Maximally Stable Extremal Regions (MSER) and Progressive Probabilistic Hough Transform (PPHT). First, MSER is applied to obtain a set of blobs including noisy pixels (e.g., trees, cars and traffic signs) and the candidate lane markings. A scanning refinement algorithm is then introduced to enhance the results of MSER and filter out noisy data. After that, to achieve the requirements of real-time systems, the PPHT is applied. Compared to Hough transform which returns the parameters ρ and Θ, PPHT returns two end-points of the detected line markings. To track lane markings, two kalman trackers are used to track both end-points. Several experiments are conducted in Ottawa roads to test the performance of our framework. The detection rate of the proposed system averages 92.7% and exceeds 84.9% in poor conditions.

Visual object tracking is challenging as target objects often undergo significant appearance changes caused by deformation, abrupt motion, background clutter and occlusion. In this paper, we exploit features extracted from deep convolutional neural networks trained on object recognition datasets to improve tracking accuracy and robustness. The outputs of the last convolutional layers encode the semantic information of targets and such representations are robust to significant appearance variations. However, their spatial resolution is too coarse to precisely localize targets. In contrast, earlier convolutional layers provide more precise localization but are less invariant to appearance changes. We interpret the hierarchies of convolutional layers as a nonlinear counterpart of an image pyramid representation and exploit these multiple levels of abstraction for visual tracking. Specifically, we adaptively learn correlation filters on each convolutional layer to encode the target appearance. We hierarchically infer the maximum response of each layer to locate targets. Extensive experimental results on a largescale benchmark dataset show that the proposed algorithm performs favorably against state-of-the-art methods.

We have implemented a GUI-based tracking testbed system in C and Intel OpenCV, running within the Microsoft Windows environment. The motivation for the testbed is to run and log tracking experiments in near real-time. The testbed allows a tracking experiment to be repeated from the same starting state using different tracking algorithms and parameter settings, thereby facilitating comparison of algorithms. We have also developed a tracking evaluation web site to encourage third-party self-evaluation of state-of-the-art tracking algorithms. The site hosts source code for the tracking testbed, a set of ground-truth datasets, and a method for on-line evaluation of uploaded tracking results.

Designing effective feature and handling the model drift problem are two important aspects for online visual tracking. For feature representation, gradient and color features are most widely used, but how to effectively combine them for visual tracking is still an open problem. In this paper, we propose a rich feature descriptor, MC-HOG, by leveraging rich gradient information across multiple color channels or spaces. Then MC-HOG features are embedded into the correlation tracking framework to estimate the state of the target. For handling the model drift problem caused by occlusion or distracter, we propose saliency proposals as prior information to provide candidates and reduce background interference. In addition to saliency proposals, a ranking strategy is proposed to determine the importance of these proposals by exploiting the learnt appearance filter, historical preserved object samples and the distracting proposals. In this way, the proposed approach could effectively explore the color-gradient characteristics and alleviate the model drift problem. Extensive evaluations performed on the benchmark dataset show the superiority of the proposed method.

We propose a method for human pose estimation based on Deep Neural Networks (DNNs). The pose estimation is formulated as a DNN-based regression problem towards body joints. We present a cascade of such DNN regres- sors which results in high precision pose estimates. The approach has the advantage of reasoning about pose in a holistic fashion and has a simple but yet powerful formula- tion which capitalizes on recent advances in Deep Learn- ing. We present a detailed empirical analysis with state-of- art or better performance on four academic benchmarks of diverse real-world images.

Object occlusion is widespread in video surveillance due to the influence of angle and environment, which brings a great impact on the target tracking and makes the development of video object tracking encountered meeting many constraints. The challenge in video object tracking is how to track accurately when the target is obscured. We use the frame difference method to detect the target and smooth the target trajectory by using Kalman filter. When the target is occluded, we can select the appropriate fitting section after smoothing trajectories and use the least square method to fit the target motion path adaptively, so we can predict the object location. By comparing the experimental results with the actual motion of the target, it shows that the algorithm can be used to track most of the occlusion targets precisely.

The goal of multi-object tracking is to estimate the number of objects and their states recursively over time. In the presence of extended objects, i.e. objects with an extent that is not negligible in comparison to sensor resolution and which give rise to multiple measurements per time step, the tracking task is even more challenging: To fully utilize all available information and to achieve accurate estimation results, measurement models that are able to represent the object shape as well as algorithms that tackle the aggravated track-to-measurements association problem are necessary. In this work, the Gaussian process measurement model is integrated in the recently developed labeled multi-Bernoulli filter for extended objects. Thus, the ability of Gaussian processes to estimate and represent a wide range of free-from shapes is combined with a principled approach to the multiple extended objects tracking problem. The filter performance is demonstrated for simulated and experimental vehicle tracking using a laser scanner. For this particular application, a new, approximately axis-symmetric covariance function is additionally introduced.

In this MTech thesis experiments will be tried out on a people counting system in an effort to enhance the accuracy when separating counting groups of people, and nonhuman objects. This system features automatic color equalization, adaptive background subtraction, shadow detection algorithm and Kalman tracking. The aim is to develop a reliable and accurate computer vision alternative to sensor or contact based mechanisms. The problem for many computer vision based systems are making good separation between the background and foreground, and teaching the computers what parts make up a scene. We also want to find features to classify the foreground moving objects, an easy task for a human, but a complex task for a computer. Video has been captured with a birds eye view close to one of the entrances at the school about ten meters above the floor. From this video troublesome parts have been selected to test the changes done to the algorithms and program code.

Most current approaches to recognition aim to be scaleinvariant. However, the cues available for recognizing a 300 pixel tall object are qualitatively different from those for recognizing a 3 pixel tall object. We argue that for sensors with finite resolution, one should instead use scale-variant, or multiresolution representations that adapt in complexity to the size of a putative detection window. We describe a multiresolution model that acts as a deformable part-based model when scoring large instances and a rigid template with scoring small instances. We also examine the interplay of resolution and context, and demonstrate that context is most helpful for detecting low-resolution instances when local models are limited in discriminative power. We demonstrate impressive results on the Caltech Pedestrian benchmark, which contains object instances at a wide range of scales. Whereas recent state-of-theart methods demonstrate missed detection rates of 86%-37% at 1 falsepositive-per-image, our multiresolution model reduces the rate to 29%.

Tracking by detection is a common approach to solving the Multiple Object Tracking problem. In this paper we show how deep metric learning can be used to improve three aspects of tracking by detection. We train a convolutional neural network to learn an embedding function in a Siamese configuration on a large person re-identification dataset offline. It is then used to improve the online performance of tracking while retaining a high frame rate. We use this learned appearance metric to robustly build estimates of pedestrian’s trajectories in the MOT16 dataset. In breaking with the tracking by detection model, we use our appearance metric to propose detections using the predicted state of a tracklet as a prior in the case where the detector fails. This method achieves competitive results in evaluation, especially among online, realtime approaches. We present an ablative study showing the impact of each of the three uses of our deep appearance metric.


Question: Real-Time Multi-human Tracking Employing a Probability Hypothesis Density Filter and Multiple Detectors

This paper explores the adaptation and application of the Optimal Sub-Pattern Assignment (OSPA) metric for the comprehensive performance evaluation of multi-target tracking algorithms. The OSPA metric, initially developed for multi-object filters, has gained recognition for its effectiveness in assessing and comparing tracking systems. Our contribution lies in extending the OSPA metric to incorporate track label error, alongside the traditional cardinality and localization errors, to more accurately reflect the data association capabilities of the tracking algorithms. This enhanced metric provides a holistic view of performance, crucial for the design and refinement of multi-target tracking systems. To illustrate the efficacy of our adapted OSPA metric, we conduct a comparative analysis of two particle filters designed for multi-target tracking. The results not only highlight the strengths and weaknesses of each algorithm but also underscore the importance of a comprehensive performance evaluation framework in the field of multi-target tracking.

We propose an approach to estimate 3D human pose in real world units from a single RGBD image and show that it exceeds performance of monocular 3D pose estimation approaches from color as well as pose estimation exclusively from depth. Our approach builds on robust human keypoint detectors for color images and incorporates depth for lifting into 3D. We combine the system with our learning from demonstration framework to instruct a service robot without the need of markers. Experiments in real world settings demonstrate that our approach enables a PR2 robot to imitate manipulation actions observed from a human teacher.

This paper presents a method for pedestrian detection and tracking using a single night-vision video camera installed on the vehicle. To deal with the nonrigid nature of human appearance on the road, a two-step detection/tracking method is proposed. The detection phase is performed by a support vector machine (SVM) with size-normalized pedestrian candidates and the tracking phase is a combination of Kalman filter prediction and mean shift tracking. The detection phase is further strengthened by information obtained by a road-detection module that provides key information for pedestrian validation. Experimental comparisons (e.g., grayscale SVM recognition versus binary SVM recognition and entire-body detection versus upper-body detection) have been carried out to illustrate the feasibility of our approach.

Correlation Filters (CFs) have recently demonstrated excellent performance in terms of rapidly tracking objects under challenging photometric and geometric variations. The strength of the approach comes from its ability to efficiently learn - on the fly - how the object is changing over time. A fundamental drawback to CFs, however, is that the background of the target is not modeled over time which can result in suboptimal performance. Recent tracking algorithms have suggested to resolve this drawback by either learning CFs from more discriminative deep features (e.g. DeepSRDCF [9] and CCOT [11]) or learning complex deep trackers (e.g. MDNet [28] and FCNT [33]). While such methods have been shown to work well, they suffer from high complexity: extracting deep features or applying deep tracking frameworks is very computationally expensive. This limits the real-time performance of such methods, even on high-end GPUs. This work proposes a Background-Aware CF based on hand-crafted features (HOG [6]) that can efficiently model how both the foreground and background of the object varies over time. Our approach, like conventional CFs, is extremely computationally efficient- and extensive experiments over multiple tracking benchmarks demonstrate the superior accuracy and real-time performance of our method compared to the state-of-the-art trackers.

In image sequence processing kalman filtering is used for an adaptive background estimation, in order to separate the foreground from the background. The presented work is an approach which takes into account that changing illumination should be considered in the background estimation, and should not be detected as foreground. The new approach assumes a stationary CCD cameras with fixed focal length and considers non-rigid objects moving non-continuously like human bodies. Furthermore, statistic based methods are used to overcome the problems caused by shadow borders and the adaptation, when the background is covered by the foreground.

Designing effective feature and handling the model drift problem are two important aspects for online visual tracking. For feature representation, gradient and color features are most widely used, but how to effectively combine them for visual tracking is still an open problem. In this paper, we propose a rich feature descriptor, MC-HOG, by leveraging rich gradient information across multiple color channels or spaces. Then MC-HOG features are embedded into the correlation tracking framework to estimate the state of the target. For handling the model drift problem caused by occlusion or distracter, we propose saliency proposals as prior information to provide candidates and reduce background interference. In addition to saliency proposals, a ranking strategy is proposed to determine the importance of these proposals by exploiting the learnt appearance filter, historical preserved object samples and the distracting proposals. In this way, the proposed approach could effectively explore the color-gradient characteristics and alleviate the model drift problem. Extensive evaluations performed on the benchmark dataset show the superiority of the proposed method.

This paper presents a monocular model-based 3D tracking approach for textureless objects. Instead of explicitly searching for 3D-2D correspondences as previous methods, which unavoidably generates individual outlier matches, we aim to minimize the holistic distance between the predicted object contour and the query image edges. We propose a method that can directly solve 3D pose parameters in unsegmented edge distance field. We derive the differentials of edge matching distance with respect to the pose parameters, and search the optimal 3D pose parameters using standard gradient-based non-linear optimization techniques. To avoid being trapped in local minima and to deal with potential large inter-frame motions, a particle filtering process with a first order autoregressive state dynamics is exploited. Occlusions are handled by a robust estimator. The effectiveness of our approach is demonstrated using comparative experiments on real image sequences with occlusions, large motions and cluttered backgrounds.

We present an unsupervised technique for visual learning, which is based on density estimation in high-dimensional spaces using an eigenspace decomposition. Two types of density estimates are derived for modeling the training data: a multivariate Gaussian (for unimodal distributions) and a Mixture-of-Gaussians model (for multimodal distributions). These probability densities are then used to formulate a maximum-likelihood estimation framework for visual search and target detection for automatic object recognition and coding. Our learning technique is applied to the probabilistic visual modeling, detection, recognition, and coding of human faces and nonrigid objects, such as hands.

Very recently tracking was approached using classification techniques such as support vector machines. The object to be tracked is discriminated by a classifier from the background. In a similar spirit we propose a novel on-line AdaBoost feature selection algorithm for tracking. The distinct advantage of our method is its capability of on-line training. This allows to adapt the classifier while tracking the object. Therefore appearance changes of the object (e.g. out of plane rotations, illumination changes) are handled quite naturally. Moreover, depending on the background the algorithm selects the most discriminating features for tracking resulting in stable tracking results. By using fast computable features (e.g. Haar-like wavelets, orientation histograms, local binary patterns) the algorithm runs in real-time. We demonstrate the performance of the algorithm on several (publically available) video sequences.

In this paper, a robust, real-time object tracking approach capable of dealing with multiple symmetric and non-symmetric objects in a real-time requirement setting is proposed. The approach relies only on depth data to track multiple objects in a dynamic environment and uses random-forest based learning to deal with problems like object occlusion, motion-blur due to camera motion and clutter. We show that the relation between object motion and the corresponding change in its 3D point cloud data can be learned using only 6 random forests. A framework that unites object pose estimation and object pose tracking to efficiently track multiple objects in 3D space is presented. The approach is robust in tracking objects even in presence of motion blur that causes noisy depth data and is capable of real-time performance with 1.8ms per frame. The experimental evaluations demonstrate the performance of the approach against robustness, accuracy and speed and compare the approach with the state of the art. A publicly available dataset with real-world data is also provided for future benchmarking.

Accurate segmentation of humans from live videos is an important problem to be solved in developing immersive video experience. We propose to extract the human segmentation information from color and depth cues in a video using multiple modeling techniques. The prior information from human skeleton data is also fused along with the depth and color models to obtain the final segmentation inside a graph-cut framework. The proposed method runs real time on live videos using single CPU and is shown to be quantitatively outperforming the methods that directly fuse color and depth data.


Question: Real-Time Multi-human Tracking Employing a Probability Hypothesis Density Filter and Multiple Detectors

This paper presents a comprehensive framework for measuring miss distances in multitarget systems, a critical yet often overlooked aspect in the design and evaluation of multiobject tracking algorithms. Building upon the established L/sub p/ norm theory, we introduce a new class of distance metrics tailored for multitarget scenarios. These metrics not only extend the conventional understanding of miss distance but also provide a robust theoretical foundation for the intuitive optimal-assignment method proposed by Drummond. This method assesses the performance of multitarget tracking algorithms by optimally pairing actual and estimated targets, calculating the distance between each pair, and aggregating these distances to determine the overall tracking accuracy. To facilitate practical implementation, we outline efficient computational methods based on standard optimal assignment algorithms or convex optimization techniques. We highlight the significant implications of these metrics for various applications, particularly in performance evaluation and sensor management. As a case study, we showcase the use of our multitarget miss-distance metrics as measures of effectiveness (MoEs) for evaluating the performance of multitarget tracking algorithms. This approach offers a more nuanced understanding of tracking accuracy and reliability, enabling the development of more effective and efficient multiobject tracking systems.

We have implemented a GUI-based tracking testbed system in C and Intel OpenCV, running within the Microsoft Windows environment. The motivation for the testbed is to run and log tracking experiments in near real-time. The testbed allows a tracking experiment to be repeated from the same starting state using different tracking algorithms and parameter settings, thereby facilitating comparison of algorithms. We have also developed a tracking evaluation web site to encourage third-party self-evaluation of state-of-the-art tracking algorithms. The site hosts source code for the tracking testbed, a set of ground-truth datasets, and a method for on-line evaluation of uploaded tracking results.

We introduce a new dataset, Human3.6M, of 3.6 Million accurate 3D Human poses, acquired by recording the performance of 5 female and 6 male subjects, under 4 different viewpoints, for training realistic human sensing systems and for evaluating the next generation of human pose estimation models and algorithms. Besides increasing the size of the datasets in the current state-of-the-art by several orders of magnitude, we also aim to complement such datasets with a diverse set of motions and poses encountered as part of typical human activities (taking photos, talking on the phone, posing, greeting, eating, etc.), with additional synchronized image, human motion capture, and time of flight (depth) data, and with accurate 3D body scans of all the subject actors involved. We also provide controlled mixed reality evaluation scenarios where 3D human models are animated using motion capture and inserted using correct 3D geometry, in complex real environments, viewed with moving cameras, and under occlusion. Finally, we provide a set of large-scale statistical models and detailed evaluation baselines for the dataset illustrating its diversity and the scope for improvement by future work in the research community. Our experiments show that our best large-scale model can leverage our full training set to obtain a 20% improvement in performance compared to a training set of the scale of the largest existing public dataset for this problem. Yet the potential for improvement by leveraging higher capacity, more complex models with our large dataset, is substantially vaster and should stimulate future research. The dataset together with code for the associated large-scale learning models, features, visualization tools, as well as the evaluation server, is available online at http://vision.imar.ro/human3.6m.

Object occlusion is widespread in video surveillance due to the influence of angle and environment, which brings a great impact on the target tracking and makes the development of video object tracking encountered meeting many constraints. The challenge in video object tracking is how to track accurately when the target is obscured. We use the frame difference method to detect the target and smooth the target trajectory by using Kalman filter. When the target is occluded, we can select the appropriate fitting section after smoothing trajectories and use the least square method to fit the target motion path adaptively, so we can predict the object location. By comparing the experimental results with the actual motion of the target, it shows that the algorithm can be used to track most of the occlusion targets precisely.

The 3D tracking of the articulation of human hands is a theoretically interesting and challenging problem with numerous and diverse applications [1]. Several attempts have been made to address the problem by considering markerless visual data. Existing approaches can be categorized into modeland appearance-based [1]. In this work, we propose a model-based approach to the problem (Fig. 1). We formulate it as an optimization problem that minimizes the discrepancy between the 3D structure and appearance of hypothesized 3D hand model instances, and actual visual observations. Optimization is performed with a variant of an existing stochastic optimization method (Particle Swarm Optimization PSO) [2]. The most computationally demanding parts of the process have been implemented to run efficiently on a GPU [3]. The resulting system performs hand articulations tracking at a rate of 15Hz.

Local image features in space-time or spatio-temporal interest points provide compact and abstract representations of patterns in a video sequence. In this paper, we present a novel human action recognition method based on multi-velocity spatio-temporal interest points (MVSTIPs) and a novel local descriptor called motion energy (ME) orientation histogram (MEOH). The MVSTIP detection includes three steps: first, filtering video frames with multi-direction ME filters at different speeds to detect significant changes at the pixel level; thereafter, a surround suppression model is employed to rectify the ME deviation caused by the camera motion and complicated backgrounds (e.g., dynamic texture); finally, MVSTIPs are obtained with local maximum filters at multispeeds. After detection, we develop MEOH descriptor to capture the motion features in local regions around interest points. The performance of the proposed method is evaluated on KTH, Weizmann, and UCF sports human action datasets. Results show that our method is robust to both simple and complex backgrounds and the method is superior to other methods that are based on local features.

[3] Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Leichter, D.K.C.R.I., Wei, A.V.Y., Krupka, D.F.P.K.E., Fitzgibbon, A., Izadi, S.: Accurate, robust, and flexible real-time hand tracking. In: CHI (2015) [4] Oberweger, M., Wohlhart, P., Lepetit, V.: Training a feedback loop for hand pose estimation. In: ICCV (2015) [18] Oberweger, M., Wohlhart, P., Lepetit, V.: Hands deep in deep learning for hand pose estimation. arXiv preprint arXiv:1502.06807 (2015) [19] Tang, D., Taylor, J., Kohli, P., Keskin, C., Kim, T.K., Shotton, J.: Opening the black box: Hierarchical sampling optimization for estimating human hand pose. In: ICCV (2015) * Indicates equal contribution. Results T θ

This paper presents a monocular model-based 3D tracking approach for textureless objects. Instead of explicitly searching for 3D-2D correspondences as previous methods, which unavoidably generates individual outlier matches, we aim to minimize the holistic distance between the predicted object contour and the query image edges. We propose a method that can directly solve 3D pose parameters in unsegmented edge distance field. We derive the differentials of edge matching distance with respect to the pose parameters, and search the optimal 3D pose parameters using standard gradient-based non-linear optimization techniques. To avoid being trapped in local minima and to deal with potential large inter-frame motions, a particle filtering process with a first order autoregressive state dynamics is exploited. Occlusions are handled by a robust estimator. The effectiveness of our approach is demonstrated using comparative experiments on real image sequences with occlusions, large motions and cluttered backgrounds.

Current state-of-the-art methods solve spatio-temporal action localisation by extending 2D anchors to 3D-cuboid proposals on stacks of frames, to generate sets of temporally connected bounding boxes called action micro-tubes. However, they fail to consider that the underlying anchor proposal hypotheses should also move (transition) from frame to frame, as the actor or the camera do. Assuming we evaluate n 2D anchors in each frame, then the number of possible transitions from each 2D anchor to he next, for a sequence of f consecutive frames, is in the order of O(n ), expensive even for small values of f . To avoid this problem we introduce a Transition-Matrix-based Network (TraMNet) which relies on computing transition probabilities between anchor proposals while maximising their overlap with ground truth bounding boxes across frames, and enforcing sparsity via a transition threshold. As the resulting transition matrix is sparse and stochastic, this reduces the proposal hypothesis search space from O(n ) to the cardinality of the thresholded matrix. At training time, transitions are specific to cell locations of the feature maps, so that a sparse (efficient) transition matrix is used to train the network. At test time, a denser transition matrix can be obtained either by decreasing the threshold or by adding to it all the relative transitions originating from any cell location, allowing the network to handle transitions in the test data that might not have been present in the training data, and making detection translationinvariant. Finally, we show that our network is able to handle sparse annotations such as those available in the DALY dataset, while allowing for both dense (accurate) or sparse (efficient) evaluation within a single model. We report extensive experiments on the DALY, UCF101-24 and Transformed-UCF101-24 datasets to support our claims.

This paper presents an algorithm implemented in real time in a laboratory robot, designed to trace and detect the color of an object in an indoor environment using the Kalman filter. The image capture is done by a kinect camera installed in the robot. The algorithm is programmed into a single board computer, BeagleBone Black that uses the ROS — OpenCV system in image processing.

The long-standing goal of localizing every object in an image remains elusive. Manually annotating objects is quite expensive despite crowd engineering innovations. Current state-of-the-art automatic object detectors can accurately detect at most a few objects per image. This paper brings together the latest advancements in object detection and in crowd engineering into a principled framework for accurately and efficiently localizing objects in images. The input to the system is an image to annotate and a set of annotation constraints: desired precision, utility and/or human cost of the labeling. The output is a set of object annotations, informed by human feedback and computer vision. Our model seamlessly integrates multiple computer vision models with multiple sources of human input in a Markov Decision Process. We empirically validate the effectiveness of our human-in-the-loop labeling approach on the ILSVRC2014 object detection dataset.


Question: A composable, deadlock-free strategy for implementing object-based isolation.

Introducing the Habanero-Java Library: A Java 8 Framework for Multicore Programming

In response to the multicore era, where application performance enhancements are increasingly dependent on parallelism, we propose the integration of parallel computing education into early stages of the Computer Science curriculum. This paper spotlights the Habanero-Java library (HJlib), a recent addition to the Java 8 framework, designed to facilitate multicore programming education and application development. HJlib, a pure Java 8 library, embodies the principles of a pedagogic parallel programming model, enabling its use in teaching a sophomore-level course on "Fundamentals of Parallel Programming" at Rice University.

HJlib contributes to the Java ecosystem by offering a robust and versatile task parallel programming model, suitable for both regular and irregular applications. It empowers Java programmers with foundational knowledge to embark on parallel programming by incorporating or modifying applications to leverage multicore architecture. The library's design hinges on straightforward, orthogonal parallel constructs that ensure safety, allowing for the introduction of parallel programming concepts without the need for advanced expertise. Its APIs, extensively utilizing lambda expressions, are compatible with any Java 8 JVM, ensuring wide applicability.

Moreover, HJlib's runtime feedback features, such as abstract execution metrics and a deadlock detector, provide programmers with insights into theoretical performance and potential bugs within their programs. This capability is particularly valuable for educators and researchers, as HJlib's pedagogic programming model serves as an effective tool for both academic instruction and research endeavors. The library is currently utilized in various research initiatives at Rice University and by external collaborators, focusing on the exploitation of homogeneous and heterogeneous multicore parallelism in big data applications developed for the Hadoop platform.

In essence, HJlib represents a significant advancement in the Java ecosystem, offering a powerful framework for multicore programming education and application development. Its combination of simplicity, safety, and versatility makes it an ideal choice for introducing parallel programming concepts to students and for facilitating research in parallel computing.

We identify three fundamental requirements for scalable network services: incremental scalability and overflow growth provisioning, 24x7 availability through fault masking, and costeffectiveness. We argue that clusters of commodity workstations interconnected by a high-speed SAN are exceptionally well-suited to meeting these challenges for Internet-server workloads, provided the software infrastructure for managing partial failures and administering a large cluster does not have to be reinvented for each new service. To this end, we propose a general, layered architecture for building cluster-based scalable network services that encapsulates the above requirements for reuse, and a service-programming model based on composable workers that perform transformation, aggregation, caching, and customization (TACC) of Internet content. For both performance and implementation simplicity, the architecture and TACC programming model exploit BASE, a weaker-than-ACID data semantics that results from trading consistency for availability and relying on soft state for robustness in failure management. Our architecture can be used as an “off the shelf” infrastructural platform for creating new network services, allowing authors to focus on the “content” of the service (by composing TACC building blocks) rather than its implementation. We discuss two real implementations of services based on this architecture: TranSend, a Web distillation proxy deployed to the UC Berkeley dialup IP population, and HotBot, the commercial implementation of the Inktomi search engine. We present detailed measurements of TranSend’s performance based on substantial client traces, as well as anecdotal evidence from the TranSend and HotBot experience, to support the claims made for the architecture.

As much of the world's computing continues to move into the cloud, the overprovisioning of computing resources to ensure the performance isolation of latency-sensitive tasks, such as web search, in modern datacenters is a major contributor to low machine utilization. Being unable to accurately predict performance degradation due to contention for shared resources on multicore systems has led to the heavy handed approach of simply disallowing the co-location of high-priority, latency-sensitive tasks with other tasks. Performing this precise prediction has been a challenging and unsolved problem.
 In this paper, we present Bubble-Up, a characterization methodology that enables the accurate prediction of the performance degradation that results from contention for shared resources in the memory subsystem. By using a bubble to apply a tunable amount of "pressure" to the memory subsystem on processors in production datacenters, our methodology can predict the performance interference between co-locate applications with an accuracy within 1% to 2% of the actual performance degradation. Using this methodology to arrive at "sensible" co-locations in Google's production datacenters with real-world large-scale applications, we can improve the utilization of a 500-machine cluster by 50% to 90% while guaranteeing a high quality of service of latency-sensitive applications.

The Chinese wall security model (CWSM) was designed to provide access controls that mitigate conflict of interest in commercial organizations, and is especially important for large-scale interenterprise workflow applications. This paper describes how to implement the CWSM in a WfMS. We first demonstrate situations in which the role-based access control model is not sufficient for this, and we then propose a security policy language to solve this problem, also providing support for the intrinsic dynamic access control mechanism defined in the CWSM (i.e., the dynamic binding of subjects and elements in the company data set). This language can also specify several requirements of the dynamic security policy that arise when applying the CWSM in WfMSs. Finally we discuss how to implement a run-time system to implement CWSM policies specified by this language in a WfMS.

This paper presents a new and very rich class of (concurrent) programming languages, based on the notion of computing with partial information, and the concomitant notions of consistency and entailment.1 In this framework, computation emerges from the interaction of concurrently executing agents that communicate by placing, checking and instantiating constraints on shared variables. Such a view of computation is interesting in the context of programming languages because of the ability to represent and manipulate partial information about the domain of discourse, in the context of concurrency because of the use of constraints for communication and control, and in the context of AI because of the availability of simple yet powerful mechanisms for controlling inference, and the promise that very rich representational/programming languages, sharing the same set of abstract properties, may be possible.
To reflect this view of computation, [Sar89] develops the cc family of languages. We present here one member of the family, cc(↓, →) (pronounced “cc with Ask and Choose”) which provides the basic operations of blocking Ask and atomic Tell and an algebra of behaviors closed under prefixing, indeterministic choice, interleaving, and hiding, and provides a mutual recursion operator. cc(↓, →) is (intentionally!) very similar to Milner's CCS, but for the radically different underlying concept of communication, which, in fact, provides a general—and elegant—alternative approach to “value-passing” in CCS. At the same time it adequately captures the framework of committed choice concurrent logic programming languages. We present the rules of behavior for cc agents, motivate a notion of “visible action” for them, and develop the notion of c-trees and reactive congruence analogous to Milner's synchronization trees and observational congruence. We also present an equational characterization of reactive congruence for Finitary cc(↓, →).

Dynamic binary instrumentation is a technique that consists of translating the binary code of executables on-the-fly, including all the loaded libraries. The original binary code is instrumented in the process of translation. Applications of this technique range from sandboxes to debugging and profiling tools. In this thesis we describe lMem, a lightweight binary translation framework for the IA-32/x64 architectures. One of the challenges when writing instrumentation tools is that resources such as registers and memory have to be shared between the instrumented program and the instrumentation logic. A possible way to approach this problem is to translate a given machine code snippet in an intermediate representation (IR), add instrumentation code written in this IR and then recompile the snippet to machine code and run it. Tools such as Valgrind successfully make use of this technique. These IR-based methods however are heavyweight in the sense that a significant amount of work done by the compiler has to be replicated at runtime. IR-based recompilation increases the runtime overhead and the complexity of the binary translation framework itself. A second approach consists of saving and restoring some of the state before and after the inserted instrumentation code. This approach, which is used for example by the selDebug debugger, is simple but it suffers from overhead due to frequent saving and restoring of state. A third approach is to let the instrumentation code make use of hardware resources that are not used by the instrumented program. The minemu taint checking tool achieves good performance by exploiting the fact that some 32 bit machines lack SSE registers. This enables it to fool the application into thinking that it is running on such a machine while in reality these registers are used by the instrumentation code. lMem translates IA-32 programs to x64 and uses additional registers and memory for instrumentation. To gauge the performance of this approach we implement a simple application on top of lMem which enables to insert an arbitrary number of watchpoints. We observe that this lMem-based application has overheads of only 50% to 200% on SPEC CPU 2006 benchmarks.

|As both the number of processors and the complexity of problems to be solved increase, programming multiprocessing systems becomes more di cult and errorprone. This paper discusses programming assistance and automation concepts and their application to a program development tool for message-passing systems called Hypertool. It performs scheduling and handles the communication primitive insertion automatically. Two algorithms, based on the critical-path method, are presented for scheduling processes statically. Hypertool also generates the performance estimates and other program quality measures to help programmers in improving their algorithms and programs.

Two key ideas in garbage collection are generational collection and conservative pointer-finding. Generational collection and conservative pointer-finding are hard to use together, because generational collection is usually expressed in terms of copying objects, while conservative pointer-finding precludes copying. We present a new framework for defining garbage collectors. When applied to generational collection, it generalizes the notion of younger/older to a partial order. It can describe traditional generational and conservative techniques, and lends itself to combining different techniques in novel ways. We study in particular two new garbage collectors inspired by this framework. Both these collectors use conservative pointer-finding. The first one is based on a rewrite of an existing trace-and-sweep collector to use one level of generation. The second one has a single parameter, which controls how objects are partitioned into generations: the value of this parameter can be changed dynamically with no overhead. We have implemented both collectors and present measurements of their performance in practice.

Efforts to build "secure" computer systems have now been underway for more than a decade. Many designs have been proposed, some prototypes have been constructed, and a few systems are approaching the production stage. A small number of systems are even operating in what the Department of Defense calls the "multilevel" mode some information contained m these computer systems may have a clasmfication higher than the clearance of some of the users of those systems. This paper revmws the need for formal security models, describes the structure and operation of military security controls, considers how automation has affected security problems, surveys models that have been proposed and applied to date, and suggests possible d~rectlons for future models

Monitoring is an essential part of many program development tools, and plays a central role in debugging, optimization, status reporting, and reconfiguration. Traditional monitoring techniques are inadequate when monitoring complex systems such as multiprocessors or distributed systems. A new approach is described in which a historical database forms the conceptual basis for the information processed by the monitor. This approach permits advances in specifying the low-level data collection, specifying the analysis of the collected data, performing the analysis, and displaying the results. Two prototype implementations demonstrate the feasibility of the approach.

The importance of software security cannot be overstated. In the past, researchers have applied program analysis techniques to automatically detect security vulnerabilities and verify security properties. However, such techniques have limited success in reality because they require manually provided code-level security specifications. Manually writing and generating these code-level security specifications are tedious and error-prone. Additionally, they seldom exist in production software. In this paper, we propose a novel method and tool, called AutoISES, which Automatically Infers Security Specifications by statically analyzing source code, and then directly use these specifications to automatically detect security violations. Our experiments with the Linux kernel and Xen demonstrated the effectiveness of this approach – AutoISES automatically generated 84 security specifications and detected 8 vulnerabilities in the Linux kernel and Xen, 7 of which have already been confirmed by the corresponding developers.


Question: A composable, deadlock-free method for implementing object-based isolation.

This paper advocates for the development of a versatile and lightweight infrastructure designed to enhance parallel graph analytics, capable of accommodating several domain-specific languages (DSLs) currently in use. We propose that these DSLs can be effectively executed atop a comprehensive framework that (i) enables the execution of very fine-grained tasks, (ii) facilitates autonomous and speculative task execution, and (iii) offers customizable task scheduling policies tailored to specific applications. To substantiate our argument, we present an implementation, dubbed the Galois system, which embodies these characteristics.

The Galois system's potential is showcased through three distinct avenues. Firstly, we employ it to implement more advanced algorithms for graph analytics tasks that have been previously addressed by DSLs. This results in substantial performance enhancements, often by several orders of magnitude, even on complex power-law graphs, due to the enriched algorithmic capabilities facilitated by a more flexible programming model. Secondly, we illustrate that when algorithms can be articulated within existing DSLs, their implementation within a more generalized system, such as Galois, can achieve significantly faster execution, particularly on graphs with high diameters like road networks, owing to enhanced scheduling techniques. Lastly, we demonstrate the interoperability of Galois by implementing the application programming interfaces (APIs) of three established graph DSLs within just a few hundred lines of code. We then compare the performance of these implementations to the original DSL systems, and interestingly, find that Galois often outperforms them, even on power-law graphs, due to its lightweight and efficient infrastructure.

Attacks on host computers by malicious peripherals are a growing problem. Inexpensive and powerful peripherals, which attach to plug-and-play buses, have made such attacks easy to mount. Making matters worse, commodity operating systems lack systematic defenses, and users are often not aware of the scope of the problem. We present Cinch, a pragmatic response to this threat. Cinch uses virtualization to place the hardware in a logically separate, untrusted machine, and includes an interposition layer between the untrusted machine and the protected one. This layer accepts or rejects interaction with devices and enforces security policies that are easily configured and extended by users. We show that Cinch integrates with existing OSes, enforces policies that thwart real world attacks, and has low overhead.

Efforts to build "secure" computer systems have now been underway for more than a decade. Many designs have been proposed, some prototypes have been constructed, and a few systems are approaching the production stage. A small number of systems are even operating in what the Department of Defense calls the "multilevel" mode some information contained m these computer systems may have a clasmfication higher than the clearance of some of the users of those systems. This paper revmws the need for formal security models, describes the structure and operation of military security controls, considers how automation has affected security problems, surveys models that have been proposed and applied to date, and suggests possible d~rectlons for future models

A virtual machine can support individual processes or a complete system depending on the abstraction level where virtualization occurs. Some VMs support flexible hardware usage and software isolation, while others translate from one instruction set to another. Virtualizing a system or component -such as a processor, memory, or an I/O device - at a given abstraction level maps its interface and visible resources onto the interface and resources of an underlying, possibly different, real system. Consequently, the real system appears as a different virtual system or even as multiple virtual systems. Interjecting virtualizing software between abstraction layers near the HW/SW interface forms a virtual machine that allows otherwise incompatible subsystems to work together. Further, replication by virtualization enables more flexible and efficient and efficient use of hardware resources.

Rich client-side applications written in HTML5 proliferate diverse platforms such as mobile devices, commodity PCs, and the web platform. These client-side HTML5 applications are increasingly accessing sensitive data, including users’ personal and social data, sensor data, and capability-bearing tokens. To fulfill their security and privacy guarantees, these applications need to maintain certain data-confinement invariants. These invariants are not explicitly stated in today’s HTML5 applications and are enforced using ad-hoc mechanisms. The complexity of web applications, coupled with hard-to-analyze client-side languages, leads to low-assurance data-confinement mechanisms in which the whole application needs to be in the TCB to ensure data-confinement in-

The level of demand for bare-metal cloud services has increased rapidly because such services are cost-effective for several types of workloads, and some cloud clients prefer a single-tenant environment due to the lower security vulnerability of such enviornments. However, as the bare-metal cloud does not utilize a virtualization layer, it cannot use live migration. Thus, there is a lack of manageability with the bare-metal cloud. Live migration support can improve the manageability of bare-metal cloud services significantly.
 This paper suggests an on-demand virtualization technique to improve the manageability of bare-metal cloud services. A thin virtualization layer is inserted into the bare-metal cloud when live migration is requested. After the completion of the live migration process, the thin virtualization layer is removed from the host. We modified BitVisor [19] to implement on-demand virtualization and live migration on the x86 architecture.
 The elapsed time of on-demand virtualization was negligible. It takes about 20 ms to insert the virtualization layer and 30 ms to remove the one. After removing the virtualization layer, the host machine works with bare-metal performance.

Android applications often include third-party libraries written in native code. However, current native components are not well managed by Android's security architecture. We present NativeGuard, a security framework that isolates native libraries from other components in Android applications. Leveraging the process-based protection in Android, NativeGuard isolates native libraries of an Android application into a second application where unnecessary privileges are eliminated. NativeGuard requires neither modifications to Android nor access to the source code of an application. It addresses multiple technical issues to support various interfaces that Android provides to the native world. Experimental results demonstrate that our framework works well with a set of real-world applications, and incurs only modest overhead on benchmark programs.

Information security can be compromised by leakage via low-level hardware features. One recently prominent example is cache probing attacks, which rely on timing channels created by caches. We introduce a hardware design language, SecVerilog, which makes it possible to statically analyze information flow at the hardware level. With SecVerilog, systems can be built with verifiable control of timing channels and other information channels. SecVerilog is Verilog, extended with expressive type annotations that enable precise reasoning about information flow. It also comes with rigorous formal assurance: we prove that SecVerilog enforces timing-sensitive noninterference and thus ensures secure information flow. By building a secure MIPS processor and its caches, we demonstrate that SecVerilog makes it possible to build complex hardware designs with verified security, yet with low overhead in time, space, and HW designer effort.

The real-time traffic control of railway networks authorizes movements of the trains and imposes safety constraints. The paper deals with the real time traffic control focusing on deadlock prevention problem. Colored Petri nets are used to model the dynamics of the railway network system: places represent tracks and stations, tokens are trains. The prevention policy is expressed by a set of linear inequality constraints, called colored Generalized Mutual Exclusion Constraints that are enforced by adding appropriate monitor places. Using digraph tools, deadlock situations are characterized and a strategy is established to define off-line a set of Generalized Mutual Exclusion Constraints that prevent deadlock. An example shows in detail the design of the proposed control logic.

We present a new programming model for deterministic parallel computation in a pure functional language. The model is monadic and has explicit granularity, but allows dynamic construction of dataflow networks that are scheduled at runtime, while remaining deterministic and pure. The implementation is based on monadic concurrency, which has until now only been used to simulate concurrency in functional languages, rather than to provide parallelism. We present the API with its semantics, and argue that parallel execution is deterministic. Furthermore, we present a complete work-stealing scheduler implemented as a Haskell library, and we show that it performs at least as well as the existing parallel programming models in Haskell.

This paper presents a systematic solution to the persistent problem of buffer overflow attacks. Buffer overflow attacks gained notoriety in 1988 as part of the Morris Worm incident on the Internet. While it is fairly simple to fix individual buffer overflow vulnerabilities, buffer overflow attacks continue to this day. Hundreds of attacks have been discovered, and while most of the obvious vulnerabilities have now been patched, more sophisticated buffer overflow attacks continue to emerge. We describe StackGuard: a simple compiler technique that virtually eliminates buffer overflow vulnerabilities with only modest performance penalties. Privileged programs that are recompiled with the StackGuard compiler extension no longer yield control to the attacker, but rather enter a fail-safe state. These programs require no source code changes at all, and are binary-compatible with existing operating systems and libraries. We describe the compiler technique (a simple patch to gcc), as well as a set of variations on the technique that tradeoff between penetration resistance and performance. We present experimental results of both the penetration resistance and the performance impact of this technique. This research is partially supported by DARPA contracts F3060296-1-0331 and F30602-96-1-0302. Ryerson Polytechnic University


Question: A composable, deadlock-free method for implementing object-based isolation.

This study delves into the multifaceted nature of user influence on Twitter, challenging the simplistic notion that follower count is the sole indicator of influence. By analyzing a substantial dataset from Twitter, we compare and contrast three key metrics—follower count (indegree), retweets, and mentions—to gauge user influence across different topics and over time. Our findings reveal several intriguing patterns. Firstly, we observe that users with a high number of followers (indegree) do not necessarily translate that popularity into high retweet or mention rates, indicating that influence is more complex than mere follower count. Secondly, the most influential users often wield significant sway across a diverse range of topics, demonstrating a broad-based impact. Thirdly, we find that achieving influence is not a random occurrence but requires strategic effort, such as focusing tweets on a specific topic to build expertise and engagement. These insights offer a nuanced understanding of influence on social media, suggesting that topological metrics like follower count alone do not fully capture a user's true influence, particularly in the context of viral marketing strategies.

**Question**: A Novel Approach for Predicting Stock Prices Using Machine Learning Algorithms

**Documents**: A Deep Learning Approach for Brain Tumor Detection in MRI Images
Brain tumor detection is a challenging task for radiologists, especially in the early stages when the tumor is not clearly visible. This paper presents a deep learning approach for brain tumor detection in MRI images. The proposed method uses a convolutional neural network (CNN) for feature extraction and a support vector machine (SVM) for classification. The CNN is trained on a large dataset of labeled MRI images to learn the features that are discriminative for tumor detection. The SVM is then trained on the extracted features to classify the images as tumor or non-tumor. The proposed method is evaluated on a dataset of 220 MRI images, including 110 tumor and 110 non-tumor images. The results show that the proposed method achieves an accuracy of 97.7% in tumor detection, which is higher than the accuracy of traditional machine learning methods such as random forest and decision tree. The proposed method can assist radiologists in early detection of brain tumors and improve the accuracy of diagnosis.

**Similar Documents**: This research introduces a cutting-edge deep learning methodology for the detection of brain tumors in magnetic resonance imaging (MRI) scans, addressing the diagnostic challenges faced by radiologists, particularly in identifying tumors at their early, less discernible stages. Our approach combines the power of convolutional neural networks (CNNs) for feature extraction

Technological progress in integrated, low-power, CMOS communication devices and sensors makes a rich design space of networked sensors viable. They can be deeply embedded in the physical world and spread throughout our environment like smart dust. The missing elements are an overall system architecture and a methodology for systematic advance. To this end, we identify key requirements, develop a small device that is representative of the class, design a tiny event-driven operating system, and show that it provides support for efficient modularity and concurrency-intensive operation. Our operating system fits in 178 bytes of memory, propagates events in the time it takes to copy 1.25 bytes of memory, context switches in the time it takes to copy 6 bytes of memory and supports two level scheduling. The analysis lays a groundwork for future architectural advances.

Commercial buildings are attractive targets for introducing innovative cyber-physical control systems, because they are already highly instrumented distributed systems which consume large quantities of energy. However, they are not currently programmable in a meaningful sense because each building is constructed with vertically integrated, closed subsystems and without uniform abstractions to write applications against. We develop a set of operating system services called BOSS, which supports multiple portable, fault-tolerant applications on top of the distributed physical resources present in large commercial buildings. We evaluate our system based on lessons learned from deployments of many novel applications in our test building, a four-year-old, 140,000sf building with modern digital controls, as well as partial deployments at other sites.

Sego is a hypervisor-based system that gives strong privacy and integrity guarantees to trusted applications, even when the guest operating system is compromised or hostile. Sego verifies operating system services, like the file system, instead of replacing them. By associating trusted metadata with user data across all system devices, Sego verifies system services more efficiently than previous systems, especially services that depend on data contents. We extensively evaluate Sego's performance on real workloads and implement a kernel fault injector to validate Sego's file system-agnostic crash consistency and recovery protocol.

Many would argue that future breakthroughs in software productivity will depend on our ability to combine existing pieces of software to produce new applications. An important step towards this goal is the development of new techniques to detect and cope with mismatches in the assembled parts. Some problems of composition are due to low-level issues of interoperability, such as mismatches in programming languages or database schemas. However, in this paper we highlight a different, and in many ways more pervasive, class of problem: architectural mismatch. Specifically, we use our experience in building a family of software design environments from existing parts to illustrate a variety of types of mismatch that center around the assumptions a reusable part makes about the structure of the application in which is to appear. Based on this experience we show how an architectural view of the mismatch problem exposes some fundamental, thorny problems for software composition and suggests possible research avenues needed to solve them.

We present a new Inter-Process Communication (IPC) mechanism and library. Ach is uniquely suited for coordinating perception, control drivers, and algorithms in real-time systems that sample data from physical processes. Ach eliminates the Head-of-Line Blocking problem for applications that always require access to the newest message. Ach is efficient, robust, and formally verified. It has been tested and demonstrated on a variety of physical robotic systems. Finally, the source code for Ach is available under an Open Source BSD-style license.

This paper presents a modular introduction to crash-tolerant and Byzantine-tolerant protocols for reaching consensus that use the method introduced by the Paxos algorithm of Lamport and by the viewstamped replication algorithm of Oki and Liskov. The consensus protocol runs a sequence of epoch abstractions as governed by an epoch-change abstraction. Implementations of epoch and epoch-change that tolerate crash faults yield the consensus algorithm in Paxos and in viewstamped replication. Implementations of epoch and epoch-change that tolerate Byzantine faults yield the consensus algorithm in the PBFT protocol of Castro and Liskov.

At the CAiSE conference in Heidelberg in 1999, Wasim Sadiq and Maria Orlowska presented an algorithm to verify workflow graphs [19]. The algorithm uses a set of reduction rules to detect structural conflicts. This paper shows that the set of reduction rules presented in [19] is not complete and proposes an alternative algorithm. The algorithm translates workflow graphs into so-called WF-nets. WF-nets are a class of Petri nets tailored towards workflow analysis. As a result, Petri-net theory and tools can be used to verify workflow graphs. In particular, our workflow verification tool Woflan [21] can be used to detect design errors. It is shown that the absence of structural conflicts, i.e., deadlocks and lack of synchronization, conforms to soundness of the corresponding WF-net [2]. In contrast to the algorithm presented in [19], the algorithm presented in this paper is complete. Moreover, the complexity of this alternative algorithm is given.

Bugs due to data races in multithreaded programs often exhibit non-deterministic symptoms and are notoriously difficult to find. This paper describes RaceTrack, a dynamic race detection tool that tracks the actions of a program and reports a warning whenever a suspicious pattern of activity has been observed. RaceTrack uses a novel hybrid detection algorithm and employs an adaptive approach that automatically directs more effort to areas that are more suspicious, thus providing more accurate warnings for much less over-head. A post-processing step correlates warnings and ranks code segments based on how strongly they are implicated in potential data races. We implemented RaceTrack inside the virtual machine of Microsoft's Common Language Runtime (product version v1.1.4322) and monitored several major, real-world applications directly out-of-the-box,without any modification. Adaptive tracking resulted in a slowdown ratio of about 3x on memory-intensive programs and typically much less than 2x on other programs,and a memory ratio of typically less than 1.2x. Several serious data race bugs were revealed, some previously unknown.

Our analysis shows that many "big-memory" server workloads, such as databases, in-memory caches, and graph analytics, pay a high cost for page-based virtual memory. They consume as much as 10% of execution cycles on TLB misses, even using large pages. On the other hand, we find that these workloads use read-write permission on most pages, are provisioned not to swap, and rarely benefit from the full flexibility of page-based virtual memory.
 To remove the TLB miss overhead for big-memory workloads, we propose mapping part of a process's linear virtual address space with a direct segment, while page mapping the rest of the virtual address space. Direct segments use minimal hardware---base, limit and offset registers per core---to map contiguous virtual memory regions directly to contiguous physical memory. They eliminate the possibility of TLB misses for key data structures such as database buffer pools and in-memory key-value stores. Memory mapped by a direct segment may be converted back to paging when needed.
 We prototype direct-segment software support for x86-64 in Linux and emulate direct-segment hardware. For our workloads, direct segments eliminate almost all TLB misses and reduce the execution time wasted on TLB misses to less than 0.5%.

In order to guarantee recoverable transaction execution, database systems permit a transaction’s writes to be observable only at the end of its execution. As a consequence, there is generally a delay between the time a transaction performs a write and the time later transactions are permitted to read it. This delayed write visibility can significantly impact the performance of serializable database systems by reducing concurrency among conflicting transactions. This paper makes the observation that delayed write visibility stems from the fact that database systems can arbitrarily abort transactions at any point during their execution. Accordingly, we make the case for database systems which only abort transactions under a restricted set of conditions, thereby enabling a new recoverability mechanism, early write visibility, which safely makes transactions’ writes visible prior to the end of their execution. We design a new serializable concurrency control protocol, piece-wise visibility (PWV), with the explicit goal of enabling early write visibility. We evaluate PWV against state-of-the-art serializable protocols and a highly optimized implementation of read committed, and find that PWV can outperform serializable protocols by an order of magnitude and read committed by 3X on high contention workloads.


Question: A composable, deadlock-free method for implementing object-based isolation.

The Emergence of a Unified Parallel Processing Paradigm: The Bulk-Synchronous Parallel Model

The widespread adoption of the von Neumann model in sequential computation is largely due to its role as a robust bridge between software and hardware. High-level programming languages can be efficiently compiled for this model, while hardware implementations are equally efficient, facilitating seamless execution. To achieve a similar level of success in parallel computation, a comparable bridge is necessary. This paper posits the bulk-synchronous parallel (BSP) model as a potential solution to this need, serving as an effective conduit between software and hardware for parallel processing. The BSP model is evaluated for its efficiency in both executing high-level language constructs and algorithms, as well as its implementability in hardware. Results demonstrate its capability to bridge the gap, suggesting that the BSP model may become a unifying paradigm in the realm of parallel computation, much like the von Neumann model in sequential processing.

Previous research on kernel monitoring and protection widely relies on higher privileged system components, such as hardware virtualization extensions, to isolate security tools from potential kernel attacks. These approaches increase both the maintenance effort and the code base size of privileged system components, which consequently increases the risk of having security vulnerabilities. SKEE, which stands for Secure Kernellevel Execution Environment, solves this fundamental problem. SKEE is a novel system that provides an isolated lightweight execution environment at the same privilege level of the kernel. SKEE is designed for commodity ARM platforms. Its main goal is to allow secure monitoring and protection of the kernel without active involvement of higher privileged software. SKEE provides a set of novel techniques to guarantee isolation. It creates a protected address space that is not accessible to the kernel, which is challenging to achieve when both the kernel and the isolated environment share the same privilege level. SKEE solves this challenge by preventing the kernel from managing its own memory translation tables. Hence, the kernel is forced to switch to SKEE to modify the system’s memory layout. In turn, SKEE verifies that the requested modification does not compromise the isolation of the protected address space. Switching from the OS kernel to SKEE exclusively passes through a well-controlled switch gate. This switch gate is carefully designed so that its execution sequence is atomic and deterministic. These properties combined guarantee that a potentially compromised kernel cannot exploit the switching sequence to compromise the isolation. If the kernel attempts to violate these properties, it will only cause the system to fail without exposing the protected address space. SKEE exclusively controls access permissions of the entire OS memory. Hence, it prevents attacks that attempt to inject unverified code into the kernel. Moreover, it can be easily extended to intercept other system events in order to support various intrusion detection and integrity verification tools. This paper presents a SKEE prototype that runs on both 32-bit ARMv7 and 64-bit ARMv8 architectures. Performance evaluation results demonstrate that SKEE is a practical solution for real world systems. 1These authors contributed equally to this work

Existing dynamic determinacy race detectors for task-parallel programs are limited to programs with strict computation graphs, where a task can only wait for its descendant tasks to complete. In this paper, we present the first known determinacy race detector for non-strict computation graphs with futures. The space and time complexity of our algorithm are similar to those of the classical SP-bags algorithm, when using only structured parallel constructs such as spawn-sync and async-finish. In the presence of point-to-point synchronization using futures, the complexity of the algorithm increases by a factor determined by the number of future operations, which includes future task creation and future get operations. The experimental results show that the slowdown factor observed for our algorithm relative to the sequential version is in the range of 1.00x to 9.92x, which is very much in line with slowdowns experienced for fully strict computation graphs.

We have designed and implemented a new operating system facility for I/O buffer management and data transferacross protection domain boundaries on shared memory machines. This facility, called fast buffers (fbufs), combines virtual page remapping with shared virtual memory, and exploits locality in I/O traffic to achieve high throughput without compromising protection, security, or modularity. goal is to help deliver the high bandwidth afforded by emerging high-speed networks to user-level processes, both in monolithic and microkernel-based operating systems.This paper outlines the requirements for a cross-domain transfer facility, describes the design of the fbuf mechanism that meets these requirements, and experimentally quantifies the impact of fbufs on network performance.

As the number of system vulnerabilities multiplies in recent years, vulnerability assessment has emerged as a powerful system security administration tool that can identify vulnerabilities in existing systems before they are exploited. Although there are many commercial vulnerability assessment tools in the market, none of them can formally guarantee that the assessment process never compromises the computer systems being tested. This paper proposes a featherweight virtual machine (FVM) technology to address the safety issue associated with vulnerability testing. Compared with other virtual machine technologies, FVM is designed to facilitate sharing between virtual machines but still provides strong protection between them. The FVM technology allows a vulnerability assessment tool to test an exact replica of a production-mode network service, including both hardware and system software components, while guaranteeing that the production-mode network service is fully isolated from the testing process. In addition to safety, the vulnerability assessment support system described in this paper can also automate the entire process of vulnerability testing and thus for the first time makes it feasible to run vulnerability testing autonomously and frequently. Experiments on a Windows-based prototype show that Nessus assessment results against an FVM virtual machine are identical to those against a real machine. Furthermore, modifications to the file system and registry state made by vulnerability assessment runs are completely isolated from the host machine. Finally, the performance impact of vulnerability assessment runs on production network services is as low as 3%

This paper presents Bolt, a compiler-directed soft error recovery scheme, that provides fine-grained and guaranteed recovery without excessive performance and hardware overhead. To get rid of expensive hardware support, the compiler protects the architectural inputs during their entire liveness period by safely checkpointing the last updated value in idempotent regions. To minimize the performance overhead, Bolt leverages a novel compiler analysis that eliminates those checkpoints whose value can be reconstructed by other checkpointed values without compromising the recovery guarantee. As a result, Bolt incurs only 4.7% performance overhead on average which is 57% reduction compared to the state-of-the-art scheme that requires expensive hardware support for the same recovery guarantee as Bolt.

C4, the Continuously Concurrent Compacting Collector, an updated generational form of the Pauseless GC Algorithm [7], is introduced and described, along with details of its implementation on modern X86 hardware. It uses a read barrier to support concur- rent compaction, concurrent remapping, and concurrent incremental update tracing. C4 differentiates itself from other generational garbage collectors by supporting simultaneous-generational concurrency: the different generations are collected using concurrent (non stop-the-world) mechanisms that can be simultaneously and independently active. C4 is able to continuously perform concurrent young generation collections, even during long periods of concurrent full heap collection, allowing C4 to sustain high allocation rates and maintain the efficiency typical to generational collectors, without sacrificing response times or reverting to stop-the-world operation. Azul systems has been shipping a commercial implementation of the Pauseless GC mechanism, since 2005. Three successive generations of Azul's Vega series systems relied on custom multi-core processors and a custom OS kernel to deliver both the scale and features needed to support Pauseless GC. In 2010, Azul released its first software-only commercial implementation of C4 for modern commodity X86 hardware, using Linux kernel enhancements to support the required feature set. We discuss implementa- tion details of C4 on X86, including the Linux virtual and physical memory management enhancements that were used to support the high rate of virtual memory operations required for sustained pauseless operation. We discuss updates to the collector's manage- ment of the heap for efficient generational collection and provide throughput and pause time data while running sustained workloads.

Data containers enable users to control access to their data while untrusted applications compute on it. However, they require replicating an application inside each container - compromising functionality, programmability, and performance. We propose DATS - a system to run web applications that retains application usability and efficiency through a mix of hardware capability enhanced containers and the introduction of two new primitives modeled after the popular model-view-controller (MVC) pattern. (1) DATS introduces a templating language to create views that compose data across data containers. (2) DATS uses authenticated storage and confinement to enable an untrusted storage service, such as memcached and deduplication, to operate on plain-text data across containers. These two primitives act as robust declassifiers that allow DATS to enforce non-interference across containers, taking large applications out of the trusted computing base (TCB). We showcase eight different web applications including Gitlab and a Slack-like chat, significantly improve the worst-case overheads due to application replication, and demonstrate usable performance for common-case usage.

This paper presents SiRiUS, a secure file system designed to be layered over insecure network and P2P file systems such as NFS, CIFS, OceanStore, and Yahoo! Briefcase. SiRiUS assumes the network storage is untrusted and provides its own read-write cryptographic access control for file level sharing. Key management and revocation is simple with minimal out-of-band communication. File system freshness guarantees are supported by SiRiUS using hash tree constructions. SiRiUS contains a novel method of performing file random access in a cryptographic file system without the use of a block server. Extensions to SiRiUS include large scale group sharing using the NNL key revocation construction. Our implementation of SiRiUS performs well relative to the underlying file system despite using cryptographic operations.

This paper presents a new and very rich class of (concurrent) programming languages, based on the notion of computing with partial information, and the concomitant notions of consistency and entailment.1 In this framework, computation emerges from the interaction of concurrently executing agents that communicate by placing, checking and instantiating constraints on shared variables. Such a view of computation is interesting in the context of programming languages because of the ability to represent and manipulate partial information about the domain of discourse, in the context of concurrency because of the use of constraints for communication and control, and in the context of AI because of the availability of simple yet powerful mechanisms for controlling inference, and the promise that very rich representational/programming languages, sharing the same set of abstract properties, may be possible.
To reflect this view of computation, [Sar89] develops the cc family of languages. We present here one member of the family, cc(↓, →) (pronounced “cc with Ask and Choose”) which provides the basic operations of blocking Ask and atomic Tell and an algebra of behaviors closed under prefixing, indeterministic choice, interleaving, and hiding, and provides a mutual recursion operator. cc(↓, →) is (intentionally!) very similar to Milner's CCS, but for the radically different underlying concept of communication, which, in fact, provides a general—and elegant—alternative approach to “value-passing” in CCS. At the same time it adequately captures the framework of committed choice concurrent logic programming languages. We present the rules of behavior for cc agents, motivate a notion of “visible action” for them, and develop the notion of c-trees and reactive congruence analogous to Milner's synchronization trees and observational congruence. We also present an equational characterization of reactive congruence for Finitary cc(↓, →).

Rich client-side applications written in HTML5 proliferate diverse platforms such as mobile devices, commodity PCs, and the web platform. These client-side HTML5 applications are increasingly accessing sensitive data, including users’ personal and social data, sensor data, and capability-bearing tokens. To fulfill their security and privacy guarantees, these applications need to maintain certain data-confinement invariants. These invariants are not explicitly stated in today’s HTML5 applications and are enforced using ad-hoc mechanisms. The complexity of web applications, coupled with hard-to-analyze client-side languages, leads to low-assurance data-confinement mechanisms in which the whole application needs to be in the TCB to ensure data-confinement in-