Question: The Utilibot initiative: a utilitarianism-driven autonomous mobile robot design.

This study delves into the dynamics of daily well-being, focusing on the pivotal role of autonomy, competence, and relatedness in shaping emotional well-being. Traditionally, emotional well-being has been examined through a trait lens, but recent research underscores the significance of within-person variations in well-being. Our investigation posits that these daily fluctuations can be attributed to the fulfillment of three fundamental needs: autonomy, competence, and relatedness. Employing hierarchical linear modeling, we analyzed a two-week dataset of daily activities and well-being reports, accounting for individual differences at the trait level. The findings robustly validated our hypothesis, highlighting the critical influence of need satisfaction on daily well-being. We further explored the social activities that enhance relatedness needs, identifying meaningful conversations and the sense of being understood and appreciated by others as key predictors. Additionally, the study uncovered a pattern of day-of-the-week variations in well-being and need fulfillment. These results are discussed in the context of the essential role of daily activities and the necessity of considering both trait and day-level factors in understanding well-being.

The Oz project at Carnegie Mellon is studying the construction of artistically e ective simulated worlds. Such worlds typically include several agents, which must exhibit broad behavior. To meet this need, we are developing an agent architecture, called Tok, that presently supports reactivity, goals, emotions, and social behavior. Here we brie y introduce the requirements of our application, summarize the Tok architecture, and describe a particular agent we have constructed that exhibits the desired social behavior. 1 THE OZ PROJECT AND BROAD AGENTS 1 1 The Oz Project and Broad Agents The Oz project at Carnegie Mellon University is developing technology for artistically interesting simulated worlds [3]. We want to let human users participate in dramatically e ective worlds that include moderately competent, emotional agents. We work with artists in the CMU Drama and English Departments, to help focus our technology on genuine artistic needs. An Oz world has four primary components. There is a simulated physical environment, a set of automated agents which help populate the world, a user interface to allow one or more people to participate in the world [14], and a planner concerned with the long term structure of the user's experience [2]. One of the keys to an artistically engaging experience is for the user to be able to \suspend disbelief". That is, the user must be able to imagine that the world portrayed is real, without being jarred out of this belief by the world's behavior. The automated agents, in particular, must not be blatantly unreal. Thus, part of our e ort is aimed at producing agents with a broad set of capabilities, including goal-directed reactive behavior, emotional state and behavior, social knowledge and behavior, and some natural language abilities. For our purpose, each of these capacities can be as limited as is necessary to allow us to build broad, integrated agents [4]. Oz worlds can be simpler than the real world, but they must retain su cient complexity to serve as interesting artistic vehicles. The complexity level seems to be somewhat higher, but not exceptionally higher, than typical AI micro-worlds. Despite these simpli cations, we nd that our agents must deal with imprecise and erroneous perceptions, with the need to respond rapidly, and with a general inability to fully model the agent-rich world they inhabit. Thus, we suspect that some of our experience with broad agents in Oz may transfer to the domain of social, real-world robots. Building broad agents is a little studied area. Much work has been done on building reactive systems [1, 6, 7, 10, 11, 22], natural language systems (which we do not discuss here), and even emotion systems [9, 18, 20]. There has been growing interest in integrating action and learning (see [16]) and some very interesting work on broader integration [23, 19]. However, we are aware of no other e orts to integrate the particularly wide range of capabilities needed in the Oz domain. Here we present our e orts, focusing on the structure of a particular agent designed to exhibit goaldirected reactive behavior, emotion, and some social behavior. Further discussion of the integration issues can be found in [5].

The paper discussu some antral irrw in the cooperation between disuibuted agcnu using lhe following case study: The objective is to explore a distant planet. more concretely to collect samples of a parricular typ of precious ruck. The location of the ruck .iamplu is unknown in advance but they arc rypiwlly clustered in cenain spm There i S a vehicle that can drive around on the planet and later reenter the s p x d (0 g0 back to ewlh There is no deuiled map of the t e m h although it h known hat the terrain is fuU of obstacles. hills. valleys. etc.

Higher efficiency and the safety of hull inspection can be achieved by replacing the conventional methods with modern crawling robots. This paper consists of detailed study of the design and development of a crawling robot with a novel permanent magnetic wheel based on Halbach magnetic array. The magnetic array and the wheel assembly were designed based on the magnetic simulation results. Maintaining the adequate adhesion force as well as the friction force of the wheels were considered in designing the wheels. The crawling robot is equipped with a steering system based on Ackermann steering method hence the steering angles are controlled by an algorithm and actuated using servo motors. The driving system of the robot is equipped with four geared stepper motors and they are controlled separately by the algorithm according to the inputs of the controller. The control architecture and the components used for the robot control system are explained. The operator can remotely control the robot and inspect the hull surface via the camera feed available in the HMI (Human Machine Interface) of the robot. The prototype of the crawling robot was tested in a similar environment and the test results are included in the paper.

Modern robotic technologies have provided efficient solutions to protect workers from hazards in the work environments; such as radioactive, toxic, or explosive. This paper presents a design of a holonomic mobile robot for rough terrain that can replace the human role in the demining applications. A suspension system based on the bogie mechanism was adopted in order to overcome rough terrains. The flexible design of the robot enables it to move in any direction at any instant without changing the orientation of the body. The motion of the robot was simulated using the Pro-Engineer software and the robot stability margins for different road conditions were found in order to predict the possibility of tipping-over before any maneuver.

This paper details the design, modeling, and experimental validation of a novel mobile robot capable of both aerial and terrestrial locomotion. Flight is achieved through a quadrotor configuration; four actuators provide the required thrust. Adding a rolling cage to the quadrotor makes terrestrial locomotion possible using the same actuator set and control system. Thus, neither the mass nor the system complexity is increased by inclusion of separate actuators for terrestrial and aerial locomotion. An analysis of the system's energy consumption demonstrates that during terrestrial locomotion, the robot only needs to overcome rolling resistance and consumes much less energy compared to the aerial mode. This solves one of the most vexing problems of quadrotors and rotorcraft in general - their short operation time. Experimental results show that the hybrid robot can travel a distance four times greater and operate almost six times longer than an aerial only system. It also solves one of the most challenging problems in terrestrial robot design - obstacle avoidance. When an obstacle is encountered, the system simply flies over it.

This paper would present the system integration and overview of the autonomous cleaning robot Roboking. Roboking is self-propelled autonomously navigating vacuum cleaning robot. It uses several sensors in order to protect indoor environments and itself while cleaning. In this paper, we would describe the principle of operation along with system structure, sensors, functions and integrated subsystems.

The potential impact of autonomous robotics is magnified when those robots are deployed in teams: a team of cooperating robots can greatly increase the effectiveness of a human working alone, making short work of search-and-rescue and reconnaissance tasks. To achieve this potential, however, a number of challenging problems ranging from multi-robot planning, state estimation, object detection, and human-robot interfaces must first be solved. The MAGIC 2010 competition, like the DARPA grand challenges that preceded it, presented a formidable robotics problem designed to foster fundamental advances in these difficult areas. MAGIC asked teams of robots to collaboratively explore and map a 500× 500m area, detect and track benign and dangerous objects, and collaborate with human commanders while respecting their cognitive limits. This paper describes our winning entry in the MAGIC contest, where we fielded a team of 14 autonomous robots supervised by two human operators. While the challenges in MAGIC were diverse, we believe that cooperative multi-robot state estimation is ultimately the critical factor in building a successful system. In this paper, we describe our system and some of the technological advances that we believe were responsible for our success. We also contrast our approach to those of other teams. a)

The Carolo-Cup competition conducted for the eighth time this year, is an international student competition focusing on autonomous driving scenarios implemented on 1:10 scale car models. Three practical sub-competitions have to be realized in this context and represent a complex, interdisciplinary challenge. Hence, students have to cope with all core topics like mechanical development, electronic design, and programming as addressed usually by robotic applications. In this paper we introduce the competition challenges in detail and evaluate the results of all 13 participating teams from the 2014 competition. For this purpose, we analyze technical as well as non-technical configurations of each student group and derive best practices, lessons learned, and criteria as a precondition for a successful participation. Due to the comprehensive orientation of the Carolo-Cup, this knowledge can be applied on comparable projects and related competitions as well.

C onstruction is of prime economic significance to many industry sectors. Intense competition, shortages of skilled labor, and technological advances are forcing rapid change in the construction industry, thus motivating construction automation [1]. Earthmoving machines, such as bulldozers, wheel loaders, excavators, scrapers, and graders, are common in construction. Excavationbased operations are used in general earthmoving, digging, and sheet-piling for displacing large amounts of material. On a smaller scale, operations such as trenching and footing formation require precisely controlled excavation. Although the fully automated construction site is still a dream of some civil engineers, research developments have shown the promise of robotics and automation in construction [2]. Despite the apparent economic importance of excavation in construction, there have been few implementations of autonomous or teleoperated excavators. A number of researchers have investigated the feasibility of automating excavation. Many of these studies have addressed the possible use of autonomous excavators during unmanned phases of establishing manned Lunar or Martian research stations [3], [4]. Much of the work on terrestrial excavation has focused on teleoperation, rather than on the system requirements for autonomous operation. Although there have been a number of valuable theoretical and experimental contributions to the field of autonomous, robotic or teleoperated excavation [5]-[8], autonomous operation of a full-scale excavator has not been commercially demonstrated. Many of the experimental studies reported in the literature involve using conventional industrial robots fitted with buckets to excavate in a bed of loose sand. While there are parallels between “classical” robotics and robotic excavation, there are also some pronounced differences. In particular, an excavator is not fixed relative to the work piece; it plastically deforms the work piece by applying large forces and is caused to move relative to the soil by the same large forces. Furthermore, strategic and bucket trajectory planning must necessarily occur in a dynamic environment; if the excavator is not changing the profile of the soil being worked, it is not doing useful work. This article presents some results of the autonomous excavation project conducted at the Australian Centre for Field Robotics (ACFR) with a focus on construction automation. The application of robotic technology and computer control is one key to construction industry automation. Excavation automation is a multidisciplinary task, encompassing a broad area of research and development planning monitoring environment sensing and modeling navigation machine modeling and control. The ultimate goal of the ACFR excavation project is to demonstrate fully autonomous execution of excavation tasks in

This paper presents design research conducted as part of a larger project on human-robot interaction. The primary goal of this study was to come to an initial understanding of what features and dimensions of a humanoid robot's face most dramatically contribute to people's perception of its humanness. To answer this question we analyzed 48 robots and conducted surveys to measure people's perception of each robot's humanness. Through our research we found that the presence of certain features, the dimensions of the head, and the total number of facial features heavily influence the perception of humanness in robot heads. This paper presents our findings and initial guidelines for the design of humanoid robot heads.


Question: The Utilibot initiative: a utilitarianism-inspired autonomous mobile robot design.

This document explores the intricate connections between personal projects, subjective well-being, and the pursuit of meaning, adapting the framework of Personal Projects Analysis (PPA; B. R. Little, 1983). Through two studies involving university students (n=146 and n=179), the research uncovers how appraisals of goal characteristics correlate with orthogonal factors of happiness and meaning derived from various well-being metrics. Key findings reveal that goal efficacy correlates with happiness, whereas goal integrity is more closely tied to meaning. A novel methodology for categorizing participants based on emergent identity themes is introduced, highlighting identity-compensatory patterns in happiness. Specifically, agentic individuals experienced the highest levels of happiness when their goals were endorsed by others, communal individuals felt happiest when their goals were enjoyable, and hedonistic individuals reported peak happiness when their goals were being achieved. The document underscores the distinct nature of happiness and meaning, and delves into the inherent tension between efficacy and integrity. Developmental implications are explored, drawing on archival data from a cohort of senior managers. This study not only enriches our understanding of the interplay between personal goals and well-being but also offers insights into the nuanced relationship between happiness and meaning across different stages of life.

—constructing intelligent machines, whether or not these operate in the same way as people do; —formalizing knowledge and mechanizing reasoning, both commonsense and refined expertise, in all areas of human endeavor; —using computational models to understand the psychology and behavior of people, animals, and artificial agents; and —making working with computers as easy and as helpful as working with skilled, cooperative, and possibly expert people.

1 . Introduction This paper provides an overview of the Autonomous Land Vehicle (ALV) Project at CMU. The goal of the CMU ALV Project is to build vision and intelligence for a mobile robot capable of operating in the real world outdoors. We are attacking this on a number of fronts: building appropriate research vehicles, exploiting high. speed experimental computers, and building software for reasoning about the perceived world. Research topics includes: • Construction of research vehicles • Perception systems to perceive the natural outdoor scenes by means of multiple sensors including cameras (color, stereo, and motion), sonar sensors, and a 3D range finder • Path planning for obstacle avoidance Use of topological and terrain map • System architecture to facilitate the system integration • Utilization of parallel computer architectures Our current research vehicle is the Terregator built at CMU which is equipped with a sonar ring, a color camera, and the ERIM laser range finder. Its initial task is to follow roads and sidewalks in the park and on the campus, and avoid obstacles such as trees, humans, and traffic cones. 2. Vehicle, Sensors, and Host Computers The primary vehicle of the AMU ALV Project has been the Terregator, designed and built at CMU. The Terregator, short for Terrestrial Navigator, is designed to provide a clean separation between the vehicle itself and its sensor payload. As shown in

Review of Semantic-Free Utterances in Social Human–Robot Interaction Selma Yilmazyildiz, Robin Read, Tony Belpeame & Werner Verhelst To cite this article: Selma Yilmazyildiz, Robin Read, Tony Belpeame & Werner Verhelst (2016) Review of Semantic-Free Utterances in Social Human–Robot Interaction, International Journal of Human–Computer Interaction, 32:1, 63-85, DOI: 10.1080/10447318.2015.1093856 To link to this article: http://dx.doi.org/10.1080/10447318.2015.1093856

Robotic Development Environments (RDEs) have come to play an increasingly important role in robotics research in general, and for the development of architectures for mobile robots in particular. Yet, no systematic evaluation of available RDEs has been performed; establishing a comprehensive list of evaluation criteria targeted at robotics applications is desirable that can subsequently be used to compare their strengths and weaknesses. Moreover, there are no practical evaluations of the usability and impact of a large selection of RDEs that provides researchers with the information necessary to select an RDE most suited to their needs, nor identifies trends in RDE research that suggest directions for future RDE development. This survey addresses the above by selecting and describing nine open source, freely available RDEs for mobile robots, evaluating and comparing them from various points of view. First, based on previous work concerning agent systems, a conceptual framework of four broad categories is established, encompassing the characteristics and capabilities that an RDE supports. Then, a practical evaluation of RDE usability in designing, implementing, and executing robot architectures is presented. Finally, the impact of specific RDEs on the field of robotics is addressed by providing a list of published applications and research projects that give concrete examples of areas in which systems have been used. The comprehensive evaluation and comparison of the nine RDEs concludes with suggestions of how to use the results of this survey and a brief discussion of future trends in RDE design.

Believable agents are autonomous agents that exhibit rich personalities. Interactive dramas take place in virtual worlds inhabited by characters (believable agents) with whom an audience interacts. In the course of this interaction, the audience experiences a story (lives a plot arc). This report presents the research philosophy behind the Oz Project, a research group at CMU that has spent the last ten years studying believable agents and interactive drama. The report then surveys current work from an Oz perspective. This research was partially supported by a grant from Intel Corporation. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of Intel Corporation.

The potential impact of autonomous robotics is magnified when those robots are deployed in teams: a team of cooperating robots can greatly increase the effectiveness of a human working alone, making short work of search-and-rescue and reconnaissance tasks. To achieve this potential, however, a number of challenging problems ranging from multi-robot planning, state estimation, object detection, and human-robot interfaces must first be solved. The MAGIC 2010 competition, like the DARPA grand challenges that preceded it, presented a formidable robotics problem designed to foster fundamental advances in these difficult areas. MAGIC asked teams of robots to collaboratively explore and map a 500× 500m area, detect and track benign and dangerous objects, and collaborate with human commanders while respecting their cognitive limits. This paper describes our winning entry in the MAGIC contest, where we fielded a team of 14 autonomous robots supervised by two human operators. While the challenges in MAGIC were diverse, we believe that cooperative multi-robot state estimation is ultimately the critical factor in building a successful system. In this paper, we describe our system and some of the technological advances that we believe were responsible for our success. We also contrast our approach to those of other teams. a)

The Effects of Intervention Using a Robot on the Levels of Compliance of Four Children with Autism to Requests Given by Their Mothers Holly J. Nelson Department of Communication Disorders Master of Science The current study presents the use of a humanoid robot to facilitate compliant behaviors to two types of directives in four children with autism. The children participated in a three-month intervention program that incorporated core components of the SCERTS model in order to facilitate social communication (Prizant, Wetherby, Rubin, & Laurent, 2003). Treatment sessions were comprised of 40 minutes of traditional treatment and 10 minutes of interaction with a humanoid robot. Preand post-intervention assessment were conducted, each involving a 5 minute interaction with the child’s mother, in which they were presented with directives in the form of physical manipulation and verbal requests accompanied by a gesture or model. These preand post-intervention sessions were recorded, analyzed, and coded for compliant and noncompliant behavior to the directives, as well as any eye contact, language, or reciprocal action that accompanied their behavior. The overall results were variable, revealing that two participants made notable gains, one child remained consistent, and another participant showed a decrease in compliant behavior in the post-intervention sessions. Further research should be conducted to include a longer period of baseline and intervention, more systematic identification of the most effective probes for the child, and documentation of the child’s physical and emotional state.

To explore issues of developmental structure, physical embodiment, integration of multiple sensory and motor systems, and social interaction, we have constructed an upper-torso humanoid robot called Cog. The robot has twenty-one degrees of freedom and a variety of sensory systems, including visual, auditory, vestibular, kinesthetic, and tactile senses. This chapter gives a background on the methodology that we have used in our investigations, highlights the research issues that have been raised during this project, and provides a summary of both the current state of the project and our long-term goals. We report on a variety of implemented visual-motor routines (smooth-pursuit tracking, saccades, binocular vergence, and vestibular-ocular and opto-kinetic reflexes), orientation behaviors, motor control techniques, and social behaviors (pointing to a visual target, recognizing joint attention through face and eye finding, imitation of head nods, and regulating interaction through expressive feedback). We further outline a number of areas for future research that will be necessary to build a complete embodied system.

The Painting Fool is software that we hope will one day be taken seriously as a creative artist in its own right. This aim is being pursued as an Artificial Intelligence (AI) project, with the hope that the technical difficulties overcome along the way will lead to new and improved generic AI techniques. It is also being pursued as a sociological project, where the effect of software which might be deemed as creative is tested in the art world and the wider public. In this chapter, we summarise our progress so far in The Painting Fool project. To do this, we first compare and contrast The Painting Fool with software of a similar nature arising from AI and graphics projects. We follow this with a discussion of the guiding principles from Computational Creativity research that we adhere to in building the software. We then describe five projects with The Painting Fool where our aim has been to produce increasingly interesting and culturally valuable pieces of art. We end by discussing the issues raised in building an automated painter, and describe further work and future prospects for the project. By studying both the technical difficulties and sociological issues involved in engineering software for creative purposes, we hope to help usher in a new era where computers routinely act as our creative collaborators, as well as independent and creative artists, musicians, writers, designers, engineers and scientists, and contribute in meaningful and interesting ways to human culture.

A wheeled mobile robot based on Arduino was described in this article. Hardware selection, Simultaneous Localization and Mapping (SLAM), Path Planning and the principle of the algorithm applied in this process are introduced in detail. The algorithm used in ROS is further verified by experiments. In order to verify the feasibility of two different 2D SLAM algorithms, we use the wheeled robot built in the lab to do the experiment in the same indoor environment and compare their results.


Question: An iterative deep convolutional encoder-decoder framework for segmenting medical images.

This paper presents a comprehensive overview of deep learning's transformative impact on medical imaging, underscoring its potential to revolutionize the field. Deep learning, a burgeoning trend in data analysis, has been hailed as one of the decade's breakthrough technologies [1]. As an evolution of artificial neural networks, deep learning's multi-layered structure enables higher levels of abstraction and enhanced predictive capabilities from raw data [2]. In the realm of imaging and computer vision, it has emerged as the leading machine-learning technique. Convolutional neural networks (CNNs), a subset of deep learning models, have demonstrated exceptional prowess in various computer vision tasks. Notably, deep CNNs can autonomously learn mid-level and high-level abstractions from images, making them highly effective in object recognition and localization in natural images. Medical imaging researchers are swiftly integrating CNNs and other deep learning methodologies into a plethora of applications, yielding promising outcomes. In the medical domain, the precision of disease diagnosis and assessment hinges on both image acquisition and interpretation. While advancements in image acquisition have significantly accelerated data collection and enhanced resolution, the image interpretation process has only recently started to leverage computer technology. Human interpretation of medical images is inherently subjective, prone to variability among interpreters, and susceptible to fatigue. Diagnostic tasks often necessitate an initial search to detect anomalies and quantify changes over time. Computerized tools, specifically those involving image analysis and machine learning, are pivotal in enhancing diagnostic accuracy by aiding in the identification of critical findings and streamlining the expert's workflow. Among these tools, deep learning is rapidly becoming the gold standard, driving unprecedented advancements in data analysis accuracy and efficiency.

The attempts to segment medical ultrasound images have had limited success than the attempts to segment images from other medical imaging modalities. In this project, we attempt to segment medical ultrasound images using convolutional neural networks (CNNs) with a group of noisy activation functions which have recently been demonstrated to improve the performance of neural networks. We report on the segmentation results using a U-Net-like CNN with noisy rectified linear unit (NReLU) functions, noisy hard sigmoid (NHSigmoid) functions, and noisy hard tanh (NHTanh) function on a small data set.

We present a technique for adding global context to deep convolutional networks for semantic segmentation. The approach is simple, using the average feature for a layer to augment the features at each location. In addition, we study several idiosyncrasies of training, significantly increasing the performance of baseline networks (e.g. from FCN [19]). When we add our proposed global feature, and a technique for learning normalization parameters, accuracy increases consistently even over our improved versions of the baselines. Our proposed approach, ParseNet, achieves state-of-the-art performance on SiftFlow and PASCAL-Context with small additional computational cost over baselines, and near current state-of-theart performance on PASCAL VOC 2012 semantic segmentation with a simple approach. Code is available at https: //github.com/weiliu89/caffe/tree/fcn.

In this paper we describe a new technique for general purpose interactive segmentation of N-dimensional images . The user marks certain pixels as “object” or “background” to provide hard constraints for segmentation. Additional soft constraints incorporate both boundary and region information. Graph cuts are used to find the globally optimal segmentation of the N-dimensional image. The obtained solution gives the best balance of boundary and region properties among all segmentations satisfying the constraints . The topology of our segmentation is unrestricted and both “object” and “background” segments may consist of several isolated parts. Some experimental results are present ed in the context of photo/video editing and medical image segmentation. We also demonstrate an interesting Gestalt example. A fast implementation of our segmentation method is possible via a new max-flow algorithm in [2].

Ulcerative Colitis (UC) is a chronic inflammatory disease characterized by periods of relapses and remissions affecting more than 500,000 people in the United States. The therapeutic goals of UC are to first induce and then maintain disease remission. However, it is very difficult to evaluate the severity of UC objectively because of non-uniform nature of symptoms and large variations in their patterns. To address this, we already developed an approach using the image textures in our previous work. But, we found that it could not handle larger number of variations in their patterns. In this paper, we propose a different approach using CNN (Convolutional Neural Network) to measure and classify objectively the severity of UC presented in optical colonoscopy video frames. We call the proposed approach using CNN as Ulcerative Colitis Severity CNN (UCS-CNN) which utilizes endoscopic domain knowledge and convolutional neural network to classify different UC severity of colonoscopy images. The experimental results show that the proposed UCS-CNN can evaluate the severity of UC reasonably.

The objective of this paper is the unsupervised segmentation of image training sets into foreground and background in order to improve image classification performance. To this end we introduce a new scalable, alternation-based algorithm for co-segmentation, BiCoS, which is simpler than many of its predecessors, and yet has superior performance on standard benchmark image datasets.

The finite mixture (FM) model is the most commonly used model for statistical segmentation of brain magnetic resonance (MR) images because of its simple mathematical form and the piecewise constant nature of ideal brain MR images. However, being a histogram-based model, the FM has an intrinsic limitation-no spatial information is taken into account. This causes the FM model to work only on well-defined images with low levels of noise; unfortunately, this is often not the the case due to artifacts such as partial volume effect and bias field distortion. Under these conditions, FM model-based methods produce unreliable results. Here, the authors propose a novel hidden Markov random field (HMRF) model, which is a stochastic process generated by a MRF whose state sequence cannot be observed directly but which can be indirectly estimated through observations. Mathematically, it can be shown that the FM model is a degenerate version of the HMRF model. The advantage of the HMRF model derives from the way in which the spatial information is encoded through the mutual influences of neighboring sites. Although MRF modeling has been employed in MR image segmentation by other researchers, most reported methods are limited to using MRF as a general prior in an FM model-based approach. To fit the HMRF model, an EM algorithm is used. The authors show that by incorporating both the HMRF model and the EM algorithm into a HMRF-EM framework, an accurate and robust segmentation can be achieved. More importantly, the HMRF-EM framework can easily be combined with other techniques. As an example, the authors show how the bias field correction algorithm of Guillemaud and Brady (1997) can be incorporated into this framework to achieve a three-dimensional fully automated approach for brain MR image segmentation.

In this paper, we propose a deep learning approach for image registration by predicting deformation from image appearance. Since obtaining ground-truth deformation fields for training can be challenging, we design a fully convolutional network that is subject to dual-guidance: (1) Coarse guidance using deformation fields obtained by an existing registration method; and (2) Fine guidance using image similarity. The latter guidance helps avoid overly relying on the supervision from the training deformation fields, which could be inaccurate. For effective training, we further improve the deep convolutional network with gap filling, hierarchical loss, and multi-source strategies. Experiments on a variety of datasets show promising registration accuracy and efficiency compared with state-of-the-art methods.

Normally, lesions are detected using supervised learning techniques that require labelled training data. We explore the use of Bayesian autoencoders to learn the variability of healthy tissue and detect lesions as unlikely events under the normative model. As a proof-of-concept, we test our method on registered 2D midaxial slices from CT imaging data. Our results indicate that our method achieves best performance in detecting lesions caused by bleeding compared to baselines.

Convolutional Neural Networks (CNNs) are typically trained in the RGB color space. However, in medical imaging, we believe that pixel stain quantities offer a fundamental view of the interaction between tissues and stain chemicals. Since the optical density (OD) colorspace allows to compute pixel stain quantities from pixel RGB intensities using the Beer-Lambert’s law, we propose a stain deconvolutional layer, hereby named as SD-Layer, affixed at the front of CNN that performs two functions: (1) it transforms the input RGB microscopic images to Optical Density (OD) space and (2) this layer deconvolves OD image with the stain basis learned through backpropagation and provides tissue-specific stain absorption quantities as input to the following CNN layers. With the introduction of only nine additional learnable parameters in the proposed SD-Layer, we obtain a considerably improved performance on two standard CNN architectures: AlexNet and T-CNN. Using the T-CNN architecture prefixed with the proposed SD-Layer, we obtain 5-fold crossvalidation accuracy of 93.2% in the problem of differentiating malignant immature White Blood Cells (WBCs) from normal immature WBCs for cancer detection.

All clustering algorithms process unlabeled data and, consequently, suffer from two problems: (P1) choosing and validating the correct number of clusters and (P2) insuring that algorithmic labels correspond to meaningful physical labels. Clustering algorithms such as hard and fuzzy c-means, based on optimizing sums of squared errors objective functions, suffer from a third problem: (P3) a tendency to recommend solutions that equalize cluster populations. The semi-supervised c-means algorithms introduced in this paper attempt to overcome these three problems for problem domains where a few data from each class can be labeled. Segmentation of magnetic resonance images is a problem of this type and we use it to illustrate the new algorithm. Our examples show that the semi-supervised approach provides MRI segmentations that are superior to ordinary fuzzy c-means and to the crisp k-nearest neighbor rule and further, that the new method ameliorates (P1)-(P3). Cluster analysis Fuzzy c-means Partial supervision Image segmentation Magnetic resonance images


Question: An iterative deep convolutional encoder-decoder framework for the segmentation of medical images.

This paper introduces the PH2 database, a comprehensive collection of dermoscopic images aimed at advancing the field of computer-aided diagnosis for melanoma. The growing prevalence of melanoma has spurred the development of numerous systems designed to classify dermoscopic images. However, the lack of a standard benchmarking dataset has hindered the direct comparison of these systems' performance. To address this issue, the PH2 database has been compiled. It features 200 dermoscopic images, each accompanied by detailed annotations. These annotations include manual segmentations, clinical diagnoses, and the identification of key dermoscopic structures, all provided by experienced dermatologists. The PH2 database is intended to serve as a valuable resource for researchers and will be freely accessible for use in the evaluation and benchmarking of various computer-aided diagnosis systems.

The state-of-the-art interactive image segmentation algorithms are sensitive to the user inputs and often unable to produce an accurate boundary with a small amount of user interaction. They frequently rely on laborious user editing to refine the segmentation boundary. In this paper, we propose a robust and accurate interactive method based on the recently developed continuous-domain convex active contour model. The proposed method exhibits many desirable properties of an effective interactive image segmentation algorithm, including robustness to user inputs and different initializations, the ability to produce a smooth and accurate boundary contour, and the ability to handle topology changes. Experimental results on a benchmark data set show that the proposed tool is highly effective and outperforms the state-of-the-art interactive image segmentation algorithms.

Convolutional autoencoders have emerged as popular methods for unsupervised defect segmentation on image data. Most commonly, this task is performed by thresholding a per-pixel reconstruction error based on an `-distance. This procedure, however, leads to large residuals whenever the reconstruction includes slight localization inaccuracies around edges. It also fails to reveal defective regions that have been visually altered when intensity values stay roughly consistent. We show that these problems prevent these approaches from being applied to complex real-world scenarios and that they cannot be easily avoided by employing more elaborate architectures such as variational or feature matching autoencoders. We propose to use a perceptual loss function based on structural similarity that examines inter-dependencies between local image regions, taking into account luminance, contrast, and structural information, instead of simply comparing single pixel values. It achieves significant performance gains on a challenging real-world dataset of nanofibrous materials and a novel dataset of two woven fabrics over state-of-the-art approaches for unsupervised defect segmentation that use per-pixel reconstruction

Automatic skin lesion segmentation on dermoscopic images is an essential step in computer-aided diagnosis of melanoma. However, this task is challenging due to significant variations of lesion appearances across different patients. This challenge is further exacerbated when dealing with a large amount of image data. In this paper, we extended our previous work by developing a deeper network architecture with smaller kernels to enhance its discriminant capacity. In addition, we explicitly included color information from multiple color spaces to facilitate network training and thus to further improve the segmentation performance. We participated and extensively evaluated our method on the ISBI 2017 skin lesion segmentation challenge. By training with the 2000 challenge training images, our method achieved an average Jaccard Index of 0.765 on the 600 challenge testing images, which ranked itself in the first place among 21 final submissions in the challenge.

Categorization of medical images means selecting the appropriate class for a given image out of a set of pre-defined categories. This is an important step for data mining and content-based image retrieval (CBIR). So far, published approaches are capable to distinguish up to 10 categories. In this paper, we evaluate automatic categorization into more than 80 categories describing the imaging modality and direction as well as the body part and biological system examined. Based on 6231 reference images from hospital routine, 85.5% correctness is obtained combining global texture features with scaled images. With a frequency of 97.7%, the correct class is within the best ten matches, which is sufficient for medical CBIR applications.

Binarization plays a key role in the automatic information retrieval from document images. This process is usually performed in the first stages of documents analysis systems, and serves as a basis for subsequent steps. Hence it has to be robust in order to allow the full analysis workflow to be successful. Several methods for document image binarization have been proposed so far, most of which are based on hand-crafted image processing strategies. Recently, Convolutional Neural Networks have shown an amazing performance in many disparate duties related to computer vision. In this paper we discuss the use of convolutional auto-encoders devoted to learning an end-to-end map from an input image to its selectional output, in which activations indicate the likelihood of pixels to be either foreground or background. Once trained, documents can therefore be binarized by parsing them through the model and applying a global threshold. This approach has proven to outperform existing binarization strategies in a number of document types.

This paper introduces a new approach for the segmentation of skin lesions in dermoscopic images based on wavelet network (WN). The WN presented here is a member of fixed-grid WNs that is formed with no need of training. In this WN, after formation of wavelet lattice, determining shift and scale parameters of wavelets with two screening stage and selecting effective wavelets, orthogonal least squares algorithm is used to calculate the network weights and to optimize the network structure. The existence of two stages of screening increases globality of the wavelet lattice and provides a better estimation of the function especially for larger scales. R, G, and B values of a dermoscopy image are considered as the network inputs and the network structure formation. Then, the image is segmented and the skin lesions exact boundary is determined accordingly. The segmentation algorithm were applied to 30 dermoscopic images and evaluated with 11 different metrics, using the segmentation result obtained by a skilled pathologist as the ground truth. Experimental results show that our method acts more effectively in comparison with some modern techniques that have been successfully used in many medical imaging problems.

Screening mammography is an important front-line tool for the early detection of breast cancer, and some 39 million exams are conducted each year in the United States alone. Here, we describe a multi-scale convolutional neural network (CNN) trained with a curriculum learning strategy that achieves high levels of accuracy in classifying mammograms. Specifically, we first train CNN-based patch classifiers on segmentation masks of lesions in mammograms, and then use the learned features to initialize a scanning-based model that renders a decision on the whole image, trained end-to-end on outcome data. We demonstrate that our approach effectively handles the “needle in a haystack” nature of full-image mammogram classification, achieving 0.92 AUROC on the DDSM dataset.

Computerized Tomography Angiography (CTA) based follow-up of Abdominal Aortic Aneurysms (AAA) treated with Endovascular Aneurysm Repair (EVAR) is essential to evaluate the progress of the patient and detect complications. In this context, accurate quantification of post-operative thrombus volume is required. However, a proper evaluation is hindered by the lack of automatic, robust and reproducible thrombus segmentation algorithms. We propose a new fully automatic approach based on Deep Convolutional Neural Networks (DCNN) for robust and reproducible thrombus region of interest detection and subsequent fine thrombus segmentation. The DetecNet detection network is adapted to perform region of interest extraction from a complete CTA and a new segmentation network architecture, based on Fully Convolutional Networks and a Holistically-Nested Edge Detection Network, is presented. These networks are trained, validated and tested in 13 post-operative CTA volumes of different patients using a 4-fold cross-validation approach to provide more robustness to the results. Our pipeline achieves a Dice score of more than 82% for post-operative thrombus segmentation and provides a mean relative volume difference between ground truth and automatic segmentation that lays within the experienced human observer variance without the need of human intervention in most common cases.

In this work, we evaluate the use of superpixel pooling layers in deep network architectures for semantic segmentation. Superpixel pooling is a flexible and efficient replacement for other pooling strategies that incorporates spatial prior information. We propose a simple and efficient GPU-implementation of the layer and explore several designs for the integration of the layer into existing network architectures. We provide experimental results on the IBSR and Cityscapes dataset, demonstrating that superpixel pooling can be leveraged to consistently increase network accuracy with minimal computational overhead. Source code is available at https://github.com/bermanmaxim/superpixPool.

The ability to automatically monitor agricultural fields is an important capability in precision farming, enabling steps towards more sustainable agriculture. Precise, high-resolution monitoring is a key prerequisite for targeted intervention and the selective application of agro-chemicals. In this paper, we present a novel weed segmentation and mapping framework that processes multispectral images obtained from an unmanned aerial vehicle (UAV) using a deep neural network (DNN). Most studies on crop/weed semantic segmentation only consider single images for processing and classification. Images taken by UAVs often cover only a few hundred square meters with either color only or color and near-infrared (NIR) channels. Although a map can be generated by processing single segmented images incrementally, this requires additional complex information fusion techniques which struggle to handle high fidelity maps due to their computational costs and problems in ensuring global consistency. Moreover, computing a single large and accurate vegetation map (e.g., crop/weed) using a DNN is non-trivial due to difficulties arising from: (1) limited ground sample distances (GSDs) in high-altitude datasets, (2) sacrificed resolution resulting from downsampling high-fidelity images, and (3) multispectral image alignment. To address these issues, we adopt a stand sliding window approach that operates on only small portions of multispectral orthomosaic maps (tiles), which are channel-wise aligned and calibrated radiometrically across the entire map. We define the tile size to be the same as that of the DNN input to avoid resolution loss. Compared to our baseline model (i.e., SegNet with 3 channel RGB inputs) yielding an area under the curve (AUC) of [background=0.607, crop=0.681, weed=0.576], our proposed model with 9 input channels achieves [0.839, 0.863, 0.782]. Additionally, we provide an extensive analysis of 20 trained models, both qualitatively and quantitatively, in order to evaluate the effects of varying input channels and tunable network hyperparameters. Furthermore, we release a large sugar beet/weed aerial dataset with expertly guided annotations for further research in the fields of remote sensing, precision agriculture, and agricultural robotics.


Question: An iterative deep convolutional encoder-decoder framework for the segmentation of medical images.

This paper introduces an advanced technique for the automated segmentation of blood vessels in two-dimensional color images of the retina, a critical step in computer-assisted analyses for conditions like diabetic retinopathy. The proposed system is centered on the identification of image ridges, which are closely aligned with the centerlines of retinal vessels. These ridges serve as the foundation for constructing primitives in the form of line segments. By utilizing these line segments, the image is partitioned into distinct patches, with each pixel attributed to the nearest line segment. This process establishes a local coordinate system for each patch, associated with its corresponding line segment. For every pixel, a set of feature vectors is calculated, leveraging the attributes of the patches and line segments. These feature vectors are then classified using a k-nearest neighbors (kNN) classifier, augmented by sequential forward feature selection. The algorithm's efficacy was evaluated on a dataset comprising 40 manually annotated retinal images. The results demonstrate an impressive area under the receiver operating characteristic curve of 0.952. This method is contrasted with two contemporary rule-based approaches by Hoover et al. and Jiang et al. Comparative analysis reveals that our method significantly outperforms these rule-based methods (p<0.01), with an accuracy of 0.944 compared to 0.947 for a second human observer.

The segmentation of infant brain tissue images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) plays an important role in studying early brain development. In the isointense phase (approximately 6–8 months of age), WM and GM exhibit similar levels of intensity in both T1 and T2 MR images, resulting in extremely low tissue contrast and thus making the tissue segmentation very challenging. The existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single T1, T2 or fractional anisotropy (FA) modality or their simply-stacked combinations without fully exploring the multi-modality information. To address the challenge, in this paper, we propose to use fully convolutional networks (FCNs) for the segmentation of isointense phase brain MR images. Instead of simply stacking the three modalities, we train one network for each modality image, and then fuse their high-layer features together for final segmentation. Specifically, we conduct a convolution-pooling stream for multimodality information from T1, T2, and FA images separately, and then combine them in high-layer for finally generating the segmentation maps as the outputs. We compared the performance of our approach with that of the commonly used segmentation methods on a set of manually segmented isointense phase brain images. Results showed that our proposed model significantly outperformed previous methods in terms of accuracy. In addition, our results also indicated a better way of integrating multi-modality images, which leads to performance improvement.

Magnetic resonance (MR) imaging is increasingly being used to assess brain growth and development in infants. Such studies are often based on quantitative analysis of anatomical segmentations of brain MR images. However, the large changes in brain shape and appearance associated with development, the lower signal to noise ratio and partial volume effects in the neonatal brain present challenges for automatic segmentation of neonatal MR imaging data. In this study, we propose a framework for accurate intensity-based segmentation of the developing neonatal brain, from the early preterm period to term-equivalent age, into 50 brain regions. We present a novel segmentation algorithm that models the intensities across the whole brain by introducing a structural hierarchy and anatomical constraints. The proposed method is compared to standard atlas-based techniques and improves label overlaps with respect to manual reference segmentations. We demonstrate that the proposed technique achieves highly accurate results and is very robust across a wide range of gestational ages, from 24 weeks gestational age to term-equivalent age.

The segmentation of liver from abdominal images has gained huge importance in medical imaging field since it is fundamental step to diagnose liver tumor and other liver diseases, liver volume measurement and 3D liver volume rendering. In this paper, recent automatic methods for liver image segmentation proposed in literature are reviewed. Generally, Liver image segmentation techniques can be semi-automatic and fully automatic. Under fully automatic segmentation category, various methods, approaches, recent advances and related problems has been explained. Brief summary of all discussed methods has been provided. Finally it is concluded that liver image segmentation is still an open problem since various drawbacks of proposed methods can be addressed. Keywords— Liver image Segmentation, Automatic Segmentation, Neural Network, Support Vector Machine, Clustering, Hybrid, Review.

Image processing and pixel-wise dense prediction have been advanced by harnessing the capabilities of deep learning. One central issue of deep learning is the limited capacity to handle joint upsampling. We present a deep learning building block for joint upsampling, namely guided filtering layer. This layer aims at efficiently generating the high-resolution output given the corresponding low-resolution one and a high-resolution guidance map. The proposed layer is composed of a guided filter, which is reformulated as a fully differentiable block. To this end, we show that a guided filter can be expressed as a group of spatial varying linear transformation matrices. This layer could be integrated with the convolutional neural networks (CNNs) and jointly optimized through end-to-end training. To further take advantage of end-to-end training, we plug in a trainable transformation function that generates task-specific guidance maps. By integrating the CNNs and the proposed layer, we form deep guided filtering networks. The proposed networks are evaluated on five advanced image processing tasks. Experiments on MIT-Adobe FiveK Dataset demonstrate that the proposed approach runs 10-100Ã— faster and achieves the state-of-the-art performance. We also show that the proposed guided filtering layer helps to improve the performance of multiple pixel-wise dense prediction tasks. The code is available at https://github.com/wuhuikai/DeepGuidedFilter.

Imaging roots of growing plants in a non-invasive and affordable fashion has been a long-standing problem in image-assisted plant breeding and phenotyping. One of the most affordable and diffuse approaches is the use of mesocosms, where plants are grown in soil against a glass surface that permits the roots visualization and imaging. However, due to soil and the fact that the plant root is a 2D projection of a 3D object, parts of the root are occluded. As a result, even under perfect root segmentation, the resulting images contain several gaps that may hinder the extraction of finely grained root system architecture traits. We propose an effective deep neural network to recover gaps from disconnected root segments. We train a fully supervised encoder-decoder deep CNN that, given an image containing gaps as input, generates an inpainted version, recovering the missing parts. Since in real data ground-truth is lacking, we use synthetic root images [10] that we artificially perturb by introducing gaps to train and evaluate our approach. We show that our network can work both in dicot and monocot cases in reducing root gaps. We also show promising exemplary results in real data from chickpea root architectures.

In this paper, the automatic medical annotation task of the 2007 CLEF cross-language image retrieval campaign (ImageCLEF) is described. The paper focusses on the images used, the task setup, and the results obtained in the evaluation campaign. Since 2005, the medical automatic image annotation task exists in ImageCLEF with increasing complexity to evaluate the performance of state-of-the-art methods for completely automatic annotation of medical images based on visual properties. The paper also describes the evolution of the task from its origin in 2005 to 2007. The 2007 task, comprising 11,000 fully annotated training images and 1,000 test images to be annotated, is a realistic task with a large number of possible classes at different levels of detail. Detailed analysis of the methods across participating groups is presented with respect to the (i) image representation, (ii) classification method, and (iii) use of the class hierarchy. The results show that methods which build on local image descriptors and discriminative models are able to provide good predictions of the image classes, mostly by using techniques that were originally developed in the machine learning and computer vision domain for object recognition in non-medical images.

Categorization of medical images means selecting the appropriate class for a given image out of a set of pre-defined categories. This is an important step for data mining and content-based image retrieval (CBIR). So far, published approaches are capable to distinguish up to 10 categories. In this paper, we evaluate automatic categorization into more than 80 categories describing the imaging modality and direction as well as the body part and biological system examined. Based on 6231 reference images from hospital routine, 85.5% correctness is obtained combining global texture features with scaled images. With a frequency of 97.7%, the correct class is within the best ten matches, which is sufficient for medical CBIR applications.

The finite mixture (FM) model is the most commonly used model for statistical segmentation of brain magnetic resonance (MR) images because of its simple mathematical form and the piecewise constant nature of ideal brain MR images. However, being a histogram-based model, the FM has an intrinsic limitation-no spatial information is taken into account. This causes the FM model to work only on well-defined images with low levels of noise; unfortunately, this is often not the the case due to artifacts such as partial volume effect and bias field distortion. Under these conditions, FM model-based methods produce unreliable results. Here, the authors propose a novel hidden Markov random field (HMRF) model, which is a stochastic process generated by a MRF whose state sequence cannot be observed directly but which can be indirectly estimated through observations. Mathematically, it can be shown that the FM model is a degenerate version of the HMRF model. The advantage of the HMRF model derives from the way in which the spatial information is encoded through the mutual influences of neighboring sites. Although MRF modeling has been employed in MR image segmentation by other researchers, most reported methods are limited to using MRF as a general prior in an FM model-based approach. To fit the HMRF model, an EM algorithm is used. The authors show that by incorporating both the HMRF model and the EM algorithm into a HMRF-EM framework, an accurate and robust segmentation can be achieved. More importantly, the HMRF-EM framework can easily be combined with other techniques. As an example, the authors show how the bias field correction algorithm of Guillemaud and Brady (1997) can be incorporated into this framework to achieve a three-dimensional fully automated approach for brain MR image segmentation.

Pixelwise segmentation of the left ventricular (LV) myocardium and the four cardiac chambers in 2-D steady state free precession (SSFP) cine sequences is an essential preprocessing step for a wide range of analyses. Variability in contrast, appearance, orientation, and placement of the heart between patients, clinical views, scanners, and protocols makes fully automatic semantic segmentation a notoriously difficult problem. Here, we present Ω-Net (Omega-Net): a novel convolutional neural network (CNN) architecture for simultaneous localization, transformation into a canonical orientation, and semantic segmentation. First, an initial segmentation is performed on the input image; second, the features learned during this initial segmentation are used to predict the parameters needed to transform the input image into a canonical orientation; and third, a final segmentation is performed on the transformed image. In this work, Ω-Nets of varying depths were trained to detect five foreground classes in any of three clinical views (short axis, SA; four-chamber, 4C; two-chamber, 2C), without prior knowledge of the view being segmented. This constitutes a substantially more challenging problem compared with prior work. The architecture was trained on a cohort of patients with hypertrophic cardiomyopathy (HCM, N = 42) and healthy control subjects (N = 21). Network performance, as measured by weighted foreground intersection-over-union (IoU), was substantially improved for the best-performing Ω-Net compared with U-Net segmentation without localization or orientation (0.858 vs 0.834). In addition, to be comparable with other works, Ω-Net was retrained from scratch on the publicly available 2017 MICCAI Automated Cardiac Diagnosis Challenge (ACDC) dataset. The Ω-Net outperformed the state-of-the-art method in segmentation of the LV and RV bloodpools, and performed slightly worse in segmentation of the LV myocardium. We conclude that this architecture represents a substantive advancement over prior approaches, with implications for biomedical image segmentation more generally.

Ulcerative Colitis (UC) is a chronic inflammatory disease characterized by periods of relapses and remissions affecting more than 500,000 people in the United States. The therapeutic goals of UC are to first induce and then maintain disease remission. However, it is very difficult to evaluate the severity of UC objectively because of non-uniform nature of symptoms and large variations in their patterns. To address this, we already developed an approach using the image textures in our previous work. But, we found that it could not handle larger number of variations in their patterns. In this paper, we propose a different approach using CNN (Convolutional Neural Network) to measure and classify objectively the severity of UC presented in optical colonoscopy video frames. We call the proposed approach using CNN as Ulcerative Colitis Severity CNN (UCS-CNN) which utilizes endoscopic domain knowledge and convolutional neural network to classify different UC severity of colonoscopy images. The experimental results show that the proposed UCS-CNN can evaluate the severity of UC reasonably.


Question: An iterative deep convolutional encoder-decoder framework for medical image segmentation.

This paper introduces an advanced active contour technique designed for object detection in vector-valued images, including RGB and multispectral formats. The proposed model expands upon the Chan–Vese algorithm, originally developed for scalar images, to accommodate vector-valued scenarios [1]. It optimizes a Mumford–Shah functional, minimizing the contour length and the fitting error across all components of the vector-valued image. Similar to the Chan–Vese framework, the vector-valued model is capable of detecting edges in the absence of a gradient. The method's efficacy is demonstrated through examples where it successfully identifies objects that remain undetectable in scalar representations. Notably, the algorithm can completely detect objects with missing parts in different channels, such as those obscured by occlusion. Additionally, it excels in color images, where objects invisible in individual channels or intensity levels are accurately detected. A significant advantage of the model is its robustness against noise, eliminating the need for preliminary denoising steps. The research underscores the model's versatility and superior performance in object detection within complex vector-valued images.

This paper addresses the problem of weakly supervised semantic image segmentation. Our goal is to label every pixel in a new image, given only image-level object labels associated with training images. Our problem statement differs from common semantic segmentation, where pixel-wise annotations are typically assumed available in training. We specify a novel deep architecture which fuses three distinct computation processes toward semantic segmentation – namely, (i) the bottom-up computation of neural activations in a CNN for the image-level prediction of object classes, (ii) the top-down estimation of conditional likelihoods of the CNNs activations given the predicted objects, resulting in probabilistic attention maps per object class, and (iii) the lateral attention-message passing from neighboring neurons at the same CNN layer. The fusion of (i)-(iii) is realized via a conditional random field as recurrent network aimed at generating a smooth and boundary-preserving segmentation. Unlike existing work, we formulate a unified end-to-end learning of all components of our deep architecture. Evaluation on the benchmark PASCAL VOC 2012 dataset demonstrates that we outperform reasonable weakly supervised baselines and state-of-the-art approaches.

The variation between images obtained with different scanners or different imaging protocols presents a major challenge in automatic segmentation of biomedical images. This variation especially hampers the application of otherwise successful supervised-learning techniques which, in order to perform well, often require a large amount of labeled training data that is exactly representative of the target data. We therefore propose to use transfer learning for image segmentation. Transfer-learning techniques can cope with differences in distributions between training and target data, and therefore may improve performance over supervised learning for segmentation across scanners and scan protocols. We present four transfer classifiers that can train a classification scheme with only a small amount of representative training data, in addition to a larger amount of other training data with slightly different characteristics. The performance of the four transfer classifiers was compared to that of standard supervised classification on two magnetic resonance imaging brain-segmentation tasks with multi-site data: white matter, gray matter, and cerebrospinal fluid segmentation; and white-matter-/MS-lesion segmentation. The experiments showed that when there is only a small amount of representative training data available, transfer learning can greatly outperform common supervised-learning approaches, minimizing classification errors by up to 60%.

In the recent years, deep learning has gained huge fame in solving problems from various fields including medical image analysis. This work proposes a deep convolutional neural network based pipeline for the diagnosis of Alzheimer's disease and its stages using magnetic resonance imaging (MRI) scans. Alzheimer's disease causes permanent damage to the brain cells associated with memory and thinking skills. The diagnosis of Alzheimer's in elderly people is quite difficult and requires a highly discriminative feature representation for classification due to similar brain patterns and pixel intensities. Deep learning techniques are capable of learning such representations from data. In this paper, a 4-way classifier is implemented to classify Alzheimer's (AD), mild cognitive impairment (MCI), late mild cognitive impairment (LMCI) and healthy persons. Experiments are performed using ADNI dataset on a high performance graphical processing unit based system and new state-of-the-art results are obtained for multiclass classification of the disease. The proposed technique results in a prediction accuracy of 98.8%, which is a noticeable increase in accuracy as compared to the previous studies and clearly reveals the effectiveness of the proposed method.

Although image compression has been actively studied for decades, there has been relatively little research on learning to compress images with modern neural networks. Standard approaches, such as those employing patch-based autoencoders, have shown a great deal of promise but cannot compete with popular image codecs because they fail to address three questions: 1) how to effectively binarize activations: in the absence of binarization, a bottleneck layer alone tends not to lead to efficient compression; 2) how to achieve variable-rate encoding: a standard autoencoder generates a fixed-length code for each fixed-resolution input patch, resulting in the same cost for lowand high-entropy patches, and requiring the network to be completely retrained to achieve different compression rates; and 3) how to avoid block artifacts: patch-based approaches are prone to block discontinuities. We propose a general framework for variable-rate image compression and a novel architecture based on convolutional and deconvolutional recurrent networks, including LSTMs, that address these issues and report promising results compared to existing baseline codecs. We evaluate the proposed methods on a large-scale benchmark consisting of tiny images (32× 32), which proves to be very challenging for all the methods.

Detection of brain tumor using a segmentation based approach is critical in cases, where survival of a subject depends on an accurate and timely clinical diagnosis. Gliomas are the most commonly found tumors having irregular shape and ambiguous boundaries, making them one of the hardest tumors to detect. The automation of brain tumor segmentation remains a challenging problem mainly due to significant variations in its structure. An automated brain tumor segmentation algorithm using deep convolutional neural network (DCNN) is presented in this paper. A patch based approach along with an inception module is used for training the deep network by extracting two co-centric patches of different sizes from the input images. Recent developments in deep neural networks such as drop-out, batch normalization, non-linear activation and inception module are used to build a new ILinear nexus architecture. The module overcomes the over-fitting problem arising due to scarcity of data using drop-out regularizer. Images are normalized and bias field corrected in the pre-processing step and then extracted patches are passed through a DCNN, which assigns an output label to the central pixel of each patch. Morphological operators are used for post-processing to remove small false positives around the edges. A two-phase weighted training method is introduced and evaluated using BRATS 2013 and BRATS 2015 datasets, where it improves the performance parameters of state-of-the-art techniques under similar settings.

We propose a novel multiple-input multiple-output convolution neural network (MIMO-Net) for cell segmentation in fluorescence microscopy images. The proposed network trains the network parameters using multiple resolutions of the input image, connects the intermediate layers for better localization and context and generates the output using multi-resolution deconvolution filters. The MIMO-Net allows us to deal with variable intensity cell boundaries and highly variable cell size in the mouse pancreatic tissue by adding extra convolutional layers which bypass the max-pooling operation. The results show that our method outperforms state-of-the-art deep learning based approaches for segmentation.

In this paper we describe a new technique for general purpose interactive segmentation of N-dimensional images . The user marks certain pixels as “object” or “background” to provide hard constraints for segmentation. Additional soft constraints incorporate both boundary and region information. Graph cuts are used to find the globally optimal segmentation of the N-dimensional image. The obtained solution gives the best balance of boundary and region properties among all segmentations satisfying the constraints . The topology of our segmentation is unrestricted and both “object” and “background” segments may consist of several isolated parts. Some experimental results are present ed in the context of photo/video editing and medical image segmentation. We also demonstrate an interesting Gestalt example. A fast implementation of our segmentation method is possible via a new max-flow algorithm in [2].

Abstruct-The problem of segmenting an image into separate regions and tracking them over time is one of the most significant problems in vision. Tenopoulos et al have proposed an approach to detect the contour regions of complex shapes, assuming a user selected initial contour not very far from the desired solution. We propose to further explore the information provided by the user’s selected points and applyan optimal method to detect contours which allows a segmentation of the image. The method is based on dynamic programming @P), and applies to a wide variety of shapes. It is exact and not iterative. We also consider a multiscale approach capable of speeding up the algorithm by a factor of 20, although at the expense of losing the guaranteed optimality characteristic. The problem of tracking and matching these contours is addressed. For tracking, the final contour obtained at one frame is sampled and used as initial points for the next frame. Then, the same DP process is applied. For matching, a novel strategy is proposed where the solution is a smooth displacement field in which unmatched regions are allowed while cross vectors are not. The algorithm is again based on DP and the optimal solution is guaranteed. We have demonstrated the algorithms on natural objects in a large spectrum of applications, including interactive segmentation and automatic tracking of the regions of interest in medical images.

We propose a novel end-to-end deep architecture for face landmark detection, based on a deep convolutional and deconvolutional network followed by carefully designed recurrent network structures. The pipeline of this architecture consists of three parts. Through the first part, we encode an input face image to resolution-preserved deconvolutional feature maps via a deep network with stacked convolutional and deconvolutional layers. Then, in the second part, we estimate the initial coordinates of the facial key points by an additional convolutional layer on top of these deconvolutional feature maps. In the last part, by using the deconvolutional feature maps and the initial facial key points as input, we refine the coordinates of the facial key points by a recurrent network that consists of multiple long short-term memory components. Extensive evaluations on several benchmark data sets show that the proposed deep architecture has superior performance against the state-of-the-art methods.

We propose a deep learning method for single image superresolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deep convolutional neural network (CNN) [15] that takes the lowresolution image as the input and outputs the high-resolution one. We further show that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network. But unlike traditional methods that handle each component separately, our method jointly optimizes all layers. Our deep CNN has a lightweight structure, yet demonstrates state-of-the-art restoration quality, and achieves fast speed for practical on-line usage.


Question: An iterative deep convolutional encoder-decoder framework for segmenting medical images.

This study delves into the impact of convolutional network depth on its precision within the domain of large-scale image recognition. Our primary contribution lies in a comprehensive assessment of networks with increasing depth, revealing that a notable enhancement over previous configurations can be achieved by escalating the depth to 16–19 weight layers. These insights underpinned our submission to the ImageNet Challenge 2014, where our team clinched the top positions in both the localisation and classification categories. Moreover, we demonstrate that our representations are highly adaptable to other datasets, where we attain state-of-the-art outcomes. Significantly, we have made our two most successful ConvNet models openly accessible to encourage further exploration into the application of deep visual representations in the field of computer vision.

In particular, we have developed an Iterative Deep Convolutional Encoder-Decoder Network (IDCEN) for medical image segmentation, which leverages the power of deep convolutional architectures. The IDCEN is designed to handle the complexities of medical images, offering a robust solution for segmentation tasks. By iterating through a series of deep convolutional and deconvolutional layers, the network is able to refine its understanding of the image content, leading to more accurate and detailed segmentations. This architecture has been rigorously tested on various medical imaging datasets, showcasing its superior performance compared to traditional segmentation methods. Additionally, our IDCEN is adaptable to different types of medical images, making it a versatile tool in the medical imaging community. We have also made our IDCEN model publicly available to facilitate collaborative research and innovation in medical image analysis.

In past years, deep convolutional neural networks (DCNN) have achieved big successes in image classification and object detection, as demonstrated on ImageNet in academic field. However, There are some unique practical challenges remain for real-world image recognition applications, e.g., small size of the objects, imbalanced data distributions, limited labeled data samples, etc. In this work, we are making efforts to deal with these challenges through a computational framework by incorporating latest developments in deep learning. In terms of two-stage detection scheme, pseudo labeling, data augmentation, cross-validation and ensemble learning, the proposed framework aims to achieve better performances for practical image recognition applications as compared to using standard deep learning methods. The proposed framework has recently been deployed as the key kernel for several image recognition competitions organized by Kaggle. The performance is promising as our final private scores were ranked 4 out of 2293 teams for fish recognition on the challenge "The Nature Conservancy Fisheries Monitoring" and 3 out of 834 teams for cervix recognition on the challenge "Intel &MobileODT Cervical Cancer Screening", and several others. We believe that by sharing the solutions, we can further promote the applications of deep learning techniques.

Many approaches to Bayesian image segmentation have used maximum a posteriori (MAP) estimation in conjunction with Markov random fields (MRF). Although this approach performs well, it has a number of disadvantages. In particular, exact MAP estimates cannot be computed, approximate MAP estimates are computationally expensive to compute, and unsupervised parameter estimation of the MRF is difficult. The authors propose a new approach to Bayesian image segmentation that directly addresses these problems. The new method replaces the MRF model with a novel multiscale random field (MSRF) and replaces the MAP estimator with a sequential MAP (SMAP) estimator derived from a novel estimation criteria. Together, the proposed estimator and model result in a segmentation algorithm that is not iterative and can be computed in time proportional to MN where M is the number of classes and N is the number of pixels. The also develop a computationally efficient method for unsupervised estimation of model parameters. Simulations on synthetic images indicate that the new algorithm performs better and requires much less computation than MAP estimation using simulated annealing. The algorithm is also found to improve classification accuracy when applied to the segmentation of multispectral remotely sensed images with ground truth data.

This paper presents an overview of the ImageCLEF 2017 caption tasks on the analysis of images from the biomedical literature. Two subtasks were proposed to the participants: a concept detection task and caption prediction task, both using only images as input. The two subtasks tackle the problem of providing image interpretation by extracting concepts and predicting a caption based on the visual information of an image alone. A dataset of 184,000 figure-caption pairs from the biomedical open access literature (PubMed Central) are provided as a testbed with the majority of them as trainign data and then 10,000 as validation and 10,000 as test data. Across two tasks, 11 participating groups submitted 71 runs. While the domain remains challenging and the data highly heterogeneous, we can note some surprisingly good results of the difficult task with a quality that could be beneficial for health applications by better exploiting the visual content of biomedical figures.

Biomedical images usually contain complex objects, which will vary in appearance significantly from one image to another. Attempting to measure or detect the presence of particular structures in such images can be a daunting task. The inherent variability will thwart naive schemes. However, by using models which can cope with the variability it is possible to successfully analyse complex images. Here we will consider a number of methods where the model represents the expected shape and local grey-level structure of a target object in an image. Model-based methods make use of a prior model of what is expected in the image, and typically attempt to find the best match of the model to the data in a new image. Having matched the model, one can then make measurements or test whether the target is actually present. This approach is a ‘top-down’ strategy, and differs significantly from ‘bottomup’ methods. In the latter the image data is examined at a low level, looking for local structures such as edges or regions, which are assembled into groups in an attempt to identify objects of interest. Without a global model of what to expect, this approach is difficult and prone to failure. A wide variety of model based approaches have been explored (see the review below). This chapter will concentrate on a statistical approach, in which a model is built from analysing the appearance of a set of labelled examples. Where structures vary in shape or texture, it is possible to learn what are plausible variations and what are not. A new image can be interpreted by finding the best plausible match of the model to the image data. The advantages of such a method are that

Image inpainting is a challenging problem as it needs to fill the information of the corrupted regions. Most of the existing inpainting algorithms assume that the positions of the corrupted regions are known. Different from the existing methods that usually make some assumptions on the corrupted regions, we present an efficient blind image inpainting algorithm to directly restore a clear image from a corrupted input. Our algorithm is motivated by the residual learning algorithm which aims to learn the missing information in corrupted regions. However, directly using existing residual learning algorithms in image restoration does not well solve this problem as little information is available in the corrupted regions. To solve this problem, we introduce an encoder and decoder architecture to capture more useful information and develop a robust loss function to deal with outliers. Our algorithm can predict the missing information in the corrupted regions, thus facilitating the clear image restoration. Both qualitative and quantitative experimental demonstrate that our algorithm can deal with the corrupted regions of arbitrary shapes and performs favorably against state-of-the-art methods.

The track 2 and track 3 of ChaLearn 2016 can be considered as Multi-Label Classification problems. We present a framework of learning deep binary encoding (DeepBE) to deal with multi-label problems by transforming multi-labels to single labels. The transformation of DeepBE is in a hidden pattern, which can be well addressed by deep convolutions neural networks (CNNs). Furthermore, we adopt an ensemble strategy to enhance the learning robustness. This strategy is inspired by its effectiveness in fine-grained image recognition (FGIR) problem, while most of face related tasks such as track 2 and track 3 are also FGIR problems. By DeepBE, we got 5.45% and 10.84% mean square error for track 2 and track 3 respectively. Additionally, we proposed an algorithm adaption method to treat the multiple labels of track 2 directly and got 6.84% mean square error.

medical fields has led to the automated processing ofpictorial data. Here, a device called the cytocomputer searches for genetic mutations. A computer revolution has occurred not only in technical fields but also in medicine, where vast amounts of information must be processed quickly and accurately. Nowhere is the need for image processing techniques more apparent than in clinical diagnosis or mass screening applications where data take the form of digital images. New high-resolution scanning techniques such as computed tomography, nuclear magnetic resonance, po-sitron emission tomography, and digital radiography produce images containing immense amounts of relevant information for medical analysis. But as these scanning techniques become more vital to clinical diagnosis, the work for specialists who must visually examine the resultant images increases. In many cases, quantitative data in the form of measurements and counts are needed to supplement nonimage patient data, and the manual extraction of these data is a time-consuming and costly step in an otherwise automated process. Furthermore, subtle variants of shade and shape can be the earliest clues to a diagnosis, placing the additional burden of complete thoroughness on the examining specialist. For the last five years, the University of Michigan and the Environmental Research Institute of Michigan have conducted a unique series of studies that involve the processing of biomedical imagery on a highly parallel computer specifically designed for image processing. System designers have incorporated the requirements of extracting a verifiable answer from an image in a reasonable time into an integrated approach to hardware and software design. The system includes a parallel pipelined image processor, called a cytocomputer, and a high-level language specifically created for image processing, C-3PL, the cytocomputer parallel picture processing language. These studies have involved a great many people from both the medical and engineering communities and have highlighted the interdisciplinary aspects of biomedical image processing. The methods have been tested in anatomy, developmental biology, nuclear medicine, car-diology, and transplant rejection. The general consensus is that quantification by automated image analysis not only increases diagnostic accuracy but also provides significant data not obtainable from qualitative analysis alone. One study in particular, on which descriptions in this article are based, involves a joint effort by the University of Michigan's human genetics and electrical and computer engineering departments and is supported by a grant from the National Cancer Institute. Basically, automated image analysis is being applied via sophisticated biochemical and computer techniques to derive …

We explore design principles for general pixel-level prediction problems, from low-level edge detection to midlevel surface normal estimation to high-level semantic segmentation. Convolutional predictors, such as the fullyconvolutional network (FCN), have achieved remarkable success by exploiting the spatial redundancy of neighboring pixels through convolutional processing. Though computationally efficient, we point out that such approaches are not statistically efficient during learning precisely because spatial redundancy limits the information learned from neighboring pixels. We demonstrate that stratified sampling of pixels allows one to (1) add diversity during batch updates, speeding up learning; (2) explore complex nonlinear predictors, improving accuracy; and (3) efficiently train state-of-the-art models tabula rasa (i.e., “from scratch”) for diverse pixel-labeling tasks. Our single architecture produces state-of-the-art results for semantic segmentation on PASCAL-Context dataset, surface normal estimation on NYUDv2 depth dataset, and edge detection on BSDS.

We propose a partially learned approach for the solution of ill posed inverse problems with not necessarily linear forward operators. The method builds on ideas from classical regularization theory and recent advances in deep learning to perform learning while making use of prior information about the inverse problem encoded in the forward operator, noise model and a regularizing functional. The method results in a gradient-like iterative scheme, where the “gradient” component is learned using a convolutional network that includes the gradients of the data discrepancy and regularizer as input in each iteration. We present results of such a partially learned gradient scheme on a non-linear tomographic inversion problem with simulated data from both the Sheep-Logan phantom as well as a head CT. The outcome is compared against filtered backprojection and total variation reconstruction and the proposed method provides a 5.4 dB PSNR improvement over the total variation reconstruction while being significantly faster, giving reconstructions of 512× 512 pixel images in about 0.4 s using a single graphics processing unit (GPU).

In this tutorial, we will provide an introduction to the main Python software tools used for applying machine learning techniques to medical data. The focus will be on open-source software that is freely available and is cross platform. To aid the learning experience, a companion GitHub repository is available so that you can follow the examples contained in this paper interactively using Jupyter notebooks. The notebooks will be more exhaustive than what is contained in this chapter, and will focus on medical datasets and healthcare problems. Briefly, this tutorial will first introduce Python as a language, and then describe some of the lower level, general matrix and data structure packages that are popular in the machine learning and data science communities, such as NumPy and Pandas. From there, we will move to dedicated machine learning software, such as SciKit-Learn. Finally we will introduce the Keras deep learning and neural networks library. The emphasis of this paper is readability, with as little jargon used as possible. No previous experience with machine learning is assumed. We will use openly available medical datasets throughout.