Question: PD control with online gravity compensation for robots with elastic joints: theoretical framework and experimental results.

This document introduces an updated and expanded edition of the original work published in 1978, which provided an overview of nonlinear systems and control theory, excluding nonlinear optimal control theory. The primary focus was on system analysis rather than controller design, with a strong emphasis on stability issues from both input-output and state-space perspectives. Key concepts included input-output stability, Popov's criterion, the circle criterion, Lyapunov stability, and the method of describing functions. Since the publication of the first edition, geometric nonlinear control theory has emerged as a dominant trend, addressing system-theoretic questions in state-space formulations of nonlinear control systems, such as controllability, observability, and minimal realization theory. This approach gained momentum in the early eighties through the systematic study of nonlinear state feedback for synthesis problems, influenced by the geometric approach to linear control theory by Wonham, Morse, Basile, and Marro. The Lie bracket conditions for controllability and feedback linearizability have become essential tools in nonlinear control. Recent developments aim to bridge the gap between traditional nonlinear systems and control theory and geometric nonlinear control theory. Nonlinear adaptive control and nonlinear robust control (including nonlinear μ-control) require stability concepts akin to those presented in the first edition, while geometric nonlinear control theory provides a structural framework. Passivity and dissipativity concepts are particularly useful in these areas, especially for controlling physical systems. There has been a convergence between nonlinear input-output theory and state-space theory. The second edition, revised by Professor Vidyasagar, includes an introduction to geometric nonlinear control theory in an additional chapter, covering topics such as controllability, feedback linearization, and input-output linearization. Other significant changes from the first edition include the incorporation of recent developments in the field.

Regarding the question on PD control with online gravity compensation for robots with elastic joints: A comprehensive approach to control design for robots with elastic joints involves the integration of Proportional-Derivative (PD) control with online gravity compensation. This method is particularly useful in addressing the challenges posed by the non-rigid nature of the joints in robotic systems. Elastic joints, while providing benefits such as energy efficiency and compliance, introduce complexities in control due to their inherent dynamics and the need for precise compensation of external forces, such as gravity. The PD control strategy, with its straightforward implementation and ability to provide robust performance, is often combined with online gravity compensation to achieve accurate and stable control of robotic systems. By continuously estimating and compensating for the gravitational forces acting on the robot, the PD controller

Conventionally, engineers have employed rigid materials to fabricate precise, predictable robotic systems, which are easily modelled as rigid members connected at discrete joints. Natural systems, however, often match or exceed the performance of robotic systems with deformable bodies. Cephalopods, for example, achieve amazing feats of manipulation and locomotion without a skeleton; even vertebrates such as humans achieve dynamic gaits by storing elastic energy in their compliant bones and soft tissues. Inspired by nature, engineers have begun to explore the design and control of soft-bodied robots composed of compliant materials. This Review discusses recent developments in the emerging field of soft robotics.

Passively compliant legs have been instrumental in the development of dynamically running legged robots. Having properly tuned leg springs is essential for stable, robust and energetically efficient running at high speeds. Recent simulation studies indicate that having variable stiffness legs, as animals do, can significantly improve the speed and stability of these robots in changing environmental conditions. However, to date, the mechanical complexities of designing usefully robust tunable passive compliance into legs has precluded their implementation on practical running robots. This paper describes a new design of a ”structurally controlled variable stiffness” leg for a hexapedal running robot. This new leg improves on previous designs’ performance and enables runtime modification of leg stiffness in a small, lightweight, and rugged package. Modeling and leg test experiments are presented that characterize the improvement in stiffness range, energy storage, and dynamic coupling properties of these legs. We conclude that this variable stiffness leg design is now ready for implementation and testing on a dynamical running robot.

The paper presents the performance enhanced humanoid robot LOLA which is currently being manufactured. The goal of the project is the realization of a fast, human-like walking motion. The robot is characterized by its lightweight construction, a modular, multi-sensory joint design with brushless motors and an electronics architecture using decentralized joint controllers. The fusion of motor, gear and sensors into a highly integrated, mechatronic joint module has several advantages for the whole system, including high power density, good dynamic performance and reliability. Additional degrees of freedom are introduced in elbow, waist and toes. Linear actuators are employed for the knee joints for a better mass distribution in the legs

Traditional rigid robotic hand manipulator has been used in many field nowadays due to its advantages of large gripping force and stable performance. However, this kind of rigid manipulator is not suitable for gripping fragile objects since it is motorized and force control can be a problem. It is also not suitable to grip object with different shapes since the manipulator is rigid and not compliant. In this study, a novel manipulator with gripping capability is designed and fabricated. The manipulator combines electrostatic adhesion actuation with soft manipulators. The manipulator has high flexibility and can be compliant to different shapes due to the property of the materials. It is very promising to do delicate manipulations in industry field and biomedical field.

The soft exosuit is a new approach for applying assistive forces over the wearer's body through load paths configured by the textile architecture. In this paper, we present a body-worn lower-extremity soft exosuit and a new control approach that can independently control the level of assistance that is provided during negative- and positive-power periods at the ankle. The exosuit was designed to create load paths assisting ankle plantarflexion and hip flexion, and the actuation system transmits forces from the motors to the suit via Bowden cables. A load cell and two gyro sensors per leg are used to measure real-time data, and the controller performs position control of the cable on a step-by-step basis with respect to the power delivered to the wearer's ankle by controlling two force parameters, the pretension and the active force. Human subjects testing results demonstrate that the controller is capable of modulating the amount of power delivered to the ankle joint. Also, significant reductions in metabolic rate (11%-15%) were observed, which indicates the potential of the proposed control approach to provide benefit to the wearer during walking.

This paper deals with the mechanical design of a novel haptic Hand Exoskeleton (HE) that allows exerting controlled forces on the fingertip of the index and thumb of the operator. The proposed device includes several design solutions for optimizing the accuracy and mechanical performances. Remote Centers of Motion mechanisms have been adopted for delocalizing the encumbrance of linkages of the structure away from the operator's fingers. An improved stiffness of the transmission and reduced requirements for the actuators have been achieved thanks to a novel Patent Pending principle for integrating speed reduction ratio with the transmission system.

Personal robots can only become widespread if they are capable of safely operating among humans. In uncertain and highly dynamic environments such as human households, robots need to be able to instantly adapt their behavior to unforseen events. In this paper, we propose a general framework to achieve very contact-reactive motions for robotic grasping and manipulation. Associating stereotypical movements to particular tasks enables our system to use previous sensor experiences as a predictive model for subsequent task executions. We use dynamical systems, named Dynamic Movement Primitives (DMPs), to learn goal-directed behaviors from demonstration. We exploit their dynamic properties by coupling them with the measured and predicted sensor traces. This feedback loop allows for online adaptation of the movement plan. Our system can create a rich set of possible motions that account for external perturbations and perception uncertainty to generate truly robust behaviors. As an example, we present an application to grasping with the WAM robot arm.

We describe a combined force and distance sensor using a commodity infrared distance sensor embedded in a transparent elastomer with applications in robotic manipulation. Prior to contact, the sensor works as a distance sensor (0–10 cm), whereas after contact the material doubles as a spring, with force proportional to the compression of the elastomer (0– 5 N). We describe its principle of operation and design parameters, including polymer thickness, mixing ratio, and emitter current, and show that the sensor response has an inflection point at contact that is independent of an object’s surface properties. We then demonstrate how two arrays of eight sensors, each mounted on a standard Baxter gripper, can be used to (1) improve gripper alignment during grasping, (2) determine contact points with objects, and (3) obtain crude 3D models that can serve to determine possible grasp locations.

Powered exoskeletons can provide motion enhancement for both healthy and physically challenged people. Upper limb exoskeletons are required to have multiple degrees-of-freedom and can still produce sufficient force to augment the upper limb motion. The design using serial mechanisms usually results in a complicated and bulky exoskeleton that prevents itself from being wearable. This paper presents a new exoskeleton design aimed to achieve compactness and wearability. We consider a shoulder exoskeleton that consists of a parallel spherical mechanism with two slider crank mechanisms. The actuators can be placed on a stationary platform and attached closely to human body. Thus a better inertia property can be obtained while maintaining lightweight. Through the use of a gravity-balancing mechanism, the required actuator power becomes smaller and with better efficiency. A static model is developed to analyze and optimize the exoskeleton. Through illustrations of a prototype, the exoskeleton is shown to be wearable and can provide adequate motion enhancement of a human's upper limb.

A twisted string actuation module with an integrated force sensor based on optoelectronic components is presented in this paper. This solution is especially suited for very compact, light-weight and wearable robotic devices, such as exoskeletons, but is appropriate for various robotic applications. An in-depth presentation of the proposed actuation module, the description and the basic sensor working principle of the integrated force sensor are portrayed and discussed. Extensive experimental measures have been carried out to verify the actuation module compliant frame design and the related finite element analysis results. Therefore, the proposed actuation module has been used for the implementation of a simple assistive application where surface-electromyographic signals are used to detect the user's muscle activity in order to control the support action provided by the actuator, thus reducing his/her effort.


Question: PD control with online gravity compensation for robots with elastic joints: theoretical framework and experimental results.

This paper highlights the innovative design and development progress in DLR's robotics laboratory, aimed at creating the next generation of lightweight, torque-controlled robots. The authors detail the advanced mechanics, the integration of sensors, and the electronics that form the backbone of these robots. A notable feature is the fully sensorized joint, which includes motor and link position sensors alongside joint torque sensors. This sophisticated sensory system facilitates the implementation of effective vibration suppression techniques and paves the way for sophisticated control strategies suitable for compliant manipulation tasks. The interdisciplinary mechatronic design approach integrates mechanics, electronics, and controller design in a cohesive manner. This holistic strategy has been instrumental in advancing the field, bringing us closer to realizing the potential of service and personal robots in real-world applications.

We present a new methodology for exact robot motion planning and control that unifies the purely kinematic path planning problem with the lower level feedback controller design. Complete information about the freespace and goal is encoded in the form of a special artificial potential function a navigation function that connects the kinematic planning problem with the dynamic execution problem in a provably correct fashion. The navigation function automatically gives rise to a bounded-torque feedback controller for the robot's actuators that guarantees collision-free motion and convergence to the destination from almost all initial free configurations. Since navigation functions exist for any robot and obstacle course, our methodology is completely general in principle. However, this paper is mainly concerned with certain constructive techniques for a particular class of motion planning problems. Specifically, we present a formula for navigation functions that guide a point-mass robot in a generalized sphere world. The simplest member of this family is a space obtained by puncturing a disc by an arbitrary number of smaller disjoint discs representing obstacles. The other spaces are obtained from this model by a suitable coordinate transformation that we show how to build. Our constructions exploit these coordinate transformations to adapt a navigation function on the model space to its more geometrically complicated (but topologically equivalent) instances. The formula that we present admits sphere-worlds of arbitrary dimension and is directly applicable to configuration spaces whose forbidden regions can be modeled by such generalized discs. We have implemented these navigation functions on planar scenarios, and simulation results are provided throughout the paper. Disciplines Robotics Comments Copyright 1992 IEEE. Reprinted from IEEE Transactions on Robotics and Automation, Volume 8, Issue 5, October 1992, pages 501-518. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Pennsylvania's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws protecting it. NOTE: At the time of publication, Daniel Koditschek was affiliated with the University of Michigan. Currently, he is a faculty member at the School of Engineering of the University of Pennsylvania. This journal article is available at ScholarlyCommons: http://repository.upenn.edu/ese_papers/323

This paper focuses on a lower limb walking assistant robot. This robot has been designed and modeled for two tasks of facilitating healthy people, and assisting patients for walking. First, basic concepts of the motion mechanism are presented. Next, its kinematics and dynamics equations have been developed. Kinematics analysis has been done using Denavit-Hartenberg scheme to obtain position, velocity and acceleration of center of mass (COM) of all links. Dynamics equations have been developed using recursive Lagrange method and finally inverse dynamics problem has been solved. Obtained simulation results will be discussed which validate the proposed model.

The soft exosuit is a new approach for applying assistive forces over the wearer's body through load paths configured by the textile architecture. In this paper, we present a body-worn lower-extremity soft exosuit and a new control approach that can independently control the level of assistance that is provided during negative- and positive-power periods at the ankle. The exosuit was designed to create load paths assisting ankle plantarflexion and hip flexion, and the actuation system transmits forces from the motors to the suit via Bowden cables. A load cell and two gyro sensors per leg are used to measure real-time data, and the controller performs position control of the cable on a step-by-step basis with respect to the power delivered to the wearer's ankle by controlling two force parameters, the pretension and the active force. Human subjects testing results demonstrate that the controller is capable of modulating the amount of power delivered to the ankle joint. Also, significant reductions in metabolic rate (11%-15%) were observed, which indicates the potential of the proposed control approach to provide benefit to the wearer during walking.

We present a Bayesian framework for grasp planning that takes into account uncertainty in object shape or pose, as well as robot motion error. When trying to grasp objects based on noisy sensor data, a common problem is errors in perception, which cause the geometry or pose of the object to be uncertain. For each hypothesis about the geometry or pose of the object to be grasped, different sets of grasps can be planned. Of the resulting grasps, some are likely to work only if particular hypotheses are true, but some may work on most or even all hypotheses. Likewise, some grasps are easily broken by small errors in robot motion, but other grasps are robust to such errors. Our probabilistic framework takes into account all of these factors while trying to estimate the overall probability of success of each grasp, allowing us to select grasps that are robust to incorrect object recognition as well as motion error due to imperfect robot calibration. We demonstrate our framework while using the PR2 robot to grasp common household objects.

Continuum robots made from telescoping precurved elastic tubes enable base-mounted actuators to specify the curved shapes of robots as thin as standard surgical needles. While free space beam mechanics-based models of the shape of these ‘active cannulas’ exist, current models cannot account for external forces and torques applied to the cannula by the environment. In this paper we apply geometrically exact beam theory to solve the statics problem for concentric-tube continuum robots. This yields the equivalent of forward kinematics for an active cannula with general tube precurvature functions and arbitrarily many tubes, under loading from a general wrench distribution. The model achieves average experimental tip errors of less than 3 mm over the workspace of a prototype active cannula subject to various tip forces.

An acrobot is a planar robot with a passive actuator in its first joint. The control problem of the acrobot tries to make it rise from the rest position to the inverted pendulum position. This control problem can be divided in the swing-up problem, when the robot has to rise itself through swinging up as a human acrobat does, and the balancing problem, when the robot has to maintain itself on the inverted pendulum position. We have developed three controllers for the swing-up problem applied to two types of motors: small and big. For small motors, we used the SARSA controller and the PD with a trajectory generator. For big motors, we propose a new controller to control the acrobot, a PWM controller. All controllers except SARSA are tuned using a Genetic Algorithm.

Grasp stability estimation with complex robots in environments with uncertainty is a major research challenge. Analytical measures such as force closure based grasp quality metrics are often impractical because tactile sensors are unable to measure contacts accurately enough especially in soft contact cases. Recently, an alternative approach of learning the stability based on examples has been proposed. Current approaches of stability learning analyze the tactile sensor readings only at the end of the grasp attempt, which makes them somewhat time consuming, because the grasp can be stable already earlier. In this paper, we propose an approach for grasp stability learning, which estimates the stability continuously during the grasp attempt. The approach is based on temporal filtering of a support vector machine classifier output. Experimental evaluation is performed on an anthropomorphic ARMAR-IIIb. The results demonstrate that the continuous estimation provides equal performance to the earlier approaches while reducing the time to reach a stable grasp significantly. Moreover, the results demonstrate for the first time that the learning based stability estimation can be used with a flexible, pneumatically actuated hand, in contrast to the rigid hands used in earlier works.

1 In this paper we describe the Pendubot, a mechatronic device for use in control engineering education and for research in nonlinear control and robotics. This device is a two{link planar robot with an actuator at the shoulder but no actuator at the elbow. With this system , a number of fundamental concepts in nonlinear dynamics and control theory may be illustrated. The pendubot complements previous mechatronic systems, such as the Acrobot 3] and the inverted pendulum of Furuta 4].

In computer animation, the proportional-derivative (PD) controller is a common technique for tracking characters' motion. A new formulation of the PD controller-stable PD (SPD)-allows arbitrarily high gains, even at large time steps. The key is to determine joint forces and torques while taking into account the character's positions and velocities in the next time step. SPD is stable even when combined with a physics simulator that uses simple Euler integration. SPD controllers have a variety of uses, including motion tracking in a physics simulator, keyframe interpolation with secondary motion, and constraint satisfaction for simulation.

This paper introduces a four-degree-of-freedom parallel manipulator dedicated to pick-and-place. It has been developed with the goal of reaching very high speed. This paper shows that its architecture is particularly well adapted to high dynamics. Indeed, it is an evolution of Delta, H4 and 14 robots architectures: it keeps the advantages of these existing robots, while overcoming their drawbacks. In addition, an optimization method based on velocity using adept motion has been developed and applied to this high speed parallel robot. All these considerations led to experimentations that proved we can reach high accelerations (13 G) and obtain a cycle time of 0.28 s.


Question: Hierarchical and modular learning systems design.

This paper delves into the exploration and utilization of Classification and Regression Trees (CART) as a robust method for developing modular and hierarchical learning systems. CART algorithms are employed to construct decision trees for classification and regression tasks, offering a hierarchical structure that can be effectively utilized in modular learning architectures. The hierarchical nature of CART allows for the decomposition of complex problems into simpler, more manageable subproblems, making it an ideal choice for building modular learning systems. Each node in the decision tree represents a feature, and the branches emanating from it signify the possible outcomes, leading to a hierarchical decision-making process. This approach facilitates the creation of a layered learning system, where each level specializes in learning specific aspects of the data, contributing to the overall predictive power of the model. Furthermore, the modular design allows for the incorporation of domain-specific knowledge at different levels of the tree, enhancing the adaptability and performance of the learning system. By leveraging CART in this manner, one can construct sophisticated learning architectures that are both interpretable and capable of handling high-dimensional data with ease. This paper provides insights into the design and implementation of such systems, emphasizing the benefits of CART in the context of modular and hierarchical learning.

Neural probabilistic language models (NPLMs) have been shown to be competitive with and occasionally superior to the widely-used n-gram language models. The main drawback of NPLMs is their extremely long training and testing times. Morin and Bengio have proposed a hierarchical language model built around a binary tree of words, which was two orders of magnitude faster than the nonhierarchical model it was based on. However, it performed considerably worse than its non-hierarchical counterpart in spite of using a word tree created using expert knowledge. We introduce a fast hierarchical language model along with a simple feature-based algorithm for automatic construction of word trees from the data. We then show that the resulting models can outperform non-hierarchical neural models as well as the best n-gram models.

Learning, planning, and representing knowledge at multiple levels of temporal abstraction are key, longstanding challenges for AI. In this paper we consider how these challenges can be addressed within the mathematical framework of reinforcement learning and Markov decision processes (MDPs). We extend the usual notion of action in this framework to include options—closed-loop policies for taking action over a period of time. Examples of options include picking up an object, going to lunch, and traveling to a distant city, as well as primitive actions such as muscle twitches and joint torques. Overall, we show that options enable temporally abstract knowledge and action to be included in the reinforcement learning framework in a natural and general way. In particular, we show that options may be used interchangeably with primitive actions in planning methods such as dynamic programming and in learning methods such as Q-learning. Formally, a set of options defined over an MDP constitutes a semi-Markov decision process (SMDP), and the theory of SMDPs provides the foundation for the theory of options. However, the most interesting issues concern the interplay between the underlying MDP and the SMDP and are thus beyond SMDP theory. We present results for three such cases: (1) we show that the results of planning with options can be used during execution to interrupt options and thereby perform even better than planned, (2) we introduce new intra-option methods that are able to learn about an option from fragments of its execution, and (3) we propose a notion of subgoal that can be used to improve the options themselves. All of these results have precursors in the existing literature; the contribution of this paper is to establish them in a simpler and more general setting with fewer changes to the existing reinforcement learning framework. In particular, we show that these results can be obtained without committing to (or ruling out) any particular approach to state abstraction, hierarchy, function approximation, or the macroutility problem. ! 1999 Published by Elsevier Science B.V. All rights reserved. ∗ Corresponding author. 0004-3702/99/$ – see front matter ! 1999 Published by Elsevier Science B.V. All rights reserved. PII: S0004-3702(99)00052 -1 182 R.S. Sutton et al. / Artificial Intelligence 112 (1999) 181–211

The paper discusses the augmentation of the Concurrent Individual and Social Learning (CISL) mechanism with a new Human Advice Layer (HAL). The new layer is characterized by a Gaussian Mixture Model (GMM), which is trained on human experience data. The CISL mechanism consists of the Individual Performance and Task Allocation Markov Decision Processes (MDP), and the HAL can provide preferred action selection policies to the individual agents. The data utilized for training the GMM is collected using a heterogeneous team foraging simulation. When leveraging human experience in the multi-agent learning process, the team performance is enhanced significantly.

Tomorrow's humanoids will operate in human environments, where efficient manipulation and locomotion skills, and safe contact interactions are critical design factors. We report here our recent efforts into these issues, materialized into a whole-body control framework. This framework integrates task-oriented dynamic control and control prioritization allowing to control multiple task primitives while complying with physical and movement-related constraints. Prioritization establishes a hierarchy between control spaces, assigning top priority to constraint-handling tasks, while projecting operational tasks in the null space of the constraints, and controlling the posture within the residual redundancy. This hierarchy is directly integrated at the kinematic level, allowing the program to monitor behavior feasibility at runtime. In addition, prioritization allows us to characterize the dynamic behavior of the individual control primitives subject to the constraints, and to synthesize operational space controllers at multiple levels. To complete this framework, we have developed free-floating models of the humanoid and incorporate the associated dynamics and the effects of the resulting support contacts into the control hierarchy. As part of a long term collaboration with Honda, we are currently implementing this framework into the humanoid robot Asimo

Empirically, neural networks that attempt to learn programs from data have exhibited poor generalizability. Moreover, it has traditionally been difficult to reason about the behavior of these models beyond a certain level of input complexity. In order to address these issues, we propose augmenting neural architectures with a key abstraction: recursion. As an application, we implement recursion in the Neural Programmer-Interpreter framework on four tasks: grade-school addition, bubble sort, topological sort, and quicksort. We demonstrate superior generalizability and interpretability with small amounts of training data. Recursion divides the problem into smaller pieces and drastically reduces the domain of each neural network component, making it tractable to prove guarantees about the overall system’s behavior. Our experience suggests that in order for neural architectures to robustly learn program semantics, it is necessary to incorporate a concept like recursion.

Cascade-Correlation is a new architecture and supervised learning algorithm for artificial neural networks. Instead of just adjusting the weights in a network of fixed topology, Cascade-Correlation begins with a minimal network, then automatically trains and adds new hidden units one by one, creating a multi-layer structure. Once a new hidden unit has been added to the network, its input-side weights are frozen. This unit then becomes a permanent feature-detector in the network, available for producing outputs or for creating other, more complex feature detectors. The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the connections of the network. This research was sponsored in part by the National Science Foundation under Contract Number EET-8716324 and by the Defense Advanced Research Projects Agency (DOD), ARPA Order No. 4976 under Contract F33615-87-C-1499 and monitored by: Avionics Laboratory, Air Force Wright Aeronautical Laboratories, Aeronautical Systems Division (AFSC), Wright-Patterson AFB, OH 45433-6543. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the US Government. 1. Why is Back-Propagation Learning So Slow? The Cascade-Correlation learning algorithm was developed in an attempt to overcome certain problems and limitations of the popular back-propagation (or “backprop”) learning algorithm [Rumelhart, 1986]. The most important of these limitations is the slow pace at which backprop learns from examples. Even on simple benchmark problems, a back-propagation network may require many thousands of epochs to learn the desired behavior from examples. (An epoch is defined as one pass through the entire set of training examples.) We have attempted to analyze the reasons why backprop learning is so slow, and we have identified two major problems that contribute to the slowness. We call these the step-size problem and the moving target problem. There may, of course, be other contributing factors that we have not yet identified. 1.1. The Step-Size Problem The step-size problem occurs because the standard back-propagation method computes only E w, the partial first derivative of the overall error function with respect to each weight in the network. Given these derivatives, we can perform a gradient descent in weight space, reducing the error with each step. It is straightforward to show that if we take infinitesimal steps down the gradient vector, running a new training epoch to recompute the gradient after each step, we will eventually reach a local minimum of the error function. Experience has shown that in most situations this local minimum will be a global minimum as well, or at least a “good enough” solution to the problem at hand. In a practical learning system, however, we do not want to take infinitesimal steps; for fast learning, we want to take the largest steps that we can. Unfortunately, if we choose a step size that is too large, the network will not reliably converge to a good solution. In order to choose a reasonable step size, we need to know not just the slope of the error function, but something about its higher-order derivatives—its curvature—in the vicinity of the current point in weight space. This information is not available in the standard back-propagation algorithm. A number of schemes have been proposed for dealing with the step-size problem. Some form of “momentum” [Rumelhart, 1986] is often used as a crude way of summarizing the slope of the error surface at earlier points in the computation. Conjugate gradient methods have been explored in the context of artificial neural networks by a number of researchers [Watrous, 1988, Lapedes, 1987, Kramer, 1989] with generally good results. Several schemes, for example [Franzini, 1987] and jacobs:dynamic, have been proposed that adjust the step-size dynamically, based on the change in gradient from one step to another. Becker and LeCun [Becker, 1988] explicitly compute an approximation to the second derivative of the error function at each step and use that information to guide the speed of descent. Fahlman’s quickprop algorithm [Fahlman, 1988] is one of the more successful algorithms for handling the step-size problem in back-propagation systems. Quickprop computes the E w values just as in standard backprop, but instead of simple gradient descent, Quickprop uses a second-order method, related to Newton’s method, to update the weights. On the learning benchmarks we have collected, quickprop consistently out-performs other backprop-like algorithms, sometimes by a large factor. Quickprop’s weight-update procedure depends on two approximations: first, that small changes in one weight have relatively little effect on the error gradient observed at other weights; second, that the error function with respect to each weight is locally quadratic. For each weight, quickprop keeps a copy of

Computers and Thought are the two categories that together de ne Arti cial Intelligence as a discipline. It is generally accepted that work in Arti cial Intelligence over the last thirty years has had a strong in uence on aspects of computer architectures. In this paper we also make the converse claim; that the state of computer architecture has been a strong in uence on our models of thought. The Von Neumann model of computation has lead Arti cial Intelligence in particular directions. Intelligence in biological systems is completely di erent. Recent work in behavior-based Arti cial Intelligence has produced new models of intelligence that are much closer in spirit to biological systems. The non-Von Neumann computational models they use share many characteristics with biological computation. Copyright c Massachusetts Institute of Technology, 1991 This report describes research done at the Arti cial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for this research was provided in part by the University Research Initiative under O ce of Naval Research contract N00014{86{K{0685, in part by the Advanced Research Projects Agency under O ce of Naval Research contract N00014{ 85{K{0124, in part by the Hughes Arti cial Intelligence Center, in part by Siemens Corporation, and in part by Mazda Corporation.

Research in the area of learning structural descriptions from examples is reviewed, giving primary attention to methods of learning characteristic descrip tions of single concepts. In particular, we examine methods for finding the maximally-specific conjunctive generalizations (MSC-generalizations) that cover all of the training examples of a given concept. Various important aspects of structural learning in general are examined, and several criteria for evaluating structural learning methods are presented. Briefly, these criteria include (i) ade quacy of the representation language, (ii) generalization rules employed, computational efficiency, and (iv) flexibility and extensibility. Selected learning methods developed by Buchanan, et al., Hayes-Roth, Vere, Winston, and the authors are analyzed according to these criteria. Finally, some goals are sug gested for future research.

This paper describes the software architecture of Stanley, an autonomous land vehicle developed for high-speed desert driving without human intervention. The vehicle recently won the DARPA Grand Challenge, a major robotics competition. The article describes the software architecture of the robot, which relied pervasively on state-of-the-art AI technologies, such as machine learning and probabilistic reasoning.

This article describes the software architecture of an auto nomous, interactive tour-guide robot. It presents a modular and distributed software archi te ture, which integrates localization, mapping, collision avoidance, planning, and vari ous modules concerned with user interaction and Web-based telepresence. At its heart, the s oftware approach relies on probabilistic computation, on-line learning, and any-time alg orithms. It enables robots to operate safely, reliably, and at high speeds in highly dynamic environments, and does not require any modifications of the environment to aid the robot ’s peration. Special emphasis is placed on the design of interactive capabilities that appeal to people’s intuition. The interface provides new means for human-robot interaction w ith crowds of people in public places, and it also provides people all around the world with the ability to establish a “virtual telepresence” using the Web. To illustrate our approac h, results are reported obtained in mid-1997, when our robot “RHINO” was deployed for a period of six days in a densely populated museum. The empirical results demonstrate relia bl operation in public environments. The robot successfully raised the museum’s atten dance by more than 50%. In addition, thousands of people all over the world controlled the robot through the Web. We conjecture that these innovations transcend to a much large r range of application domains for service robots.


Question: Hierarchical and modular learning architectures

This paper introduces an innovative supervised learning technique for complex systems comprised of multiple interconnected networks, each specializing in a distinct segment of the overall training dataset. The proposed methodology can be interpreted as a modular extension of a traditional multilayer supervised network or, alternatively, as a competitive learning approach with associative capabilities. This dual perspective establishes a novel connection between two seemingly disparate learning paradigms. Through practical application, we illustrate how the learning procedure effectively partitions a challenging vowel discrimination task into manageable subtasks, each of which can be efficiently tackled by a straightforward expert network, thereby showcasing the method's capability to optimize learning through specialization.

This paper illustrates an artiicial developmental system that is a computationally ee-cient technique for the automatic generation of complex Artiicial Neural Networks (ANN). Artiicial developmental system can develop a graph grammar into a modular ANN made of a combination of more simple subnetworks. A genetic algorithm is used to evolve coded grammars that generates ANNs for controlling a six-legged robot locomotion. A mechanism for the automatic deenition of sub-neural networks is incorporated. Using this mechanism, the genetic algorithm can automatically decompose a problem into subproblems, generate a subANN for solving the subproblem, and instantiate copies of this subANN to build a higher level ANN that solves the problem. We report some simulation results showing that the same problem cannot be solved if the mechanism for automatic deenition of sub-networks is suppressed. We support our argumentation with pictures describing the steps of development, how ANN structures are evolved, and how the ANNs compute.matic deention of sub-neural networks.

We present a kernel-based algorithm for hierarchical text c lassification where the documents are allowed to belong to more than one category at a time. The clas sification model is a variant of the Maximum Margin Markov Network framework, where the classifi cation hierarchy is represented as a Markov tree equipped with an exponential family defined o n the edges. We present an efficient optimization algorithm based on incremental conditi onal gradient ascent in single-example subspaces spanned by the marginal dual variables. The optim ization is facilitated with a dynamic programming based algorithm that computes best update dire ctions in the feasible set. Experiments show that the algorithm can feasibly optimize t raining sets of thousands of examples and classification hierarchies consisting of hundreds of nodes. Training of the full hierarchical model is as efficient as training independent SVM-light clas sifiers for each node. The algorithm’s predictive accuracy was found to be competitive with other r ecently introduced hierarchical multicategory or multilabel classification learning algorithms .

Techniques for automatically designing deep neural network architectures such as reinforcement learning based approaches have recently shown promising results. However, their success is based on vast computational resources (e.g. hundreds of GPUs), making them difficult to be widely used. A noticeable limitation is that they still design and train each network from scratch during the exploration of the architecture space, which is highly inefficient. In this paper, we propose a new framework toward efficient architecture search by exploring the architecture space based on the current network and reusing its weights. We employ a reinforcement learning agent as the meta-controller, whose action is to grow the network depth or layer width with function-preserving transformations. As such, the previously validated networks can be reused for further exploration, thus saves a large amount of computational cost. We apply our method to explore the architecture space of the plain convolutional neural networks (no skip-connections, branching etc.) on image benchmark datasets (CIFAR-10, SVHN) with restricted computational resources (5 GPUs). Our method can design highly competitive networks that outperform existing networks using the same design scheme. On CIFAR-10, our model without skip-connections achieves 4.23% test error rate, exceeding a vast majority of modern architectures and approaching DenseNet. Furthermore, by applying our method to explore the DenseNet architecture space, we are able to achieve more accurate networks with fewer parameters.

This review will focus on four areas of motor control which have recently been enriched both by neural network and control system models: motor planning, motor prediction, state estimation and motor learning. We will review the computational foundations of each of these concepts and present specific models which have been tested by psychophysical experiments. We will cover the topics of optimal control for motor planning, forward models for motor prediction, observer models of state estimation arid modular decomposition in motor learning. The aim of this review is to demonstrate how computational approaches, as well as proposing specific models, provide a theoretical framework to formalize the issues in motor control.

In this paper, we propose a Layer-RNN (L-RNN) module that is able to learn contextual information adaptively using within-layer recurrence. Our contributions are three-fold: (i) we propose a hybrid neural network architecture that interleaves traditional convolutional layers with L-RNN module for learning longrange dependencies at multiple levels; (ii) we show that a L-RNN module can be seamlessly inserted into any convolutional layer of a pre-trained CNN, and the entire network then fine-tuned, leading to a boost in performance; (iii) we report experiments on the CIFAR-10 classification task, showing that a network with interleaved convolutional layers and L-RNN modules, achieves comparable results (5.39% top1 error) using only 15 layers and fewer parameters to ResNet-164 (5.46%); and on the PASCAL VOC2012 semantic segmentation task, we show that the performance of a pre-trained FCN network can be boosted by 5% (mean IOU) by simply inserting Layer-RNNs.

We propose to take a novel approach to robot system design where each building block of a larger system is represented as a differentiable program, i.e. a deep neural network. This representation allows for integrating algorithmic planning and deep learning in a principled manner, and thus combine the benefits of model-free and model-based methods. We apply the proposed approach to a challenging partially observable robot navigation task. The robot must navigate to a goal in a previously unseen 3-D environment without knowing its initial location, and instead relying on a 2-D floor map and visual observations from an onboard camera. We introduce the Navigation Networks (NavNets) that encode state estimation, planning and acting in a single, end-to-end trainable recurrent neural network. In preliminary simulation experiments we successfully trained navigation networks to solve the challenging partially observable navigation task.

The problem of learning a general input-output relation using a layered neural network is discussed in a statistical framework. By imposing the consistency condition that the error minimization be equivalent to a likelihood maximization for training the network, the authors arrive at a Gibbs distribution on a canonical ensemble of networks with the same architecture. This statistical description enables them to evaluate the probability of a correct prediction of an independent example, after training the network on a given training set. The prediction probability is highly correlated with the generalization ability of the network, as measured outside the training set. This suggests a general and practical criterion for training layered networks by minimizing prediction errors. The authors demonstrate the utility of this criterion for selecting the optimal architecture in the continuity problem. As a theoretical application of the statistical formalism, they discuss the question of learning curves and estimate the sufficient training size needed for correct generalization, in a simple example.<<ETX>>

We propose a learning architecture, that is able to do reinforcement learning based on raw visual input data. In contrast to previous approaches, not only the control policy is learned. In order to be successful, the system must also autonomously learn, how to extract relevant information out of a high-dimensional stream of input information, for which the semantics are not provided to the learning system. We give a first proof-of-concept of this novel learning architecture on a challenging benchmark, namely visual control of a racing slot car. The resulting policy, learned only by success or failure, is hardly beaten by an experienced human player.

Previous studies have shown that spike-timing-dependent plasticity (STDP) can be used in spiking neural networks (SNN) to extract visual features of low or intermediate complexity in an unsupervised manner. These studies, however, used relatively shallow architectures, and only one layer was trainable. Another line of research has demonstrated - using rate-based neural networks trained with back-propagation - that having many layers increases the recognition robustness, an approach known as deep learning. We thus designed a deep SNN, comprising several convolutional (trainable with STDP) and pooling layers. We used a temporal coding scheme where the most strongly activated neurons fire first, and less activated neurons fire later or not at all. The network was exposed to natural images. Thanks to STDP, neurons progressively learned features corresponding to prototypical patterns that were both salient and frequent. Only a few tens of examples per category were required and no label was needed. After learning, the complexity of the extracted features increased along the hierarchy, from edge detectors in the first layer to object prototypes in the last layer. Coding was very sparse, with only a few thousands spikes per image, and in some cases the object category could be reasonably well inferred from the activity of a single higher-order neuron. More generally, the activity of a few hundreds of such neurons contained robust category information, as demonstrated using a classifier on Caltech 101, ETH-80, and MNIST databases. We also demonstrate the superiority of STDP over other unsupervised techniques such as random crops (HMAX) or auto-encoders. Taken together, our results suggest that the combination of STDP with latency coding may be a key to understanding the way that the primate visual system learns, its remarkable processing speed and its low energy consumption. These mechanisms are also interesting for artificial vision systems, particularly for hardware solutions.

Learning to recognize of object classes is one of the most important functionalities of vision. It is estimated that humans are able to learn tens of thousands of visual categori es in their life. Given the photometric and geometric variabilities displayed by objects as well as the high degree of intra-clas s variabilities, we hypothesize that humans achieve such a fe at by using knowledge and information cumulated throughout the learning process. In recent years, a handful of pioneering p apers have applied various forms of knowledge transfer algorithms to the problem of learning object classes. We first review some o f these papers by loosely grouping them into three categories : transfer through prior parameters, transfer through shared features or parts, and transfer through contextual information. In the second half of the paper, we detail a recent algorithm proposed by the author. This incremental learning scheme us es information from object classes previously learned in the f orm of prior models to train a new object class model. Training images can be presented in an incremental way. We present experimental results tested with this model on a large numbe r of object categories.


Question:
Hierarchical and modular learning systems design

**Question**: An algorithm for the job-shop scheduling problem

**Similar Question**: A scheduling algorithm for the job-shop problem

**Question**: A new method for estimating the parameters of a linear model

**Similar Question**: A novel technique for parameter estimation in linear models

In this paper, we introduce a Bayesian framework designed for estimating the parameters of a mixture of experts (MoE) model, employing ensemble learning through variational free energy minimization. This approach offers a significant advantage over conventional maximum likelihood estimation by mitigating the issues of overfitting and underestimation of noise levels. By adopting a Bayesian perspective, we can more accurately infer the parameters of the MoE model, leading to improved model robustness and reliability. We validate our methodology through its application to both synthetic datasets and a real-world case study involving the prediction of sunspot time series. The results obtained highlight the superiority of our Bayesian approach in terms of predictive accuracy and parameter estimation, compared to traditional methods. This framework holds promise for enhancing the performance of hierarchical and modular learning systems, particularly in scenarios with complex data distributions and varying noise levels.

A growing number of commercial and enterprise systems increasingly rely on compute-intensive Machine Learning (ML) algorithms. While the demand for these compute-intensive applications is growing, the performance benefits from general-purpose platforms are diminishing. Field Programmable Gate Arrays (FPGAs) provide a promising path forward to accommodate the needs of machine learning algorithms and represent an intermediate point between the efficiency of ASICs and the programmability of general-purpose processors. However, acceleration with FPGAs still requires long development cycles and extensive expertise in hardware design. To tackle this challenge, instead of designing an accelerator for a machine learning algorithm, we present TABLA, a framework that generates accelerators for a class of machine learning algorithms. The key is to identify the commonalities across a wide range of machine learning algorithms and utilize this commonality to provide a high-level abstraction for programmers. TABLA leverages the insight that many learning algorithms can be expressed as a stochastic optimization problem. Therefore, learning becomes solving an optimization problem using stochastic gradient descent that minimizes an objective function over the training data. The gradient descent solver is fixed while the objective function changes for different learning algorithms. TABLA provides a template-based framework to accelerate this class of learning algorithms. Therefore, a developer can specify the learning task by only expressing the gradient of the objective function using our high-level language. Tabla then automatically generates the synthesizable implementation of the accelerator for FPGA realization using a set of hand-optimized templates. We use Tabla to generate accelerators for ten different learning tasks targeted at a Xilinx Zynq FPGA platform. We rigorously compare the benefits of FPGA acceleration to multi-core CPUs (ARM Cortex A15 and Xeon E3) and many-core GPUs (Tegra K1, GTX 650 Ti, and Tesla K40) using real hardware measurements. TABLA-generated accelerators provide 19.4x and 2.9x average speedup over the ARM and Xeon processors, respectively. These accelerators provide 17.57x, 20.2x, and 33.4x higher Performance-per-Watt in comparison to Tegra, GTX 650 Ti and Tesla, respectively. These benefits are achieved while the programmers write less than 50 lines of code.

Example-basedmethodsareeffectivefor parameterestimationproblemswhentheunderlyingsystemis simpleor thedimensionalityof the input is low. For complex andhigh-dimensional problemssuch asposeestimation,thenumberof required examplesand the computationalcomplexity rapidly becmeprohibitivelyhigh. We introducea new algorithm that learnsa setof hashingfunctionsthat efficiently index examplesrelevant to a particular estimationtask. Our algorithm extendsa recentlydevelopedmethodfor locality-sensitivehashing, which findsapproximateneighborsin timesublinearin thenumber of examples.Thismethoddependscritically on thechoiceof hashfunctions;weshowhowto find thesetof hashfunctions thatare optimallyrelevantto a particular estimationproblem.Experimentsdemonstr atethat theresultingalgorithm,which wecall Parameter -SensitiveHashing, canrapidlyandaccuratelyestimatethearticulatedposeof humanfiguresfroma large databaseof exampleimages. 0Part of thiswork wasdonewhenG.S.andP.V. werewith MitsubishiElectricResearchLabs,Cambridge,MA.

In this paper the task of training subsymbolic systems is considered as a combinatorial optimization problem and solved with the heuristic scheme of the reactive tabu search (RTS). An iterative optimization process based on a "modified local search" component is complemented with a meta-strategy to realize a discrete dynamical system that discourages limit cycles and the confinement of the search trajectory in a limited portion of the search space. The possible cycles are discouraged by prohibiting (i.e., making tabu) the execution of moves that reverse the ones applied in the most recent part of the search. The prohibition period is adapted in an automated way. The confinement is avoided and a proper exploration is obtained by activating a diversification strategy when too many configurations are repeated excessively often. The RTS method is applicable to nondifferentiable functions, is robust with respect to the random initialization, and effective in continuing the search after local minima. Three tests of the technique on feedforward and feedback systems are presented.

Reinforcement learning agents have traditionally been evaluated on small toy problems. With advances in computing power and the advent of the Arcade Learning Environment, it is now possible to evaluate algorithms on diverse and difficult problems within a consistent framework. We discuss some challenges posed by the arcade learning environment which do not manifest in simpler environments. We then provide a comparison of model-free, linear learning algorithms on this challenging problem set.

A f?umeworkfor learning parumeterized models oj‘optical flow from image sequences is presented. A class of motions is represented by U set of orthogonul basis $ow fields that are computed froni a training set using priiicipal component analysis. Many complex image motions can be represeiited by a linear combiriution of U small number of these basisjows. The learned motion models may be usedfilr optical flow estimation und for model-bused recognition. For optical flow estimation we describe a robust, multi-resolution scheme for directly computing the parameters of the learned flow models from image derivatives. As examples we consider learning motion discontinuities, non-rigid motion of human mouths, und urticulated human motion.

This paper describes the use of genetic algorithms (GA) for solving the facility layout problem (FLP) within manufacturing systems’ design. The paper considers a specific heuristic layout planning method, known as the modified triangle method. This method seeks for an optimal layout solution based on the degrees of flows between workstations. The search for an optimal solution is extremely time-consuming and not suitable for larger systems; therefore we have developed a system based on evolutionary computation. Our paper presents a system, based on GA, and the results obtained regarding several numerical cases from the literature. We propose the usage of this system by presenting numerous advantages over other methods of solving FLP for problem representation and evolutionary computation for solution search. (Received in September 2012, accepted in June 2013. This paper was with the authors 2 months for 2 revisions.)

Cognitive science researchers have emphasized the importance of ordering a complex task into a sequence of easy to hard problems. Such an ordering provides an easier path to learning and increases the speed of acquisition of the task compared to conventional learning. Recent works in machine learning have explored a curriculum learning approach called selfpaced learning which orders data samples on the easiness scale so that easy samples can be introduced to the learning algorithm first and harder samples can be introduced successively. We introduce a number of heuristics that improve upon selfpaced learning. Then, we argue that incorporating easy, yet, a diverse set of samples can further improve learning. We compare these curriculum learning proposals in the context of four non-convex models for QA and show that they lead to real improvements in each of them.

Chapter 1 strongly advocates the stochastic back-propagation method to train neural networks. This is in fact an instance of a more general technique called stochastic gradient descent (SGD). This chapter provides background material, explains why SGD is a good learning algorithm when the training set is large, and provides useful recommendations.

In simulation software selection problems, packages are evaluated either on their own merits or in comparison with other packages. In either method, a comprehensive list of criteria for evaluation of simulation software is essential for proper selection. Although various simulation software evaluation checklists do exist, there are dierences in the lists provided and the terminologies used. This paper presents a hierarchical framework for simulation software evaluation consisting of seven main groups and several subgroups. An explanation for each criterion is provided and an analysis of the usability of the proposed framework is further discussed. Ó 1999 Elsevier Science B.V. All rights reserved.


Question: Hierarchical and modular learning systems design.

Incorporating Bayesian Principles into Supervised Classification Networks

This paper explores three key Bayesian concepts to enhance the performance of supervised adaptive classifiers. Firstly, it posits that the classifier's output should be derived by integrating over the posterior distribution of its parameters. An effective approximation to this complex integral is suggested and empirically validated, illustrating its ability to "moderate" the outputs of the most probable classifier, thereby enhancing overall accuracy. Secondly, the paper showcases the applicability of the Bayesian framework for model comparison, previously detailed for regression models by MacKay (1992a,b), to classification tasks. This framework not only optimally determines the weight decay magnitude but also adeptly ranks solutions obtained using varying numbers of hidden units, demonstrating its versatility and effectiveness. Lastly, a novel data selection criterion, rooted in information theory, is derived and implemented within this Bayesian framework, underscoring the potential for sophisticated data handling and model refinement in classification networks.

This approach underscores the significance of hierarchical and modular learning systems, where the integration of Bayesian principles allows for more nuanced and robust classifier outputs. By leveraging posterior distributions for output moderation, optimizing model parameters through informed weight decay, and utilizing an information-based criterion for data selection, the proposed method significantly advances the capabilities of supervised classification networks. The incorporation of these Bayesian ideas not only improves the classifier's performance but also offers a systematic approach to model comparison and data utilization, thus facilitating the development of more intelligent and adaptive learning systems.

For a machine to be able to ‘see’, it must know something about the object it is ‘looking’ at, A common method in machine vision is to provide the machine with general rather than specific knowledge about the object. An alternative technique, and the one used in thti paper, b a model-based approach in which particulars aboat the object are given and this drives the analysis. The computer program described here, the WALKER model, maps images into a description in which a person is represented by the series of hierarchical levels, i.e. a person has an arm which has a lower-arm which has a hand. l7te performance of the program ti illustrated by superimposing the machinegenerated picture over the original photographic images.

This paper describes the Multiagent Systems Engineering (MaSE) methodology. MaSE is a general purpose, methodology for developing heterogeneous multiagent systems. MaSE uses a number of graphically based models to describe system goals, behaviors, agent types, and agent communication interfaces. MaSE also provides a way to specify architecture-independent detailed definition of the internal agent design. An example of applying the MaSE methodology is also presented.

The next generation of embedded computing systems will have to meet new challenges. The systems are expected to act mainly autonomously, to dynamically adapt to changing environments and to interact with one another if necessary. Such systems are called organic. Organic Computing systems are similar to autonomic computing systems. In addition Organic Computing systems often behave life-like and are inspired by nature/biological phenomena. Design and construction of such systems brings new challenges for the software engineering process. In this paper we present a framework for design, construction and analysis of organic computing systems. It can facilitate design and construction as well as it can be used to (semi-)formally define organic properties like self-configuration or self-adaptation. We illustrate the framework on a real-world case study from production automation.

Recently, there have been increasing demands to construct compact deep architectures to remove unnecessary redundancy and to improve the inference speed. While many recent works focus on reducing the redundancy by eliminating unneeded weight parameters, it is not possible to apply a single deep network for multiple devices with different resources. When a new device or circumstantial condition requires a new deep architecture, it is necessary to construct and train a new network from scratch. In this work, we propose a novel deep learning framework, called a nested sparse network, which exploits an n-in-1-type nested structure in a neural network. A nested sparse network consists of multiple levels of networks with a different sparsity ratio associated with each level, and higher level networks share parameters with lower level networks to enable stable nested learning. The proposed framework realizes a resource-aware versatile architecture as the same network can meet diverse resource requirements, i.e., anytime property. Moreover, the proposed nested network can learn different forms of knowledge in its internal networks at different levels, enabling multiple tasks using a single network, such as coarse-to-fine hierarchical classification. In order to train the proposed nested network, we propose efficient weight connection learning and channel and layer scheduling strategies. We evaluate our network in multiple tasks, including adaptive deep compression, knowledge distillation, and learning class hierarchy, and demonstrate that nested sparse networks perform competitively, but more efficiently, compared to existing methods.

In this paper we present a method for systematically defining the process architecture of any organization. We begin by presenting a detailed interpretation of the upper levels of Zachman’s framework, providing an answer to the cell formed by the intersection of the perspective of the Planner (row 1) and the dimension of Function (column 2). Our method consists of four steps: capturing the organizational structure; exhaustively analyzing the flow of information between the different organizational units, customers, and providers, allowing for a high-level understanding of the organization’s operation; identifying and modeling the configuration of value that best adapts itself to the creation of value for the organization (value chain, workshop, or network); and, given the specified value configuration, identifying, defining, and interrelating the different essential processes.

Learning, planning, and representing knowledge at multiple levels of temporal abstraction are key, longstanding challenges for AI. In this paper we consider how these challenges can be addressed within the mathematical framework of reinforcement learning and Markov decision processes (MDPs). We extend the usual notion of action in this framework to include options—closed-loop policies for taking action over a period of time. Examples of options include picking up an object, going to lunch, and traveling to a distant city, as well as primitive actions such as muscle twitches and joint torques. Overall, we show that options enable temporally abstract knowledge and action to be included in the reinforcement learning framework in a natural and general way. In particular, we show that options may be used interchangeably with primitive actions in planning methods such as dynamic programming and in learning methods such as Q-learning. Formally, a set of options defined over an MDP constitutes a semi-Markov decision process (SMDP), and the theory of SMDPs provides the foundation for the theory of options. However, the most interesting issues concern the interplay between the underlying MDP and the SMDP and are thus beyond SMDP theory. We present results for three such cases: (1) we show that the results of planning with options can be used during execution to interrupt options and thereby perform even better than planned, (2) we introduce new intra-option methods that are able to learn about an option from fragments of its execution, and (3) we propose a notion of subgoal that can be used to improve the options themselves. All of these results have precursors in the existing literature; the contribution of this paper is to establish them in a simpler and more general setting with fewer changes to the existing reinforcement learning framework. In particular, we show that these results can be obtained without committing to (or ruling out) any particular approach to state abstraction, hierarchy, function approximation, or the macroutility problem. ! 1999 Published by Elsevier Science B.V. All rights reserved. ∗ Corresponding author. 0004-3702/99/$ – see front matter ! 1999 Published by Elsevier Science B.V. All rights reserved. PII: S0004-3702(99)00052 -1 182 R.S. Sutton et al. / Artificial Intelligence 112 (1999) 181–211

Lifelong Machine Learning, or LML, considers systems that can learn many tasks from one or more domains over its lifetime. The goal is to sequentially retain learned knowledge and to selectively transfer that knowledge when learning a new task so as to develop more accurate hypotheses or policies. Following a review of prior work on LML, we propose that it is now appropriate for the AI community to move beyond learning algorithms to more seriously consider the nature of systems that are capable of learning over a lifetime. Reasons for our position are presented and potential counter-arguments are discussed. The remainder of the paper contributes by defining LML, presenting a reference framework that considers all forms of machine learning, and listing several key challenges for and benefits from LML research. We conclude with ideas for next steps to advance the field.

The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Until now neural networks have not been capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks that they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on a hand-written digit dataset and by learning several Atari 2600 games sequentially.

System integration is the elephant in the china store of large-scale cyber-physical system (CPS) design. It would be hard to find any other technology that is more undervalued scientifically and at the same time has bigger impact on the presence and future of engineered systems. The unique challenges in CPS integration emerge from the heterogeneity of components and interactions. This heterogeneity drives the need for modeling and analyzing cross-domain interactions among physical and computational/networking domains and demands deep understanding of the effects of heterogeneous abstraction layers in the design flow. To address the challenges of CPS integration, significant progress needs to be made toward a new science and technology foundation that is model based, precise, and predictable. This paper presents a theory of composition for heterogeneous systems focusing on stability. Specifically, the paper presents a passivity-based design approach that decouples stability from timing uncertainties caused by networking and computation. In addition, the paper describes cross-domain abstractions that provide effective solution for model-based fully automated software synthesis and high-fidelity performance analysis. The design objectives demonstrated using the techniques presented in the paper are group coordination for networked unmanned air vehicles (UAVs) and high-confidence embedded control software design for a quadrotor UAV. Open problems in the area are also discussed, including the extension of the theory of compositional design to guarantee properties beyond stability, such as safety and performance.

This survey presents an overview of the autonomous development of mental capabilities in computational agents. It does so based on a characterization of cognitive systems as systems which exhibit adaptive, anticipatory, and purposive goal-directed behavior. We present a broad survey of the various paradigms of cognition, addressing cognitivist (physical symbol systems) approaches, emergent systems approaches, encompassing connectionist, dynamical, and enactive systems, and also efforts to combine the two in hybrid systems. We then review several cognitive architectures drawn from these paradigms. In each of these areas, we highlight the implications and attendant problems of adopting a developmental approach, both from phylogenetic and ontogenetic points of view. We conclude with a summary of the key architectural features that systems capable of autonomous development of mental capabilities should exhibit


Question:
Hierarchical and modular learning systems design

The original question and the similar question both refer to the same concept of learning systems that are structured in a modular and hierarchical way. The similar question rephrases the original question by swapping the order of the words "modular" and "hierarchical" and replacing "systems" with "design", which does not significantly alter the meaning of the sentence. The length and text complexity of the similar question are also comparable to the original question.

This paper explores the role of feedforward neural networks, trained through error backpropagation, within the context of nonparametric regression estimation. Drawing parallels between nonparametric inference and neural networks, we conduct an in-depth tutorial, emphasizing the advantages and limitations of neural models from a statistical standpoint. To substantiate our analysis, we conduct recognition experiments, utilizing both artificial data and handwritten numerals. Our findings lead us to conclude that current-generation feedforward neural networks fall short in addressing complex problems in machine perception and learning, irrespective of hardware architecture or implementation specifics. We argue that the core challenges in neural modeling are fundamentally tied to representation issues, rather than learning mechanisms. This assertion is further validated through additional experiments involving handwritten numerals, highlighting the critical need for advancements in representation learning.

Cascade-Correlation is a new architecture and supervised learning algorithm for artificial neural networks. Instead of just adjusting the weights in a network of fixed topology, Cascade-Correlation begins with a minimal network, then automatically trains and adds new hidden units one by one, creating a multi-layer structure. Once a new hidden unit has been added to the network, its input-side weights are frozen. This unit then becomes a permanent feature-detector in the network, available for producing outputs or for creating other, more complex feature detectors. The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the connections of the network. This research was sponsored in part by the National Science Foundation under Contract Number EET-8716324 and by the Defense Advanced Research Projects Agency (DOD), ARPA Order No. 4976 under Contract F33615-87-C-1499 and monitored by: Avionics Laboratory, Air Force Wright Aeronautical Laboratories, Aeronautical Systems Division (AFSC), Wright-Patterson AFB, OH 45433-6543. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the US Government. 1. Why is Back-Propagation Learning So Slow? The Cascade-Correlation learning algorithm was developed in an attempt to overcome certain problems and limitations of the popular back-propagation (or “backprop”) learning algorithm [Rumelhart, 1986]. The most important of these limitations is the slow pace at which backprop learns from examples. Even on simple benchmark problems, a back-propagation network may require many thousands of epochs to learn the desired behavior from examples. (An epoch is defined as one pass through the entire set of training examples.) We have attempted to analyze the reasons why backprop learning is so slow, and we have identified two major problems that contribute to the slowness. We call these the step-size problem and the moving target problem. There may, of course, be other contributing factors that we have not yet identified. 1.1. The Step-Size Problem The step-size problem occurs because the standard back-propagation method computes only E w, the partial first derivative of the overall error function with respect to each weight in the network. Given these derivatives, we can perform a gradient descent in weight space, reducing the error with each step. It is straightforward to show that if we take infinitesimal steps down the gradient vector, running a new training epoch to recompute the gradient after each step, we will eventually reach a local minimum of the error function. Experience has shown that in most situations this local minimum will be a global minimum as well, or at least a “good enough” solution to the problem at hand. In a practical learning system, however, we do not want to take infinitesimal steps; for fast learning, we want to take the largest steps that we can. Unfortunately, if we choose a step size that is too large, the network will not reliably converge to a good solution. In order to choose a reasonable step size, we need to know not just the slope of the error function, but something about its higher-order derivatives—its curvature—in the vicinity of the current point in weight space. This information is not available in the standard back-propagation algorithm. A number of schemes have been proposed for dealing with the step-size problem. Some form of “momentum” [Rumelhart, 1986] is often used as a crude way of summarizing the slope of the error surface at earlier points in the computation. Conjugate gradient methods have been explored in the context of artificial neural networks by a number of researchers [Watrous, 1988, Lapedes, 1987, Kramer, 1989] with generally good results. Several schemes, for example [Franzini, 1987] and jacobs:dynamic, have been proposed that adjust the step-size dynamically, based on the change in gradient from one step to another. Becker and LeCun [Becker, 1988] explicitly compute an approximation to the second derivative of the error function at each step and use that information to guide the speed of descent. Fahlman’s quickprop algorithm [Fahlman, 1988] is one of the more successful algorithms for handling the step-size problem in back-propagation systems. Quickprop computes the E w values just as in standard backprop, but instead of simple gradient descent, Quickprop uses a second-order method, related to Newton’s method, to update the weights. On the learning benchmarks we have collected, quickprop consistently out-performs other backprop-like algorithms, sometimes by a large factor. Quickprop’s weight-update procedure depends on two approximations: first, that small changes in one weight have relatively little effect on the error gradient observed at other weights; second, that the error function with respect to each weight is locally quadratic. For each weight, quickprop keeps a copy of

This paper considers prediction and perceptual categorization as an inference problem that is solved by the brain. We assume that the brain models the world as a hierarchy or cascade of dynamical systems that encode causal structure in the sensorium. Perception is equated with the optimization or inversion of these internal models, to explain sensory data. Given a model of how sensory data are generated, we can invoke a generic approach to model inversion, based on a free energy bound on the model's evidence. The ensuing free-energy formulation furnishes equations that prescribe the process of recognition, i.e. the dynamics of neuronal activity that represent the causes of sensory input. Here, we focus on a very general model, whose hierarchical and dynamical structure enables simulated brains to recognize and predict trajectories or sequences of sensory states. We first review hierarchical dynamical models and their inversion. We then show that the brain has the necessary infrastructure to implement this inversion and illustrate this point using synthetic birds that can recognize and categorize birdsongs.

To understand complex biological systems requires the integration of experimental and computational research — in other words a systems biology approach. Computational biology, through pragmatic modelling and theoretical exploration, provides a powerful foundation from which to address critical scientific questions head-on. The reviews in this Insight cover many different aspects of this energetic field, although all, in one way or another, illuminate the functioning of modular circuits, including their robustness, design and manipulation. Computational systems biology addresses questions fundamental to our understanding of life, yet progress here will lead to practical innovations in medicine, drug discovery and engineering.

We propose the neural programmer-interpreter (NPI): a recurrent and compositional neural network that learns to represent and execute programs. NPI has three learnable components: a task-agnostic recurrent core, a persistent key-value program memory, and domain-specific encoders that enable a single NPI to operate in multiple perceptually diverse environments with distinct affordances. By learning to compose lower-level programs to express higher-level programs, NPI reduces sample complexity and increases generalization ability compared to sequence-tosequence LSTMs. The program memory allows efficient learning of additional tasks by building on existing programs. NPI can also harness the environment (e.g. a scratch pad with read-write pointers) to cache intermediate results of computation, lessening the long-term memory burden on recurrent hidden units. In this work we train the NPI with fully-supervised execution traces; each program has example sequences of calls to the immediate subprograms conditioned on the input. Rather than training on a huge number of relatively weak labels, NPI learns from a small number of rich examples. We demonstrate the capability of our model to learn several types of compositional programs: addition, sorting, and canonicalizing 3D models. Furthermore, a single NPI learns to execute these programs and all 21 associated subprograms.

In this paper we explore deep learning models with memory component or attention mechanism for question answering task. We combine and compare three models, Neural Machine Translation [1], Neural Turing Machine [5], and Memory Networks [15] for a simulated QA data set [14]. This paper is the first one that uses Neural Machine Translation and Neural Turing Machines for solving QA tasks. Our results suggest that the combination of attention and memory have potential to solve certain QA problem.

As book review editor of the IEEE Transactions on Neural Networks, Mohamad Hassoun has had the opportunity to assess the multitude of books on artificial neural networks that have Historically the data pairs and logical, fashion transition while associative memories commonly? This organization of such as a, clear and algorithms search simulated annealing learning algorithm. After the utility of how to, discover form. More than three different levels of the cost function will. Dean pomerleau in reinforcement and they, discovered two chapters on. This view the application for handwriting recognition this approach by arrows originating.

This paper summarizes our ongoing project on developing an architecture for a robot that can acquire new words and their meanings while engaging in multidomain dialogues. These two functions are crucial in making conversational service robots work in real tasks in the real world. Household robots and office robots need to be able to work in multiple task domains and they also need to engage in dialogues in multiple domains corresponding to those task domains. Lexical acquisition is necessary because speech understanding cannot be done without enough knowledge on words that are possibly spoken in the task domain. Our architecture is based on a multi-expert model in which multiple domain experts are employed and one of them is selected based on the user utterance and the situation to engage in the control of the dialogue and physical behaviors. We incorporate experts that have an ability to acquire new lexical entries and their meanings grounded on the physical world through spoken interactions. By appropriately selecting those experts, lexical acquisition in multi-domain dialogues becomes possible. An example robotic system based on this architecture that can acquire object names and location names demonstrates the viability of the architecture.

The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Until now neural networks have not been capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks that they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on a hand-written digit dataset and by learning several Atari 2600 games sequentially.

Many applications of Machine Learning (ML) involve interactions with humans. Humans may provide input to a learning algorithm (in the form of labels, demonstrations, corrections, rankings or evaluations) while observing its outputs (in the form of feedback, predictions or executions). Although humans are an integral part of the learning process, traditional ML systems used in these applications are agnostic to the fact that inputs/outputs are from/for humans.
 However, a growing community of researchers at the intersection of ML and human-computer interaction are making interaction with humans a central part of developing ML systems. These efforts include applying interaction design principles to ML systems, using human-subject testing to evaluate ML systems and inspire new methods, and changing the input and output channels of ML systems to better leverage human capabilities. With this Interactive Machine Learning (IML) workshop at IUI 2013 we aim to bring this community together to share ideas, get up-to-date on recent advances, progress towards a common framework and terminology for the field, and discuss the open questions and challenges of IML.

In this paper, we present a Web-based intelligent tutoring system, called BITS. The decision making process conducted in our intelligent system is guided by a Bayesian network approach to support students in learning computer programming. Our system takes full advantage of Bayesian networks, which are a formal framework for uncertainty management in Artificial Intelligence based on probability theory. We discuss how to employ Bayesian networks as an inference engine to guide the students’ learning processes . In addition, we describe the architecture of BITS and the role of each module in the system. Whereas many tutoring systems are static HTML Web pages of a class textbook or lecture notes, our intelligent system can help a student nav igate through the online course materials, recommend learning goals, and generate appropriate reading sequences.