Question: RRT* with bidirectional guidance for rapid optimal path planning in cluttered settings.

This paper recounts the success story of Stanley, the autonomous robot that triumphed in the DARPA Grand Challenge, a significant milestone in the evolution of robotic navigation and artificial intelligence. Led by Sebastian Thrun and a team of dedicated researchers and engineers from Stanford University, Stanford Racing Team, Volkswagen Group, and other contributing organizations, Stanley showcased cutting-edge capabilities in autonomous driving. The robot was equipped with a suite of sensors and algorithms that enabled it to navigate complex, cluttered environments, demonstrating remarkable agility and decision-making skills. Stanley's victory was not merely a technical achievement but also a testament to the collaborative spirit and innovative thinking of its creators. The team's multidisciplinary approach, combining expertise in robotics, computer science, engineering, and artificial intelligence, was pivotal in overcoming the challenges posed by the DARPA Grand Challenge. Stanley's success has since inspired further advancements in autonomous vehicle technology and has paved the way for the development of self-driving cars and robots capable of operating in diverse and challenging terrains.

This work deals with navigation of an omni-directional mobile robot with active caster wheels. Initially, the posture of the omni-directional mobile robot is calculated by using the odometry information. Next, the position accuracy of the mobile robot is measured through comparison of the odometry information and the external sensor measurement. Finally, for successful navigation of the mobile robot, a motion planning algorithm that employs kinematic redundancy resolution method is proposed. Through experiments for multiple obstacles and multiple moving obstacles, the feasibility of the proposed navigation algorithm was verified.

We propose a new approach to the problem of searching a space of policies for a Markov decision process (MDP) or a partially observable Markov decision process (POMDP), given a model. Our approach is based on the following observation: Any (PO)MDP can be transformed into an “equivalent” POMDP in which all state transitions (given the current state and action) are deterministic. This reduces the general problem of policy search to one in which we need only consider POMDPs with deterministic transitions. We give a natural way of estimating the value of all policies in these transformed POMDPs. Policy search is then simply performed by searching for a policy with high estimated value. We also establish conditions under which our value estimates will be good, recovering theoretical results similar to those of Kearns, Mansour and Ng [7], but with “sample complexity” bounds that have only a polynomial rather than exponential dependence on the horizon time. Our method applies to arbitrary POMDPs, including ones with infinite state and action spaces. We also present empirical results for our approach on a small discrete problem, and on a complex continuous state/continuous action problem involving learning to ride a bicycle.

Evolution strategies (ES) are a family of black-box optimization algorithms able to train deep neural networks roughly as well as Q-learning and policy gradient methods on challenging deep reinforcement learning (RL) problems, but are much faster (e.g. hours vs. days) because they parallelize better. However, many RL problems require directed exploration because they have reward functions that are sparse or deceptive (i.e. contain local optima), and it is unknown how to encourage such exploration with ES. Here we show that algorithms that have been invented to promote directed exploration in small-scale evolved neural networks via populations of exploring agents, specifically novelty search (NS) and quality diversity (QD) algorithms, can be hybridized with ES to improve its performance on sparse or deceptive deep RL tasks, while retaining scalability. Our experiments confirm that the resultant new algorithms, NS-ES and two QD algorithms, NSR-ES and NSRA-ES, avoid local optima encountered by ES to achieve higher performance on Atari and simulated robots learning to walk around a deceptive trap. This paper thus introduces a family of fast, scalable algorithms for reinforcement learning that are capable of directed exploration. It also adds this new family of exploration algorithms to the RL toolbox and raises the interesting possibility that analogous algorithms with multiple simultaneous paths of exploration might also combine well with existing RL algorithms outside ES.

Mobile robots are increasingly populating our human environments. To interact with humans in a socially compliant way, these robots need to understand and comply with mutually accepted rules. In this paper, we present a novel approach to model the cooperative navigation behavior of humans. We model their behavior in terms of a mixture distribution that captures both the discrete navigation decisions, such as going left or going right, as well as the natural variance of human trajectories. Our approach learns the model parameters of this distribution that match, in expectation, the observed behavior in terms of user-defined features. To compute the feature expectations over the resulting high-dimensional continuous distributions, we use Hamiltonian Markov chain Monte Carlo sampling. Furthermore, we rely on a Voronoi graph of the environment to efficiently explore the space of trajectories from the robot’s current position to its target position. Using the proposed model, our method is able to imitate the behavior of pedestrians or, alternatively, to replicate a specific behavior that was taught by tele-operation in the target environment of the robot. We implemented our approach on a real mobile robot and demonstrate that it is able to successfully navigate in an office environment in the presence of humans. An extensive set of experiments suggests that our technique outperforms state-of-the-art methods to model the behavior of pedestrians, which makes it also applicable to fields such as behavioral science or computer graphics.

Multiagent path finding is the problem of finding paths for a set of agents—one for each agent—in a graph that the agents navigate simultaneously. Agents must navigate from their individual start to goal vertices without any collision. We argue that the prevalent treatment of path conflicts in the literature is incomplete for applications, such as computer games and crowd simulations, and extend the definition of path conflicts to accommodate cases where agents persist in their intermediate locations and even after reaching their goals. We show that an existing algorithm, conflict-based search (CBS), can be extended to handle these cases while preserving its optimality. Experiments show that our variant of CBS generates fewer nodes and runs faster than a competing algorithm.

Many relevant tasks require an agent to reach a certain state, or to manipulate objects into a desired configuration. For example, we might want a robot to align and assemble a gear onto an axle or insert and turn a key in a lock. These tasks present considerable difficulties for reinforcement learning approaches, since the natural reward function for such goal-oriented tasks is sparse and prohibitive amounts of exploration are required to reach the goal and receive a learning signal. Past approaches tackle these problems by manually designing a task-specific reward shaping function to help guide the learning. Instead, we propose a method to learn these tasks without requiring any prior task knowledge other than obtaining a single state in which the task is achieved. The robot is trained in “reverse”, gradually learning to reach the goal from a set of starting positions increasingly far from the goal. Our method automatically generates a curriculum of starting positions that adapts to the agent’s performance, leading to efficient training on such tasks. We demonstrate our approach on difficult simulated fine-grained manipulation problems, not solvable by state-of-the-art reinforcement learning methods.

In recent years, deep generative models have been shown to ‘imagine’ convincing high-dimensional observations such as images, audio, and even video, learning directly from raw data. In this work, we ask how to imagine goal-directed visual plans – a plausible sequence of observations that transition a dynamical system from its current configuration to a desired goal state, which can later be used as a reference trajectory for control. We focus on systems with high-dimensional observations, such as images, and propose an approach that naturally combines representation learning and planning. Our framework learns a generative model of sequential observations, where the generative process is induced by a transition in a low-dimensional planning model, and an additional noise. By maximizing the mutual information between the generated observations and the transition in the planning model, we obtain a low-dimensional representation that best explains the causal nature of the data. We structure the planning model to be compatible with efficient planning algorithms, and we propose several such models based on either discrete or continuous states. Finally, to generate a visual plan, we project the current and goal observations onto their respective states in the planning model, plan a trajectory, and then use the generative model to transform the trajectory to a sequence of observations. We demonstrate our method on imagining plausible visual plans of rope manipulation3.

Learning, planning, and representing knowledge at multiple levels of temporal abstraction are key, longstanding challenges for AI. In this paper we consider how these challenges can be addressed within the mathematical framework of reinforcement learning and Markov decision processes (MDPs). We extend the usual notion of action in this framework to include options—closed-loop policies for taking action over a period of time. Examples of options include picking up an object, going to lunch, and traveling to a distant city, as well as primitive actions such as muscle twitches and joint torques. Overall, we show that options enable temporally abstract knowledge and action to be included in the reinforcement learning framework in a natural and general way. In particular, we show that options may be used interchangeably with primitive actions in planning methods such as dynamic programming and in learning methods such as Q-learning. Formally, a set of options defined over an MDP constitutes a semi-Markov decision process (SMDP), and the theory of SMDPs provides the foundation for the theory of options. However, the most interesting issues concern the interplay between the underlying MDP and the SMDP and are thus beyond SMDP theory. We present results for three such cases: (1) we show that the results of planning with options can be used during execution to interrupt options and thereby perform even better than planned, (2) we introduce new intra-option methods that are able to learn about an option from fragments of its execution, and (3) we propose a notion of subgoal that can be used to improve the options themselves. All of these results have precursors in the existing literature; the contribution of this paper is to establish them in a simpler and more general setting with fewer changes to the existing reinforcement learning framework. In particular, we show that these results can be obtained without committing to (or ruling out) any particular approach to state abstraction, hierarchy, function approximation, or the macroutility problem. ! 1999 Published by Elsevier Science B.V. All rights reserved. ∗ Corresponding author. 0004-3702/99/$ – see front matter ! 1999 Published by Elsevier Science B.V. All rights reserved. PII: S0004-3702(99)00052 -1 182 R.S. Sutton et al. / Artificial Intelligence 112 (1999) 181–211

Exploration remains a significant challenge to reinforcement learning methods, especially in environments where reward signals are sparse. Recent methods of learning from demonstrations have shown to be promising in overcoming exploration difficulties but typically require considerable highquality demonstrations that are difficult to collect. We propose to effectively leverage available demonstrations to guide exploration through enforcing occupancy measure matching between the learned policy and current demonstrations, and develop a novel Policy Optimization from Demonstration (POfD) method. We show that POfD induces implicit dynamic reward shaping and brings provable benefits for policy improvement. Furthermore, it can be combined with policy gradient methods to produce state-of-the-art results, as demonstrated experimentally on a range of popular benchmark sparse-reward tasks, even when the demonstrations are few and imperfect.

With the goal to generate more scalable algorithms with high er efficiency and fewer open parameters, reinforcement learning (RL) has recently moved towar ds combining classical techniques from optimal control and dynamic programming with modern learni ng techniques from statistical estimation theory. In this vein, this paper suggests to use the fr amework of stochastic optimal control with path integrals to derive a novel approach to RL with para meterized policies. While solidly grounded in value function estimation and optimal control b ased on the stochastic Hamilton-JacobiBellman (HJB) equations, policy improvements can be transf ormed into an approximation problem of a path integral which has no open algorithmic parameters o ther than the exploration noise. The resulting algorithm can be conceived of as model-based, sem i-model-based, or even model free, depending on how the learning problem is structured. The upd ate equations have no danger of numerical instabilities as neither matrix inversions nor g radient learning rates are required. Our new algorithm demonstrates interesting similarities with previous RL research in the framework of probability matching and provides intuition why the slig htly heuristically motivated probability matching approach can actually perform well. Empirical eva luations demonstrate significant performance improvements over gradient-based policy learnin g a d scalability to high-dimensional control problems. Finally, a learning experiment on a simul ated 12 degree-of-freedom robot dog illustrates the functionality of our algorithm in a complex robot learning scenario. We believe that Policy Improvement withPath Integrals ( PI2) offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL based o n trajectory roll-outs.


Question: RRT* with bidirectional guidance for rapid optimal path planning in cluttered settings.

This paper presents an innovative application of probabilistic roadmaps for efficient path planning in high-dimensional configuration spaces, particularly relevant for real-time robot motion planning. Inspired by rasterizing computer graphics hardware, the proposed method significantly enhances planning efficiency. Drawing from prior work in retraction methods for disc motion planning and local approaches for manipulators with a high number of degrees of freedom, the authors integrate these concepts into a robust path planning algorithm suitable for complex airbus panel programming and motion planning scenarios. The algorithm's performance is further optimized through the inclusion of an iterative collision checker, which ensures the planner's effectiveness without compromising speed. A critical research direction is to adapt the method for dynamic environments, addressing questions such as updating roadmaps for incremental changes in the workspace and adjusting learning and query phases for moving obstacles along known trajectories. The roadmap is envisioned to operate in the configuration-time space of the robot, necessitating a directed graph structure to account for temporal progression and constraints on velocity and acceleration. This approach is scalable, having been successfully applied to a wide range of holonomic robots, including planar and spatial articulated robots with various joint types and kinematic chains. Additionally, a variant of the method has proven effective for nonholonomic car-like robots. Comparative studies highlight the method's superiority, particularly in terms of consistency and efficiency, when solving complex problems that challenge other existing path planning techniques. For instance, in the case of planar articulated robots with multiple degrees of freedom, the customized implementation outperforms the Randomized Path Planner (RPP) by offering more reliable and faster solutions to difficult problems.

When designing state-of-the-art, domain-independent planning systems, many decisions have to be made with respect to the domain analysis or compilation performed during preprocessing, the heuristic functions used during search, and other features of the search algorithm. These design decisions can have a large impact on the performance of the resulting planner. By providing many alternatives for these choices and exposing them as parameters, planning systems can in principle be configured to work well on different domains. However, usually planners are used in default configurations that have been chosen because of their good average performance over a set of benchmark domains, with limited experimentation of the potentially huge range of possible configurations. In this work, we propose a general framework for automatically configuring a parameterized planner, showing that substantial performance gains can be achieved. We apply the framework to the well-known LPG planner, which has 62 parameters and over 6.5 × 10 possible configurations. We demonstrate that by using this highly parameterized planning system in combination with the off-the-shelf, state-of-the-art automatic algorithm configuration procedure ParamILS, the planner can be specialized obtaining significantly improved performance.

We introduce a new image-based visual navigation algorithm that allows the Cartesian velocity of a robot to be defined with respect to a set of visually observed features corresponding to previously unseen and unmapped world points. The technique is well suited to mobile robot tasks such as moving along a road or flying over the ground. We describe the algorithm in general form and present detailed simulation results for an aerial robot scenario using a spherical camera and a wide angle perspective camera, and present experimental results for a mobile ground robot.

This paper shows how the recent breakthroughs in reinforcement learning (RL) that have enabled robots to learn to play arcade video games, walk, or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear partial differential equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. Cyberphysical systems (e.g., hydraulic channels, transportation systems, the energy grid, and electromagnetic systems) are commonly modeled by PDEs, which historically have been a reliable way to enable engineering applications in these domains. However, it is known that the control of these PDE models is notoriously difficult. We show how neural network-based RL enables the control of discretized PDEs whose parameters are unknown, random, and time-varying. We introduce an algorithm of mutual weight regularization (MWR), which alleviates the curse of dimensionality of multi-agent control schemes by sharing experience between agents while giving each agent the opportunity to specialize its action policy so as to tailor it to the local parameters of the part of the system it is located in. A discretized PDE, such as the scalar Lighthill–Whitham–Richards PDE can indeed be considered as a macroscopic freeway traffic simulator and which presents the most salient challenges for learning to control large cyberphysical system with multiple agents. We consider two different discretization procedures and show the opportunities offered by applying deep reinforcement for continuous control on both. Using a neural RL PDE controller on a traffic flow simulation based on a Godunov discretization of the San Francisco Bay Bridge, we are able to achieve precise adaptive metering without model calibration thereby beating the state of the art in traffic metering. Furthermore, with the more accurate BeATS simulator, we manage to achieve a control performance on par with ALINEA, a state-of-the-art parametric control scheme, and show how using MWR improves the learning procedure.

This paper addresses the real-time control of autonomous vehicles under a minimum traveling time objective. Control inputs for the vehicle are computed from a nonlinear model predictive control (MPC) scheme. The time-optimal objective is reformulated such that it can be tackled by existing efficient algorithms for real-time nonlinear MPC that build on the generalized Gauss-Newton method. We numerically validate our approach in simulations and present a real-world hardware setup of miniature race cars that is used for an experimental comparison of different approaches.

We demonstrate the fully autonomous collaboration of an aerial and a ground robot in a mock-up disaster scenario. Within this collaboration, we make use of the individual capabilities and strengths of both robots. The aerial robot first maps an area of interest, then it computes the fastest mission for the ground robot to reach a spotted victim and deliver a first-aid kit. Such a mission includes driving and removing obstacles in the way while being constantly monitored and commanded by the aerial robot. Our mission-planning algorithm distinguishes between movable and fixed obstacles and considers both the time for driving and removing obstacles. The entire mission is executed without any human interaction once the aerial robot is launched and requires a minimal amount of communication between the robots. We describe both the hardware and software of our system and detail our mission-planning algorithm. We present exhaustive results of both simulation and real experiments. Our system was successfully demonstrated more than 20 times at a trade fair.

There are many situations for which a feasible nonholonomic motion plan must be generated immediately based on real-time perceptual information. Parametric trajectory representations limit computation because they reduce the search space for solutions (at the cost of potentially introducing suboptimality). The use of any parametric trajectory model converts the optimal control formulation into an equivalent nonlinear programming problem. In this paper, curvature polynomials of arbitrary order are used as the assumed form of solution. Polynomials sacrifice little in terms of spanning the set of feasible controls while permitting an expression of the general solution to the system dynamics in terms of decoupled quadratures. These quadratures are then readily linearized to express the necessary conditions for optimality. Resulting trajectories are convenient to manipulate and execute in vehicle controllers and they can be computed with a straightforward numerical procedure in real time. KEY WORDS—mobile robots, car-like robots, trajectory generation, curve generation, nonholonomic, clothoid, cornu spiral, optimal control

Recent developments have clari ed the process of generating partially ordered, partially speci ed sequences of actions whose execution will achive an agent's goal. This paper summarizes a progression of least commitment planners, starting with one that handles the simple strips representation, and ending with one that manages actions with disjunctive precondition, conditional e ects and universal quanti cation over dynamic universes. Along the way we explain how Chapman's formulation of the Modal Truth Criterion is misleading and why his NP-completeness result for reasoning about plans with conditional e ects does not apply to our planner. 1 I thank Franz Amador, Tony Barrett, Darren Cronquist, Denise Draper, Ernie Davis, Oren Etzioni, Nort Fowler, Rao Kambhampati, Craig Knoblock, Nick Kushmerick, Neal Lesh, Karen Lochbaum, Drew McDermott, Ramesh Patil, Kari Pulli, Ying Sun, Austin Tate and Mike Williamson for helpful comments, but retain sole responsibility for errors. This research was funded in part by O ce of Naval Research Grant 90-J-1904 and by National Science Foundation Grant IRI-8957302

Exploration remains a significant challenge to reinforcement learning methods, especially in environments where reward signals are sparse. Recent methods of learning from demonstrations have shown to be promising in overcoming exploration difficulties but typically require considerable highquality demonstrations that are difficult to collect. We propose to effectively leverage available demonstrations to guide exploration through enforcing occupancy measure matching between the learned policy and current demonstrations, and develop a novel Policy Optimization from Demonstration (POfD) method. We show that POfD induces implicit dynamic reward shaping and brings provable benefits for policy improvement. Furthermore, it can be combined with policy gradient methods to produce state-of-the-art results, as demonstrated experimentally on a range of popular benchmark sparse-reward tasks, even when the demonstrations are few and imperfect.

We propose to take a novel approach to robot system design where each building block of a larger system is represented as a differentiable program, i.e. a deep neural network. This representation allows for integrating algorithmic planning and deep learning in a principled manner, and thus combine the benefits of model-free and model-based methods. We apply the proposed approach to a challenging partially observable robot navigation task. The robot must navigate to a goal in a previously unseen 3-D environment without knowing its initial location, and instead relying on a 2-D floor map and visual observations from an onboard camera. We introduce the Navigation Networks (NavNets) that encode state estimation, planning and acting in a single, end-to-end trainable recurrent neural network. In preliminary simulation experiments we successfully trained navigation networks to solve the challenging partially observable navigation task.

Traditionally routing has focused on finding shortest paths in networks with positive, static edge costs representing the distance between two nodes. Energy-optimal routing for electric vehicles creates novel algorithmic challenges, as simply understanding edge costs as energy values and applying standard algorithms does not work. First, edge costs can be negative due to recuperation, excluding Dijkstra-like algorithms. Second, edge costs may depend on parameters such as vehicle weight only known at query time, ruling out existing preprocessing techniques. Third, considering battery capacity limitations implies that the cost of a path is no longer just the sum of its edge costs. This paper shows how these challenges can be met within the framework of A* search. We show how the specific domain gives rise to a consistent heuristic function yielding an O(n) routing algorithm. Moreover, we show how battery constraints can be treated by dynamically adapting edge costs and hence can be handled in the same way as parameters given at query time, without increasing run-time complexity. Experimental results with real road networks and vehicle data demonstrate the advantages of our solution.


Question: Two distinct populations of dopamine neurons separately transmit positive and negative motivational cues.

This paper outlines a conceptual framework for understanding how mesencephalic dopamine systems might distribute predictive signals related to future expectations to their target areas. Specifically, it explores how cerebral cortex activity can forecast the likelihood of future rewards and how deviations in the activity levels of neurons within the diffuse dopamine systems, both above and below baseline, could signify prediction errors. These errors are then communicated to both cortical and subcortical regions. The paper proposes a model for the construction of these error signals in the brain that aligns with physiological findings for a subgroup of dopaminergic neurons found in the ventral tegmental area and its adjacent regions. Additionally, the framework puts forth testable hypotheses regarding human decision-making behavior in a straightforward choice task. It further elucidates that through a basic modulation of synaptic plasticity, variations in dopamine release can adjust predictions in a manner that is both efficient and adaptive. This comprehensive approach integrates the physiological mechanisms of dopamine systems with the cognitive processes of prediction and decision-making, providing insights into how these systems contribute to learning and behavior.

Brain mechanisms that control human sexual behavior in general, and ejaculation in particular, are poorly understood. We used positron emission tomography to measure increases in regional cerebral blood flow (rCBF) during ejaculation compared with sexual stimulation in heterosexual male volunteers. Manual penile stimulation was performed by the volunteer's female partner. Primary activation was found in the mesodiencephalic transition zone, including the ventral tegmental area, which is involved in a wide variety of rewarding behaviors. Parallels are drawn between ejaculation and heroin rush. Other activated mesodiencephalic structures are the midbrain lateral central tegmental field, zona incerta, subparafascicular nucleus, and the ventroposterior, midline, and intralaminar thalamic nuclei. Increased activation was also present in the lateral putamen and adjoining parts of the claustrum. Neocortical activity was only found in Brodmann areas 7/40, 18, 21, 23, and 47, exclusively on the right side. On the basis of studies in rodents, the medial preoptic area, bed nucleus of the stria terminalis, and amygdala are thought to be involved in ejaculation, but increased rCBF was not found in any of these regions. Conversely, in the amygdala and adjacent entorhinal cortex, a decrease in activation was observed. Remarkably strong rCBF increases were observed in the cerebellum. These findings corroborate the recent notion that the cerebellum plays an important role in emotional processing. The present study for the first time provides insight into which regions in the human brain play a primary role in ejaculation, and the results might have important implications for our understanding of how human ejaculation is brought about, and for our ability to improve sexual function and satisfaction in men.

Mirror neurons in macaque area F5 fire when an animal performs an action, such as a mouth or limb movement, and also when the animal passively observes an identical or similar action performed by another individual. Brain-imaging studies in humans conducted over the last 20 years have repeatedly attempted to reveal analogous brain regions with mirror properties in humans, with broad and often speculative claims about their functional significance across a range of cognitive domains, from language to social cognition. Despite such concerted efforts, the likely neural substrates of these mirror regions have remained controversial, and indeed the very existence of a distinct subcategory of human neurons with mirroring properties has been questioned. Here we used activation likelihood estimation (ALE), to provide a quantitative index of the consistency of patterns of fMRI activity measured in human studies of action observation and action execution. From an initial sample of more than 300 published works, data from 125 papers met our strict inclusion and exclusion criteria. The analysis revealed 14 separate clusters in which activation has been consistently attributed to brain regions with mirror properties, encompassing 9 different Brodmann areas. These clusters were located in areas purported to show mirroring properties in the macaque, such as the inferior parietal lobule, inferior frontal gyrus and the adjacent ventral premotor cortex, but surprisingly also in regions such as the primary visual cortex, cerebellum and parts of the limbic system. Our findings suggest a core network of human brain regions that possess mirror properties associated with action observation and execution, with additional areas recruited during tasks that engage non-motor functions, such as auditory, somatosensory and affective components.

Personality neuroscience involves the use of neuroscience methods to study individual differences in behavior, motivation, emotion, and cognition. Personality psychology has contributed much to identifying the important dimensions of personality, but relatively little to understanding the biological sources of those dimensions. However, the rapidly expanding field of personality neuroscience is increasingly shedding light on this topic. This article provides a survey of progress in the use of neuroscience to study personality traits, organized using a hierarchical model of traits based on the Big Five dimensions: Extraversion, Neuroticism, Agreeableness, Conscientiousness, and Openness ⁄ Intellect. Evidence is reviewed for hypotheses about the biological systems involved in each trait. The mission of personality psychology is ‘‘to provide an integrative framework for understanding the whole person’’ (McAdams & Pals, 2006; p. 204), and many different methods must be brought to bear to accomplish such an ambitious goal. The field of personality neuroscience rests on the premise that the whole person cannot be understood without understanding the brain. In this article, I discuss the role that neuroscience can play in personality research and review the progress of this rapidly expanding field. (For more in depth review of the field and its influential theories see DeYoung & Gray, 2009; and Zuckerman, 2005.) Personality psychology’s attempt to understand the whole person calls for a broad conception of personality itself, such as the one provided by McAdams and Pals (2006, p. 212): Personality is an individual’s unique variation on the general evolutionary design for human nature, expressed as a developing pattern of dispositional traits, characteristic adaptations, and integrative life stories, complexly and differentially situated in culture. This definition describes three levels at which personality can be analyzed: traits, characteristic adaptations, and life stories. Personality neuroscience has focused primarily on traits, which are relatively stable patterns of behavior, motivation, emotion, and cognition (Pytlik Zillig, Hemenover, & Dienstbier, 2002; Wilt & Revelle, 2009) that are not specific to a particular social milieu or culture. This is not to say that traits are evident to the same extent or with identical manifestations in all cultures, but rather that any trait can be observed in a subset of situations in any culture. In contrast to traits, characteristic adaptations and life stories describe the individual’s specific responses to his or her particular life circumstances. Obviously, the latter two levels of analysis are crucial for understanding any individual, but their complexity renders them less amenable to study by neuroscience, and this article focuses on the biology of traits. Traits can be considered probabilistic descriptions of the frequency and intensity with which individuals exhibit various behavioral, motivational, emotional, and cognitive states Social and Personality Psychology Compass 4/12 (2010): 1165–1180, 10.1111/j.1751-9004.2010.00327.x a 2010 The Author Social and Personality Psychology Compass a 2010 Blackwell Publishing Ltd (Fleeson, 2001; Fleeson & Gallagher, 2009). Individuals who are high in some trait will experience the states associated with that trait more often and more intensely than individuals low in that trait. For example, someone high in Extraversion will be talkative, outgoing, and excited more often than someone low in Extraversion, but even the person low in Extraversion may experience those states occasionally. The aim of personality neuroscience is to understand both the biological systems that are responsible for the states associated with traits and the parameters of those systems that cause them to function differently in different individuals. The systems themselves are presumed to be present in every intact human brain – hence McAdams and Pals’ (2006) reference to ‘‘the general evolutionary design for human nature’’ – but their parameters will vary from person to person. (For example, all people have brain systems that respond to rewards, but in different individuals these systems will respond with different degrees of vigor to a particular reward, and the systems’ average level of response may be associated with some personality trait.) When considering the biological sources of personality, one must distinguish between proximal and distal sources. The proximal sources, just described, consist of stable differences in the functioning of the neural systems that produce the states associated with traits. The distal sources are both genetic and environmental, as indicated by the fact that heritability estimates for personality traits are in the range of 40% to 80%, depending on trait and method (Bouchard & McGue, 2003; Riemann, Angleitner, & Strelau, 1997). (The heritability of a trait indicates the amount of its variation in a population due to genetic, rather than environmental, variation.) It is important to remember that when either genes or situations have lasting effects on traits, they must do so by changing the brain; thus, personality differences are ‘biological’ regardless of their heritability, in the sense that they must be proximally generated by the brain no matter whether they originated in genes or environment. Methods in Personality Neuroscience Scientific investigation of the biological basis of personality has been limited, until relatively recently, by a lack of technology to examine brain structure and function in living human beings. Prior to the development of neuroimaging, the sole means for assessing brain activity was the electroencephalogram (EEG), which measures the brain’s electrical activity at the scalp. Today a number of methods are available for personality neuroscience. Five important categories of neuroscientific methods are (1) neuroimaging (e.g., magnetic resonance imaging [MRI] or positron emission tomography [PET]), which allows assessment of brain structure and function with a relatively high spatial resolution; (2) molecular genetics, which allows assessment of variation in specific genes that are expressed in the brain; (3) EEG, which provides the highest temporal resolution of neural activity of any available method; (4) assays of endogenous psychoactive substances or their byproducts (e.g., hormone levels in saliva or neurotransmitter metabolites in cerebrospinal fluid); and (5) psychopharmocological manipulation (e.g., tryptophan depletion or augmentation to alter levels of the neurotransmitter serotonin). Personality neuroscience employs these methods in conjunction with the methods of personality psychology, attempting to link biological variables to traits. Measurement of traits is typically accomplished by questionnaire, through self-report and ⁄ or ratings by peers or other knowledgeable informants. Questionnaires provide a convenient and reliable method for assessing a broad range of stable individual differences, drawing on raters’ experiences over a far greater span of time than is available in the laboratory. However, 1166 Personality Neuroscience a 2010 The Author Social and Personality Psychology Compass 4/12 (2010): 1165–1180, 10.1111/j.1751-9004.2010.00327.x Social and Personality Psychology Compass a 2010 Blackwell Publishing Ltd traits can also be assessed in other ways, such as through laboratory tasks, behavioral observation, or experience sampling, and personality should not be identified exclusively with the variables provided by personality questionnaires. What we want to explain in personality neuroscience is not how people answer questionnaires, but rather why they exhibit stable patterns of behavior, motivation, emotion, and cognition. Most studies discussed in this review begin with some psychological trait and attempt to discover its biological basis. Another valid and useful approach is to begin with stable individual differences in some biological parameter (such as asymmetry in the level of left versus right cortical hemisphere activity, measured at rest by EEG) and then to attempt to discover what psychological trait or traits are associated with that biological trait. An important caveat for anyone exploring the literature on personality neuroscience or contemplating entering the field as an investigator is that much inconsistency exists in the findings to date. In neuroimaging research, inconsistency is probably due in large part to the use very small samples, due to cost. Unfortunately, small samples often lack statistical power to detect true effects (Yarkoni, 2009). Worse still, low power has the unfortunate additional consequence of increasing the proportion of false positives among significant findings (Green et al., 2008). In genetic studies, though larger samples are typically used, power may still be an issue, and inconsistency may arise, because each trait is influenced by many genes, each accounting for only a very small amount of the variance of the trait (Green et al., 2008). These difficulties highlight the importance of testing reasonably focused hypotheses, rather than simply exploring associations with biological variables in the absence of any theory of the causal mechanisms that might underlie a given trait. The Structure of Personality Traits Personality neuroscience can usefully be guided by existing knowledge about the structure of personality – that is, knowledge about how various traits relate to one another and to the major dimensions of personality. In order to produce a coherent overview of the progress of personality neuroscience, one needs to relate findings to a reasonably comprehensive taxonomy of traits, such as that provided by the Five Factor Model or Big Five, which categorizes the majority of traits within five broad domains, typically labeled Extraversion, Neuroticism, Agreeableness, Conscientiousness, and Openness ⁄ Intellect (Digman, 199

A wealth of animal data implicates the amygdala in aspects of emotional processing. In recent years, functional neuroimaging and neuropsychological studies have begun to refine our understanding of the functions of the amygdala in humans. This literature offers insights into the types of stimuli that engage the amygdala and the functional consequences that result from this engagement. Specific conclusions and hypotheses include: (1) the amygdala activates during exposure to aversive stimuli from multiple sensory modalities; (2) the amygdala responds to positively valenced stimuli, but these responses are less consistent than those induced by aversive stimuli; (3) amygdala responses are modulated by the arousal level, hedonic strength or current motivational value of stimuli; (4) amygdala responses are subject to rapid habituation; (5) the temporal characteristics of amygdala responses vary across stimulus categories and subject populations; (6) emotionally valenced stimuli need not reach conscious awareness to engage amygdala processing; (7) conscious hedonic appraisals do not require amygdala activation; (8) activation of the amygdala is associated with modulation of motor readiness, autonomic functions, and cognitive processes including attention and memory; (9) amygdala activations do not conform to traditional models of the lateralization of emotion; and (10) the extent and laterality of amygdala activations are related to factors including psychiatric status, gender and personality. The strengths and weakness of these hypotheses and conclusions are discussed with reference to the animal literature.

In monkeys, the rostral part of ventral premotor cortex (area F5) contains neurons that discharge, both when the monkey grasps or manipulates objects and when it observes the experimenter making similar actions. These neurons (mirror neurons) appear to represent a system that matches observed events to similar, internally generated actions, and in this way forms a link between the observer and the actor. Transcranial magnetic stimulation and positron emission tomography (PET) experiments suggest that a mirror system for gesture recognition also exists in humans and includes Broca's area. We propose here that such an observation/execution matching system provides a necessary bridge from'doing' to'communicating',as the link between actor and observer becomes a link between the sender and the receiver of each message.

The discipline of affective neuroscience is concerned with the neural bases of emotion and mood. The past 30 years have witnessed an explosion of research in affective neuroscience that has addressed questions such as: which brain systems underlie emotions? How do differences in these systems relate to differences in the emotional experience of individuals? Do different regions underlie different emotions, or are all emotions a function of the same basic brain circuitry? How does emotion processing in the brain relate to bodily changes associated with emotion? And, how does emotion processing in the brain interact with cognition, motor behaviour, language and motivation?

Mammals respond to reduced food availability by becoming more wakeful and active, yet the central pathways regulating arousal and instinctual motor programs (such as food seeking) according to homeostatic need are not well understood. We demonstrate that hypothalamic orexin neurons monitor indicators of energy balance and mediate adaptive augmentation of arousal in response to fasting. Activity of isolated orexin neurons is inhibited by glucose and leptin and stimulated by ghrelin. Orexin expression of normal and ob/ob mice correlates negatively with changes in blood glucose, leptin, and food intake. Transgenic mice, in which orexin neurons are ablated, fail to respond to fasting with increased wakefulness and activity. These findings indicate that orexin neurons provide a crucial link between energy balance and arousal.

The tragic health effects of nicotine addiction highlight the importance of investigating the cellular mechanisms of this complex behavioral phenomenon. The chain of cause and effect of nicotine addiction starts with the interaction of this tobacco alkaloid with nicotinic acetylcholine receptors (nAChRs). This interaction leads to activation of reward centers in the CNS, including the mesoaccumbens DA system, which ultimately leads to behavioral reinforcement and addiction. Recent findings from a number of laboratories have provided new insights into the biologic processes that contribute to nicotine self-administration. Examination of the nAChR subtypes expressed within the reward centers has identified potential roles for these receptors in normal physiology, as well as the effects of nicotine exposure. The high nicotine sensitivity of some nAChR subtypes leads to rapid activation followed in many cases by rapid desensitization. Assessing the relative importance of these molecular phenomena in the behavioral effects of nicotine presents an exciting challenge for future research efforts.

Metaphors linking spatial location and affect (e.g., feeling up or down) may have subtle, but pervasive, effects on evaluation. In three studies, participants evaluated words presented on a computer. In Study 1, evaluations of positive words were faster when words were in the up rather than the down position, whereas evaluations of negative words were faster when words were in the down rather than the up position. In Study 2, positive evaluations activated higher areas of visual space, whereas negative evaluations activated lower areas of visual space. Study 3 revealed that, although evaluations activate areas of visual space, spatial positions do not activate evaluations. The studies suggest that affect has a surprisingly physical basis.

For more than half a century, emotion researchers have attempted to establish the dimensional space that most economically accounts for similarities and differences in emotional experience. Today, many researchers focus exclusively on two-dimensional models involving valence and arousal. Adopting a theoretically based approach, we show for three languages that four dimensions are needed to satisfactorily represent similarities and differences in the meaning of emotion words. In order of importance, these dimensions are evaluation-pleasantness, potency-control, activation-arousal, and unpredictability. They were identified on the basis of the applicability of 144 features representing the six components of emotions: (a) appraisals of events, (b) psychophysiological changes, (c) motor expressions, (d) action tendencies, (e) subjective experiences, and (f) emotion regulation.


Question: Two distinct populations of dopamine neurons separately transmit positive and negative motivational cues.

Dopamine Neurons: Key Players in Motivational Signaling

Dopamine neurons, primarily located in the midbrain, play a pivotal role in processing reward information and modulating approach behaviors. These neurons exhibit a range of responses to stimuli, reflecting their complex function in the brain's reward system. Typically, dopamine neurons display phasic activations in response to primary rewards, such as food or liquids, as well as to conditioned stimuli that predict such rewards. Interestingly, they also show biphasic responses—activation followed by depression—to stimuli that are novel, salient, or resemble reward-predicting cues. However, aversive stimuli generally elicit fewer phasic activations. This suggests that dopamine neurons selectively tag stimuli with appetitive value, predict rewards, and signal events that require alertness and motivation.

A key feature of dopamine neuron responses is their dependence on event predictability. These neurons are activated by rewards that surpass expectations, remain neutral in response to expected rewards, and show decreased activity when outcomes are less favorable than anticipated. By encoding rewards based on prediction errors, dopamine responses align with the theoretical teaching signals proposed by reinforcement learning models. As learning progresses, dopamine responses shift from primary rewards to reward-predicting stimuli, potentially underpinning the retrograde action of rewards, a crucial aspect of reinforcement learning.

The dopamine signal is not isolated but works in conjunction with other neuronal systems. Neurons in the striatum, frontal cortex, and amygdala process specific reward information without generating a global reward prediction error signal. This collaborative network ensures that specific rewards can selectively reinforce behaviors. Noradrenaline neurons, on the other hand, primarily support attentional mechanisms, while nucleus basalis neurons encode rewards in a more heterogeneous manner. Cerebellar climbing fibers, meanwhile, signal errors in motor performance or aversive event predictions to cerebellar Purkinje cells.

Dopamine-depleting lesions often lead to deficits that are not solely attributable to a compromised reward signal but may also indicate the loss of a general enabling function provided by the tonic levels of extracellular dopamine. Therefore, dopamine systems seem to serve dual roles: the phasic transmission of reward information and the tonic facilitation of postsynaptic neuronal activity.

In summary, dopamine neurons are essential for the encoding and prediction of rewards, facilitating learning and approach behaviors through their dynamic signaling patterns. Their interactions with other neuronal systems underscore the complex and multifaceted nature of reward processing in the brain.

Recent technological developments, such as single unit recordings coupled to optogenetic approaches, have provided unprecedented knowledge about the precise neuronal circuits contributing to the expression and recovery of conditioned fear behavior. These data have provided an understanding of the contributions of distinct brain regions such as the amygdala, prefrontal cortex, hippocampus, and periaqueductal gray matter to the control of conditioned fear behavior. Notably, the precise manipulation and identification of specific cell types by optogenetic techniques have provided novel avenues to establish causal links between changes in neuronal activity that develop in dedicated neuronal structures and the short and long-lasting expression of conditioned fear memories. In this review, we provide an update on the key neuronal circuits and cell types mediating conditioned fear expression and recovery and how these new discoveries might refine therapeutic approaches for psychiatric conditions such as anxiety disorders and posttraumatic stress disorder.

The pallido-recipient thalamus transmits information from the basal ganglia to the cortex and is critical for motor initiation and learning. Thalamic activity is strongly inhibited by pallidal inputs from the basal ganglia, but the role of nonpallidal inputs, such as excitatory inputs from cortex, remains unclear. We simultaneously recorded from presynaptic pallidal axon terminals and postsynaptic thalamocortical neurons in a basal ganglia–recipient thalamic nucleus that is necessary for vocal variability and learning in zebra finches. We found that song-locked rate modulations in the thalamus could not be explained by pallidal inputs alone and persisted following pallidal lesion. Instead, thalamic activity was likely driven by inputs from a motor cortical nucleus that is also necessary for singing. These findings suggest a role for cortical inputs to the pallido-recipient thalamus in driving premotor signals that are important for exploratory behavior and learning.

Distortions in time perception and timed performance are presented by a number of different neurological and psychiatric conditions (e.g. Parkinson's disease, schizophrenia, attention deficit hyperactivity disorder and autism). As a consequence, the primary focus of this review is on factors that define or produce systematic changes in the attention, clock, memory and decision stages of temporal processing as originally defined by Scalar Expectancy Theory. These findings are used to evaluate the Striatal Beat Frequency Theory, which is a neurobiological model of interval timing based upon the coincidence detection of oscillatory processes in corticostriatal circuits that can be mapped onto the stages of information processing proposed by Scalar Timing Theory.

Action, cognition, emotion and perception can be mapped in the brain by using set of techniques. Translating unimodal concepts from one modality to another is an important step towards understanding the neural mechanisms. This paper provides a comprehensive survey of multimodal analysis of brain signals such as fMRI, EEG, MEG, NIRS and motivations, assumptions and pitfalls associated with it. All these non-invasive brain modalities complement and restrain each other and hence improve our understating of functional and neuronal organization. By combining the various modalities together, we can exploit the strengths and flaws of individual brain imaging methods. Integrated anatomical analysis and functional measurements of human brain offer a powerful paradigm for the brain mapping. Here we provide the brief review on non invasive brain modalities, describe the future of co-analysis of these

Picking up an object requires two basic motor operations: reaching and grasping. Neurophysiological studies in monkeys have suggested that the visuomotor transformations necessary for these two operations are carried out by separate parietofrontal circuits and that, for grasping, a key role is played by a specific sector of the ventral premotor cortex: area F5. The aim of the present study was to test the validity of this hypothesis by reversibly inactivating area F5 in monkeys trained to grasp objects of different shape, size and orientation. In separate sessions, the hand field of the primary motor cortex (area F1 or area 4) was also reversibly inactivated. The results showed that after inactivation of area F5 buried in the bank of the arcuate sulcus (the F5 sector where visuomotor neurones responding to object presentation are located), the hand shaping preceding grasping was markedly impaired and the hand posture was not appropriate for the object size and shape. The monkeys were eventually able to grasp the objects, but only after a series of corrections made under tactile control. With small inactivations the deficits concerned the contralesional hand, with larger inactivations the ipsilateral hand as well. In addition, there were signs of peripersonal neglect in the hemispace contralateral to the inactivation site. Following inactivation of area F5 lying on the cortical convexity (the F5 sector where visuomotor neurones responding to action observation, 'mirror neurones', are found) only a motor slowing was observed, the hand shaping being preserved. The inactivation of the hand field of area F1 produced a severe paralysis of contralateral finger movements with hypotonia. The results of this study indicate the crucial role of the ventral premotor cortex in visuomotor transformations for grasping movements. More generally, they provide strong support for the notion that distal and proximal movement organization relies upon distinct cortical circuits. Clinical data on distal movement deficits in humans are re-examined in the light of the present findings.

Metaphors linking spatial location and affect (e.g., feeling up or down) may have subtle, but pervasive, effects on evaluation. In three studies, participants evaluated words presented on a computer. In Study 1, evaluations of positive words were faster when words were in the up rather than the down position, whereas evaluations of negative words were faster when words were in the down rather than the up position. In Study 2, positive evaluations activated higher areas of visual space, whereas negative evaluations activated lower areas of visual space. Study 3 revealed that, although evaluations activate areas of visual space, spatial positions do not activate evaluations. The studies suggest that affect has a surprisingly physical basis.

Adult hippocampal neurogenesis is a unique example of structural plasticity, the functional role of which has been a matter of intense debate. New transgenic models have recently shown that neurogenesis participates in hippocampus-mediated learning. Here, we show that transgenic animals, in which adult hippocampal neurogenesis has been specifically impaired, exhibit a striking increase in anxiety-related behaviors. Our results indicate that neurogenesis plays an important role in the regulation of affective states and could be the target of new treatments for anxiety disorders.

Emotion enhances our ability to form vivid memories of even trivial events. Norepinephrine (NE), a neuromodulator released during emotional arousal, plays a central role in the emotional regulation of memory. However, the underlying molecular mechanism remains elusive. Toward this aim, we have examined the role of NE in contextual memory formation and in the synaptic delivery of GluR1-containing alpha-amino-3-hydroxy-5-methyl-4-isoxazoleproprionic acid (AMPA)-type glutamate receptors during long-term potentiation (LTP), a candidate synaptic mechanism for learning. We found that NE, as well as emotional stress, induces phosphorylation of GluR1 at sites critical for its synaptic delivery. Phosphorylation at these sites is necessary and sufficient to lower the threshold for GluR1 synaptic incorporation during LTP. In behavioral experiments, NE can lower the threshold for memory formation in wild-type mice but not in mice carrying mutations in the GluR1 phosphorylation sites. Our results indicate that NE-driven phosphorylation of GluR1 facilitates the synaptic delivery of GluR1-containing AMPARs, lowering the threshold for LTP, thereby providing a molecular mechanism for how emotion enhances learning and memory.

We investigate neural circuits in the exacting setting that (i) the acquisition of a piece of knowledge can occur from a single interaction, (ii) the result of each such interaction is a rapidly evaluatable subcircuit, (iii) hundreds of thousands of such subcircuits can be acquired in sequence without substantially degrading the earlier ones, and (iv) recall can be in the form of a rapid evaluation of a composition of subcircuits that have been so acquired at arbitrary different earlier times.We develop a complexity theory, in terms of asymptotically matching upper and lower bounds, on the capacity of a neural network for executing, in this setting, the following action, which we call {\it association}: Each action sets up a subcircuit so that the excitation of a chosen set of neurons A will in future cause the excitation of another chosen set B.% As model of computation we consider the neuroidal model, a fully distributed model in which the quantitative resources n, the neuron numbers, d, the number of other neurons each neuron is connected to, and k, the inverse of the maximum synaptic strength, are all accounted for.A succession of experiences, possibly over a lifetime, results in the realization of a complex set of subcircuits. The composability requirement constrains the model to ensure that, for each association as realized by a subcircuit, the excitation in the triggering set of neurons A is quantitatively similar to that in the triggered set B, and also that the unintended excitation in the rest of the system is negligible. These requirements ensure that chains of associations can be triggeredWe first analyze what we call the Basic Mechanism, which uses only direct connections between neurons in the triggering set A and the target set B. We consider random networks of n neurons with expected number d of connections to and from each. We show that in the composable context capacity growth is limited by d^2, a severe limitation if the network is sparse, as it is in cortex. We go on to study the Expansive Mechanism, that additionally uses intermediate relay neurons which have high synaptic weights. For this mechanism we show that the capacity can grow as dn, to within logarithmic factors. From these two results it follows that in the composable regime, for the realistic cortical estimate of d=n^{\frac{1}{2}, superlinear capacity of order n^{\frac{3}{2}} in terms of the neuron numbers can be realized by the Expansive Mechanism, instead of the linear order n to which the Basic Mechanism is limited. More generally, for both mechanisms, we establish matching upper and lower bounds on capacity in terms of the parameters n, d, and the inverse maximum synaptic strength k.The results as stated above assume that in a set of associations, a target B can be triggered by at most one set A. It can be shown that the capacities are similar if the number m of As that can trigger a B is greater than one but small, but become severely constrained if m exceeds a certain threshold.

Psychologists and neuroscientists have had a long-standing interest in the P3, a prominent component of the event-related brain potential. This review aims to integrate knowledge regarding the neural basis of the P3 and to elucidate its functional role in information processing. The authors review evidence suggesting that the P3 reflects phasic activity of the neuromodulatory locus coeruleus-norepinephrine (LC-NE) system. They discuss the P3 literature in the light of empirical findings and a recent theory regarding the information-processing function of the LC-NE phasic response. The theoretical framework emerging from this research synthesis suggests that the P3 reflects the response of the LC-NE system to the outcome of internal decision-making processes and the consequent effects of noradrenergic potentiation of information processing.


Question: Two distinct populations of dopamine neurons separately transmit positive and negative motivational cues.

This study employs functional magnetic resonance imaging (fMRI) to visualize the brain activity of 12 healthy participants during a monetary incentive delay task. The aim was to explore the activation of striatal and mesial forebrain regions, known for their roles in generating autonomic, endocrine, and behavioral responses to incentives. The analysis of time-series data, both at the individual and group levels, demonstrated significant activation of the striatal and mesial forebrain structures, encompassing the insula, caudate, putamen, and mesial prefrontal cortex, during trials involving both rewards and punishments. Interestingly, during punishment trials, the group analysis also revealed activation in the anterior cingulate and thalamus. These findings support previous comparative studies that highlight the involvement of striatal and mesial forebrain circuitry in incentive-driven behaviors. Additionally, this study introduces a novel approach for assessing the functional integrity of these circuits in human subjects, contributing to our understanding of the neural mechanisms underlying reward and punishment processing.

BACKGROUND
Serotonin function has been implicated in both major depressive disorder and neuroticism. In the current investigation, we examined the hypothesis that any change in depression severity is mediated through the reduction of neuroticism, but only for those compounds which target serotonin receptors.

METHODS
Ninety-three outpatients in the midst of a major depressive episode received one of three antidepressant medications, classified into two broad types: selective serotonin reuptake inhibitors (SSRIs) and non-SSRIs (i.e. reversible monoamine oxidase inhibitors [RIMAs] and noradrenergic and dopaminergic reuptake blockers [NDMs]). Patients completed the Hamilton Rating Scale for Depression, Beck Depression Inventory II and Revised NEO Personality Inventory prior to and following approximately 16 weeks of treatment. Structural equation modeling was used to test two models: a mediation model, in which neuroticism change is the mechanism by which SSRIs exert a therapeutic effect upon depressive symptoms, and a complication model, in which neuroticism change is a mere epiphenomenon of depression reduction in response to SSRIs.

RESULTS
The mediation model provided a good fit to the data; the complication model did not. Patients treated with SSRIs demonstrated greater neuroticism change than those treated with non-SSRIs, and greater neuroticism change was associated with greater depressive symptom change. These effects held for both self-reported and clinician-rated depressive symptom severity.

LIMITATIONS
Replication within a randomized control trial with multiple assessment periods is required.

CONCLUSION
Neuroticism mediates changes in depression in response to treatment with SSRIs, such that any treatment effect of SSRIs occurs through neuroticism reduction.

Brain mechanisms that control human sexual behavior in general, and ejaculation in particular, are poorly understood. We used positron emission tomography to measure increases in regional cerebral blood flow (rCBF) during ejaculation compared with sexual stimulation in heterosexual male volunteers. Manual penile stimulation was performed by the volunteer's female partner. Primary activation was found in the mesodiencephalic transition zone, including the ventral tegmental area, which is involved in a wide variety of rewarding behaviors. Parallels are drawn between ejaculation and heroin rush. Other activated mesodiencephalic structures are the midbrain lateral central tegmental field, zona incerta, subparafascicular nucleus, and the ventroposterior, midline, and intralaminar thalamic nuclei. Increased activation was also present in the lateral putamen and adjoining parts of the claustrum. Neocortical activity was only found in Brodmann areas 7/40, 18, 21, 23, and 47, exclusively on the right side. On the basis of studies in rodents, the medial preoptic area, bed nucleus of the stria terminalis, and amygdala are thought to be involved in ejaculation, but increased rCBF was not found in any of these regions. Conversely, in the amygdala and adjacent entorhinal cortex, a decrease in activation was observed. Remarkably strong rCBF increases were observed in the cerebellum. These findings corroborate the recent notion that the cerebellum plays an important role in emotional processing. The present study for the first time provides insight into which regions in the human brain play a primary role in ejaculation, and the results might have important implications for our understanding of how human ejaculation is brought about, and for our ability to improve sexual function and satisfaction in men.

Parenting behavior critically shapes human infants' current and future behavior. The parent-infant relationship provides infants with their first social experiences, forming templates of what they can expect from others and how to best meet others' expectations. In this review, we focus on the neurobiology of parenting behavior, including our own functional magnetic resonance imaging (fMRI) brain imaging experiments of parents. We begin with a discussion of background, perspectives and caveats for considering the neurobiology of parent-infant relationships. Then, we discuss aspects of the psychology of parenting that are significantly motivating some of the more basic neuroscience research. Following that, we discuss some of the neurohormones that are important for the regulation of social bonding, and the dysregulation of parenting with cocaine abuse. Then, we review the brain circuitry underlying parenting, proceeding from relevant rodent and nonhuman primate research to human work. Finally, we focus on a study-by-study review of functional neuroimaging studies in humans. Taken together, this research suggests that networks of highly conserved hypothalamic-midbrain-limbic-paralimbic-cortical circuits act in concert to support aspects of parent response to infants, including the emotion, attention, motivation, empathy, decision-making and other thinking that are required to navigate the complexities of parenting. Specifically, infant stimuli activate basal forebrain regions, which regulate brain circuits that handle specific nurturing and caregiving responses and activate the brain's more general circuitry for handling emotions, motivation, attention, and empathy--all of which are crucial for effective parenting. We argue that an integrated understanding of the brain basis of parenting has profound implications for mental health.

Self-regulation strategies improve self-discipline in adolescents: benefits of mental contrasting and implementation intentions Angela Lee Duckwortha; Heidi Grantb; Benjamin Loewa; Gabriele Oettingenac; Peter M. Gollwitzerad a Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA b Department of Psychology, Lehigh University, Bethlehem, PA, USA c Department of Psychology, University of Hamburg, Hamburg, Germany d Department of Psychology, University of Konstanz, Konstanz, Germany

It has long been assumed that sensory neurons are adapted, through both evolutionary and developmental processes, to the statistical properties of the signals to which they are exposed. Attneave (1954)Barlow (1961) proposed that information theory could provide a link between environmental statistics and neural responses through the concept of coding efficiency. Recent developments in statistical modeling, along with powerful computational tools, have enabled researchers to study more sophisticated statistical models for visual images, to validate these models empirically against large sets of data, and to begin experimentally testing the efficient coding hypothesis for both individual neurons and populations of neurons.

In the present study we provide the first empirical evidence that viscero-sensory feedback from an internal organ is associated with decision-making processes. Participants with accurate vs. poor perception of their heart activity were compared with regard to their performance in the Iowa Gambling Task. During this task, participants have to choose between four card decks. Decks A and B yield high gains and high losses, and if played continuously, result in net loss. In contrast, decks C and D yield small gains and also small losses, but result in net profit if they are selected continuously. Accordingly, participants have to learn to avoid the net loss options in favor of the net gain options. In our study, participants with good cardiac perception chose significantly more of the net gain and fewer of the net loss options. Our findings document the substantial role of visceral feedback in decision-making processes in complex situations.

The basal ganglia have been shown to contribute to habit and stimulus-response (S-R) learning. These forms of learning have the property of slow acquisition and, in humans, can occur without conscious awareness. This paper proposes that one aspect of basal ganglia-based learning is the recoding of cortically derived information within the striatum. Modular corticostriatal projection patterns, demonstrated experimentally, are viewed as producing recoded templates suitable for the gradual selection of new input-output relations in cortico-basal ganglia loops. Recordings from striatal projection neurons and interneurons show that activity patterns in the striatum are modified gradually during the course of S-R learning. It is proposed that this recoding within the striatum can chunk the representations of motor and cognitive action sequences so that they can be implemented as performance units. This scheme generalizes Miller's notion of information chunking to action control. The formation and the efficient implementation of action chunks are viewed as being based on predictive signals. It is suggested that information chunking provides a mechanism for the acquisition and the expression of action repertoires that, without such information compression would be biologically unwieldy or difficult to implement. The learning and memory functions of the basal ganglia are thus seen as core features of the basal ganglia's influence on motor and cognitive pattern generators.

UNLABELLED
Basal forebrain cholinergic neurons are the main source of cortical acetylcholine, and their activation by histamine elicits cortical arousal. TWIK-like acid-sensitive K(+) (TASK) channels modulate neuronal excitability and are expressed on basal forebrain cholinergic neurons, but the role of TASK channels in the histamine-basal forebrain cholinergic arousal circuit is unknown. We first expressed TASK channel subunits and histamine Type 1 receptors in HEK cells. Application of histamine in vitro inhibited the acid-sensitive K(+) current, indicating a functionally coupled signaling mechanism. We then studied the role of TASK channels in modulating electrocortical activity in vivo using freely behaving wild-type (n = 12) and ChAT-Cre:TASK(f/f) mice (n = 12), the latter lacking TASK-1/3 channels on cholinergic neurons. TASK channel deletion on cholinergic neurons significantly altered endogenous electroencephalogram oscillations in multiple frequency bands. We then identified the effect of TASK channel deletion during microperfusion of histamine into the basal forebrain. In non-rapid eye movement sleep, TASK channel deletion on cholinergic neurons significantly attenuated the histamine-induced increase in 30-50 Hz activity, consistent with TASK channels contributing to histamine action on basal forebrain cholinergic neurons. In contrast, during active wakefulness, histamine significantly increased 30-50 Hz activity in ChAT-Cre:TASK(f/f) mice but not wild-type mice, showing that the histamine response depended upon the prevailing cortical arousal state. In summary, we identify TASK channel modulation in response to histamine receptor activation in vitro, as well as a role of TASK channels on cholinergic neurons in modulating endogenous oscillations in the electroencephalogram and the electrocortical response to histamine at the basal forebrain in vivo.

SIGNIFICANCE STATEMENT
Attentive states and cognitive function are associated with the generation of γ EEG activity. Basal forebrain cholinergic neurons are important modulators of cortical arousal and γ activity, and in this study we investigated the mechanism by which these neurons are activated by the wake-active neurotransmitter histamine. We found that histamine inhibited a class of K(+) leak channels called TASK channels and that deletion of TASK channels selectively on cholinergic neurons modulated baseline EEG activity as well as histamine-induced changes in γ activity. By identifying a discrete brain circuit where TASK channels can influence γ activity, these results represent new knowledge that enhances our understanding of how subcortical arousal systems may contribute to the generation of attentive states.

Multiple factors simultaneously affect the spiking activity of individual neurons. Determining the effects and relative importance of these factors is a challenging problem in neurophysiology. We propose a statistical framework based on the point process likelihood function to relate a neuron's spiking probability to three typical covariates: the neuron's own spiking history, concurrent ensemble activity, and extrinsic covariates such as stimuli or behavior. The framework uses parametric models of the conditional intensity function to define a neuron's spiking probability in terms of the covariates. The discrete time likelihood function for point processes is used to carry out model fitting and model analysis. We show that, by modeling the logarithm of the conditional intensity function as a linear combination of functions of the covariates, the discrete time point process likelihood function is readily analyzed in the generalized linear model (GLM) framework. We illustrate our approach for both GLM and non-GLM likelihood functions using simulated data and multivariate single-unit activity data simultaneously recorded from the motor cortex of a monkey performing a visuomotor pursuit-tracking task. The point process framework provides a flexible, computationally efficient approach for maximum likelihood estimation, goodness-of-fit assessment, residual analysis, model selection, and neural decoding. The framework thus allows for the formulation and analysis of point process models of neural spiking activity that readily capture the simultaneous effects of multiple covariates and enables the assessment of their relative importance.

The human brain is adept at anticipating upcoming events, but in a rapidly changing world, it is essential to detect and encode events that violate these expectancies. Unexpected events are more likely to be remembered than predictable events, but the underlying neural mechanisms for these effects remain unclear. We report intracranial EEG recordings from the hippocampus of epilepsy patients, and from the nucleus accumbens of depression patients. We found that unexpected stimuli enhance an early (187 ms) and a late (482 ms) hippocampal potential, and that the late potential is associated with successful memory encoding for these stimuli. Recordings from the nucleus accumbens revealed a late potential (peak at 475 ms), which increases in magnitude during unexpected items, but no subsequent memory effect and no early component. These results are consistent with the hypothesis that activity in a loop involving the hippocampus and the nucleus accumbens promotes encoding of unexpected events.


Question: Two distinct types of dopamine neurons separately transmit positive and negative motivational signals.

This study explores the acute effects of cocaine on human brain activity and emotional states using functional magnetic resonance imaging (fMRI). Cocaine-dependent individuals were administered a double-blind infusion of cocaine (0.6 mg/kg) and saline, and their entire brain activity was scanned for five minutes before and thirteen minutes after each infusion. Participants rated their experiences on scales measuring rush, high, low, and craving. Cocaine administration led to significant signal increases in various brain regions, including the nucleus accumbens/subcallosal cortex (NAc/SCC), caudate, putamen, basal forebrain, thalamus, insula, hippocampus, parahippocampal gyrus, cingulate, lateral prefrontal and temporal cortices, parietal cortex, striate/extrastriate cortices, ventral tegmentum, and pons. Signal decreases were observed in the amygdala, temporal pole, and medial frontal cortex. Saline infusions produced minimal activations, primarily localized in the lateral prefrontal cortex and temporo-occipital cortex. Participants who underwent repeated studies showed consistent fMRI activation patterns following cocaine and saline infusions, with saline retest activations possibly indicating the influence of expectancy. Regions that exhibited early and short-lived signal maxima correlated more closely with rush ratings, encompassing the ventral tegmentum, pons, basal forebrain, caudate, cingulate, and most areas of the lateral prefrontal cortex. In contrast, regions with early but sustained signal maxima were more strongly associated with craving than rush ratings, including the NAc/SCC, right parahippocampal gyrus, and certain areas of the lateral prefrontal cortex. The amygdala displayed sustained negative signal changes, which correlated with craving ratings. These findings highlight the capability of fMRI to map the dynamic patterns of brain activation following cocaine infusion in cocaine-dependent individuals, providing insights into the evolving brain networks associated with cocaine-induced euphoria and craving.

The relation between the structure of the nervous system and its function is more poorly understood than the relation between structure and function in any other organ system. We explore why bridging the structure-function divide is uniquely difficult in the brain. These difficulties also explain the thrust behind the enormous amount of innovation centered on microscopy in neuroscience. We highlight some recent progress and the challenges that remain.

The aim of this study was to identify the neural substrates of online gaming addiction through evaluation of the brain areas associated with the cue-induced gaming urge. Ten participants with online gaming addiction and 10 control subjects without online gaming addiction were tested. They were presented with gaming pictures and the paired mosaic pictures while undergoing functional magnetic resonance imaging (fMRI) scanning. The contrast in blood-oxygen-level dependent (BOLD) signals when viewing gaming pictures and when viewing mosaic pictures was calculated with the SPM2 software to evaluate the brain activations. Right orbitofrontal cortex, right nucleus accumbens, bilateral anterior cingulate and medial frontal cortex, right dorsolateral prefrontal cortex, and right caudate nucleus were activated in the addicted group in contrast to the control group. The activation of the region-of-interest (ROI) defined by the above brain areas was positively correlated with self-reported gaming urge and recalling of gaming experience provoked by the WOW pictures. The results demonstrate that the neural substrate of cue-induced gaming urge/craving in online gaming addiction is similar to that of the cue-induced craving in substance dependence. The above-mentioned brain regions have been reported to contribute to the craving in substance dependence, and here we show that the same areas were involved in online gaming urge/craving. Thus, the results suggest that the gaming urge/craving in online gaming addiction and craving in substance dependence might share the same neurobiological mechanism.

The frontopolar cortex (FPC), the most anterior part of the frontal lobes, forms the apex of the executive system underlying decision-making. Here, we review empirical evidence showing that the FPC function enables contingent interposition of two concurrent behavioral plans or mental tasks according to respective reward expectations, overcoming the serial constraint that bears upon the control of task execution in the prefrontal cortex. This function is mechanistically explained by interactions between FPC and neighboring prefrontal regions. However, its capacity appears highly limited, which suggests that the FPC is efficient for protecting the execution of long-term mental plans from immediate environmental demands and for generating new, possibly more rewarding, behavioral or cognitive sequences, rather than for complex decision-making and reasoning.

The hypothalamus plays a central role in the integrated control of feeding and energy homeostasis. We have identified two novel neuropeptides, both derived from the same precursor by proteolytic processing, that bind and activate two closely related (previously) orphan G protein-coupled receptors. These peptides, termed orexin-A and -B, have no significant structural similarities to known families of regulatory peptides. prepro-orexin mRNA and immunoreactive orexin-A are localized in neurons within and around the lateral and posterior hypothalamus in the adult rat brain. When administered centrally to rats, these peptides stimulate food consumption. prepro-orexin mRNA level is up-regulated upon fasting, suggesting a physiological role for the peptides as mediators in the central feedback mechanism that regulates feeding behavior.

An unresolved question in neuroscience and psychology is how the brain monitors performance to regulate behavior. It has been proposed that the anterior cingulate cortex (ACC), on the medial surface of the frontal lobe, contributes to performance monitoring by detecting errors. In this study, event-related functional magnetic resonance imaging was used to examine ACC function. Results confirm that this region shows activity during erroneous responses. However, activity was also observed in the same region during correct responses under conditions of increased response competition. This suggests that the ACC detects conditions under which errors are likely to occur rather than errors themselves.

Theorists have proposed that depression is associated with abnormalities in the behavioral activation (BAS) and behavioral inhibition (BIS) systems. In particular, depressed individuals are hypothesized to exhibit deficient BAS and overactive BIS functioning. Self-reported levels of BAS and BIS were examined in 62 depressed participants and 27 nondepressed controls. Clinical functioning was assessed at intake and at 8-month follow-up. Relative to nondepressed controls, depressed participants reported lower BAS levels and higher BIS levels. Within the depressed group, lower BAS levels were associated with greater concurrent depression severity and predicted worse 8-month outcome. Levels of both BIS and BAS showed considerable stability over time and clinical state. Overall, results suggest that BAS dysregulation exacerbates the presentation and course of depressive illness.

What chiefly distinguishes cerebral cortex from other parts of the central nervous system is the great diversity of its cell types and inter-connexions. It would be astonishing if such a structure did not profoundly modify the response patterns of fibres coming into it. In the cat's visual cortex, the receptive field arrangements of single cells suggest that there is indeed a degree of complexity far exceeding anything yet seen at lower levels in the visual system. In a previous paper we described receptive fields of single cortical cells, observing responses to spots of light shone on one or both retinas (Hubel & Wiesel, 1959). In the present work this method is used to examine receptive fields of a more complex type (Part I) and to make additional observations on binocular interaction (Part II). This approach is necessary in order to understand the behaviour of individual cells, but it fails to deal with the problem of the relationship of one cell to its neighbours. In the past, the technique of recording evoked slow waves has been used with great success in studies of functional anatomy. It was employed by Talbot & Marshall (1941) and by Thompson, Woolsey & Talbot (1950) for mapping out the visual cortex in the rabbit, cat, and monkey. Daniel & Whitteiidge (1959) have recently extended this work in the primate. Most of our present knowledge of retinotopic projections, binocular overlap, and the second visual area is based on these investigations. Yet the method of evoked potentials is valuable mainly for detecting behaviour common to large populations of neighbouring cells; it cannot differentiate functionally between areas of cortex smaller than about 1 mm2. To overcome this difficulty a method has in recent years been developed for studying cells separately or in small groups during long micro-electrode penetrations through nervous tissue. Responses are correlated with cell location by reconstructing the electrode tracks from histological material. These techniques have been applied to CAT VISUAL CORTEX 107 the somatic sensory cortex of the cat and monkey in a remarkable series of studies by Mountcastle (1957) and Powell & Mountcastle (1959). Their results show that the approach is a powerful one, capable of revealing systems of organization not hinted at by the known morphology. In Part III of the present paper we use this method in studying the functional architecture of the visual cortex. It helped us attempt to explain on anatomical …

Julian C. Motzkin,1 Joseph P. Newman,2 Kent A. Kiehl,3,4 and Michael Koenigs1 1Department of Psychiatry, University of Wisconsin-Madison, Madison, Wisconsin 53719, 2Department of Psychology, University of Wisconsin-Madison, Madison, Wisconsin 53706, 3The MIND Research Network, Albuquerque, New Mexico 87131, and 4Departments of Psychology and Neuroscience, University of New Mexico, Albuquerque, New Mexico 87131

Blood flow in the brain is regulated by neurons and astrocytes. Knowledge of how these cells control blood flow is crucial for understanding how neural computation is powered, for interpreting functional imaging scans of brains, and for developing treatments for neurological disorders. It is now recognized that neurotransmitter-mediated signalling has a key role in regulating cerebral blood flow, that much of this control is mediated by astrocytes, that oxygen modulates blood flow regulation, and that blood flow may be controlled by capillaries as well as by arterioles. These conceptual shifts in our understanding of cerebral blood flow control have important implications for the development of new therapeutic approaches.


Question: Representation of abstract ideas: positive and negative in left-handed and right-handed individuals.

This investigation explores the impact of language on thought, specifically focusing on how English and Mandarin speakers conceptualize time. Given that English predominantly describes time horizontally, whereas Mandarin incorporates both horizontal and vertical dimensions, these linguistic differences appear to influence cognitive representations of time. In one experiment, Mandarin speakers, even when using English, were quicker to verify temporal sequences after viewing vertical object arrangements compared to horizontal arrangements, a pattern reversed among English speakers. Another study revealed that the inclination of Mandarin-English bilinguals to visualize time vertically correlates with the age at which they initiated English learning. In a subsequent experiment, English speakers were instructed to utilize vertical spatial terminology when discussing time, similar to Mandarin usage. These English speakers subsequently demonstrated a bias towards visualizing time vertically, akin to that observed in Mandarin speakers. These findings suggest that (1) language significantly shapes thought concerning abstract concepts and (2) one's mother tongue plays a crucial role in molding habitual thinking patterns, such as the spatial representation of time, although it does not rigidly dictate all aspects of thought as per the strong Whorfian hypothesis.

he aim of the paper is fourfold: (a) show why humor scholars should study irony, (b) explore the need for considering interindividual differences in healthy adults’ irony performance, (c) stress the necessity for developing tools assessing habitual differences in irony performance, and (d) indicate future directions for joint irony and humor research and outline possible applications. Verbal irony is often employed with a benevolent humorous intent by speakers, but can also serve as a means of disparagement humor. In both cases, encoding and decoding activities entailing irony need to be considered in the context of the psychology of humor. We argue that verbal irony performance can be considered a phenomenon native to the realm of humor and individual differences. We point out that research has widely neglected the meaningfulness of variance in irony performance within experimental groups when looking at determinants of irony detection and production. Based on theoretical considerations and previous empirical findings we show that this variance can be easily related to individual-differences variables such as the sense of humor, dispositions toward laughter and ridicule (e.g., gelotophobia), and general mental ability. Furthermore, we hypothesize that there is an enduring trait determining irony performance we will label the sense of irony. The sense of irony possibly goes along with inclinations toward specific affective and cognitive processing patterns when dealing with verbal irony. As an application, novel irony performance tests can help to study psychological and neurophysiological correlates of irony performance more feasibly, that is, in nonclinical groups. DOI: https://doi.org/10.1037/tps0000054 Posted at the Zurich Open Repository and Archive, University of Zurich ZORA URL: https://doi.org/10.5167/uzh-124094 Accepted Version Originally published at: Bruntsch, Richard; Hofmann, Jennifer; Ruch, Willibald (2016). Virgin soil in irony research: Personality, humor, and the “sense of irony”. Translational Issues in Psychological Science, 2(1):25-34. DOI: https://doi.org/10.1037/tps0000054 VIRGIN SOIL IN IRONY RESEARCH 1 Running head: VIRGIN SOIL IN IRONY RESEARCH Virgin soil in irony research: Personality, humor, and the “sense of irony”