Selection of Behavioral Parameters: Integration of Case-Based ...

Selection of Behavioral Parameters: Integration of Case-Based ...

Selection of Behavioral Parameters: Integration of Case-Based Reasoning with Learning Momentum Brian Lee, Maxim Likhachev, and Ronald C. Arkin Mobile Robot Laboratory Georgia Tech Atlanta, GA This research was funded under the DARPA MARS program. Integrated Multi-layered Learning CBR Wizardry Guide the operator Probabilistic Planning Manage complexity for the operator RL for Behavioral Assemblage Selection

Learn what works for the robot CBR for Behavior Transitions Adapt to situations the robot can recognize Learning Momentum Vary robot parameters in real time THE LEARNING CONTINUUM: Deliberative (pre-mission) . .

. Behavioral switching . . . Reactive (online adaptation) . . . Motivation Its hard to manually derive behavioral controller parameters. The parameter space increases exponentially with the number of parameters.

You dont always have a priori knowledge of the environment. Without prior knowledge, a user cant confidently derive appropriate parameter values, so it becomes necessary for the robot to adapt on its own to what it finds. Obstacle densities and layout in the environment may be heterogeneous. Parameters that work well for one type of environment may not work well with another type. A solution is to provide adaptability to the system while remaining fully reactive. Context for Case-based Reasoning (CBR) Spatial and temporal features are used to select stored cases from a case library. Cases contain parameters for a behavior-based reactive controller.

Selected parameters are adapted for the current situation. The controller is updated with new parameters that should be more appropriate to the current environment. CBR Module Spatial Feature Matching Temporal Feature Matching Sensors Case Library

Random Selection Process Case Application Case Adaptation Case Switching Decision Feature Identification Context for Learning

Momentum (LM) A crude form of reinforcement learning. If the robot is doing well, try doing what its doing a little more, otherwise try something different. Behavior parameters are continually changed in response to progress and obstacles. Static rules for pre-defined situations are used to update behavior parameters. Different sets of rules for parameter changes can be used (ballooning versus squeezing). LM Strategies Ballooning Alter parameters so the robot reacts to obstacles at larger distances than normal to push it out of box canyon situations. Squeezing Alter parameters so the robot reacts to obstacles only at

shorter distances than normal so it can move between closely spaced obstacles. Example ballooning rule: if ( situation == NO_PROGRESS_WITH_OBSTACLES ) obstacle_sphere_of_influence += 0.5 meters else obstacle_sphere_of_influence -= 0.5 meters LM Module Sensors Parameter Deltas Short Sensor History

Situation Matching Parameter Adaptation Adapted parameters Old parameters Behavioral Parameters Effects of CBR and LM When Used Separately Reported in ICRA 2001 Effects of CBR Distances traversed were shorter

Time taken was shorter Effects of LM Completion rates were much higher for dense obstacles Completion times were higher than those for successful non-adaptive robots Why Integrate? Want discontinuous switching + continuous searching in the parameter space. CBR is not continuous Parameter changes are triggered by environment changes or case time-outs. Case library is manually built to provide only ballpark solutions for different environment types. LM does not make large, discontinuous changes LM may take a while to adapt to large environmental changes.

LM cannot change strategies at run time The LM strategies of ballooning and squeezing are tuned for different environments. Currently Used Behaviors Move to Goal Always returns a vector pointing toward the goal position. Avoid Obstacles Returns a sum of weighted vectors pointing away from obstacles. Wander Returns vectors pointing in random directions. Bias Move Returns a vector biasing the robots movement in a

certain direction (i.e. away from high obstacle densities), and is set by the CBR module. Only used when CBR is present. Adjustable Behavioral Parameters Move to goal vector gain Avoid obstacle vector gain Avoid obstacle sphere of influence Radius around the robot inside of which obstacles reacted to Wander vector gain Wander persistence The number of consecutive steps the wander vector points in the same direction Bias Move vector gain Bias Move X, Bias Move Y

These are the components of the vector returned by Bias Move Integration Base System Core Behavior-Based Controller Sensors Behavioral Parameters Actuators Integration Addition of CBR Module Core Behavior-Based Controller Sensors Actuators

CBR Module Behavioral Parameters Updated Parameters Integration Addition of LM Module Core Behavior-Based Controller Sensors Actuators CBR Module LM Module Updated Deltas and Parameter Bounds

Behavioral Parameters Updated Parameters Simulation Setup Heterogeneous Environments varying obstacles density, order, and size 350 x 350 meters Homogeneous Environments even obstacle distribution random obstacle placement and size two environments with 15% density and two environments with 20% density 150 x 150 meters CBR-LM in Simulation

Simulation Results For a Heterogeneous Environment Percent Complete Completed Runs 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% non-adaptive

LM CBR adaptation algorithm CBR-LM Simulation Results For a Heterogeneous Environment Average Steps to Completion 4000 3500 Steps 3000 2500

2000 1500 1000 500 0 non-adaptive LM CBR adaptation algorithm CBR-LM Simulation Results For a Homogeneous Environment

Percent Complete Completed Runs 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% non-adaptive LM CBR CBR-LM

15% coverage 20% coverage adaptation algorithm Simulation Results For a Homogeneous Environment Average Steps to Completion 14000.00 12000.00 Steps 10000.00 non-adaptive

8000.00 LM 6000.00 CBR LM-CBR 4000.00 2000.00 0.00 15% coverage 20% coverage adaptation algorithm

Simulation Observations Beneficial Attributes of CBR are Preserved. We see quick, radical changes in behavior. Time taken is about the same as CBR only. Beneficial Attributes of LM are not always apparent. Results can probably be attributed to a well-tuned case library. If the case library is good enough, LM should not be needed. Physical Robot Experiments RWI ATRV-Jr robot Forward and rear LMS SICK laser scanners Odometry, compass, and gyroscope for localization Straight-line start to goal

distance of about 46 meters Outdoor environment with trees and man-made obstacles CBR-LM, CBR, LM, and non-adaptive systems were compared The squeezing strategy was used in the LM-only experiments. Data was averaged over 10 runs per adaptation algorithm Outdoor Run Physical Experiments Results Average Steps to Completion 1400 1200 1000 Steps

All valid runs were able to reach the goal. Both CBR and LM beat the non-adaptive system. The CBR-LM integrated system gave the best performance. 800 600 400 200 0 nonadaptive

LM CBR adaptation algorithm CBR-LM Difference From Simulation CBR-LM outperformed CBR on the physical robot more than in simulation. The case library for the real robot may not have been as well tuned as the simulation library. Time Improvement of CBR-LM Over CBR 35% 30% 25%

Heterogeneous Env. 20% Homogeneous Env. (15% coverage) 15% 10% 5% 0% -5% Homogeneous Env. (20% coverage) Physical Robot Conclusions A performance increase is not guaranteed.

For a well-tuned case library, there may be little for LM to do. Integration of CBR and LM can result in a performance increase observed up to 29% improvement in steps over CBR Benefits of LM are more likely to be apparent when the CBR case library is not well-tuned (which is likely to be the case for real robots.) LM could be used to dynamically update the case library with better sets of parameters.

Recently Viewed Presentations

  • Enterprise Content Management

    Enterprise Content Management

    Enterprise Content Management Author: Ian Palangio Last modified by: DDembo ... Services The relevance of Microsoft in Health and Human Services Strategy Knowledge Driven Health Acquisitions and HSG How clinicians spend time Time value of information Slide 10 Slide 11...
  • Acids, pH, and Buffers: Some Basic Chemistry for

    Acids, pH, and Buffers: Some Basic Chemistry for

    Acids, pH, and Buffers: Some Basic Chemistry for Biological Science Terry Platt University of Rochester * Paul Mathews has just learned about acidity in his chemistry class, and… surprises his parents with questions about the acidity of common substances, such...
  • Exercise in older adults with chronic disease

    Exercise in older adults with chronic disease

    Positive affect. Epidemiologic studies- pleasurable emotions (i.e., happiness, joy, pleasure, contentment) Intervention studies- positive mood state induced by small events, such as receiving a small, unexpected gift, seeing a few minutes of comedy, or receiving a report of success on...
  • The Scientific Literatre - University of Michigan

    The Scientific Literatre - University of Michigan

    Times New Roman Default Design CS ChemDraw Drawing The Scientific Literature General 216 Announcements The Honors Cup The Proposal Goals for your Proposal Interesting Molecule or Process What is NOT a Synthetic Transformation? Extraction PowerPoint Presentation Where do I get...
  • Public Relations - American Legion

    Public Relations - American Legion

    Public Relations. New PR/Membership Recruiting Tools: Legiontown USA. The centerpiece. A publication that highlights the programs and services of American Legion Posts in communities from coast to coast.
  • PIPA Implementation - Pipeline Safety Trust

    PIPA Implementation - Pipeline Safety Trust

    Pipeline Safety Trust Other titles: Century Gothic MS Pゴシック Arial Wingdings 2 Calibri Impact Franklin Gothic Medium Plaza 1_Plaza 2_Plaza 3_Plaza 4_Plaza 5_Plaza 6_Plaza 7_Plaza 8_Plaza 9_Plaza 10_Plaza 11_Plaza 12_Plaza 13_Plaza 14_Plaza 15_Plaza 16_Plaza 17_Plaza 18_Plaza 19_Plaza PIPA IMPLEMENTATION Pre-PIPA...
  • The Magnetic Origins of Solar Eruptions

    The Magnetic Origins of Solar Eruptions

    Times New Roman Blank Presentation Magnetic Topology and the Corona-Heliosphere Connection Basic Assumptions Slide 3 Conjecture: One CH per Polarity Region Slide 5 Slide 6 Slide 7 Slide 8 Slide 9 Topology of 3D Breakout Eruption Topology of Fully 3D...
  • DESKTOP PUBLISHING  Desktop publishing is the use of

    DESKTOP PUBLISHING Desktop publishing is the use of

    DESKTOP PUBLISHING. Desktop publishing is the use of the computer and software to create visual displays of ideas and information. Publications may be for desktop or commercial printing or electronic distribution such as email newsletters, electronic books ….