Understanding Models of Choice in Animal Behavior

Other Models of Choice

Other models of Choice Matching law = choice model Describes and predicts how animals make choices A performance model, not a learning model Describes how they will allocate their responses when given two choices of 2 different quantities or ratios The matching law does not describe or predict how animals choose How delay of a reward might affect choice: Later vs. sooner rewards How qualitatively different reinforcers affect choice How behavioral momentum affects choices

So, we have other models for those: Molecular Maximizing Molar Maximizing Melioration Temporal discounting: how time to the reward affects choice Self control: how organisms learn to have self control:

Two levels of behavior change Molar models: Look at overall (across sessions) behavior Assume animal is making choices based on longer time horizons Aggregate of responses: Overall GPA rather than just grade in 1 class Allows for momentary fluctuations in behavior Is the end result that is important

Two levels of behavior change Molecular models: Look at moment x moment behavior Assume animal is making choices based on what choice it just made rather than on longer time horizons Examine patterns of individual responses: If just received a bad grade on assignment- then study more If just received a good grade on assignment- then study less Allows for examination of momentary fluctuations in behavior in producing aggregate result

Molecular Maximizing Idea that animals maximize their responding moment by moment Most reinforcement for least effort Maximize over a long time horizon Assumes: Organisms always chose whichever response alternative is most likely to be reinforced at the time of the response Example: Two schedules: A and B In effect concurrently Subject switches from schedule A to schedule B as the probability of the reinforcement for schedule B INCREASES This model can explain why subjects switch with concurrent VI-VI schedules but not with concurrent ratio schedules Hinson and Staddon, 1983

Molecular Maximizing Vaughn suggests is more complicated than simple 2- way choice: 4 choices at any time: Staying on Schedule A Switching from Schedule A to Schedule B Staying on Schedule B Switching from Schedule B to Schedule A Must take into consideration the effects of switching as well as the rate of payoff Data not strongly support this model

Molar Maximizing Assume organisms distribute their responses among various alternative so as to maximize the amount of reinforcement they earn OVER A LONG TIME HORIZON Not clearly specify a long time horizon Developed to explain exclusive choice on VR-VR and VR VI schedules Assume that animals work for maximum return

Melioration To meliorate means to make better Assume animal makes best overall goal Has to be good enough , not necessarily best just have to make it better Proposes that subjects shift responding toward one or other alternative until both alternatives are equally attractive in terms of reinforcement per unit of effort or time Is an model that is in between molar and molecular in focus.

Melioration Model assumes that organism change from 1 response alternative to another to improve the LOCAL rate of reinforcement they are receiving Remember: local rate = responding/time Adjustments in distribution of behavior between alternatives assumed to continue until organism is obtaining the same rate of reinforcement on BOTH alternatives This, then, defaults to the matching law. But: Explains the Conc VI VR or Mult VI VR problem Animal stays on schedule combination that gets it the most reinforcement Only occasionally, if ever, sample the VI, as VR pays off more the faster you respond on that schedule; stop to respond on the VI and lose reinforcers! In a way, is an attempt to explain how animals match.

Complex Choice What about situations with MORE than 2 choices? In real world, have more than 2 choices And have time limit in which to decide which to choose Study concurrent chain schedules to examine this problem These involve 2 stages or links First stage = choice link Second link = terminal link

Used in studies of self control Choose first link: Choose between 2 schedule alternatives Smaller but sooner Larger but later Must make a choice Can use commitment criterion (no redos!) Can also allow a change to other schedule within x seconds Second link, then, depends on first choice If chose smaller and sooner: smaller requirement for smaller reinforcer If chose larger and later: larger Sr after a longer time or more work Use a black out period between DISCRETE TRIALS Thus can t get more by working shorter choice faster Controls for trial duration

Concurrent Chain schedules

Now can study self control and choice! Temporal discounting: Model that assumes the value of a reinforcer decreases over wait time Self control examines how people make the more valuable choices rather than impulsive choices

Temporal discounting

Temporal Discounting The tendency of people to discount rewards as they approach a temporal horizon in the future or the past The time to reward becomes so distant in time that reward ceases to be valuable or to have additive effects A tendency to give greater value to rewards as they move away from their temporal horizons and towards the "now". A nicotine deprived smoker may highly value a cigarette available any time in the next 6 hours In contrast, that same smoker would assign little or no value to a cigarette available in 6 months

Temporal Discounting Len Green; Mark Dixon; Tom Critchfield Theory or model which examines time frame under which people choose larger/later (LL) vs. Sooner/smaller (SS) How long are you willing to wait for a reward Assumes that the value of a reward is discounting (decreases) as time between response and reinforcer increase http://www.youtube.com/watch?v=G7LN96jEXHc

Temporal Discounting Accounts for commitment to choice in a two-choice paradigm. Commitment to choice is considered a form of self control, and corresponds with low levels of impulsiveness Is explained by a hyperbolic relationship between value and delay of reinforcer ? 1+?? ? = V is the present value of future reward M is the magnitude (value, amount) of the reward D is the delay to reinforcement k is the value of discounting (Rachlin & Green, 1972; Rachlin and Logue, 1976; Mazur, 1987, 1988)

Value of a smaller/sooner and a larger/later reinforcer are quite different depending on the time

Temporal Discounting as a Measure of Impulsivity in Humans Shiels et al., 2009 Measured delay discounting in children with ADHD Children were measured both on and off medication Three groups More inattentive ADHD More hyperactive ADHD Normal control All three groups were tested on and off both doses of ADHD medication Low dose Concerta (39 mg) High dose Concerta (73 mg)

Shiels et al., 2009 cont. Children were exposed to a discounting task Choose 5 cents now (SS) Choose 30 cents later (LL) Delayed option was manipulated probabilistically 7s, 14s, or 28s delay With a constant 35% of rewards being delivered, 65% not being delivered Results concluded that: Children with ADHD significantly chose SS over LL Each dose of Concerta significantly lowered preference for SS in ADHD children With medication, the ADHD children similarly to their typical peer controls

Practical Uses? Used to understand and predict Gambling behavior Addiction Impulsivity Self Control Important in that can predict the value of reward across different individuals and settings

Self-Control Or ..give up and let some one else control your contingencies!

Self-Control Process by which individual deliberately alters or changes his or her behavior to achieve a specific goal Self Administration of rewards Individual controls their own contingency Should optimize: get the most gain for the least effort

Self Control Learning to choose delayed/ later but larger consequences over shorter but smaller consequences Assumes that LL reward has greater value Very difficult for animals to do May be biological: take what can get, not wait for unknown Important human (and animal) behavior We can learn to delay reward Question is: how long are we willing to wait

Causes of Self-Control Problems Problems of Behavioral Excesses Immediate reinforcers versus delayed punishers for a behavior Immediate reinforcers versus cumulatively significant punishers for a behavior Immediate reinforcers for problem behavior versus delayed reinforcers for alternative desired behavior

Causes of Self-Control Problems Problems of Behavioral Deficiencies Immediate small punishers for a behavior versus reinforcers that are cumulatively significant Immediate small punisher for behavior versus immediate but highly improbable major punisher if the behavior does not occur Immediate small punisher for a behavior versus a delayed major punisher if the behavior does not occur

Behavioral Contracts Serves at least four important stimulus control functions: Ensures all parties agree to goals and procedures Ensures that all parties agree on how close they are to reaching the goals Provides the client with a realistic estimate of the cost of the program to him or her in time, effort, and money Signatures help ensure that all parties with follow procedures

Bottom Line Reinforcers lose their value if you have to wait for them too long Self control = commitment to a choice Need immediate as well as delayed reinforcement Contracting can help Biggest way to have people exhibit self control? Teach them strategies for waiting!