decker_2016_from

two-step-task psych-201

Children, adolescents, and adults (N=59: 20 children ages 8–12, 20 adolescents ages 13–17, 19 adults ages 18–25) performed a two-step Markov decision task. On each paper-trial, participants first chose one of two spaceships (first stage), which probabilistically (70/30 common/rare) transitioned to one of two planets, then chose one of two aliens for a stochastic reward. The study tested whether model-based reinforcement learning increases across development by estimating the weighting parameter ω from a hybrid model-based/model-free RL model. Response type is binary choice across ~200 trials per participant (exp0).

Responses
23,246
1,978 held out
CORE effects
–
no effects simulated
CORE test NLL (avg)
0.481
rank 3 of 7 models
Best model
CORE (no phonemes)
test NLL (avg) 0.475

Likelihood of human responses

modelPNLLtest NLL (avg)test responses train NLL (avg)train responses
CORE951.50.48101,9780.521321,268
CORE (no prediction errors)955.60.48311,9780.521721,268
CORE (no phonemes) best939.40.47491,9780.517621,268
CORE (no forgetting)977.50.49421,9780.536521,268
CORE (no hierarchy)1,158.90.58591,9780.608021,268
Transformer992.00.50151,9780.555421,268
Centaur941.80.47621,978––

Experiments

exptask classparticipantsresponsesconditionssimulatedeffects
exp0TwoStepTask5923,2463no–

Behavioral effects

No behavioral effects were simulated for this study.

Transcript

You are a space explorer collecting space treasure. On each trial you first choose one of two spaceships. Each spaceship usually travels to one planet but sometimes to the other. After your spaceship travels, you arrive at a planet. On that planet you choose one of two aliens. If you pick the right alien, you find space treasure; otherwise you find nothing. The aliens' chances of giving treasure change slowly over time, so keep trying different choices.
Press A to choose the first spaceship or B to choose the second spaceship. On each planet, press A to choose the first alien or B to choose the second alien.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press A. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press A. You travel to planet A. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet A. You choose an alien. You press B. You find space treasure!
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find nothing.
You choose a spaceship. You press B. You travel to planet B. You choose an alien. You press B. You find space treasure!