evangelidis_2022_upscaling
Participants choose between a high-desirability (HD) and a high-feasibility (HF) option across varied consumer stimuli (backpacks, speakers, hard drives, hotels, TVs). Some choice sets add a symmetrically dominated decoy (C) and/or a "search for other options" no-choice option; the key manipulation is presence/absence of the decoy, which increases HD choice share — the upscaling effect. Public trial-level data cover Studies 1–7 plus a Study 5 follow-up: Study 1 (N=2014, one choice/participant), Study 2 (N=506, within-subject two choices), Study 3 (N=809, hotel + free-text explanations), Study 4 (N=1609, speaker + justification measures), Study 5 (N=2016, hard drive, HD-price manipulation), Study 5 Follow-up (N=302, justifiability ratings), Study 6 (N=2018, decoy placement), Study 7 (N=805, same/different page presentation). Response is discrete choice between the alternatives; Study 5 Follow-up uses a 1–7 justifiability rating. The paper reports 19 studies total; only studies 1–7 and the Study 5 follow-up publish public trial-level data.
Responses
10,610
105 held out
CORE effects
0 / 1
CORE test NLL (avg)
0.905
rank 2 of 7 models
Best model
CORE (no forgetting)
test NLL (avg) 0.896
Likelihood of human responses
| model | PNLL | test NLL (avg) | test responses | train NLL (avg) | train responses |
|---|---|---|---|---|---|
| CORE | 95.0 | 0.9050 | 105 | 0.9159 | 10,505 |
| CORE (no prediction errors) | 96.2 | 0.9157 | 105 | 0.8800 | 10,505 |
| CORE (no phonemes) | 108.3 | 1.0311 | 105 | 0.7643 | 10,505 |
| CORE (no forgetting) best | 94.1 | 0.8963 | 105 | 0.9263 | 10,505 |
| CORE (no hierarchy) | 99.6 | 0.9481 | 105 | 1.0315 | 10,505 |
| Transformer | 102.4 | 0.9757 | 105 | 0.8349 | 10,505 |
| Centaur | 106.0 | 1.0091 | 105 | – | – |
Experiments
| exp | task class | participants | responses | conditions | simulated | effects |
|---|---|---|---|---|---|---|
| exp0 | UpscalingStudy1 | 2,014 | 2,014 | 2 | yes | upscaling_basic_study1 |
| exp1 | UpscalingStudy2 | 506 | 1,339 | 1 | no | – |
| exp2 | UpscalingStudy3 | 809 | 809 | 4 | no | – |
| exp3 | UpscalingStudy4 | 1,609 | 1,609 | 4 | no | – |
| exp4 | UpscalingStudy5 | 2,016 | 2,016 | 4 | no | – |
| exp6 | UpscalingStudy6 | 2,018 | 2,018 | 2 | no | – |
| exp7 | UpscalingStudy7 | 805 | 805 | 2 | no | – |
Behavioral effects
| effect | experiment | paper | CORE | CORE (no prediction errors) | CORE (no phonemes) | CORE (no forgetting) | CORE (no hierarchy) |
|---|---|---|---|---|---|---|---|
upscaling_basic_study1 | exp0 | +0.440 | ✗ no +0.167 | ✓ yes +0.580 | ✓ yes +0.557 | ✗ no +0.050 | ✗ no -0.148 |
between-subject; adding a symmetrically dominated decoy (three_options vs two_options) increases choice share of the HD option (response==1), tested with a logistic regression (positive logit coefficient, p<.05). | |||||||
Transcript
Imagine that you consider buying an external hard drive. You have the following options: Brand A, capacity 1TB, price $39.99; Brand B, capacity 2TB, price $79.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press B.
Imagine that you consider buying a backpack. You have the following options: Backpack A, capacity 30 L, price $29.99; Backpack B, capacity 40 L, price $39.99; Backpack C, capacity 30 L, price $39.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press A.
Imagine that you consider buying an external hard drive. You have the following options: Brand A, capacity 2TB, price $39.99; Brand B, capacity 4TB, price $79.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press A. Independently of your choice, how much do you like each brand? Rate from 1 (not at all) to 20 (very much). You rate Brand A 10 and you rate Brand B 9. Next you complete a 10-item self-construal scale. For each statement, rate how much you agree from 1 (strongly disagree) to 7 (strongly agree): you respond item 1 1 item 2 1 item 3 1 item 4 1 item 5 1 item 6 7 item 7 2 item 8 6 item 9 7 item 10 7. Now, an additional option is added to the set: Brand C, capacity 2TB, price $79.99. The options are now Brand A, Brand B, and Brand C. What would you do? Press the letter of the option you choose, or S to search for other options. You press A. Which option is clearly better than Brand C? Press A if only Brand A, B if only Brand B, or AB if both are better. You press A.
Imagine that you are searching for a hotel room for your upcoming summer holidays. In your search, you find the following options: Hotel A: price $119 per night, hotel rating 4-star; Hotel B: price $89 per night, hotel rating 3-star. Before making a choice, please explain in a few sentences what motivates your decision and what reasons or factors motivate your preference for one (or none) of these options. You write: I'll want to check the location, the safety of the hotel (like does a lower rating mean security issues or not), and non-smoking room options. What would you do? Press the letter of the option you choose, or S to search for other options. You press B.
Imagine that you are searching for a hotel room for your upcoming summer holidays. In your search, you find the following options: Hotel A: price $119 per night, hotel rating 4-star; Hotel B: price $89 per night, hotel rating 3-star; Hotel C: price $119 per night, hotel rating 3-star. Before making a choice, please explain in a few sentences what motivates your decision and what reasons or factors motivate your preference for one (or none) of these options. You write: For one it's price. Two, I don't think I really care about the star rating as long as it is at least three stars. What would you do? Press the letter of the option you choose, or S to search for other options. You press B.
Imagine that you are searching for a hotel room for your upcoming summer holidays. In your search, you find the following options: Hotel A: price $89 per night, hotel rating 3-star; Hotel B: price $119 per night, hotel rating 4-star. Before making a choice, please explain in a few sentences what motivates your decision and what reasons or factors motivate your preference for one (or none) of these options. You write: I’m motivated by price, but I also don’t want a hotel so cheap I’m worried about the quality of service and room. I want to be comfortable at a reasonable price What would you do? Press the letter of the option you choose, or S to search for other options. You press B.
Imagine that you are searching for a hotel room for your upcoming summer holidays. In your search, you find the following options: Hotel A: price $89 per night, hotel rating 3-star; Hotel B: price $119 per night, hotel rating 4-star; Hotel C: price $119 per night, hotel rating 3-star. Before making a choice, please explain in a few sentences what motivates your decision and what reasons or factors motivate your preference for one (or none) of these options. You write: I prefer a 4 star hotel since I am bringing family. The price would be smarter to spend on better room if the same price What would you do? Press the letter of the option you choose, or S to search for other options. You press B.
Imagine that you consider buying a Bluetooth speaker. In your search, you find the following options: Brand A, Amazon rating 4.3 out of 5 stars, price $59.99; Brand B, Amazon rating 3.5 out of 5 stars, price $29.99; Brand C, Amazon rating 3.5 out of 5 stars, price $59.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press S. How easy is it to justify choosing Brand A? Rate from 1 (not at all easy) to 7 (very easy). You rate 3. Please explain in a few sentences how choice of Brand A could be justified. You write: Poor rating for the product How easy is it to justify choosing Brand B? Rate from 1 (not at all easy) to 7 (very easy). You rate 5. Please explain in a few sentences how choice of Brand B could be justified. You write: Good rating with slightly high cost
Imagine that you consider buying a Bluetooth speaker. In your search, you find the following options: Brand A, Amazon rating 3.5 out of 5 stars, price $29.99; Brand B, Amazon rating 4.3 out of 5 stars, price $59.99; Brand C, Amazon rating 3.5 out of 5 stars, price $59.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press A. How easy is it to justify choosing Brand A? Rate from 1 (not at all easy) to 7 (very easy). You rate 7. Please explain in a few sentences how choice of Brand A could be justified. You write: I thought the rating was high enough for the $30 difference. How easy is it to justify choosing Brand B? Rate from 1 (not at all easy) to 7 (very easy). You rate 2. Please explain in a few sentences how choice of Brand B could be justified. You write: The star value was higher so I guess you could justify it by saying it was a better brand.
Imagine that you consider buying a Bluetooth speaker. In your search, you find the following options: Brand A, Amazon rating 4.3 out of 5 stars, price $59.99; Brand B, Amazon rating 3.5 out of 5 stars, price $29.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press A. How easy is it to justify choosing Brand A? Rate from 1 (not at all easy) to 7 (very easy). You rate 4. Please explain in a few sentences how choice of Brand A could be justified. You write: differentation. sales teams that can articulate a company's purpose beyond the ususal product story are better able to create meaningful distinctions relative to the competition How easy is it to justify choosing Brand B? Rate from 1 (not at all easy) to 7 (very easy). You rate 3. Please explain in a few sentences how choice of Brand B could be justified. You write: several people choose brand b because the product price is low.
Imagine that you consider buying a Bluetooth speaker. In your search, you find the following options: Brand A, Amazon rating 3.5 out of 5 stars, price $29.99; Brand B, Amazon rating 4.3 out of 5 stars, price $59.99. How easy is it to justify choosing Brand A? Rate from 1 (not at all easy) to 7 (very easy). You rate 4. Please explain in a few sentences how choice of Brand A could be justified. You write: Brand B price and rating is high compared to Brand A How easy is it to justify choosing Brand B? Rate from 1 (not at all easy) to 7 (very easy). You rate 4. Please explain in a few sentences how choice of Brand B could be justified. You write: Brand B price and rating is low compared to Brand A What would you do? Press the letter of the option you choose, or S to search for other options. You press B.
Imagine that you consider buying an external hard drive. You have the following options: Brand A, capacity 2TB, price $39.99; Brand B, capacity 4TB, price $42.99; Brand C, capacity 2TB, price $42.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press B.
Imagine that you consider buying an external hard drive. You have the following options: Brand A, capacity 2TB, price $39.99; Brand B, capacity 4TB, price $79.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press S.
Imagine that you consider buying an external hard drive. You have the following options: Brand A, capacity 4TB, price $79.99; Brand B, capacity 2TB, price $39.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press A.
Imagine that you consider buying an external hard drive. You have the following options: Brand A, capacity 4TB, price $42.99; Brand B, capacity 2TB, price $39.99; Brand C, capacity 2TB, price $42.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press A.
Imagine that you are searching for a hotel room for your upcoming summer holidays. You have the following options: Brand A, hotel rating 3-star, price $79 per night; Brand B, hotel rating 4-star, price $109 per night. What would you do? Press the letter of the option you choose, or S to search for other options. You press A.
Imagine that you consider buying a backpack. You have the following options: Brand A, capacity 40 L, price $39.99; Brand B, capacity 30 L, price $29.99; Brand C, capacity 30 L, price $39.99. What would you do? Press the letter of the option you choose, or S to search for other options. You press A.
Imagine that you consider buying an external hard drive. Brand A is priced at $39.99 and has a storage capacity of 2TB. Brand B is priced at $79.99 and has a storage capacity of 4TB. Brand C is priced at $79.99 and has a storage capacity of 2TB. On different pages (each shown separately), you view these options and then make a choice. What would you do? Press the letter of the option you choose, or S to search for other options. You press A.
Imagine that you consider buying an external hard drive. Brand A is priced at $39.99 and has a storage capacity of 2TB. Brand B is priced at $79.99 and has a storage capacity of 4TB. At the same time, on one page, you view these options and then make a choice. What would you do? Press the letter of the option you choose, or S to search for other options. You press A.