vantiel_2021_probabilistic
Six experiments test whether quantity words (e.g., "some", "all", "most") have truth-conditional or prototype-based semantics, explained by Bayesian probabilistic-pragmatic models. Exp 1a (N=596) and 1b (N=201) collect free-text quantity-word productions in response to displays of 432 red/black circles; Exp 2a (N=50) and 2b (N=150) collect binary validity judgments (with RTs) of monotonicity arguments; Exp 3 (N=21) collects continuous slider proportion estimates of red circles; Exp 4 (N=194) collects slider adequacy ratings of quantity-word descriptions. Response types span open production, binary validity judgments, and continuous ratings. The data reproduce the paper's headline effects: approximate-number tracking in estimation, monotonicity-based validity judgments, and adequacy ratings where the observed data outperform model predictions, supporting a gradience/prototype account of quantity-word meaning.
Responses
7,600
792 held out
CORE effects
1 / 1
CORE test NLL (avg)
0.343
rank 1 of 7 models
Best model
CORE
test NLL (avg) 0.343
Likelihood of human responses
| model | PNLL | test NLL (avg) | test responses | train NLL (avg) | train responses |
|---|---|---|---|---|---|
| CORE best | 271.4 | 0.3427 | 792 | 0.4652 | 6,808 |
| CORE (no prediction errors) | 279.2 | 0.3525 | 792 | 0.4526 | 6,808 |
| CORE (no phonemes) | 308.1 | 0.3890 | 792 | 0.1619 | 6,808 |
| CORE (no forgetting) | 320.8 | 0.4050 | 792 | 0.5041 | 6,808 |
| CORE (no hierarchy) | 507.9 | 0.6412 | 792 | 0.6529 | 6,808 |
| Transformer | 335.7 | 0.4239 | 792 | 0.4666 | 6,808 |
| Centaur | 281.2 | 0.3550 | 792 | – | – |
Experiments
| exp | task class | participants | responses | conditions | simulated | effects |
|---|---|---|---|---|---|---|
| exp2 | MonotonicityPretest | 50 | 2,500 | 1 | no | – |
| exp3 | Monotonicity | 150 | 5,100 | 1 | yes | monotonicity_validity |
Behavioral effects
| effect | experiment | paper | CORE | CORE (no prediction errors) | CORE (no phonemes) | CORE (no forgetting) | CORE (no hierarchy) |
|---|---|---|---|---|---|---|---|
monotonicity_validity | exp3 | +0.400 | ✓ yes +0.211 | ✓ yes +0.293 | ✓ yes +0.473 | ✓ yes +0.167 | ✓ yes +0.044 |
validity judgments track the monotonicity of quantity words — monotone-increasing words (most, all, many, ...) are accepted for the upward (strong->weak) entailment more often than for the reverse (weak->strong). Test: per-participant mean acceptance difference strong_to_weak - weak_to_strong over the monotone-increasing quantifiers, expected positive and significant. | |||||||
Transcript
You will judge whether logical arguments are valid. An argument is valid when the conclusion is necessarily true if the premise is true. For each argument, press A if it is valid or B if it is invalid. Example: 'Angela owns a dog. Therefore, Angela owns a pet.' is valid; 'Angela owns a pet. Therefore, Angela owns a dog.' is invalid. Argument: 'is from Texas. Therefore, is from Houston.' You judge it B (A = valid, B = invalid). Argument: 'travelled to Paris. Therefore, travelled to France.' You judge it A (A = valid, B = invalid). Argument: 'was drinking a soda. Therefore, was drinking a Pepsi.' You judge it B (A = valid, B = invalid). Argument: 'saw an oak. Therefore, saw a tree.' You judge it A (A = valid, B = invalid). Argument: 'travelled to Moscow. Therefore, travelled to Russia.' You judge it A (A = valid, B = invalid). Argument: 'owns a pet. Therefore, owns a dog.' You judge it B (A = valid, B = invalid). Argument: 'read 'Moby Dick'. Therefore, read a novel.' You judge it A (A = valid, B = invalid). Argument: 'watched a movie. Therefore, watched 'Pulp Fiction'.' You judge it B (A = valid, B = invalid). Argument: 'travelled to Russia. Therefore, travelled to Moscow.' You judge it B (A = valid, B = invalid). Argument: 'bought a flower. Therefore, bought a rose.' You judge it B (A = valid, B = invalid). Argument: 'ate vegetables. Therefore, ate carrots.' You judge it B (A = valid, B = invalid). Argument: 'ordered fish. Therefore, ordered salmon.' You judge it B (A = valid, B = invalid). Argument: 'travelled to Japan. Therefore, travelled to Tokyo.' You judge it B (A = valid, B = invalid). Argument: 'travelled to Tokyo. Therefore, travelled to Japan.' You judge it A (A = valid, B = invalid). Argument: 'played a card game. Therefore, played poker.' You judge it B (A = valid, B = invalid). Argument: 'ordered meat. Therefore, ordered steak.' You judge it B (A = valid, B = invalid). Argument: 'is from San Francisco. Therefore, is from California.' You judge it A (A = valid, B = invalid). Argument: 'is from Houston. Therefore, is from Texas.' You judge it A (A = valid, B = invalid). Argument: 'saw a bird. Therefore, saw an eagle.' You judge it B (A = valid, B = invalid). Argument: 'saw an animal. Therefore, saw a lion.' You judge it B (A = valid, B = invalid). Argument: 'bought a diamond. Therefore, bought a gemstone.' You judge it A (A = valid, B = invalid). Argument: 'read a novel. Therefore, read 'Moby Dick'.' You judge it B (A = valid, B = invalid). Argument: 'saw a cockroach. Therefore, saw an insect.' You judge it A (A = valid, B = invalid). Argument: 'used a herb. Therefore, used rosemary.' You judge it B (A = valid, B = invalid). Argument: 'ate an apple. Therefore, ate a fruit.' You judge it A (A = valid, B = invalid). Argument: 'ate carrots. Therefore, ate vegetables.' You judge it A (A = valid, B = invalid). Argument: 'owns a weapon. Therefore, owns a sword.' You judge it B (A = valid, B = invalid). Argument: 'owns a vehicle. Therefore, owns a truck.' You judge it B (A = valid, B = invalid). Argument: 'used rosemary. Therefore, used a herb.' You judge it A (A = valid, B = invalid). Argument: 'is from Chicago. Therefore, is from Illinois.' You judge it A (A = valid, B = invalid). Argument: 'watched 'Pulp Fiction'. Therefore, watched a movie.' You judge it A (A = valid, B = invalid). Argument: 'saw an eagle. Therefore, saw a bird.' You judge it A (A = valid, B = invalid). Argument: 'drank wine. Therefore, drank alcohol.' You judge it A (A = valid, B = invalid). Argument: 'ordered salmon. Therefore, ordered fish.' You judge it A (A = valid, B = invalid). Argument: 'is from California. Therefore, is from San Francisco.' You judge it B (A = valid, B = invalid). Argument: 'drank alcohol. Therefore, drank wine.' You judge it B (A = valid, B = invalid). Argument: 'bought a gemstone. Therefore, bought a diamond.' You judge it B (A = valid, B = invalid). Argument: 'saw a tree. Therefore, saw an oak.' You judge it B (A = valid, B = invalid). Argument: 'played poker. Therefore, played a card game.' You judge it A (A = valid, B = invalid). Argument: 'owns a sword. Therefore, owns a weapon.' You judge it A (A = valid, B = invalid). Argument: 'ate a fruit. Therefore, ate an apple.' You judge it B (A = valid, B = invalid). Argument: 'is from Illinois. Therefore, is from Chicago.' You judge it B (A = valid, B = invalid). Argument: 'travelled to France. Therefore, travelled to Paris.' You judge it B (A = valid, B = invalid). Argument: 'ordered steak. Therefore, ordered meat.' You judge it A (A = valid, B = invalid). Argument: 'owns a truck. Therefore, owns a vehicle.' You judge it A (A = valid, B = invalid). Argument: 'saw an insect. Therefore, saw a cockroach.' You judge it B (A = valid, B = invalid). Argument: 'was drinking a Pepsi. Therefore, was drinking a soda.' You judge it A (A = valid, B = invalid). Argument: 'owns a dog. Therefore, owns a pet.' You judge it A (A = valid, B = invalid). Argument: 'bought a rose. Therefore, bought a flower.' You judge it A (A = valid, B = invalid). Argument: 'saw a lion. Therefore, saw an animal.' You judge it A (A = valid, B = invalid).
You will judge whether logical arguments are valid. An argument is valid when the conclusion is necessarily true if the premise is true. The arguments are of the form 'Q of the people P1. Therefore, Q of the people P2.', where Q is a quantity word such as 'all' or 'some'. For each argument, press A if it is valid or B if it is invalid. Example: 'All of the people ate salmon. Therefore, all of the people ate fish.' is valid; 'All of the people ate fish. Therefore, all of the people ate salmon.' is invalid. Argument: 'almost all of the people saw an eagle. Therefore, almost all of the people saw a bird.' You judge it A (A = valid, B = invalid). Argument: 'the majority of the people played a card game. Therefore, the majority of the people played poker.' You judge it A (A = valid, B = invalid). Argument: 'less than half of the people own a vehicle. Therefore, less than half of the people own a truck.' You judge it A (A = valid, B = invalid). Argument: 'more than half of the people own a weapon. Therefore, more than half of the people own a sword.' You judge it A (A = valid, B = invalid). Argument: 'many of the people saw a cockroach. Therefore, many of the people saw an insect.' You judge it A (A = valid, B = invalid). Argument: 'very few of the people are from California. Therefore, very few of the people are from San Francisco.' You judge it A (A = valid, B = invalid). Argument: 'some of the people saw a lion. Therefore, some of the people saw an animal.' You judge it A (A = valid, B = invalid). Argument: 'a lot of the people bought a flower. Therefore, a lot of the people bought a rose.' You judge it A (A = valid, B = invalid). Argument: 'most of the people bought a rose. Therefore, most of the people bought a flower.' You judge it A (A = valid, B = invalid). Argument: 'a lot of the people own a dog. Therefore, a lot of the people own a pet.' You judge it A (A = valid, B = invalid). Argument: 'about half of the people played poker. Therefore, about half of the people played a card game.' You judge it A (A = valid, B = invalid). Argument: 'almost all of the people ate a fruit. Therefore, almost all of the people ate an apple.' You judge it A (A = valid, B = invalid). Argument: 'few of the people read 'Moby Dick'. Therefore, few of the people read a novel.' You judge it A (A = valid, B = invalid). Argument: 'less than half of the people used rosemary. Therefore, less than half of the people used a herb.' You judge it A (A = valid, B = invalid). Argument: 'very few of the people are from San Francisco. Therefore, very few of the people are from California.' You judge it A (A = valid, B = invalid). Argument: 'more than half of the people drank wine. Therefore, more than half of the people drank alcohol.' You judge it A (A = valid, B = invalid). Argument: 'a few of the people saw an insect. Therefore, a few of the people saw a cockroach.' You judge it A (A = valid, B = invalid). Argument: 'several of the people travelled to Tokyo. Therefore, several of the people travelled to Japan.' You judge it A (A = valid, B = invalid). Argument: 'none of the people ordered meat. Therefore, none of the people ordered steak.' You judge it A (A = valid, B = invalid). Argument: 'half of the people are from Houston. Therefore, half of the people are from Texas.' You judge it A (A = valid, B = invalid). Argument: 'several of the people read a novel. Therefore, several of the people read 'Moby Dick'.' You judge it A (A = valid, B = invalid). Argument: 'most of the people saw a bird. Therefore, most of the people saw an eagle.' You judge it A (A = valid, B = invalid). Argument: 'few of the people drank alcohol. Therefore, few of the people drank wine.' You judge it A (A = valid, B = invalid). Argument: 'some of the people saw an animal. Therefore, some of the people saw a lion.' You judge it A (A = valid, B = invalid). Argument: 'half of the people are from Texas. Therefore, half of the people are from Houston.' You judge it A (A = valid, B = invalid). Argument: 'none of the people ordered steak. Therefore, none of the people ordered meat.' You judge it A (A = valid, B = invalid). Argument: 'all of the people ordered salmon. Therefore, all of the people ordered fish.' You judge it A (A = valid, B = invalid). Argument: 'about half of the people ordered fish. Therefore, about half of the people ordered salmon.' You judge it A (A = valid, B = invalid). Argument: 'hardly any of the people own a sword. Therefore, hardly any of the people own a weapon.' You judge it A (A = valid, B = invalid). Argument: 'the majority of the people own a truck. Therefore, the majority of the people own a vehicle.' You judge it A (A = valid, B = invalid). Argument: 'hardly any of the people used a herb. Therefore, hardly any of the people used rosemary.' You judge it A (A = valid, B = invalid). Argument: 'a few of the people ate an apple. Therefore, a few of the people ate a fruit.' You judge it A (A = valid, B = invalid). Argument: 'all of the people own a pet. Therefore, all of the people own a dog.' You judge it A (A = valid, B = invalid). Argument: 'many of the people travelled to Japan. Therefore, many of the people travelled to Tokyo.' You judge it A (A = valid, B = invalid).