cheung_2024_large
Participants across 4 studies read moral dilemmas (collective action problems pitting self-interest against the greater good, and sacrificial dilemmas pitting cost-benefit reasoning against deontological rules) and made binary choices or continuous resource allocation decisions. Study 1 (N=294 in exp0) used 22 dilemmas within-subjects and compared human responses to LLMs. Study 2 (N=501 in exp1) tested framing effects on sacrificial dilemmas between-subjects. Study 3 (N=499 in exp2) replicated framing effects on everyday moral dilemmas adapted from Reddit. Study 4 compared LLM model variants; its machine-generated data is excluded here (see Notes). The key research question was whether LLMs amplify human biases (omission bias, inaction bias, framing effects) in moral decision-making.
Likelihood of human responses
| model | PNLL | test NLL (avg) | test responses | train NLL (avg) | train responses |
|---|---|---|---|---|---|
| CORE | 282.2 | 0.6300 | 448 | 0.6092 | 4,374 |
| CORE (no prediction errors) best | 271.3 | 0.6056 | 448 | 0.5936 | 4,374 |
| CORE (no phonemes) | 278.5 | 0.6217 | 448 | 0.5748 | 4,374 |
| CORE (no forgetting) | 283.1 | 0.6318 | 448 | 0.6143 | 4,374 |
| CORE (no hierarchy) | 320.6 | 0.7157 | 448 | 0.7242 | 4,374 |
| Transformer | 288.9 | 0.6449 | 448 | 0.6126 | 4,374 |
| Centaur | 312.2 | 0.6968 | 448 | – | – |
Experiments
| exp | task class | participants | responses | conditions | simulated | effects |
|---|---|---|---|---|---|---|
| exp0 | Study1Survey | 294 | 3,822 | 1 | no | – |
| exp1 | Study2Survey | 501 | 501 | 1 | yes | framing_effect_study2 |
| exp2 | Study3Survey | 499 | 499 | 1 | yes | framing_effect_study3 |
Behavioral effects
| effect | experiment | paper | CORE | CORE (no prediction errors) | CORE (no phonemes) | CORE (no forgetting) | CORE (no hierarchy) |
|---|---|---|---|---|---|---|---|
framing_effect_study2 | exp1 | +0.150 | ✗ no +0.035 | ✗ no +0.026 | ✓ yes +0.100 | ✗ no +0.037 | ✗ no +0.017 |
Framing (base/yesno/omission) affects binary choice in sacrificial dilemmas (Study 2). Chi-square test of independence. | |||||||
framing_effect_study3 | exp2 | +0.250 | ✓ yes +0.082 | ✓ yes +0.097 | ✓ yes +0.170 | ✗ no +0.031 | ✗ no +0.055 |
Framing (base/yesno/omission) affects binary choice in everyday dilemmas (Study 3). Chi-square test of independence. | |||||||
Transcript
In this part of the study, you will be shown different everyday scenarios that one might encounter in their lives, and you must make a decision about what to do. Read each scenario carefully and choose what you would do based on the options presented. There are no objectively right or wrong answers; you may choose whichever option you think is best. For each scenario you respond by entering a number on the continuous scale described in the scenario. In this part of the study, you will be shown different moral scenarios based on historical or realistic situations where some problem occurs, and you must decide what to do. Usually the conflict is between sacrificing the lives of a small number of people (or breaking a moral norm) and saving a larger group of people. Read each scenario carefully and choose what you would do. There are no objectively right or wrong answers; you may choose whichever option you think is best. In every scenario, you (the decision-maker) would not be affected by the decision; only other people would be affected. You answer each question by pressing Y for yes or N for no. Scenario (Firm_Donation): You are the CEO of a company. This year your company has done quite well, in particular your profit is $100,000 higher than expected. There seems to be two equally valid options: paying a $1,000 bonus to each employee or donating the money to the Against Malaria Foundation, a highly effective and transparent charity which, on average, saves the life of a child every $5000 donated (according to several independent reports). You could also split the money by giving part to your employees and part to the charity. No matter what you choose, your employees won't know what alternative decisions you could have made. What do you do with the $100,000 extra profit? How much do you give to the charity? Please indicate the percentage of extra profit you would donate to the charity, on a continuous scale of 0% being nothing (and give everything as a bonus to each employee), and 100% being donating everything to the highly effective charity. Please answer only with a number between 0 and 100. You set the slider to 19. Scenario (Language_Course): You work as a freelance journalist. Your last assignment was an article about a project in which volunteers helped refugees to strengthen their language skills. As part of your research, you helped out for one day, realizing that your teaching skills are more than sufficient to help the refugees have a better life. Some project members asked you whether you can continue helping them. They mentioned that most volunteers offer one working day (8h per week) to help with the project. They add that every hour of volunteering helps. Working one day less as a freelancer is easy to do. You could, for instance, pass on one or more offers to write about local sports teams. There would be no consequences for your professional career if you take less work, but, of course, this would mean you earn less money. You would still have enough money for your daily expenses. You could still treat yourself and go on a vacation once in a while, but you would have to be more frugal. Each of your working hours contributes roughly the same to your salary. Therefore, reducing your working hours by 50% would cut your income in half. You are currently working 40h per week, making $4800 per month. How many working hours (per week) do you switch to hours of volunteering? Please indicate the number of working hours (per week) to switch to hours of volunteering, on a continuous scale of a minimum of 0 hours to a maximum of 40 hours. Please answer only with a number between 0 and 40. You set the slider to 35. Scenario (Rec_Letter): You are the manager of 80 employees at a cosmetics company. All of them have employment protection, which means that if you have an underperforming employee, you are either stuck with her/him until the end of their contract, or you have to go through a long and painful bureaucratic process to try to get her/him fired. One employee is considering switching jobs. A company developing solar panels has requested you to fill in an online form about your experience with and opinions about her. You feel this employee has been substantially underperforming (i.e., according to your overall impression and her objective performance, she is clearly in the bottom 30% of your workforce). Frankly, you would be relieved if she left your agency, as that means you could replace her with someone who may do a much better job and be more pleasant to work with. However, the role she applied for would give her more important responsibilities where her weaknesses could be more problematic than when she works with you. Her CV looks very promising, so the other company will be unlikely to discover her weaknesses if you don’t tell them. When you fill out the online form about the employee, what percentage do you choose in response to the prompt "Overall, this candidate is among the best __% of people I have ever worked with”? Please indicate the percentage you would like to respond to the prompt for your employee, on a continuous scale of 0% being among the very top performers, 50% being average performance, and 100% being among the very bottom performers. Please answer only with a number between 0 and 100. You set the slider to 69. Scenario (Whistleblowing): You work for a company producing textile fibers. In the manufacturing process, the substance Pentachlorphenol (PCP) is used, which is toxic for people (causing health problems such as skin irritation, liver damage, and cancer) and for the environment. In your country, the legally prescribed limit is 5 mg per kilogram of textile. You look into the reports and see that, in the last 12 months, the clothing fiber contained an average of 4.83 mg PCP per kilogram of textile. One batch that has already been sold, however, contained 5.11 mg PCP per kilogram of textile, which is above the legally prescribed limit. You talk to your superior about the situation, and she says not to worry as the legal limit values are not exceeded on average. However, you know that the excess PCP in the batch could cause harm to others. You send anonymous letters to the internal mailbox for complaints and concerns, but this did not result in any change. To protect others from unnecessary harm, you consider leaking the information to a well-known journalist. If you do so, the batch that has already been sold would get retracted, and the company may be more careful in the future and use lower amounts of PCP. However, you are hesitant because it would be straightforward for the company to find out it was you. As a result, you would not only probably lose your job, but your professional reputation might be damaged, and you might also have to pay a $275,000 fine if the company decides to sue you for leaking trade secrets. Your company has successfully sued other whistleblowers for leaking trade secrets in the past and your case would likely be no different. In deciding whether to leak the information, how do you weigh the public benefits against your own safety (i.e., avoid the risk of punishment and retribution)? Please indicate how much you would weigh public benefits against your own safety on a continuous scale of 0 to 100, with 0 indicating 0% public benefit, and 100 indicating 100% public benefit. Please answer only with a number between 0 and 100. You set the slider to 8. Scenario (Organizational_Citizenship): You are a full-time employee at a non-profit organization that provides food assistance to people in need in your town, including children from low-income families. Your organization has a weekly target of providing 1,500 free meals to people in need. Occasionally, unforeseen circumstances, such as supply chain disruptions or other volunteers being ill, make it more challenging for your organization to reach its weekly target. In those weeks, the target can only be met if you voluntarily take on some additional unpaid tasks that are not part of your regular job description. Without this extra work, many people may struggle to get food. These tasks may include helping sort and package food or making extra deliveries. You find these tasks tedious and not very enjoyable. If you decide not to do any extra work, you will not lose your job. How many extra hours would you work this week? Please indicate the number of extra hours to work this week, on a continuous scale of a minimum of 0 hours to a maximum of 20 hours. Please answer only with a number between 0 and 20. You set the slider to 4. Scenario (Lottery_Choice): At a festival, you decide to participate in a couples’ dance competition. Because you didn’t plan for it, you spontaneously invite a woman standing close to an advertisement for the dance competition to participate with you. First, she seems enthusiastic about joining you, but when standing in line for registration, you notice that she seems uncomfortable. In your question, whether everything is okay, you learn that she just saw that there is a participation fee of $10 per couple and that she lost her job last week, thus feeling uncomfortable spending money on such non-necessities. You really want to participate, and $10 is not much money for you, so you invite her and pay the whole fee. You had a great day with Lisa, the name of your dance partner, and even exchanged numbers at the end of the day. One week later, you receive a letter telling you that you and Lisa have taken second place and will receive a $3000 cash prize. As you were the one who put down his name and address on the registration form, you alone received the cash prize. You consider telling your dance partner, Lisa. She clearly didn’t know how much cash you might win. She mentioned that she has always wanted to go to college, but can’t financially afford to do so. Her dream is to become a physiotherapist but she is struggling to save enough money to pay for the program’s tuition. On the other hand, your kitchen is quite old, and you would like to buy a new one. How much do you give to your dance partner? Please indicate the amount to give to your dance partner, on a continuous scale of a minimum of $0 to a maximum of $3000. Please answer only with a number between 0 and 3000. You set the slider to 1503. Scenario (Common_Goods): You are a bank employee. Because your bank was really successful last year, each employee gets a one-time bonus of $1000. In addition, they announced that there is the option to donate a fraction of this bonus to a charity. Moreover, the bank will match each employee’s donation, thereby doubling the money given to the charity. The charity provides bed nets to prevent malaria in sub-Saharan Africa. You looked up the charity. As malaria is transferred by mosquito bites, the charity provides long-lasting insecticide-treated bed nets to protect people from getting bitten by mosquitoes in the first place. Each bed net costs about $2.00, lasts for 3 to 4 years, and protects two people on average. Furthermore, you think the charity is trustworthy. How much of the $1000 bonus do you donate to the charity? Please indicate the amount to donate to the charity, on a continuous scale of a minimum of $0 to a maximum of $1000. Please answer only with a number between 0 and 1000. You set the slider to 301. Scenario (Drought): Your community is experiencing a moderate drought, and the government has asked residents to reduce their water usage to ensure there is enough for everyone - specifically, it recommends halving the amount of water used for gardening. However, the government is not actively controlling or monitoring water use. You have a beautiful garden that requires a lot of water. The lawn is very well maintained, and you have many beautiful flowers and a few trees that provide delicious peaches and cherries. If you keep watering the garden as usual, all plants may survive, but they may wilt, and the fruits may be puny. If you water them less, the trees may be unable to grow any fruits this year. If you reduce their water to -50% or less, most plants could even die under substantial water stress. Your kids love spending time in your garden, and you love showing off your garden to your neighbors and hosting parties in it. You could either continue watering your garden, which would enhance the aesthetics of your property and bear large delicious fruits, but risk exacerbating the water shortage, or you could reduce your water usage to help preserve the community's limited water supply. The weather is getting hotter, so your precious plants may need about twice as much water to flourish than they have been getting so far. Since it has not been raining for weeks, rainwater collection is not an option, and there is also no system to use gray water to water your garden. How much water do you use for your garden during the drought compared to what you are doing currently? Please indicate how much water you would use for your garden from a continuous scale of -100% to 100%, where -100% indicates that you stop watering the garden entirely, 0 indicates that you continue to water the garden as much as before, and 100% indicates that you water the garden twice as often as before. Please answer only with a number between -100 and 100. You set the slider to -68. Scenario (Note_Taking): You are a student who takes excellent notes during class. If you keep your notes to yourself, you will get the highest grade in the class and therefore receive a $500 discount on next year's tuition fee (reducing the fee from $2000 to $1500). You overheard a few classmates commenting how good your notes are and how happy they would be if they would have access to them, but no one dared to ask for them. If you share your notes with your classmates, most of them may learn more and achieve higher grades thanks to your notes. Even a few students who would otherwise fail the exam, could pass thanks to your good notes. However, this also means that someone else may get even better grades than you and therefore receive the discount instead of you. Do you share your notes with your classmates? Please indicate how willing you are to share your notes with your classmates on a scale of 0 to 5, where 0 indicates definitely not, 3 indicates unsure, and 5 indicates definitely yes. Please answer only with a number between 0 and 5. You set the slider to 1. Dilemma (Ladder): It is 1987 and you are on a ferry from Belgium to England. Suddenly, the ferry starts tilting and water begins to pour in. You and some other passengers are trying to get to the deck by a rope ladder. You climb up the ladder and stand on the deck, looking at the other passengers making their way up behind you. Directly below you, a man who is midway up the ladder seems frozen into immobility by fear or cold and is blocking the way. You try to speak and reach to him, but he does not react. People behind you are jostling. The ship seems to be sinking fast and the man is still blocking the ladder. From the crowd below, someone shouts that you should push the man off. If you push the man off the ladder, he will probably die, but the other people will be able to climb on deck. If you do not push the man off the ladder, he will probably continue blocking the way so that many of the people behind you will not be able to get on deck and therefore will drown. Do you push the man off the ladder? Please answer only with yes or no. You press Y. Dilemma (Tyran): You are an army officer in the Dominican Republic in 1961. For decades now, your country has been governed by a dictator who has had thousands of people killed because of their ethnicity or their political views. You are a member of a group that wants a new start for your country without a civil war. To achieve this goal, you consider it unavoidable to kill the dictator in order to disempower him, his family, and his followers. For this, you and six co-conspirators are asked to ambush and kill the dictator. If you kill the dictator, the people in your country will have a chance for a new start. If you do not kill him, the regime will continue its killings. Do you kill the dictator? Please answer only with yes or no. You press Y. Dilemma (Lifeboat): It is 1841, and you are the captain of a ship. After colliding with an iceberg, the ship sank. You, some other crew members, and passengers were able to save yourselves onto two lifeboats. The next day, it starts to rain. You navigate while the others scoop out the water and oar. Someone from the other lifeboat calls to you, and you say that they are overcrowded with 41 people. The large waves could make their lifeboat sink at any time and the passengers ask you what to do as you are the highest ranking officer. You think that approximately 10 adults would have to leave the boat to substantially increase the chance that the other passengers will survive. However, your lifeboat is already at full capacity and you cannot accept any more passengers. The 10 passengers would have to go overboard, and you know that the water is so cold that they would freeze to death. If you tell them not to throw anybody overboard, the boat will most likely sink at some point within the next few days and all 41 people will die. If you instead tell them to throw 10 people overboard, the rest of the passengers will be able to keep rowing towards land and will have a good chance of being rescued. Do you ask them to throw 10 people overboard? Please answer only with yes or no. You press Y. Dilemma (Suicide): You are a member of the Canadian Senate and have to vote on a new law. Canada’s healthcare expenses are increasing steadily, mainly because the proportion of old people is increasing. Recent studies indicate that the legalization of medically assisted suicide would strongly decrease medical expenses. The state considers legalizing medically assisted suicide for the patients who are terminally ill and whose life expectancy is 6 months or less. If they wish, these patients could then choose to be killed by a doctor with an overdose of tranquillizers. Experts estimate that the new law would save 139 million dollars. This money could be used for other medical treatments. If you allow medically assisted suicide, many more people will get better medical treatment improving their quality of life and life expectancy. However, you would legalize a form of killing. If you do not allow medically assisted suicide, the healthcare system will have to keep on saving money in every possible way. Do you change the law and make medically assisted suicide legal? Please answer only with yes or no. You press Y. Dilemma (Medicine): You are the Belgian minister of health. A 7-year-old boy who suffers from a very rare immunological disease asks you for a refund for his medicine. Without taking this medicine regularly, the boy will die. Currently, the medicine is not covered by the health care system, and it is so expensive that the family, who has been buying the medicine for years, will not be able to raise enough money for much longer (approximately 200,000 euros per year). Thus, the boy will die if you refuse the refund. If you refuse to pay for the medicine, the boy will probably die within the next few years. However, if you pay, this money will have to be saved elsewhere; thus, the quality of care for other patients would be reduced, increasing the risk that their treatments will be less effective. An independent agency estimated that funding the boy will therefore likely lead to the death of more people. Do you refund the boy’s medicine? Please answer only with yes or no. You press N. Dilemma (RAF): You are the German head of state in 1977 and a far-left terrorist group kidnapped the President of the Federation of the German Employers’ Association. In exchange for his life, the terrorist group demands the release of 11 imprisoned members. A few years ago, the terrorist group had successfully obtained the release of 5 members by another kidnapping. 4 of the released members have since then committed further terrorist attacks that killed several innocent people. If you consent to the terrorist group’s demands, 11 former terrorists will be released and will likely commit further terrorist attacks killing several people. If you refuse, the hostage will be killed. Do you release the former terrorists? Please answer only with yes or no. You press N. Dilemma (Rwanda): You are a project leader in a development aid team in Rwanda. All over the country, members of the ethnic group of Hutu attack members of the Tutsi group. The family of one of your Tutsi employees is hiding in your agency’s center. Another employee arrives and tells you that he just witnessed his whole family and another of your employees being killed by an armed group simply for being Tutsis. Shortly afterward, a crowd of armed people gathers at the center. The leader threatens to attack the building and kill all the Tutsi civilians inside unless you hand over your employee who just arrived. If you protect your employee, the mob will probably attack the center and kill many people. If you let the mob take your employee, he will be killed but the other people under your care will be safe. Do you protect your employee? Please answer only with yes or no. You press N. Dilemma (Ransom): You are the German head of state and 10 of your citizens are taken hostage in the Sahara Desert by a group of terrorists. You have to decide whether to pay a ransom of several million euros. Other measures to free the hostages have been tried but did not succeed; by now, it has been going on for half a year and one woman has died. Similar situations in the past have shown that terrorists use the money to buy weapons and kill even more people. Furthermore, future tourists will more likely be kidnapped if you pay. If you pay the ransom, you can save the hostages. If you do not pay, the 10 hostages will be killed, but you will prevent future terrorism being financed, which will probably save more than 10 lives. Do you pay the ransom? Please answer only with yes or no. You press Y. Dilemma (Tax): You are the state minister of finance of a German federal state. You have to decide whether or not to buy data that was obtained illegally. You have been given an offer to buy data about tax evaders. The data was compiled by hackers who have illegally invaded the servers of several foreign banks, where some of your wealthy citizens transfer their money to evade taxes. The data is probably reliable and consists of up to 50,000 transactions. If you buy the data, you will use illegally obtained information and reward hackers for data theft, which might promote cybercrime in the future - but you will be able to prosecute tax evaders, whose fines will amount to millions. If you do not buy the data, you will miss out on millions of euros of additional tax payments and fines. Do youbuy the data? Please answer only with yes or no. You press Y. Dilemma (Operations): You are a physician in a hospital in a town in a rural area. Your hospital needs to perform at least 50 lung surgeries per year in order to have them refunded by health insurance. Until this December, the hospital has only conducted 48 surgeries; thus, maybe none of the surgeries will be covered by insurance. The clinic has already been in debt the previous year and will have to file for bankruptcy if there will be another debt when the lung surgeries are not refunded. Without this hospital, the region will not have access to fast intensive emergency care. Sooner or later, having no fast access to intensive emergency care will lead to avoidable deaths in the region. The only way of sustaining the clinic would be by performing two lung surgeries that are medically not indicated. If you perform two unnecessary surgeries, you can save the hospital from bankruptcy and the region will continue to have fast emergency care, but you will perform surgery on two patients for whom other treatments would be more appropriate. If you do not perform the surgeries, the clinic will be closed leading to deaths sooner or later. Do you perform the two surgeries that are not medically indicated? Please answer only with yes or no. You press Y. Dilemma (Vet): You are a European veterinarian conducting animal experiments on highly contagious germs. On a daily basis, you cause suffering to animals by infecting them with deadly diseases. You are aware that this work provides important insights that, in the long run, can save many other animals. Furthermore, you are a specialist and because of your expertise you can achieve more than other veterinarians. However, you have become a vet in order to heal animals and you are considering quitting your job. If you quit your job, fewer animals will die in the lab, but there will also be fewer findings about animal diseases. As a result, the development of medicines for healing sick animals will be delayed, leading to many preventable deaths. If you continue your job of infecting animals with diseases for medical research, you will keep causing animal suffering, but your research will probably save the lives of even more animals in the future. Do you quit your job? Please answer only with yes or no. You press N. Dilemma (Endow): After a long-term imprisonment, a convicted child kidnapper and murderer wants to organise a foundation for young victims of crime. You are the president of the agency who decides about the foundation. The founder would donate 25,000 euros to the foundation. However, the foundation would be named after the offender, and it is unclear whether he is acting out of remorse or simply to clear his name. The family of his victim opposes such a foundation and considers it immoral, derogatory, and an exploitation of the victim. If you approve and accept the money for the foundation, crime victims will benefit from the 25,000 euros and additional fund raising. If you oppose the foundation, the money will be lost. Do you approve the foundation? Please answer only with yes or no. You press Y. Dilemma (RobinHood): You are an employee of a bank in Germany. You have many loyal customers who trust you and your bank. Among them are some customers who are in debt and do not fulfill the conditions for being granted a loan. Interviews with them are very stressful for you and for the customers. Currently, there is one very severe case where a customer and his family would be ruined without a loan. You see an undetectable way to temporarily redirect some deposits and investments from other customers to his bank account, which would improve his credit rating sufficiently. This way you could help him as well as other customers in difficulties. At the same time, you would abuse the trust of the wealthy customers and of your superiors and commit fraud. If you temporarily redirect a part of the wealthy customers’ money, you will be able to help indebted customers but abuse trust. If you follow the rules, your indebted customers and their families will go bankrupt. Do you redirect money from wealthy customers? Please answer only with yes or no. You press Y.
In this study, you will be shown a moral scenario and you must make a decision about what to do. Read the scenario carefully and choose what you would do. There is no objectively right or wrong answer; you may choose whichever option you think is best. You answer by pressing Y for yes or N for no. Dilemma (Vet, yesno framing): You are a European veterinarian conducting animal experiments on highly contagious germs. On a daily basis, you cause suffering to animals by infecting them with deadly diseases. You are aware that this work provides important insights that, in the long run, can save many other animals. Furthermore, you are a specialist and because of your expertise you can achieve more than other veterinarians. However, you have become a vet in order to heal animals and you are considering quitting your job. If you quit your job, fewer animals will die in the lab, but there will also be fewer findings about animal diseases. As a result, the development of medicines for healing sick animals will be delayed, leading to many preventable deaths. If you continue your job of infecting animals with diseases for medical research, you will keep causing animal suffering, but your research will probably save the lives of even more animals in the future. Do you continue working at your job? You press N.
In this study, you will be shown an everyday scenario where some dilemma or problem occurs, and you must make a decision about what to do. Read the scenario carefully and choose what you would do. There is no objectively right or wrong answer; you may choose whichever option you think is best. You answer by pressing Y for yes or N for no. Dilemma (Pregnant, omission framing): Your wife is currently 8 months pregnant. You’ve taken care of her and did most of her chores for the past month. You feel like you're doing all the chores in the house and helping her with cravings, and you really stepped up your game. Your friends have this monthly game night. It’s always the first Friday of the month, and it’s tonight, you’re there right now with them. You didn’t go to the last one because you wanted to focus on your wife. You really hate to say it, but you think you’re getting tired of caring for her. If you could go on without a break, you would, but you’ve just had a lot of stress, due to the pregnancy and your dad being sick. You just think getting drunk with your friends would help out. It’s just going to be a day. Well, less than a day. Probably 8 hours. But your wife just called and really wants you home, even though everything is fine. You 100% love her with every ounce of your being, but you think she is overbearing now. You get that her hormones are intense right now, and that you really need to be there for her, and you need to excuse some of these traits, but you just want a day off. Your wife has said some compelling arguments, like, “You don’t get days off when you have a kid.” You can’t argue with that, but you think once he’s born, you’ll probably change. Do you leave the game night to be with your wife? You press Y.