https://orcid.org/0000-0002-3246-9984
Figures
Abstract
The Holt and Laury [1] task is arguably the most widely employed method for eliciting individual risk attitudes in economics. However, although around 15% of standard experimental subjects fail to complete the task consistently, this fraction increases to an alarming 60-70% in the case of children, teenagers, and other non-standard subject pools. This paper explores extensively a visual version of the Holt and Laury task, where subjects choose six times between two gumball machines. Using a sample of 4,972 adolescents, we find that the Gumball Machine task decreases the presence of inconsistencies substantially to 20%. Cognitive skills appear to be a strong predictor of consistency and risk preferences, but gender differences are not found.
Citation: Vasco M, Francisco MJV-D (2025) Holt-Laury as a gumball machine. PLoS ONE 20(4): e0320576. https://doi.org/10.1371/journal.pone.0320576
Editor: Nicolas Jacquemet, Paris School of Economics, FRANCE
Received: November 1, 2024; Accepted: February 20, 2025; Published: April 10, 2025
Copyright: © 2025 Vasco and Francisco. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The dataset can be found in Zenodo (https://zenodo.org/records/13720112).
Funding: This research was supported by the Spanish Minirrstry of Economy and Competitiveness (PID2021-126892NB-I00), the Spanish Ministry of Education (FPU20/05846), the Andalusian Agency for International Development Cooperation (AACID-0I008/2020), and Grant B-SEJ-280-UGR20 funded by the Department of University, Research and Innovation of the Junta de Andalucía and by “ERDF A way of making Europe”. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
Since risk aversion is a fundamental concept in decision-making under uncertainty, it has received considerable attention in economics at least since Cramer [2] and Bernoulli [3]. The seminal papers of Pratt [4] and Arrow [5] provide influential definitions and frameworks to quantify and measure risk aversion. Attitudes toward risk have become more prominent empirically since the 80’s and 90’s with the development of experimental techniques allowing for a simple and systematic elicitation of individual risk attitudes [6]. Nevertheless, risk-aversion elicitation generates certain methodological issues [7].
Arguably, the most commonly employed experimental method to elicit individual risk preferences is the multiple price list (MPL, hereafter) by Holt and Laury [1], where ten pairwise choices between lotteries with fixed payoffs but changing associated probabilities are presented to subjects. The distinction between different levels of risk-seeking behavior is an advantage of the Holt-Laury task (HL, hereafter) over other tasks, such as Gneezy and Potters [8] and Eckel and Grossman [9].
The main issue with the HL is related to the complexity of the task and the cognitive demands required, and therefore to the cognitive abilities of the subjects [10]. Indeed, one particular problem is that non-standard subject pools (e.g. children, teenagers, people with low levels of education among others) struggle to fill the task consistently: while most of the experiments with standard experimental subject pools (e.g. adult volunteers, university students) report that around 15% fail to complete the HL task consistently [1,11–14], this number becomes alarming with non-standard participants [15–17]. Since inconsistent subjects might systematically differ in terms of cognitive abilities, literacy, and other characteristics, the common practice of discarding their answers from the data might biashe elicited distributions of risk preferences. In addition, power is reduced at higher inconsistency rates, as inconsistent subjects are excluded [1,7,14,18–20]. In line with these arguments, Charness et al. [21] document that both the complexity and the structure of the employed mechanism affect the elicited risk preferences (see also Andersson et al. [22]). Moreover, Amador-Hidalgo et al. [23] show that lower-ability subjects are more likely to be inconsistent. Indeed, there is considerable evidence that individuals with lower cognitive abilities tend to exhibit inconsistent decision-making behavior in risk-taking tasks [24,25]. Although intuitive tasks, such as BART (Balloon Analogue Risk Task) proposed by Lejuez et al. [26], have been developed from a very different approach taken by psychologists, they do not provide comparable data and the task is slightly difficult to implement in the field [7]. Besides, Crosetto and Filippin [27] proposed BRET (Bomb Risk Elicitation Task) as an alternative measure of individual risk preferences. However, it seems to generate results that are not comparable with HL [28].
Given that many human preferences and behaviors emerge early in life, comprehending the behavior of children and adolescents is essential for understanding adult behavior. Consequently, numerous experimental studies focusing on children and adolescents have developed (e.g., [20,29,30]). Regarding risk attitudes, Sutter, Zoller, and Glätzle-Rützler [31] suggests that the results of the studies analyzed differ substantially (e.g. Sutter et al. [32–34]). Moreover, Defoe et al. [35] demonstrate that whether adolescents exhibit riskier decision-making than adults largely depends on the task used to assess these preferences. Experimental economists thus need a task that elicits children’s risk attitudes reliably and consistently.
In this paper, we explore exhaustively a new risk-elicitation task, the Gumball Machine introduced by Alfonso et al. [36] (gumball, henceforth). This task is a variation of the HL task, modifying two main features: it is shorter and, more importantly, visual. In Alfonso et al. [36], three waves of adolescents are used to discuss different experimental designs and methods of data collection in schools. Regarding individual risk attitudes elicitation, in its first wave HL is used (715 adolescents), while in its third wave, gumball is introduced with 959 observations. We elicit now the risk preferences in a sample of 4,972 adolescents using the gumball task in order to: i) compare the gumball outcomes in terms of inconsistencies and risk-taking types (averse, neutral, lover) with other adolescents who performed the original HL, ii) explore the determinants of consistency and, iii) explore potential order effects – in terms of the position of the task within the experiment and the position of the balls within the machine (design).
We find that the gumball increases the consistency among adolescents to nearly 80%, a number comparable with standard experimental subjects but dramatically higher compared to the results of adolescents using the original HL list format. The comparative analysis was carried out with a sample of 715 adolescents included in the first wave of Alfonso et al. [36], using HL plus an eleventh decision (see Table A2 in S1 Appendix). We also observe that the distribution of risk attitudes slightly shifts towards risk neutrality, a result consistent with Bosch-Domènech and Silvestre [37], who analyze reduced versions of HL, and Estepa-Mohedano and Espinosa [19], who added a different visual element to the task (such as bills, beans, and pie chart). Concerning individual heterogeneity, we find no gender effects, but we observe that older teenagers and those with higher cognitive abilities are more likely to exhibit consistent behavior. For consistent subjects, we also find a positive correlation between cognitive abilities and risk aversion. In addition, we find no order effects. Hence, we support the use of the gumball task for eliciting individual risk preferences among adolescents.
The paper is organized as follows. The next section introduces the gumball task. Section 3 outlines the procedures of the experiment. Section 4 presents the results and the last section concludes.
2 The Gumball Machine task
We explore a new elicitation task of attitudes toward risk based on a gumball machine, which retains the core features of HL. In this new task, the classic list of lottery pairs is replaced by pairs of gumball machine figures, where the payoffs are displayed as balls. However, unlike the original HL, this version is shorter. Each subject is asked to make six decisions between options A and B, where option A is safer. In addition, it includes an initial control question, which contains a dominated option. Table 1 provides a summary of the payoffs and the expected values of each decision, as well as the coefficient of relative risk aversion that would make a subject indifferent between A and B for the choice problem, considering that subjects follow a CRRA utility function with the form: .
Fig 1 displays a screen of a decision presented to the subject, which corresponds to decision #3 in Table 1. In every decision, each machine contains five balls with the same probability of coming out (1/5). The six decisions are presented to the subjects one by one on different screens, but in the same order (without randomization). Both options are also fixed, A on the left and B on the right. The complete task is included in S3 Appendix, where Fig A1 displays the original version. In order to test the possible effects of the distribution of the balls within the machine, we ran some sessions in which the balls were placed in different positions (see Fig A2 in S3 Appendix).
Decision-makers can be classified as either consistent or inconsistent with their decisions. We assume for a consistent subject to start by choosing option A and finish by choosing option B, thus switching from A to B only once. In contrast, following the literature, we consider a subject to exhibit a inconsistent behavior in this task if she switches back from B to A, selects option B in the first decision (#1), or selects option A in the sixth decision (#6) (both decision problems under certainty) [10,21]. Indeed, we do not measure the number of total inconsistencies. Instead, we use it to nested classify subjects considered as inconsistent. We propose a classification for individuals who appear to exhibit inconsistent behavior:
- Lack of Understanding (LoU): subjects who select B in the first decision (see screen 1 in Fig A1 and A2 in S3 Appendix). Although there is no uncertainty in this decision, the option with the lowest expected value is chosen. If a subject chooses a dominated option in the first decision, we can assume that she started the task without fully understanding it. Therefore, they are considered as inconsistent subjects, regardless of their subsequent decisions.
- Switch Back (SB): those who do not satisfy LoU but switch back from B to A in any of the following decisions. That is, once they move to option B, they return to option A.
- Lack of Attention (LoA): those who neither satisfy LoU nor switch back along the decisions but select A in the sixth decision (see screen 6 in Fig A1 and Fig A2 in S3 Appendix). As in the first type - LoU, this decision is under certainty and option A is dominated. Nevertheless, we consider that the subjects may have lost their concentration (or attention) at this stage of the task since they were displaying consistent behavior in the previous decisions.
Risk attitudes are determined by how many safe choices are made [1]. It must be considered that decision number S (Fig 1) is critical because the expected values of decisions A and B are fairly similar in both cases ( = 8.8 and
= 9.2) and require substantial computation. If a subject decides to select A in this decision, she would be classified as risk-averse, while if she selects B, she would be classified as risk-neutral, as long as she had chosen A in the previous decision. Thus, consistent subjects can be classified into three risk levels based on their switch point:
- Risk-averse individuals: select A three times or more: AAABBB, AAAABB or AAAAAB.
- Risk-neutral individuals: select A two times: AABBBB.
- Risk-loving individuals: select A once: ABBBBB.
This specification implies risk neutrality (risk-prone people included) for I, risk aversion for C, and risk seeking for R. However, these coefficients differ from the original HL, where the risk-neutral range is − 0 . 15 > r > 0 . 15 (see S2 Appendix).
3 Experimental Procedures
The sample was collected between October 1, 2021, and January 27, 2023, as part of a larger project involving adolescents [38]. The entire experiment was approved by the Ethical Committee of Loyola University Andalusia (No. 20211216, 20200709 and 20190318) and parental consents were obtained in conformity with regulations protecting minors (see Vasco et al. [39] for more details on data collection). We implemented the task using a sample of 4,972 adolescents aged 11 to 17 years, from 22 high schools in Spain. Of these, 4,787 completed the risk task, and only 4,694 completed the entire survey. A detailed summary of the sample is provided in S3 Appendix. All participants completed the experiment in class using a tailored platform to run online web-based experiments. It was conducted in supervised sessions included in their regular scholar schedule, minimizing self-selection effects [31,32]. In experiments involving children and adolescents, attrition must be considered, as opposed to those conducted in the laboratory. They might be too young to pay attention and complete all the tasks [20].
The entire experiment consisted of the elicitation of class networks, time and risk preferences, and cognitive abilities. In order to check potential order effects, we ran additional sessions with a different order (networks-risk-time-abilities). The risk task, as previously mentioned and central to this paper, involved six sequential decisions presented one at a time on separate screens, always in the same predetermined order. Subjects completed the tasks using their mobile phones or tablets. In addition, hypothetical payments are provided as incentives. Alfonso et al. [36] test the impact of mobile phone use among adolescents while Brañas-Garza et al. [40] compare results using real and hypothetical incentives including a reduced version of HL. They find no significant differences, as well as, Kühberger, Schulte-Mecklenbeck, and Perner [41] report that both real and hypothetical decisions lead to similar choice patterns. Additionally, Chuang and Schechter [42] present results from risk studies using hypothetical incentives (see also Read [43] Beattie and Loomes [44] and Horn and Freund [45]).
4 Results
4.1 Types of inconsistency
Fig 2 shows the number and fractions of observations that are inconsistent following our classification and those that are consistent. Our task achieves a higher consistency ratio, reaching values close to those typically observed in adult samples [1,11–14]. This result contrasts previous studies using the list format with children and adolescents, where the consistency ratio was notably lower (e.g., Alfonso et al. [36] reported only 34.83% consistency, see S1 Appendix for further details).
Not only it is observed a high percentage of consistent responses, but also, the level of attention is high, almost all of those who show consistent behavior up to decision 5 maintain their level of attention (99.30%). In addition, by decreasing the cases of LoU, we can confirm the advantage of using a visual task in terms of understanding among young people. The effect is combined with the advantage of a shorter task for remaining inconsistencies types. Hence, the gumball task greatly reduces all types of inconsistency.
Determinants of consistent behavior. We also analyze the potential determinants of consistent behavior. Table 2 presents the estimated results of the regression models (Logit) in consistency, which takes a value of 1 if the subject behaves consistently or 0 if the subject is classified in any of the three categories of inconsistent behavior (LoU, SB or LoA). The models include school fixed effects, as well as controlling for female, age, and several cognitive skills items that are explained below.
Firstly, based on the value of pseudo , we find that model (3) improves by including all the tasks to measure cognitive skills (r=0.3, P-value < 0.001). Secondly, a positive effect is observed between consistency and the following cognitive skills of the subjects: i) in line with Amador-Hidalgo et al. [23], a more reflective behavior measured by the adapted cognitive reflection test (CRT) from Thomson and Oppenheimer [46], increases the consistent behavior; ii) students with better financial skills (finance; which covers three exercises involving mathematical and financial calculations) are more likely to exhibit consistent behavior; iii) and adolescents with higher academic grades (Grade Point Average; GPA) tend to be consistent in the gumball task. Therefore, cognitive skills are good predictors of consistency. We also observe that consistency seems to improve with age, but we do not find robust differences between genders.
4.2 Risk profiles
Fig 3 displays the fractions of risk-averse, risk-neutral, and risk-loving individuals in the sample. The dark color represents consistent subjects.
In line with Bosch-Domènech and Silvestre [37], we observe that the fraction of risk-averse adolescents decreases, shifting towards risk neutrality. If we focus only on consistent subjects, 35.99% show risk-neutral behavior, which is considerably higher than in the original version of the HL task for teenagers. Indeed, the data of AC-HL documents less than 4% of consistent subjects classified as neutral (see Fig A1 in S5 Appendix, for further details). In contrast, the percentage of risk-loving individuals remains similarly distributed. Furthermore, Fig A2 in S5 Appendix, plots the entire distribution of safe choices for both our data and AC-HL sample (transformed into six decisions).
Finally, Table 3 explains the potential determinants of risk aversion only for consistent subjects behavior using the gumball task. We perform an OLS regression analysis to estimate the relationship between the number of safe choices (option A) in the six decisions of the gumball task. As in the previous models (1), we also control for female, age, and the cognitive skills variables.
In line with Charness et al. [21], gender does not explain risk aversion. However, it seems to change with age. Older adolescents tend to be more risk-averse. In addition, our results suggest a positive correlation between the number of safe choices and the skills of the subjects which is supported by the literature [23,47]. We observe that i) those with more reflective behaviour (CRT) tend to show more risk aversion; ii) students with better financial skills (finance) are more likely to choose safer lotteries; iii) and adolescents with higher academic scores (GPA) tend to make less risky choices.
According to Charness, Gneezy, and Imas [7], it is common to find cases where subjects switch options several times, therefore models that add these errors as a stochastic component are often reported. Because the gumball task is shorter we also verify how the effect is using this task as opposed to the original version. We applied a maximum likelihood (ML) structural estimation with the Luce error specification [48–50]. We added to our sample, 715 observations from AC-HL sample that represent adolescents who completed the HL task using a classical list format. Therefore, we created a dummy to estimate the effect of the gumball task on the risk aversion coefficient assuming a CRRA utility function. Table 4 reports the results of this model, revealing the gumball task decreases the risk aversion coefficient compared to HL.
4.3 Order effects
We test order effects. Two types are considered: i) the order in which the task appears in the protocol (order); and ii) the position of the balls within the machine (design). We do not observe a significant impact on either consistency nor on the level of risk aversion in previous results (Tables 1 and 3). In the same vein, there is no significant effect due to variations in the order of implementation of tasks, which would confirm the robustness of the gumball task to elicit risk preferences among adolescents. See S6 Appendix, for the results of the estimation of effects on the maximum likelihood estimation of the risk aversion coefficient assuming a CRRA utility function.
5 Conclusions
Risk aversion has been measured traditionally as one of the classical variables of economic preferences in the field of experimental and behavioral economics. Although adult populations have been the standard experimental subjects involved in such studies, there is a growing interest in the economic behavior of children and adolescents. This is due to the interest in predicting, explaining and, sometimes, preventing future events related to educational performance, health, or disruptive behaviors. The widely used MPL of Holt and Laury [1] to elicit individual risk preferences, presents some limitations when administered to different ability groups to standard subjects, such as children or rural populations. These groups may have difficulties due to the high computational demands, making it difficult to perform the task consistently. Consequently, high levels of inconsistent behavior are observed, which prevents drawing clear findings in such samples.
This paper contributes to the literature on risk measurement by addressing key limitations of traditional methods when applied to non-standard populations like adolescents. A short and visual method to measure risk aversion especially thought for teenagers is explored: the Gumball Machine task. Using this task, we elicited risk preferences from a large sample of adolescents in Spanish high schools. Our results show that the inconsistency level decreases greatly from 65.17% to 20.45%. Particularly, less dominated choices are observed in both the first and the sixth decisions. Visual support therefore seems to help students to understand the task better since with the first decision, regardless of the task length, the reduction of LoU behaviors is remarkable (from 20.98% to 10.28%). Likewise, the number of switch backs is reduced, also as might be expected due to the shorter task (from 39.86% to 9.61%).
Furthermore, our findings reveal a significant relationship not only between cognitive abilities and consistency but also between cognitive abilities and risk preferences. In line with the existing literature, which suggests that simpler tasks tend to promote risk neutrality, we observe a similar trend with the gumball task (e.g. Bosch-Domènech and Silvestre [37]). Additionally, as stated by Defoe et al. [35], preferences can vary depending on the task used to elicit risk attitudes. In our study, the list format in the HL task appears to lead to more averse decisions among adolescents compared to the risk attitudes observed with the gumball task. Hence, our analysis indirectly supports the notion that the original HL task might systematically skew the distribution of elicited risk attitudes. This raises concerns that certain risk attitudes might be underrepresented when inconsistent subjects are excluded from the analysis. This issue could be potentially mitigated by using the gumball task.
Although this study provides valuable insights and highlights the importance of designing adaptive tools that recognize cognitive diversity and varying ability levels to elicit risk preferences, one issue has not been yet completely disentangled and deserves further investigation: the individual effect of shortening the task versus displaying it visually. Our results reveal already for the first decision a significant decrease in the number of individuals who do not understand the task (LoU), along with an increase in consistent behavior across subjects. This suggests a direct impact of our visual tool. However, when considering the task as a whole, we cannot conclusively attribute the overall reduction in inconsistent behavior to only one of these features - the shortening of the task also plays a role. Future research should explore these two factors separately to better understand their distinct contributions. Additionally, extending this research to other non-standard populations, such as younger children or individuals with diverse cognitive abilities, could offer new insights. Finally, this task offers a promising approach to a gender-neutral method for eliciting risk preferences.
Supporting information
S1 Appendix. Samples of adults vs. adolescents (PDF)
S2 Appendix. Features of the original HL task (PDF)
S3 Appendix. Additional details of the gumball task (PDF)
S4 Appendix. Type of inconsistencies (PDF)
S5 Appendix. Versions with different decision number (PDF)
S6 Appendix. Additional estimation of order effects (PDF)
Acknowledgments
We sincerely thank Pablo Brañas and Jaromír Kovářík, for their invaluable guidance and support. Their expertise and dedication were instrumental in the success of this research. We also thank Antonio Alfonso for helping us with the development of the task, Pablo Lomas for leading the field team in high schools and Diego Jorrat and César Mantilla for their valuable feedback for the improvement of the paper.
References
- 1. Holt CA, Laury SK. Risk aversion and incentive effects. American Economic Review. 2002;92(5):1644–55.
- 2. Cramer G. Letter to N. Bernoulli. Specimen Theoriae Novae de Mensura Sortis Commentarii Academiae Scientarum Imperialis Petropolitane. Econometrica. 1728;22(1):175–92.
- 3. Bernoulli D. Exposition of a new theory on the measurement of risk. Econometrica 1954;22(1):23.
- 4. Pratt J. Risk aversion in the small and in the large. Econometrica. 1964;32:122–36.
- 5. Arrow K. Aspects of the theory of risk-bearing. Helsinki: Yrj¨o Jahnsonian S¨atïo; 1965.
- 6. Falk A, Heckman JJ. Lab experiments are a major source of knowledge in the social sciences. Science 2009;326(5952):535–8.
- 7. Charness G, Gneezy U, Imas A. Experimental methods: Eliciting risk preferences. J Econ Behav Org. 2013;87:43–51.
- 8. Gneezy U, Potters J. An experiment on risk taking and evaluation periods. Quart J Econ 1997;112(2):631–45.
- 9. Eckel CC, Grossman PJ. Sex differences and statistical stereotyping in attitudes toward financial risk. Evol Hum Behav 2002;23(4):281–95.
- 10. Bruns S, Hermann D, Mußhoff O. Investigating inconsistencies in complex lotteries: The role of cognitive skills of low-numeracy subjects. J Behav Exp Econ. 2022;97:101840.
- 11. Holt CA, Laury SK. Risk aversion and incentive effects: New data without order effects. Am Econ Rev 2005;95(3):902–4.
- 12. Dave C, Eckel CC, Johnson CA, Rojas C. Eliciting risk preferences: When is simple better? J Risk Uncertain. 2010;41(3):219–43.
- 13. Harrison G, Lau M, Rutström E. Estimating risk attitudes in Denmark: A field experiment. Scand J Econ. 2007;109(2):341–68.
- 14. Engel C, Kirchkamp O. How to deal with inconsistent choices on multiple price lists. J Econ Behav Org. 2019;160:138–57.
- 15. von Gaudecker H-M, van Soest A, WengstrÖm E. Heterogeneity in risky choice behavior in a broad population. Am Econ Rev 2011;101(2):664–94.
- 16. Charness G, Viceisza A. Three risk-elicitation methods in the field: Evidence from rural Senegal. RBE 2016;3(2):145–71.
- 17. Jacobson S, Petrie R. Learning from mistakes: What do inconsistent choices over risk tell us? J Risk Uncertain. 2009;38(2):143–58.
- 18. Hirschauer N, Musshoff O, Maart-Noelck SC, Gruener S. Eliciting risk attitudes – How to avoid mean and variance bias in Holt-and-Laury lotteries. Appl Econ Lett 2013;21(1):35–8.
- 19. Estepa-Mohedano L, Espinosa MP. Comparing risk elicitation in lotteries with visual or contextual aids. J Behav Exp Econ. 2023;103:101974.
- 20. List JA, Petrie R, Samek A. How experiments with children inform economics. J Econ Lit 2023;61(2):504–64.
- 21. Charness G, Eckel C, Gneezy U, Kajackaite A. Complexity in risk elicitation may affect the conclusions: A demonstration using gender differences. J Risk Uncertain 2018;56(1):1–17.
- 22. Andersson O, Holm HJ, Tyran J-R, Wengström E. Risk aversion relates to cognitive ability: Preferences or noise? J Eur Econ Assoc. 2016;14(5):1129–54.
- 23. Amador-Hidalgo L, Brañas-Garza P, Espín AM, García-Muñoz T, Hernández-Román A. Cognitive abilities and risk-taking: Errors, not preferences. Eur Econ Rev. 2021;134:103694.
- 24. Burks SV, Carpenter JP, Goette L, Rustichini A. Cognitive skills affect economic preferences, strategic behavior, and job attachment. Proc Natl Acad Sci U S A 2009;106(19):7745–50.
- 25. Dohmen T, Falk A, Huffman D, Sunde U. On the relationship between cognitive ability and risk preference. J Econ Perspect 2018;32(2):115–34.
- 26. Lejuez CW, Read JP, Kahler CW, Richards JB, Ramsey SE, Stuart GL, et al. Evaluation of a behavioral measure of risk taking: The Balloon Analogue Risk Task (BART). J Exp Psychol Appl 2002;8(2):75–84.
- 27. Crosetto P, Filippin A. The “bomb” risk elicitation task. J Risk Uncertain 2013;47(1):31–65.
- 28. Alexy M, Georgantzis N, K´acer M, P´eliov´a J. Risk attitude elicitation methods: do they tell similar stories? J Econ. 2016;64(9):847–77.
- 29. Czermak S, Feri F, Glätzle-Rützler D, Sutter M. How strategic are children and adolescents? Experimental evidence from normal-form games. J Econ Behav Org. 2016;128:265–85.
- 30. Angerer S, Lergetporer P, Glätzle-Rützler D, Sutter M. How to measure time preferences in children: a comparison of two methods. J Econ Sci Assoc 2015;1(2):158–69.
- 31. Sutter M, Zoller C, Glätzle-Rützler D. Economic behavior of children and adolescents – A first survey of experimental economics results. Eur Econ Rev. 2019;111:98–121.
- 32. Sutter M, Kocher MG, Glätzle-Rützler D, Trautmann ST. Impatience and uncertainty: Experimental decisions predict adolescents’ field behavior. Am Econ Rev 2013;103(1):510–31.
- 33. Eckel CC, Grossman PJ, Johnson CA, de Oliveira ACM, Rojas C, Wilson RK. School environment and risk preferences: Experimental evidence. J Risk Uncertain 2012;45(3):265–92.
- 34. Munro A, Tanaka Y. Risky rotten kids: An experiment on risk attitudes amongst adolescents in rural Uganda. National Graduate Institute for Policy Studies. Discussion Papers; 2014.
- 35. Defoe IN, Dubas JS, Figner B, van Aken MAG. A meta-analysis on age differences in risky decision making: Adolescents versus children and adults. Psychol Bull 2015;141(1):48–84.
- 36. Alfonso A, Brañas-Garza P, Jorrat D, Lomas P, Prissé B, Vasco M, et al. The adventure of running experiments with teenagers. J Behav Exp Econ. 2023;106:102048.
- 37. Bosch-Domènech A, Silvestre J. Measuring risk aversion with lists: A new bias. Theory Decis 2012;75(4):465–96.
- 38. 38. Vasco M, Alfonso A, Arenas A, Cabrales A, Cuesta JA, Espín AM, et al. TeensLab dataset; 2024. Available from: https://zenodo.org/record/13720112.
- 39. Vasco M, Alfonso A, Arenas A, Cabrales A, Cuesta JA, Espín AM, et al. Economic preferences and cognitive abilities among teenagers in Spain. Sci Data 2025;12(1):7.
- 40. Brañas-Garza P, Estepa-Mohedano L, Jorrat D, Orozco V, Rascón-Ramírez E. To pay or not to pay: Measuring risk preferences in lab and field. Judgm Decis Mak 2021;16(5):1290–313.
- 41. Kühberger A, Schulte-Mecklenbeck M, Perner J. Framing decisions: Hypothetical and real. Org Behav Hum Decis Process 2002;89(2):1162–75.
- 42. Chuang Y, Schechter L. Stability of experimental and survey measures of risk, time, and social preferences: A review and some new results. J Dev Econ. 2015;117:151–70.
- 43. Read D. Monetary incentives, what are they good for? J Econ Methodol. 2005;12(2):265–76.
- 44. Beattie J, Loomes G. The impact of incentives upon risky choice experiments. J Risk Uncert. 1997;14:155–68.
- 45. Horn S, Freund A. Adult age differences in monetary decisions with real and hypothetical reward. J Behav Decis Mak. 2022:35(2);e2253.
- 46. Thomson KS, Oppenheimer DM. Investigating an alternate form of the cognitive reflection test. Judgm Decis Mak 2016;11(1):99–113.
- 47. Frederick S. Cognitive reflection and decision making. J Econ Perspect 2005;19(4):25–42.
- 48. Harrison G. Maximum likelihood estimation of utility functions using Stata. University of Central Florida. Working Paper. 2008. p. 06–12.
- 49. Andersen S, Harrison G, Lau M, Rutström E. Eliciting risk and time preferences. Econometrica. 2008;76(3):583–618.
- 50. Loomes G, Moffatt PG, Sugden R. A microeconometric test of alternative stochastic theories of risky choice. J Risk Uncert. 2002;24:103–30.