Publications
A growing body of empirical research has developed measures of economic preferences related to risk taking and intertemporal choice. This research has documented pronounced heterogeneity in preferences across and within societies, and also provided evidence that these differences are culturally transmitted. This chapter discusses existing data sets that allow for a comparable measurement across the globe, takes stock of commonalities and differences in approaches, and presents an extended synthetic cross-country data set that combines information from existing data sets. The analysis then establishes various empirical regularities, such as broadly similar patterns of heterogeneity across the globe, revealed by the different datasets, but also some systematic divergences by measurement approach, and substantial correlations of economic preferences with country-aggregate and individual-level outcomes and traits. We also briefly discuss international data sets measuring social preferences, and end with an outlook on avenues for future research.
Creative thinking is a primary driver of innovation in science, technology, engineering, and math (STEM), allowing students and practitioners to generate novel hypotheses, flexibly connect information from diverse sources, and solve ill-defined problems. To foster creativity in STEM education, there is a crucial need for assessment tools for measuring STEM creativity that educators and researchers can apply to test how different teaching approaches impact scientific creativity in undergraduate education. In this work, we introduce the Scientific Creative Thinking Test (SCTT). The SCTT includes three subtests that assess cognitive skills important for STEM creativity: generating hypotheses, research questions, and experimental designs. In five studies with young adults, we demonstrate the reliability and validity of the SCTT – including test-retest reliability and convergent validity with measures of creativity and academic achievement – as well as measurement invariance across race/ethnicity and gender. In addition, we present a method for automatically scoring SCTT responses, training the large language model Llama 2 to produce originality scores that closely align with human ratings – demonstrating STEM-specific, automated creativity assessment for the first time. The full SCTT, along with the code to automatically score it, are available on a repository in the Open Science Framework.
Abstract Background Creativity is essential for engineering design, yet its assessment remains challenging due to the resource‐intensive nature of traditional evaluation methods. Purpose/Hypothesis(es) This study investigates the potential of automatic item generation (AIG) using large language models (LLMs) to create psychometrically sound items for the design problem task (DPT), which measures creative thinking in engineering. Design/Method We developed and validated engineering design problems across three domains: ability difference and limitations (e.g., assisting people with learning impairments), transportation and mobility (e.g., reducing traffic congestion in mega cities), and social environments and systems (e.g., improving access to clean water in remote areas). The study comprised three phases with samples matched on race and ethnicity: (1) content validation with a diverse sample of 40 engineers evaluating item clarity and validity; (2) item administration to 462 engineering students; and (3) response evaluation by 65 expert raters assessing originality and effectiveness. Results Results demonstrated that LLM‐generated items achieved comparable or higher content validity rates than expert‐written items (43% vs. 20% success). Bayesian confirmatory factor analysis supported a unidimensional model for fluency, originality, and effectiveness scores, with excellent reliability estimates (.92–.95). While fluency showed minimal correlation with originality ( r = −.11) and effectiveness ( r = −.04), originality and effectiveness were strongly positively correlated ( r = .73). Conclusions The present research advances our understanding of automated assessment generation in engineering education, provides empirical evidence for the psychometric properties of AI‐generated engineering creativity tasks, and offers a scalable approach for measuring creative thinking in engineering classrooms.
• Introduces two automated transformer models for assessing cognitive question complexity: high-performing LLaMA 2 and efficient, robust RoBERTa. • These fine-tuned transformer language models greatly outperform surface-based and semantic-distance baselines. • Both models predict open-ended question complexity and align strongly with human ratings. • Introduces PrediQT , a free web app for automated question-complexity scoring. • Bridges creativity, cognition, and AI for educational assessment innovation. Question-asking, an essential yet understudied activity, holds significant implications for learning, creativity, and cognitive development. Research shows that asking complex, open-ended questions is better for learning than closed ones. Previous research has explored open-ended question complexity through Bloom's taxonomy, but the measurement of complexity remains challenging. Recent advancements in natural language processing have enabled automated scoring of psychological tasks aligned to human-ratings. However, automatic assessment of open-ended questions remains understudied. We address this gap by fine-tuning transformer language models to predict human ratings of open-ended question complexity and comparing them to existing baseline measures (i.e., word count and semantic distance). Using previously collected human-rated responses and Bloom ratings from a creative question-asking task, we trained an encoder model (RoBERTa) and a Large Language Model (Llama-2-7B). Our results reveal that RoBERTa correlated strongly with human ratings of complexity (r = .73), exceeding baseline measures and offering an efficient, lightweight solution suitable for broad adoption. Our fine-tuned LLaMA 2 model achieved stronger performance (r = .84), establishing a new benchmark for predictive accuracy. Thus, we demonstrate how language models can be utilized to automatically score the complexity of open-ended questions. Importantly, LLaMA 2 demonstrates higher accuracy, while RoBERTa provides a replicable, accessible, and cost-effective option for everyday educational and psychological applications. Our work paves the way for automatic assessment of open-ended questions, which are critical across a wide range of cognitive domains.
Creativity is a key 21st-century skill and a consistent predictor of academic learning outcomes. Despite decades of research on creativity and learning, little is known about the cognitive mechanisms underlying their relationship. In two studies, we examined whether creativity supports associative learning through associative thinking-the ability to generate novel word associations-an ability central to creativity which has not been previously tied to associative learning. In Study 1, we found that students who generated more novel word associations learned more words on a foreign language learning test 24 h later. In Study 2, we replicated and extended the effect to naturalistic creativity tasks (i.e., writing short stories and sketching line drawings), finding associative thinking mediated the relationship between creativity and associative learning. Importantly, both studies controlled for general intelligence. Our findings suggest that creativity's contribution to learning operates partly through a shared cognitive capacity for making new connections.
ABSTRACT Creativity researchers have recently sought to standardize idea assessment via computational measures of semantic distance: the degree of conceptual dissimilarity between words. The relationship between semantic distance and creativity has traditionally been described using linear models, with the embedded assumption that as semantic distance increases, so does the creative quality of ideas. However, informal observations would suggest that distant associations may sometimes become too incoherent or nonsensical to be considered creative. Using generalized additive models (GAMs), we explored the non‐linear nature of this relationship across three divergent thinking tasks: alternate uses, question asking, and metaphor generation. Our results revealed a consistent pattern: human ratings of creativity increased with semantic distance up to a certain threshold (between 0.9 and 1), after which point, additional semantic distance did not translate into more subjectively creative ideas. These findings provide a more nuanced understanding of the interplay between semantic distance and creativity than previously available, suggesting that the relationship is best characterized as curvilinear rather than linear. This work highlights a potential “sweet spot” for semantic distance in creative ideation and holds important implications for models of creativity.
Successful problem-solving and enhanced creative ability may improve physical health, cognitive well-being, and overall independence of older adults. In general, older adults who are more creative, may be better able to cope with cognitive decline and navigate everyday tasks. While previous research on creative performance in older adulthood showed age-related stability, open questions remain regarding the specific underlying cognitive basis for this invariability across the lifespan. Mediation analyses showed that intelligence measures served as underlying cognitive mechanisms for the stability of creative thinking in older age. The broader implications of these findings provide insight into the complex relationships supporting age-related preservation in creativity.
The integration of artificial intelligence (AI) into creative work continues to expand, yet its impact on human creativity itself—beyond simply providing ideas—remains uncertain. We reposition AI's role from idea generator to idea evaluator, using trained models to provide real-time feedback on human-generated ideas. Across two studies—a preregistered online experiment involving individuals with varying levels of expertise (N = 554) and a large-scale naturalistic experiment during a year-long museum exhibit (N = 36,198)—participants generated solutions to real-world problems or created visual sketches. AI feedback significantly improved participant originality in both verbal and visual creative domains. Mediation analyses revealed these gains were partly driven by changes in individuals' self-evaluation of their own originality, implying a key role for metacognition—the ability to monitor, control and regulate one's thinking. These findings suggest that AI's potential extends beyond generation to include idea evaluation, helping humans assess and refine their ideas through real-time feedback.
Mete Ismayilzada, Antonio Laverghetta Jr., Simone A. Luchini, Reet Patel, Antoine Bosselut, Lonneke Van Der Plas, Roger E. Beaty. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025.
Linking neurobiology to relatively stable individual differences in cognition, emotion, motivation, and behavior can require large sample sizes to yield replicable results. Given the nature of between-person research, sample sizes at least in the hundreds are likely to be necessary in most neuroimaging studies of individual differences, regardless of whether they are investigating the whole brain or more focal hypotheses. However, the appropriate sample size depends on the expected effect size. Therefore, we propose four strategies to increase effect sizes in neuroimaging research, which may help to enable the detection of replicable between-person effects in samples in the hundreds rather than the thousands: (1) theoretical matching between neuroimaging tasks and behavioral constructs of interest; (2) increasing the reliability of both neural and psychological measurement; (3) individualization of measures for each participant; and (4) using multivariate approaches with cross-validation instead of univariate approaches. We discuss challenges associated with these methods and highlight strategies for improvements that will help the field to move toward a more robust and accessible neuroscience of individual differences.


