Abstract Background Creativity is essential for engineering design, yet its assessment remains challenging due to the resource‐intensive nature of traditional evaluation methods. Purpose/Hypothesis(es) This study investigates the potential of automatic item generation (AIG) using large language models (LLMs) to create psychometrically sound items for the design problem task (DPT), which measures creative thinking in engineering. Design/Method We developed and validated engineering design problems across three domains: ability difference and limitations (e.g., assisting people with learning impairments), transportation and mobility (e.g., reducing traffic congestion in mega cities), and social environments and systems (e.g., improving access to clean water in remote areas). The study comprised three phases with samples matched on race and ethnicity: (1) content validation with a diverse sample of 40 engineers evaluating item clarity and validity; (2) item administration to 462 engineering students; and (3) response evaluation by 65 expert raters assessing originality and effectiveness. Results Results demonstrated that LLM‐generated items achieved comparable or higher content validity rates than expert‐written items (43% vs. 20% success). Bayesian confirmatory factor analysis supported a unidimensional model for fluency, originality, and effectiveness scores, with excellent reliability estimates (.92–.95). While fluency showed minimal correlation with originality ( r = −.11) and effectiveness ( r = −.04), originality and effectiveness were strongly positively correlated ( r = .73). Conclusions The present research advances our understanding of automated assessment generation in engineering education, provides empirical evidence for the psychometric properties of AI‐generated engineering creativity tasks, and offers a scalable approach for measuring creative thinking in engineering classrooms.

Evaluating the originality of visual images poses enduring challenges for creativity assessment. Automated scoring using AI models has proven effective in the verbal domain, yet key questions remain about evaluating visual creativity and understanding how models arrive at their ratings. The present research asks whether multimodal large language models (LLMs) can serve as judges of visual creativity zero-shot (without any fine-tuning or examples of human ratings) and whether their "reasoning" output offers an interpretable window into their evaluation process. We tested six multimodal LLMs (Gemini 3 Flash, Gemma 4 31B IT, GPT-5.4 Mini, GLM-5v Turbo, Kimi K2.5, and Qwen 3.6 Plus) on 992 AI-generated images (based on human-written prompts) and 1,500 hand-drawn sketches scored for creativity by human raters. In Study 1, all models showed substantial alignment with human creativity ratings on both datasets (r = .57-.68 on AI-generated images; r = .29-68 on sketches). In Study 2, we analyzed the step-by-step reasoning processes of three LLMs evaluating the same images and drawings. Although reasoning made model evaluations interpretable -- showing what they attend to, how they balance originality vs. quality, and how they justify their ratings -- reasoning did not improve alignment with human ratings. In sum, our findings indicate that multimodal LLMs can match human judgments of visual creativity without any additional training, and that their reasoning reveals how AI models evaluate creativity. An open scoring app implementing this pipeline is available at https://review-visual-eval-scoring.hf.space.

Creative thinking is a primary driver of innovation in science, technology, engineering, and math (STEM), allowing students and practitioners to generate novel hypotheses, flexibly connect information from diverse sources, and solve ill-defined problems. To foster creativity in STEM education, there is a crucial need for assessment tools for measuring STEM creativity that educators and researchers can apply to test how different teaching approaches impact scientific creativity in undergraduate education. In this work, we introduce the Scientific Creative Thinking Test (SCTT). The SCTT includes three subtests that assess cognitive skills important for STEM creativity: generating hypotheses, research questions, and experimental designs. In five studies with young adults, we demonstrate the reliability and validity of the SCTT – including test-retest reliability and convergent validity with measures of creativity and academic achievement – as well as measurement invariance across race/ethnicity and gender. In addition, we present a method for automatically scoring SCTT responses, training the large language model Llama 2 to produce originality scores that closely align with human ratings – demonstrating STEM-specific, automated creativity assessment for the first time. The full SCTT, along with the code to automatically score it, are available on a repository in the Open Science Framework.

A growing body of empirical research has developed measures of economic preferences related to risk taking and intertemporal choice. This research has documented pronounced heterogeneity in preferences across and within societies, and also provided evidence that these differences are culturally transmitted. This chapter discusses existing data sets that allow for a comparable measurement across the globe, takes stock of commonalities and differences in approaches, and presents an extended synthetic cross-country data set that combines information from existing data sets. The analysis then establishes various empirical regularities, such as broadly similar patterns of heterogeneity across the globe, revealed by the different datasets, but also some systematic divergences by measurement approach, and substantial correlations of economic preferences with country-aggregate and individual-level outcomes and traits. We also briefly discuss international data sets measuring social preferences, and end with an outlook on avenues for future research.

Participants in creativity studies sometimes produce invalid data that is unusable for analysis, such as nonsensical or incomplete responses to idea generation tasks. Identifying such responses is a time-consuming yet necessary process to ensure robust results but also remains challenging to automate. We explore the efficacy of transformer language models (TLMs) for automatically detecting invalid creativity responses, using only the text prompt and response and no other metadata about the experimental session. We train a suite of transformers to detect invalid data for two creativity assessments: the Alternate Uses Task (AUT) and a design problems task (DPT). We find that transformers generally outperform other baselines for both tasks. Further, we show that TLMs' predictions are well calibrated to the quality of the participant response, ensuring that model failures will occur in a predictable way and that high-quality responses are unlikely to be labeled invalid. Finally, we conduct a fairness analysis based on language background—using an adversarial study where participants attempt to “break” the model by coming up with invalid responses that are nonetheless labeled valid. Our results demonstrate the potential of deep learning methods for cleaning creativity assessment data, using solely participant responses, in a reliable and unbiased way.

Mete Ismayilzada, Antonio Laverghetta Jr., Simone A. Luchini, Reet Patel, Antoine Bosselut, Lonneke Van Der Plas, Roger E. Beaty. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025.

ABSTRACT Creativity researchers have recently sought to standardize idea assessment via computational measures of semantic distance: the degree of conceptual dissimilarity between words. The relationship between semantic distance and creativity has traditionally been described using linear models, with the embedded assumption that as semantic distance increases, so does the creative quality of ideas. However, informal observations would suggest that distant associations may sometimes become too incoherent or nonsensical to be considered creative. Using generalized additive models (GAMs), we explored the non‐linear nature of this relationship across three divergent thinking tasks: alternate uses, question asking, and metaphor generation. Our results revealed a consistent pattern: human ratings of creativity increased with semantic distance up to a certain threshold (between 0.9 and 1), after which point, additional semantic distance did not translate into more subjectively creative ideas. These findings provide a more nuanced understanding of the interplay between semantic distance and creativity than previously available, suggesting that the relationship is best characterized as curvilinear rather than linear. This work highlights a potential “sweet spot” for semantic distance in creative ideation and holds important implications for models of creativity.

Successful problem-solving and enhanced creative ability may improve physical health, cognitive well-being, and overall independence of older adults. In general, older adults who are more creative, may be better able to cope with cognitive decline and navigate everyday tasks. While previous research on creative performance in older adulthood showed age-related stability, open questions remain regarding the specific underlying cognitive basis for this invariability across the lifespan. Mediation analyses showed that intelligence measures served as underlying cognitive mechanisms for the stability of creative thinking in older age. The broader implications of these findings provide insight into the complex relationships supporting age-related preservation in creativity.

Linking neurobiology to relatively stable individual differences in cognition, emotion, motivation, and behavior can require large sample sizes to yield replicable results. Given the nature of between-person research, sample sizes at least in the hundreds are likely to be necessary in most neuroimaging studies of individual differences, regardless of whether they are investigating the whole brain or more focal hypotheses. However, the appropriate sample size depends on the expected effect size. Therefore, we propose four strategies to increase effect sizes in neuroimaging research, which may help to enable the detection of replicable between-person effects in samples in the hundreds rather than the thousands: (1) theoretical matching between neuroimaging tasks and behavioral constructs of interest; (2) increasing the reliability of both neural and psychological measurement; (3) individualization of measures for each participant; and (4) using multivariate approaches with cross-validation instead of univariate approaches. We discuss challenges associated with these methods and highlight strategies for improvements that will help the field to move toward a more robust and accessible neuroscience of individual differences.

Creativity is fundamentally a collaborative process. Yet as generative AI becomes increasingly integrated into creative work, understanding how AI reshapes collaboration has become critical. This pre-registered study directly compares human-human and human-AI collaboration dynamics across two creative tasks: the Alternative Uses Task (AUT) and creative short story writing. Participants were randomly assigned to pairs in either human-human (N = 68 pairs) or human-AI (GPT-4o; N = 72 pairs) conditions, with partners alternating turns as first responders to examine how initiation order shapes the creative process over time. Our findings reveal that the apparent "AI advantage" in creative collaboration is illusory, driven primarily by increased AI verbosity rather than enhanced creativity. Critically, collaboration with AI partners negatively impacted humans' own creative responses compared to human-human partnerships, with human-AI collaboration failing to enhance idea originality or diversity relative to human-human collaboration. Human partners demonstrated superior collaborative effectiveness that strengthened over time, indicating that current generative AI systems, while producing more verbose outputs, do not replicate the collective creativity characteristic of human-human collaboration. These results challenge assumptions about AI's creative potential, with direct implications for AI system design and collaborative creative practice.