• Introduces two automated transformer models for assessing cognitive question complexity: high-performing LLaMA 2 and efficient, robust RoBERTa. • These fine-tuned transformer language models greatly outperform surface-based and semantic-distance baselines. • Both models predict open-ended question complexity and align strongly with human ratings. • Introduces PrediQT , a free web app for automated question-complexity scoring. • Bridges creativity, cognition, and AI for educational assessment innovation. Question-asking, an essential yet understudied activity, holds significant implications for learning, creativity, and cognitive development. Research shows that asking complex, open-ended questions is better for learning than closed ones. Previous research has explored open-ended question complexity through Bloom's taxonomy, but the measurement of complexity remains challenging. Recent advancements in natural language processing have enabled automated scoring of psychological tasks aligned to human-ratings. However, automatic assessment of open-ended questions remains understudied. We address this gap by fine-tuning transformer language models to predict human ratings of open-ended question complexity and comparing them to existing baseline measures (i.e., word count and semantic distance). Using previously collected human-rated responses and Bloom ratings from a creative question-asking task, we trained an encoder model (RoBERTa) and a Large Language Model (Llama-2-7B). Our results reveal that RoBERTa correlated strongly with human ratings of complexity (r = .73), exceeding baseline measures and offering an efficient, lightweight solution suitable for broad adoption. Our fine-tuned LLaMA 2 model achieved stronger performance (r = .84), establishing a new benchmark for predictive accuracy. Thus, we demonstrate how language models can be utilized to automatically score the complexity of open-ended questions. Importantly, LLaMA 2 demonstrates higher accuracy, while RoBERTa provides a replicable, accessible, and cost-effective option for everyday educational and psychological applications. Our work paves the way for automatic assessment of open-ended questions, which are critical across a wide range of cognitive domains.

Across development, experience has a strong impact on the way we think and adapt. School experience affects academic and social-emotional outcomes, yet whether differences in pedagogical experience modulate underlying brain network development is still unknown. In this study, we compared the brain network dynamics of students with different pedagogical backgrounds. Specifically, we characterized the diversity and stability of brain activity at rest by combining both resting-state fMRI and diffusion-weighted structural imaging data of 87 4-18 years old students experiencing either the Montessori pedagogy (i.e., student-led, trial-and-error pedagogy) or the traditional pedagogy (i.e., teacher-led, test-based pedagogy). Our results revealed spatiotemporal brain dynamics differences between students as a function of schooling experience at the whole-brain level. Students from Montessori schools showed overall higher functional integration (higher system diversity) and neural stability (lower spatiotemporal diversity) compared to traditionally schooled students. Higher integration was explained mainly through the cerebellar (CBL) functional network. In contrast, higher temporal stability was observed in the ventral attention, dorsal attention, somatomotor, frontoparietal, and CBL functional networks. This study suggests a form of experience-dependent dynamic functional connectivity plasticity, in learning-related networks.

The visual modality is central to both reception and expression of human creativity. Creativity assessment paradigms, such as structured drawing tasks Barbot (2018), seek to characterize this key modality of creative ideation. However, visual creativity assessment paradigms often rely on cohorts of expert or naive raters to gauge the level of creativity of the outputs. This comes at the cost of substantial human investment in both time and labor. To address these issues, recent work has leveraged the power of machine learning techniques to automatically extract creativity scores in the verbal domain (e.g., SemDis; Beaty & Johnson 53, 757-780, 2021). Yet, a comparably well-vetted solution for the assessment of visual creativity is missing. Here, we introduce AuDrA - an Automated Drawing Assessment platform to extract visual creativity scores from simple drawing productions. Using a collection of line drawings and human creativity ratings, we trained AuDrA and tested its generalizability to untrained drawing sets, raters, and tasks. Across four datasets, nearly 60 raters, and over 13,000 drawings, we found AuDrA scores to be highly correlated with human creativity ratings for new drawings on the same drawing task (r = .65 to .81; mean = .76). Importantly, correlations between AuDrA scores and human raters surpassed those between drawings' elaboration (i.e., ink on the page) and human creativity raters, suggesting that AuDrA is sensitive to features of drawings beyond simple degree of complexity. We discuss future directions, limitations, and link the trained AuDrA model and a tutorial (https://osf.io/kqn9v/) to enable researchers to efficiently assess new drawings.

Standard learning assessments like multiple-choice questions measure what students know but not how their knowledge is organized. Recent advances in cognitive network science provide quantitative tools for modeling the structure of semantic memory, revealing key learning mechanisms. In two studies, we examined the semantic memory networks of undergraduate students enrolled in an introductory psychology course. In Study 1, we administered a cumulative multiple-choice test of psychology knowledge, the Intro Psych Test, at the end of the course. To estimate semantic memory networks, we administered two verbal fluency tasks: domain-specific fluency (naming psychology concepts) and domain-general fluency (naming animals). Based on their performance on the Intro Psych Test, we categorized students into a high-knowledge or low-knowledge group, and compared their semantic memory networks. Study 1 (N = 213) found that the high-knowledge group had semantic memory networks that were more clustered, with shorter distances between concepts-across both the domain-specific (psychology) and domain-general (animal) categories-compared to the low-knowledge group. In Study 2 (N = 145), we replicated and extended these findings in a longitudinal study, collecting data near the start and end of the semester. In addition to replicating Study 1, we found the semantic memory networks of high-knowledge students became more interconnected over time, across both domain-general and domain-specific categories. These findings suggest that successful learners show a distinct semantic memory organization-characterized by high connectivity and short path distances between concepts-highlighting the utility of cognitive network science for studying variation in student learning.

Creative problem-solving is a naturalistic form of creative thinking involving the generation of solutions that are not only original but also of high quality (i.e., plausible and effective). Naturalistic tasks that evaluate both originality and quality are vital for the promotion of creativity in real-world settings—yet scoring such tasks remains challenging, due to costly human labor required to manually rate task responses. Past work has shown that large language models (LLMs) can be trained to predict human originality ratings of responses to tests of divergent thinking. In the present research, we extend this work to creative problem-solving, examining whether both originality and quality can be automatically scored for a naturalistic creativity task. We gathered data from 10 studies, amounting to 3,235 participants who completed a creative problem-solving task (CPST). We then fine-tuned two open-source LLMs, RoBERTa and GPT-2, to predict human ratings of originality and quality on the CPST, and compared their performance to two other scoring methods: elaboration (i.e., word count) and semantic distance. We found that RoBERTa and GPT-2 models predict solution quality (RoBERTa, r = .83; GPT-2, r = .83) better than solution originality (RoBERTa, r = .79; GPT-2, r = .80). Moreover, we found that both models outperformed elaboration and semantic distance methods and generalized to new CPST items not present in their training set. We therefore show for the first time that naturalistic creativity tasks can be automatically scored for both originality and quality. Open access is provided to the models and training data.

Creativity research commonly involves recruiting human raters to judge the originality of responses to divergent thinking tasks, such as the alternate uses task (AUT). These manual scoring practices have benefited the field, but they also have limitations, including labor-intensiveness and subjectivity, which can adversely impact the reliability and validity of assessments. To address these challenges, researchers are increasingly employing automatic scoring approaches, such as distributional models of semantic distance. However, semantic distance has primarily been studied in English-speaking samples, with very little research in the many other languages of the world. In a multilab study (N= 6,522 participants), we aimed to validate semantic distance on the AUT in 12 languages: Arabic, Chinese, Dutch, English, Farsi, French, German, Hebrew, Italian, Polish, Russian, and Spanish. We gathered AUT responses and human creativity ratings (N= 107,672 responses), as well as criterion measures for validation (e.g., creative achievement).We compared two deep learning-based semantic models—multilingual bidirectional encoder representations from transformers and cross-lingual language model RoBERTa—to compute semantic distance and validate this automated metric with human ratings and criterion measures. We found that the top-performing model for each language correlated positively with human creativity ratings, with correlations ranging from medium to large across languages. Regarding criterion validity, semantic distance showed small-to-moderate effect sizes (comparable to human ratings) for openness, creative behavior/achievement, and creative self-concept. We provide open access to our multilingual dataset for future algorithmic development, along with Python code to compute semantic distance in 12 languages. © 2023 American Psychological Association

Creativity has long been thought to involve associative processes in memory: connecting concepts to form ideas, inventions, and artworks. However, associa-tive thinking has been difficult to study due to limitations in modeling memory structure and retrieval processes. Recent advances in computational models of semantic memory allow researchers to examine how people navigate a se-mantic space of concepts when forming associations, revealing key search strat-egies associated with creativity. Here, we synthesize cognitive, computational, and neuroscience research on creativity and associative thinking. This Review highlights distinctions between free-and goal-directed association, illustrates the role of associative thinking in the arts, and links associative thinking to brain systems supporting both semantic and episodic memory - offering a new perspective on a longstanding creativity theory.

Although both creativity and humor elicit experiences of surprise followed by appreciation, it remains unknown whether shared or distinct patterns of effective connectivity are involved in their processing. The present fMRI study used dynamic causal modeling and parametrical empirical Bayes analysis to examine the effective connectivity between the amygdala and frontoparietal network during two-stage creativity and humor processing. We examined processing during the setup and punch line stages for creativity and humor, including typical forms (alternate uses for creativity and incongruity-resolution humor), atypical forms (aesthetic uses for creativity and nonsense humor), and baseline forms. Our focus was on the mesolimbic pathway during the punch line stage. We found that the amygdala plays a key role in expectation violation and appreciation. Broadly, amygdala-to-IFG connectivity was important for evaluating typical and atypical forms of both creativity and humor, while amygdala-to-precuneus connectivity was involved in evaluating typical forms. Amygdala-to-IFG connectivity was involved in the expectation violation to resolution stage of processing for typical and atypical forms of creativity and humor. Amygdala-to-precuneus connectivity was involved in processing the novelty and usefulness of typical forms of creativity (alternate uses) and understanding others' intentions in typical forms of humor (incongruity-resolution). Interestingly, VTA-to-amygdala connectivity was involved in processing the appreciation of both typical (incongruity-resolution humor) and atypical (nonsense humor) forms of humor while amygdala-to-VTA connectivity was involved in processing the appreciation of atypical (aesthetic uses) forms of creativity. Altogether, these findings suggest that the amygdala and frontoparietal circuitry are critical for creativity and humor processing.

Existing research has consistently supported a relationship between creative achievement and specific personality traits (e.g. openness to experience). However, such work has largely focused on univariate associations, potentially obscuring complex interactions among multiple personality factors, rendering an incomplete picture of the creative personality. We applied a psychometric network approach to characterize the multidimensional personality structure of highly creative individuals in the arts (artists) and sciences (scientists), using data from three samples (N = 4,015): college students, a representative adult sample, and the Big-C project of eminent creative professionals. Replicating past work, we found that artists showed reliably higher levels of openness to experience compared to scientists and a control group of less creative people. Psychometric network analysis revealed that artists were characterized by higher connectivity (i.e. co-occurrence) with other personality traits for openness, indicating that openness may be more heterogeneous in how it co-occurs with other personality traits in highly creative people. Across all three samples, we found that the scientists' personality network structure was more cohesive than the personality network of artists and the control group, indicating greater homogeneity in the personality characteristics of scientists. Our findings uncover a constellation of traits that give rise to creative achievement in the arts and sciences.