Publications
The standard definition of creativity as novel and useful describes creative products, but creativity is constituted by processes. This misalignment contributes to the oft-noted challenges of operationalizing creativity. Here, we distinguish creativity as a process from creativity as an attribute (i.e. creative-ness). Operating from a priori premises of creativity theory, we develop a process definition of creativity. Specifically, creativity is defined as internal attention constrained by a generative goal. This definition comprises three criteria: 1) attention is directed internally (toward mental representations); 2) attentional operations (e.g. search, manipulation) are constrained to fit parameters of a to-be-achieved goal state (whether or not a goal is actually achieved); and 3) the goal state is generative (not already precisely held in memory). We illustrate how these three definitional process elements align with insights yielded by creativity neuroscience, clarify ontological distinctions (e.g. from mind-wandering and retrieval), and relate the process definition to process-based models. The process definition provides minimal necessity and sufficiency criteria for whether a process should be considered creativity, but does not exclude the many other perceptual, emotional, etc. elements that can contribute to creativity processes. Researchers should specify whether they are studying creativity-as-process vs. creative-ness, and consider including process-focused assessments.
In this paper, we explore measurements of design creativity through metrics related to the processes used in designing and relate them to the metrics used in psychology for idea creativity, ie, novelty and fluency. Our goal was to test the reliability of psychometric measures of creativity to assess creativity in team design. We studied 19 teams of 3 professional engineers that engaged in a one hour-long design task. Design tasks have a greater ecological validity than single repetitive tasks like the AUT and the RAT. Engaging in a design task involves a wide range of cognitive activities, which contribute to creative ideation and to expanding the design space. This study focused on the relationship between the teams’ design idea creativity and design behaviors during the task. We explored to what extent design collaboration between teammates, design evaluation and the co-evolution of the problem-solution space relate to the psychometric measures of idea creativity. Results suggest no specific trend in the correlation between collaboration and idea creativity as measured by the metrics used in psychology, while more cognitive focus on problem-solution co-evolution negatively correlates with these measures of idea creativity. The paper concludes with potential explanations for this lack of correlation. © 2024 Taylor & Francis Group, LLC.
The purpose of this report is to present our process for establishing evidence of validity and reliability of an observation tool [PEMTP: Potentially Engaging Mathematics Teaching Practices] used to investigate mathematics teaching practices that have the potential to engage high school students. We first present our practices for establishing content-oriented evidence of validity for our observation tool. Next, we describe how we developed a framework to identify and specify potentially engaging mathematics teaching practices, including our efforts to pilot the tool to design descriptive levels for rubrics. We then report on a reliability investigation we conducted to examine the nature of coding disagreements. Our results indicate the value of reporting on validity processes for tools such as the PEMTP to illustrate the ways that the tool has been designed and revised, which can inform the use cases and potential future applications of the tool. These results also show how such validation efforts can lead to meaningful revisions – in the case of the PEMTP, the addition of two rubrics aligned with teaching mathematics for equity. Finally, the results of our reliability investigation indicate the value of using multiple raters and collaborative resolution meetings when investigating complex phenomena like classroom instruction.
Many college students drop STEM majors after struggling in gateway courses, in part because these courses place large demands on students' time. In three online experiments with two different lessons (measures of central tendency and multiple regression), we identified a promising approach to increase the efficiency of STEM instruction. When we removed lectures and taught participants exclusively with practice and feedback, they learned at least 15% faster. However, our research also showed that this instructional strategy has the potential to undermine interest in course content for less-confident students, who may be discouraged when challenged to solve problems without upfront instruction and learn from their mistakes. If researchers and educators can develop engaging and efficacy-building activities that replace lectures, STEM courses could become better learning environments.
Three-dimensional (3D) technologies such as 3D digital prototyping have recently been used in apparel engineering to reduce the number of costly physical prototypes. This technology can simulate garment drapes by digitally sewing 2D patterns and placing them on a 3D avatar. Yet, current limitations in simulation accuracy mean physical prototypes are still indispensable. Integrating AI technologies shows promise in accelerating digital transformation in the apparel industry. AI can support decision-making in apparel design, size and fit prediction, and fashion recommendations. It can also assist decision-making in technical design by predicting garment fit before making physical prototypes or offering solutions for misfit areas. Research indicates that models such as Naive Bayes (NB), Support Vector Machine (SVM), and Artificial Neural Networks (ANN) offer robust solutions for fit evaluation. However, they require further testing across diverse garment styles and demographics. Developing a nuanced classification system for fit conditions could improve model accuracy, ultimately supporting the industry's shift towards digital and AI-enhanced product development. In this study, we propose a framework to achieve such a system.
We present CPSCoach 2.0, an automated system that provides feedback, instructional scaffolding, and practice to help individuals improve three collaborative problem-solving (CPS) skills drawn from a theoretical CPS framework: construction of shared knowledge, negotiation/coordination, and maintaining team function. CPSCoach 2.0 was developed and tested in the context of computer-mediated collaboration (video conferencing) with an educational game. It automatically analyzes users' speech during a round of collaborative gameplay to provide personalized feedback and to select a target CPS skill for improvement. After multiple cycles of iterative testing and refinement, we tested CPSCoach 2.0 in a user study where 21 dyads (n = 42) completed four rounds of feedback and scaffolding embedded within five rounds of game-play in a single session. Using a quasi-experimental matching procedure, we found that the use of CPSCoach 2.0 was associated with improvement in CPS skill development compared to matched controls. Further, users found the automated feedback to be moderately accurate and had positive perceptions of the system, and these impressions were stronger for those who received higher scores overall. Results demonstrate the use of automated feedback and instructional scaffolds to support the development of CPS skills.
Metaphor is crucial in human cognition and creativity, facilitating abstract thinking, analogical reasoning, and idea generation. Typically, human raters manually score the originality of responses to creative thinking tasks - a laborious and error-prone process. Previous research sought to remedy these risks by scoring creativity tasks automatically using semantic distance and large language models (LLMs). Here, we extend research on automatic creativity scoring to metaphor generation - the ability to creatively describe episodes and concepts using nonliteral language. Metaphor is arguably more abstract and naturalistic than prior targets of automated creativity assessment. We collected 4,589 responses from 1,546 participants to various metaphor prompts and corresponding human creativity ratings. We fine-tuned two open-source LLMs (RoBERTa and GPT-2) - effectively teaching them to score metaphors like humans - before testing their ability to accurately assess the creativity of new metaphors. Results showed both models reliably predicted new human creativity ratings (RoBERTa r = .72, GPT-2 r = .70), significantly more strongly than semantic distance (r = .42). Importantly, the fine-tuned models generalized accurately to metaphor prompts they had not been trained on (RoBERTa r = .68, GPT-2 r = .63). We provide open access to the fine-tuned models, allowing researchers to assess metaphor creativity in a reproducible and timely manner.
Modern assessment demands, resulting from educational reform efforts, call for strengthening diagnostic testing capabilities to identify not only the understanding of expected learning goals but also related intermediate understandings that are steppingstones on pathways to learning goals. An accurate and nuanced way of interpreting assessment results will allow subsequent instructional actions to be targeted. An appropriate psychometric model is indispensable in this regard. In this study, we developed a new psychometric model, namely, the diagnostic facet status model (DFSM), which belongs to the general class of cognitive diagnostic models (CDM), but with two notable features: (1) it simultaneously models students’ target understanding (i.e., goal facet) and intermediate understanding (i.e., intermediate facet); and (2) it models every response option, rather than merely right or wrong responses, so that each incorrect response uniquely contributes to discovering students’ facet status. Given that some combination of goal and intermediate facets may be impossible due to facet hierarchical relationships, a regularized expectation–maximization algorithm (REM) was developed for model estimation. A log-penalty was imposed on the mixing proportions to encourage sparsity. As a result, those impermissible latent classes had estimated mixing proportions equal to 0. A heuristic algorithm was proposed to infer a facet map from the estimated permissible classes. A simulation study was conducted to evaluate the performance of REM to recover facet model parameters and to identify permissible latent classes. A real data analysis was provided to show the feasibility of the model. © The Author(s), under exclusive licence to The Psychometric Society 2024. corrected publication 2024.
A growing body of literature indicates strong associations between indentations of the cerebral cortex (i.e., sulci) and individual differences in cognitive performance. Interruptions, or gaps, of sulci (historically known as pli de passage) are particularly intriguing as previous work suggests that these interruptions have a causal effect on cognitive development. Here, we tested how the presence and morphology of sulcal interruptions in the left posterior occipitotemporal sulcus (pOTS) longitudinally impact the development of a culturally-acquired skill: reading. Forty-three children were successfully followed from age 5 in kindergarten, at the onset of literacy instruction, to ages 7 and 8 with assessments of cognitive, pre-literacy, and literacy skills, as well as MRI anatomical scans at ages 5 and 8. Crucially, we demonstrate that the presence of a left pOTS gap at 5 years is a specific and robust longitudinal predictor of better future reading skills in children, with large observed benefits on reading behavior ranging from letter knowledge to reading comprehension. The effect of left pOTS interruptions on reading acquisition accumulated through time, and was larger than the impact of benchmark cognitive and familial predictors of reading ability and disability. Finally, we show that increased local U-fiber white matter connectivity associated with such sulcal interruptions possibly underlie these behavioral benefits, by providing a computational advantage. To our knowledge, this is the first quantitative evidence supporting a potential integrative gray-white matter mechanism underlying the cognitive benefits of macro-anatomical differences in sulcal morphology related to longitudinal improvements in a culturally-acquired skill.
Assessment is central to teaching and learning, and recently there has been a substantive shift from paper-and-pencil assessments towards technology delivered assessments such as computer-adaptive tests. Fairness is an important aspect of the assessment process, including design, administration, test-score interpretation, and data utility. The Universal Design for Learning (UDL) guidelines can inform assessment development to promote fairness; however, it is not explicitly clear how UDL and fairness may be linked through students’ conceptualizations of assessment fairness. This phenomenological study explores how middle grades students conceptualize and reason about the fairness of mathematics tests, including paper-and-pencil and technology-delivered assessments. Findings indicate that (a) students conceptualize fairness through unique notions related to educational opportunities and (b) students’ reason about fairness non-linearly. Implications of this study have potential to inform test developers and users about aspects of test fairness, as well as educators data usage from fixed-form, paper-and-pencil tests, and computer-adaptive, technology-delivered tests. © The Author(s) 2024.


