Qualitative video analysis is a foundational yet labor-intensive method in the learning sciences, particularly for investigating the multimodal and collaborative dimensions of learning. This paper introduces GAIVA (Generative Artificial Intelligence for Video Analysis), a novel framework that integrates multimodal large language models (MLLMs) and large language models (LLMs) to support scalable, consistent, and theory-informed video analysis. GAIVA comprises four sequential phases: (1) video preprocessing, (2) MLLM-based behavioral transcription, (3) human-AI collaborative codebook development grounded in theoretical constructs from computer-supported collaborative learning (CSCL), and (4) LLM-based automated coding. Drawing on a case study to analyze simulation-based collaborative science learning, we demonstrate that GAIVA generates rich behavioral transcriptions, enables the construction of stable and theoretically grounded coding schemes, and achieves inter-rater reliability comparable to human annotation. Notably, findings illustrate GAIVA's capacity to support both deductive alignment and inductive discovery. The framework surfaces a behavioral construct--Peripheral Engagement--that is consistent across data batches but absent from existing frameworks. We contribute a systematic and replicable framework for integrating generative AI into qualitative video research, enabling large-scale, multimodal analysis of collaborative learning while preserving analytical rigor, theoretical coherence, and interpretive transparency.

Cocreating meaning in collaboration is challenging. Success is often determined by people's abilities to coordinate their language to converge upon shared mental representations. Here we explore one set of low-level linguistic behaviors, linguistic alignment, that both emerges from, and facilitates, outcomes of high-level convergence. Linguistic alignment captures the ways people reuse, that is, align to, the lexical, syntactic, and semantic forms of others' utterances. Our focus is on the temporal change of multi-level linguistic alignment, as well as how alignment is related to communicative outcomes within a unique collaborative problem-solving paradigm. The primary task, situated within a virtual educational video game, requires creative thinking between three people where the paths for possible solutions are highly variable. We find that over time interactions are marked by decreasing lexical and syntactic alignment, with a trade-off of increasing semantic alignment. However, greater semantic alignment does not translate into better team performance. Overall, these findings provide greater clarity on the role of linguistic coordination within complex and dynamic collaborative problem-solving tasks.

We present CPSCoach 2.0, an automated system that provides feedback, instructional scaffolding, and practice to help individuals improve three collaborative problem-solving (CPS) skills drawn from a theoretical CPS framework: construction of shared knowledge, negotiation/coordination, and maintaining team function. CPSCoach 2.0 was developed and tested in the context of computer-mediated collaboration (video conferencing) with an educational game. It automatically analyzes users' speech during a round of collaborative gameplay to provide personalized feedback and to select a target CPS skill for improvement. After multiple cycles of iterative testing and refinement, we tested CPSCoach 2.0 in a user study where 21 dyads (n = 42) completed four rounds of feedback and scaffolding embedded within five rounds of game-play in a single session. Using a quasi-experimental matching procedure, we found that the use of CPSCoach 2.0 was associated with improvement in CPS skill development compared to matched controls. Further, users found the automated feedback to be moderately accurate and had positive perceptions of the system, and these impressions were stronger for those who received higher scores overall. Results demonstrate the use of automated feedback and instructional scaffolds to support the development of CPS skills.

Shared visual attention between team members facilitates collaborative problem solving (CPS), but little is known about how team-level eye gaze dynamics influence the quality and successfulness of CPS. To better understand the role of shared visual attention during CPS, we collected eye gaze data from 279 individuals solving computer-based physics puzzles while in teams of three. We converted eye gaze into discrete screen locations and quantified team-level gaze dynamics using recurrence quantification analysis (RQA). Specifically, we used a centroid-based auto-RQA approach, a pairwise team member cross-RQAs approach, and a multi-dimensional RQA approach to quantify team-level eye gaze dynamics from the eye gaze data of team members. We find that teams differing in composition based on prior task knowledge, gender, and race show few differences in team-level eye gaze dynamics. We also find that RQA metrics of team-level eye gaze dynamics were predictive of task success (all ps < .001). However, the same metrics showed different patterns of feature importance depending on predictive model and RQA type, suggesting some redundancy in task-relevant information. These findings signify that team-level eye gaze dynamics play an important role in CPS and that different forms of RQA pick up on unique aspects of shared attention between team-members.

Collaborative problem solving (CPS) is a critical competency for the modern workforce, as many of todays' problems require groups to come together to find innovative solutions to complex problems. This has motivated increased interest in work dedicated to assessing and developing CPS skills. However, there has been limited attention in prior CPS assessment research on potential differences in how CPS behaviors are exhibited across task contexts. In the current study, we investigated associations among middle- and high-school students' displayed CPS skills across two online (i.e., via videoconferencing) tasks (Physics Playground and the T-Shirt Math Task) and the extent to which different skills were related to CPS outcomes across those tasks. Results showed variation in associations of CPS skills across the tasks, contributing further evidence to our understanding of how different CPS task designs can give students the opportunity to demonstrate different CPS skills. Our findings highlight the potential of incorporating multiple tasks during CPS assessments and can inform future research on CPS task design and computerbased CPS assessment.

We investigated the feasibility of using eye gaze to model collaborative problem solving (CPS) behaviors in 96 triads (N= 288) who used videoconferencing to remotely collaborate on an educational physics game. Trained human raters coded spoken utterances based on a theoretically grounded framework consisting of three CPS facets: constructing shared knowledge, negotiation/coordination, and maintaining team function. We then trained random forest classifiers to identify each of the CPS facets using eye gaze features pertaining to each individual (e.g., number of fixations) and/or shared across individuals (e.g., eye gaze distance between collaborators) in conjunction with information about the unfolding task context. We found that the individual gaze features outperformed the shared features, and together yielded between 6% to 8% improvements in classification accuracy above task-context baseline models, using a cross-validation scheme that generalized across teams. We discuss how our findings support CPS theories and the development of real-time intervention systems that provide actionable feedback to improve collaboration.

In collaborative problem solving (CPS), people's actions are interactive, interdependent, and temporal. However, it is unclear how actions temporally relate to each other and what are the temporal similarities and differences between successful vs. unsuccessful CPS processes. As such, we apply a temporal analysis approach, Multilevel Vector Autoregression (mlVAR) to investigate CPS processes. Our data were collected from college students who collaborated in triads via a video-conferencing tool (Zoom) to collaborately engage a physics learning game. Video recordings of their verbal interactions were transcribed, coded using a validated CPS framework, and organized into sequences of 10-second windows. Then, mlVAR was applied to the successful vs. unsuccessful CPS sequences to build temporal models for each. A comparison of the models together with a qualitative analysis of the transcripts revealed six temporal relationships common to both, six unique to successful level attempts, and another eight unique to unsuccessful level attempts only. Generally, for successful outcomes, people were likely to answer clarification questions with reasons and to ask for suggestions according to the current game situation, while for unsuccessful CPS level attempts, people were more likely to struggle with unclear instructions and to respond to inappropriate ideas. Overall, our results suggest that mlVAR is an effective approach for temporal analyses of CPS processes by identifying relationships that go beyond a coding and counting approach.

Collaborative problem solving (CPS) is an essential skill for the 21st century workforce but remains difficult to assess. Understanding how CPS skills affect CPS performance outcomes can inform CPS training, task design, feedback design, and automated assessment. We investigated CPS behaviors (individually and in co-occurring patterns) in 101 (N = 303) remote triads who collaboratively played an educational game called Physics Playground for 45-min. Team interactions consisted of open-ended speech occurring over videoconferencing with screen sharing. We coded participant's utterances relative to a CPS framework consisting of three facets (i.e., competencies such as constructing shared knowledge) manifested in 19 specific indicators (e.g., responds to others' questions/ideas). A matching technique was used to isolate the effect of CPS behaviors on CPS outcomes (quality of solution of a game level) controlling for pertinent covariates. Mixed-effects ordinal regression models indicated that proposing solution ideas and discussing results were the major predictors of CPS performance, and that team-member activities surrounding idea generation mattered. These findings highlighted the importance of both individual and collective contributions and social and cognitive skills in successful CPS outcomes.

Multi-level vector autoregression (mlVAR) is a recently developed dynamic network model for assessing multimodal temporal data streams derived from multiple users over time. Importantly, mlVAR facilitates investigations into highly complex collaborative interactions within a unifed framework. In order to demonstrate the utility of mlVAR for understanding the temporal dynamics of multimodal multi-party (MMP) interactions, we apply it to 9 signals measured from 201 users (67 triads) who engaged in a 15-minute collaborative problem solving task. Measured signals refect participants' afective states (positive valence and negative valence), physiological states (skin conductance and heart rate), attention (gaze fxation duration and gaze dispersion), nonverbal communication (head acceleration and facial expressiveness), and verbal communication (speech rate). Using node-level metrics of in-strength, out-strength, and synchrony, we show that mlVAR is capable of teasing apart complex role-based dynamics (controller, primary contributor, or secondary contributor) between participants. Our fndings also provide evidence for a complex feedback system between individuals where internal states (i.e., skin conductance) are infuenced by external signals of shared attention and communication (i.e., gaze and speech).

We investigated the generalizability of language-based analytics models across two collaborative problem solving (CPS) tasks: an educational physics game and a block programming challenge. We analyzed a dataset of 95 triads (N=285) who used videoconferencing to collaborate on both tasks for an hour. We trained supervised natural language processing classifiers on automatic speech recognition transcripts to predict the human-coded CPS facets (skills) of constructing shared knowledge, negotiation / coordination, and maintaining team function. We tested three methods for representing collaborative discourse: (1) deep transfer learning (using BERT), (2) n-grams (counts of words/phrases), and (3) word categories (using the Linguistic Inquiry Word Count [LIWC] dictionary). We found that the BERT and LIWC methods generalized across tasks with only a small degradation in performance (Transfer Ratio of.93 with 1 indicating perfect transfer), while the n-grams had limited generalizability (Transfer Ratio of.86), suggesting overfitting to task-specific language. We discuss the implications of our findings for deploying language-based collaboration analytics in authentic educational environments.