We adopt a multimodal approach to investigating team interactions in the context of remote collaborative problem solving (CPS). Our goal is to understand multimodal patterns that emerge and their relation with collaborative outcomes. We measured speech rate, body movement, and galvanic skin response from 101 triads (303 participants) who used video conferencing software to collaboratively solve challenging levels in an educational physics game. We use multi-dimensional recurrence quantification analysis (MdRQA) to quantify patterns of team-level regularity, or repeated patterns of activity in these three modalities. We found that teams exhibit significant regularity above chance baselines. Regularity was unaffected by task factors. but had a quadratic relationship with session time in that it initially increased but then decreased as the session progressed. Importantly, teams that produce more varied behavioral patterns (irregularity) reported higher emotional valence and performed better on a subset of the problem solving tasks. Regularity did not predict arousal or subjective perceptions of the collaboration. We discuss implications of our findings for the design of systems that aim to improve collaborative outcomes by monitoring the ongoing collaboration and intervening accordingly.

Collaborative problem solving (CPS) is a crucial 21st century skill; however, current technologies fall short of effectively supporting CPS processes, especially for remote, computer-enabled interactions. In order to develop next-generation computer-supported collaborative systems that enhance CPS processes and outcomes by monitoring and responding to the unfolding collaboration, we investigate automated detection of three critical CPS process – construction of shared knowledge, negotiation/coordination, and maintaining team function – derived from a validated CPS framework. Our data consists of 32 triads who were tasked with collaboratively solving a challenging visual computer programming task for 20 minutes using commercial videoconferencing software. We used automatic speech recognition to generate transcripts of 11,163 utterances, which trained humans coded for evidence of the above three CPS processes using a set of behavioral indicators. We aimed to automate the trained human-raters’ codes in a team-independent fashion (current study) in order to provide automatic real-time or offline feedback (future work). We used Random Forest classifiers trained on the words themselves (bag of n-grams) or with word categories (e.g., emotions, thinking styles, social constructs) from the Linguistic Inquiry Word Count (LIWC) tool. Despite imperfect automatic speech recognition, the n-gram models achieved AUROC (area under the receiver operating characteristic curve) scores of .85, .77, and .77 for construction of shared knowledge, negotiation/coordination, and maintaining team function, respectively; these reflect 70%, 54%, and 54% improvements over chance. The LIWC-category models achieved similar scores of .82, .74, and .73 (64%, 48%, and 46% improvement over chance). Further, the LIWC model-derived scores predicted CPS outcomes more similar to human codes, demonstrating predictive validity. We discuss embedding our models in collaborative interfaces for assessment and dynamic intervention aimed at improving CPS outcomes. Copyright © ACM

We investigated how affective states influence expository text comprehension and whether text valence moderates the effects (i.e., mood congruency). In Experiment 1 participants were randomly assigned to a happy or sad affective state (elicited via films) before reading a positive or negative version of a scientific text on animal adaptations. Participants (n = 79) in the sad (film) group had higher scores on deep-reasoning (d = .312) but not surface-level questions on a subsequent multiple-choice comprehension assessment; there was also no evidence for mood congruence. Using a neutral version of the same text, in Experiment 2 participants (n = 52) in a fearful condition performed better on surface-level comprehension questions (d = .594) compared with a sad condition, but the groups were on par for deep-reasoning questions. Experiment 3 (n = 595) did not replicate the findings from Experiment 2 (no comprehension differences between the sad and fear groups) and there were no differences between the fear and happy groups. However, the sad group outperformed the happy group on deep-reasoning questions (d = .210), thereby replicating Experiment 1. The overall findings were confirmed after pooling the data from the three experiments to increase power.

Linguistic alignment (LA) is the tendency during a conversation to reuse each other's linguistic expressions, including lexical, conceptual, or syntactic structures. LA is often argued to be a crucial driver in reciprocal understanding and interpersonal rapport, though its precise dynamics and effects are still controversial. One barrier to more systematic investigation of these effects lies in the diversity in the methods employed to analyze LA, which makes it difficult to integrate and compare results of individual studies. To overcome this issue, we have developed ALIGN (Analyzing Linguistic Interactions with Generalizable techNiques), an open-source Python package to measure LA in conversation (https://pypi.python.org/pypi/align) along with in-depth open-source tutorials hosted on ALIGN's GitHub repository (https://github.com/nickduran/align-linguistic-alignment). Here, we first describe the challenges in the study of LA and outline how ALIGN can address them. We then demonstrate how our analytical protocol can be applied to theory-driven questions using a complex corpus of dialogue (the Devil's Advocate corpus; Duran & Fusaroli, 2017). We close by identifying further challenges and point to future developments of the field.

We model coordination and coregulation patterns in 33 triads engaged in collaboratively solving a challenging computer programming task for approximately 20 minutes. Our goal is to prospectively model speech rate (words/sec) - an important signal of turn taking and active participation - of one teammate (A or B or C) from time lagged nonverbal signals (speech rate and acoustic-prosodic features) of the other two (i.e., A + B -> C; A + C -> B; B + C -> A) and task-related context features. We trained feed -forward neural networks (FFNNs) and long sshort-term memory recurrent neural networks (LSTMs) using group level nested cross-validation. LSTMs outperformed FFNNs and a chance baseline and could predict speech rate up to 6s into the future. A multimodal combination of speech rate, acoustic prosodic, and task context features outperformed unimodal and bimodal signals. The extent to which the models could predict an individual's speech rate was positively related to that individual's scores on a subsequent posttest, suggesting a link between coordination/coregulation and collaborative learning outcomes. We discuss applications of the models for real-time systems that monitor the collaborative process and intervene to promote positive collaborative outcomes.

Positive interpersonal relationships require shared understanding along with a sense of rapport. A key facet of rapport is mirroring and convergence of facial expression and body language, known as nonverbal synchrony. We examined nonverbal synchrony in a study of 29 heterosexual romantic couples, in which audio, video, and bracelet accelerometer were recorded during three conversations. We extracted facial expression, body movement, and acoustic-prosodic features to train neural network models that predicted the nonverbal behaviors of one partner from those of the other. Recurrent models (LSTMs) outperformed feed-forward neural networks and other chance baselines. The models learned behaviors encompassing facial responses, speech-related facial movements, and head movement. However, they did not capture fleeting or periodic behaviors, such as nodding, head turning, and hand gestures. Notably, a preliminary analysis of clinical measures showed greater association with our model outputs than correlation of raw signals. We discuss potential uses of these generative models as a research tool to complement current analytical methods along with real world applications (e.g., as a tool in therapy).

We model collaborative problem solving outcomes using data from 37 triads who completed a challenging computer programming task. Participants individually rated their group's performance, communication, cooperation, and agreeableness after the session, which were aggregated to produce group-level measures of subjective outcomes. We scored teams on objective task outcomes and measured individual students' learning outcomes with a posttest. Groups with similar personalities performed better on the task and had higher ratings of communication, cooperation, and agreeableness. Importantly, greater deviation in teammates' perception of group performance and higher ratings of communication, cooperation, and agreeableness negatively predicted individual learning. We discuss findings from the perspective of group work norms and consider applications to intelligent systems that support collaborative problem solving.

In this paper, we review computer-based assessment for learning (CBAfL), in elementary and secondary education, as a viable way to merge instruction and assessment of students' developing proficiencies. We begin by contextualizing our topic relative to summative and formative assessment before presenting the current literature, which we categorized into the following: (a) supplementary use in classrooms, (b) web-based, and (c) data-driven, continuous CBAfL. Examples of research studies per category are provided. Findings show that using CBAfL in the classroom, via the Internet, or embedded in a game, generally enhances learning and other outcomes across a range of content areas (e.g. biology, math, and programming). One conclusion is that feedback, to be most beneficial to learning, should not be overly complex and must be used to be effective. Findings also showed that the quality of the assessment (i.e. validity, reliability, and efficiency) is unimpaired by the inclusion of feedback. The possibilities created by advances in the learning sciences, measurement, and technology have paved the way toward new assessment approaches that will support personalized learning and that can accurately measure and support complex competencies. The next steps involve evaluating the new assessments regarding their psychometric properties and support of learning. Lay Description: What is currently known about computer-based assessment for learning (CBAfL)? Early CBAfL systems were divided into linear and branching programs with no diagnostics and evolved into systems possessing more personalized/adaptive remediation with AI. Current CBAfL can support a range of competencies in various digital environments. Advanced learning analytic methods include learning analytics and stealth assessment. What our paper adds to what is already known about CBA for learning? Trends in our review suggest CBAs will improve in personalizing learning in a variety of contexts. Innovative CBAfL techniques will move beyond the laboratory and into the mainstream. Boundaries between instruction, learning and assessment will eventually become blurred, thus removing the need for high-stake tests of learning. What are the implications of our topic for practitioners? With CBAfL advances, teachers will have more time to provide targeted support to learners. Students would not need to worry about taking exams if CBAfL is continuous and formative. Educators will be able to provide personalized learning experiences for diverse students. Students will be equipped with the knowledge and skills needed to succeed in the 21st century. © 2017 John Wiley & Sons Ltd

This paper examines the growing field of computational thinking (CT) in education. A review of the relevant literature shows a diversity in definitions, interventions, assessments, and models. After synthesizing various approaches used to develop the construct in K-16 settings, we have created the following working definition of CT: The conceptual foundation required to solve problems effectively and efficiently (i.e., algorithmically, with or without the assistance of computers) with solutions that are reusable in different contexts. This definition highlights that CT is primarily a way of thinking and acting, which can be exhibited through the use particular skills, which then can become the basis for performance-based assessments of CT skills. Based on the literature, we categorized CT into six main facets: decomposition, abstraction, algorithm design, debugging, iteration, and generalization. This paper shows examples of CT definitions, interventions, assessments, and models across a variety of disciplines, with a call for more extensive research in this area.