Whereas social visual attention has been examined in computer-mediated (e.g., shared screen) or video-mediated (e.g., FaceTime) interaction, it has yet to be studied in mixed-media interfaces that combine video of the conversant along with other UI elements. We analyzed eye gaze of 37 dyads (74 participants) who were tasked with negotiating the price of a new car (as a buyer and seller) using mixed-media video conferencing under competitive or cooperative negotiation instructions (experimental manipulation). We used multidimensional recurrence quantification analysis to extract spatio-temporal patterns corresponding to mutual gaze (individuals look at each other), joint attention (individuals focus on the same elements of the interface), and gaze aversion (an individual looks at their partner, who is looking elsewhere). Our results indicated that joint attention predicted the sum of points attained by the buyer and seller (i.e., the joint score). In contrast, gaze aversion was associated with faster time to complete the negotiation, but with a lower joint score. Unexpectedly, mutual gaze was highly infrequent and unrelated to the negotiation outcomes and none of the gaze patterns predicted subjective perceptions of the negotiation. There were also no effects of gender composition or negotiation condition on the gaze patterns or negotiation outcomes. Our results suggest that social visual attention may operate differently in mixed-media collaborative interfaces than in face-to-face interaction. As mixed-media collaborative interfaces gain prominence, our work can be leveraged to inform the design of gaze-sensitive user interfaces that support remote negotiations among other tasks.

In an increasingly globalized and service-oriented economy, people need to engage in computer-mediated collaborative problem solving (CPS) with diverse teams. However, teams routinely fail to live up to expectations, showcasing the need for technologies that help develop effective collaboration skills. We take a step in this direction by investigating how different dimensions of team diversity (demographic, personality, attitudes towards teamwork, prior domain experience) predict objective (e.g. effective solutions) and subjective (e.g. positive perceptions) collaborative outcomes. We collected data from 96 triads who engaged in a 30-minute CPS task via videoconferencing. We found that demographic diversity and differing attitudes towards teamwork predicted impressions of positive engagement, while personality diversity predicted learning outcomes. Importantly, these relationships were maintained after accounting for team makeup. None of the diversity measures predicted task performance. We discuss how our findings can be incorporated into technologies that aim to help diverse teams develop CPS skills.

Prior research undertaken for the purpose of identifying deceptive language has focused on deception as it is used for nefarious ends, such as purposeful lying. However, despite the intent to mislead, not all examples of deception are carried out for malevolent ends. In this study, we describe the linguistic features of humorous deception. Specifically, we analyzed the linguistic features of 753 news stories, 1/3 of which were truthful and 2/3 of which we categorized as examples of humorous deception. The news stories we analyzed occurred naturally as part of a segment named Bluff the Listener on the popular American radio quiz show Wait, Wait… Don’t Tell Me!. Using a combination of supervised learning and predictive modeling, we identified 11 linguistic features accounting for approximately 18% of the variance between humorous deception and truthful news stories. These linguistic features suggested the deceptive news stories were more confident and descriptive but also less cohesive when compared to the truthful new stories. We suggest these findings reflect the dual communicative goal of this unique type of discourse to simultaneously deceive and be humorous. © 2020 Stephen Skalicky, Nicholas Duran, and Scott A. Crossley This is an open-access article distributed under the terms of a Creative Commons Attribution License (http:==creativecommons:org=licenses=by=3:0=).

Modeling team phenomena from multiparty interactions inherently requires combining signals from multiple teammates, often by weighting strategies. Here, we explored the hypothesis that strategic weighting signals from individual teammates would outperform an equal weighting baseline. Accordingly, we explored role-, trait-, and behavior-based weighting of behavioral signals across team members. We analyzed data from 101 triads engaged in computer-mediated collaborative problem solving (CPS) in an educational physics game. We investigated the accuracy of machine-learned models trained on facial expressions, acoustic-prosodics, eye gaze, and task context information, computed one-minute prior to the end of a game level, at predicting success at solving that level. AUROCs for unimodal models that equally weighted features from the three teammates ranged from .54 to .67, whereas a combination of gaze, face, and task context features, achieved an AUROC of .73. The various multiparty weighting strategies did not outperform an equal-weighting baseline. However, our best nonverbal model (AUROC = .73) outperformed a language-based model (AUROC = .67), and there were some advantages to combining the two (AUROC = .75). Finally, models aimed at prospectively predicting performance on a minute-by-minute basis from the start of the level achieved a lower, but still above-chance, AUROC of .60. We discuss implications for multiparty modeling of team performance and other team constructs. © 2020 ACM.

The low cost of mobile computing devices, coupled with cheap and elastic cloud-based data storage and computing, along with advances in the fields of artificial intelligence and machine learning, has launched a data revolution in the science of learning. How do we gain knowledge about how people learn from data on hundreds of thousands of students and their teachers collected across extended time frames? This chapter considers this question, starting with a word about the data itself. How to make sense of all these data? The chapter briefly considers some of the key research areas and computational techniques used. The key research areas and computational techniques include: formative assessment; supervised, semisupervised, and unsupervised learning; relationship discovery; sequence modeling; content analysis and natural language processing; and multisensory, multimodal modeling. The chapter finally turns to four case studies from authors' lab to illustrate a broad range of research goals, data, and methods. (PsycInfo Database Record (c) 2024 APA, all rights reserved)

Collaborative problem solving (CPS) is an essential skill in the 21st century. There is a need for an appropriate framework and operationalization of CPS to guide its assessment and support and across multiple domains. Accordingly, we synthesized prior research on CPS to construct a generalized CPS competency model (i.e., skills and abilities) consisting of the following core facets: constructing shared knowledge, negotiation/coordination, and maintaining team function. Each facet has two sub-facets, which in turn, have multiple verbal and nonverbal indicators. We validated our model in two empirical studies involving triadic CPS, but in very different contexts - middle-school students playing an educational game in a 3-h, face-to-face session vs. college students engaging in a visual programming task for 20 min via videoconferencing. We used principal component analysis to investigate whether the empirical data aligned with our theorized model. Correlational analyses provided evidence on the orthogonality of the facets and their independence to individual differences in prior knowledge, intelligence and personality and regression analyses indicated that the facets predicted both subjective and objective outcome measures controlling for several covariates. Thus we provide initial evidence for the convergent, discriminant, and predictive validity of our model by using two different CPS contexts and student populations. This shows promise towards generalizing across various human-human CPS interactive environments.

Collaborative problem solving (CPS) in virtual environments is an increasingly important context of 21st century learning. However, our understanding of this complex and dynamic phenomenon is still limited. Here, we examine unimodal primitives (activity on the screen, speech, and body movements), and their multimodal combinations during remote CPS. We analyze two datasets where 116 triads collaboratively engaged in a challenging visual programming task using video conferencing software. We investigate how UI-interactions, behavioral primitives, and multimodal patterns were associated with teams' subjective and objective performance outcomes. We found that idling with limited speech (i.e., silence or backchannel feedback only) and without movement was negatively correlated with task performance and with participants' subjective perceptions of the collaboration. However, being silent and focused during solution execution was positively correlated with task performance. Results illustrate that in some cases, multimodal patterns improved the predictions and improved explanatory power over the unimodal primitives. We discuss how the findings can inform the design of real-time interventions for remote CPS.

We hypothesize that effective collaboration is facilitated when individuals and environmental components form a synergy where they work together and regulate one another to produce stable patterns of behavior, or regularity, as well as adaptively reorganize to form new behaviors, or irregularity. We tested this hypothesis in a study with 32 triads who collaboratively solved a challenging visual computer programming task for 20 min following an introductory warm-up phase. Multidimensional recurrence quantification analysis was used to examine fine-grained (i.e., every 10 s) collective patterns of regularity across team members' speech rate, body movement, and team interaction with the shared user interface. We found that teams exhibited significant patterns of regularity as compared to shuffled baselines, but there were no systematic trends in regularity across time. We also found that periods of regularity were associated with a reduction in overall behavior. Notably, the production of irregular behavior predicted expert-coded metrics of collaborative activity, such as teams' ability to construct shared knowledge and effectively negotiate and coordinate execution of solutions, net of overall behavioral production and behavioral self-similarity. Our findings support the theory that groups can interact to form interpersonal synergies and indicate that information about system-level dynamics is a viable way to understand and predict effective collaborative processes.

We investigated how affective states influence expository text comprehension and whether text valence moderates the effects (i.e., mood congruency). In Experiment 1 participants were randomly assigned to a happy or sad affective state (elicited via films) before reading a positive or negative version of a scientific text on animal adaptations. Participants (n = 79) in the sad (film) group had higher scores on deep-reasoning (d = .312) but not surface-level questions on a subsequent multiple-choice comprehension assessment; there was also no evidence for mood congruence. Using a neutral version of the same text, in Experiment 2 participants (n = 52) in a fearful condition performed better on surface-level comprehension questions (d = .594) compared with a sad condition, but the groups were on par for deep-reasoning questions. Experiment 3 (n = 595) did not replicate the findings from Experiment 2 (no comprehension differences between the sad and fear groups) and there were no differences between the fear and happy groups. However, the sad group outperformed the happy group on deep-reasoning questions (d = .210), thereby replicating Experiment 1. The overall findings were confirmed after pooling the data from the three experiments to increase power.