Multi-modal approaches in engineering and computing education are still in its early stages. With the advent of new technologies and communication platforms, understanding the principles and elements of multi-modal work will help scholars to answer complex research questions related to engineering and computing education. Multi-modal approaches consist of research principles and practices that aim to explore the multi-sensory ways humans experience the complexity and multiplicity of their surrounding world as it happens. This manuscript will elaborate on the principles of multi-modal research, highlight examples in the engineering and computing education literature, and share considerations and strategies. The manuscript’s purpose is to guide scholars who wish to capture participant experiences of phenomena naturalistically and authentically, and in near-real-time. © 2023 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.

The multi-modal approach for conducting education research is gaining traction among the engineering and computing education disciplines. This panel aims to introduce this novel and emergent approach to answering important research questions related to engineering and computing education. Additionally, the panelists will share their personal experiences on how they have incorporated a multi-modal approach for their research. The panelists will also share the best practices and common mistakes researchers could make using this methodology. © 2023 IEEE.

Background This study examined the relations between students' expectancies for success and a physiological component of test anxiety, salivary cortisol, during an authentic testing setting. Aims The aim of the study was to better understand the connection between shifts in students' control appraisals and changes in the physiological component of test anxiety. Sample The study comprised 45 undergraduate engineering majors in the United States. Methods Survey data concerning students' expectancy for success and saliva samples were taken before, during and after the practice midterm examination prior to their actual in-class examination. Results Students' expectancy for success declined during the examination while cortisol levels declined from the beginning to middle of the examination and began to increase again as a function of time. Although students' initial levels of expectancy for success and cortisol were not correlated, there was a negative relation between change in cortisol and change in expectancy for success. Conclusions Our study demonstrates a relation between salivary cortisol, a physiological component of test anxiety and students' expectancy for success in an authentic testing context. Most students saw a decrease in cortisol during the examination, suggesting anticipatory anxiety prior to the test and a return to homeostasis as the examination progressed. Some students, however, did not see a declination in cortisol, suggesting they may not have recovered from pre-examination anxiety. The negative relation between change in cortisol and expectancy for success suggests that students who had the greatest decrease in expectancy for success saw the smallest recovery in cortisol.

[No abstract available]

Multi-modal approaches have increasingly shown promise in exploring the human side of engineering via the assessment of authentic responses to learning or working environments. This study explores the utility of non-invasive physiological wrist sensors in measuring the reactive and regulatory responses of a group of 161 engineering students taking an authentic engineering practice exam. The practice exam was categorized into Conceptual problems (e.g., rote memorization) and Analytical problems (e.g., requiring application of learned concepts through equations and free-body diagrams). Responses were measured through electrodermal activity and indicators of performance. Findings identified that the type of practice exam problem, even if designed to be within the moderate range of difficulty, influenced how students reacted to and regulated their performance to the problem (as seen by stronger positive correlations in the Analytical problems) and that these may occur via multi-componential processes.

Students' academic learning, performance, and motivation are ongoing topics in engineering education. Those studies that have attempted to understand the mechanisms of motivation in authentic classroom settings and scenarios are few and limited to the methods used (e.g., self-reports, observations). This Work-in-Progress study explores the utility of electrodermal activity (EDA) and temperature sensors in accurately informing scholars about student performance during an exam in real-time. Correlations between each factor were analyzed. Initial results suggest that peripheral skin temperature has a weak, positive hut significant correlation to exam question difficulty r=0.08; p<0.001). Also, electrodermal activity and temperature showed a weak, positive, but significant correlation (r-0.13; p<0.0.5). The electrodermal activity showed a weak, positive, but significant correlation to exam question difficulty (:-0.16; p<0.01). Also, skin temperature correlations with difficulty index (did not) changed across semesters (r-0.18; p<0.001). We also developed a multiple regression model and found moderately significant relationships between EDA, difficulty index, and skin temperature (r=0.45; p<0.05). The findings suggest that performance is tied to physiological responses among students during exam taking, indicating a possible connection between emotions and cognition via physiology.

Over the past ten years, research into students' emotions in educational environments has increased. Although researchers have called for more studies that rely on objective measures of emotional experience, limitations on utilizing multi-modal data sources exist. Studies of emotion and emotional regulation in classrooms traditionally rely on survey instruments, experience-sampling, artifacts, interviews, or observational procedures. These methods, while valuable, are mainly dependent on participant or observer subjectivity and is limited in its authentic measurement of students' real-time performance to a classroom activity or task. The latter, in particular, poses a stumbling block to many scholars seeking to objectively measure emotions and other related measures in the classroom, in real-time. The purpose of this work is to present a protocol to experimentally study students' real-time responses to exam experiences during an authentic assessment situation. For this, a team of educational psychologists, engineers, and engineering education researchers designed an experimental protocol that retained the limits required for accurate physiological sensor measurement, best-practices of salivary collection, and an authentic testing environment. In particular, existing studies that rely on physiological sensors are conducted in experimental environments that are disconnected from educational settings (e.g., Trier Stress Test), detached in time (e.g., before or after a task), or introduce analysis error (e.g., use of sensors in environments where students are likely to move). This limits our understanding of students' real-time responses to classroom activities and tasks. Furthermore, recent research has called for more considerations to be covered around issues of recruitment, replicability, validity, setups, data cleaning, preliminary analysis, and particular circumstances (e.g., adding a variable in the experimental design) in academic emotions research that relies on multi-modal approaches.