Gathering Response Process Data for a Problem-Solving Measure Through Whole-Class Think Alouds

Response process validity evidence provides a window into a respondent's cognitive processing. The purpose of this study is to describe a new data collection tool called a whole-class think aloud (WCTA). This work is performed as part of test development for a series of problem-solving measures to be used in elementary and middle grades. Data from third-grade students were collected in a 1-1 think-aloud setting and compared to data from similar students as part of WCTAs. Findings indicated that students performed similarly on the items when the two think-aloud settings were compared. Respondents also needed less encouragement to share ideas aloud during the WCTA compared to the 1-1 think aloud. They also communicated feeling more comfortable in the WCTA setting compared to the 1-1 think aloud. Drawing the findings together, WCTAs functioned as well if not better, than 1-1 think alouds for the purpose of contextualizing third-grade students' cognitive processes. Future studies using WCTAs are recommended to explore their limitations and other factors that might impact their success as data gathering tools.