Bernstein, Erin E.. Using the think aloud lab method to gather student input on state-wide science assessments. Retrieved from https://doi.org/doi:10.7282/t3-5nx3-sf34
DescriptionA think aloud lab is a method used to gather evidence based on response processes to determine whether tests have construct validity. The current study sought to gather evidence based on response processes during a think aloud lab which followed a protocol that Johnstone, Bottsford-Miller, and Thompson (2006) developed. The data helped to evaluate whether TNReady science test items evoked the intended cognitive and problem-solving processes, barriers that may have impeded their access (e.g., high reading loads), and whether or not valid inferences could be drawn from the results in regard to students’ knowledge of science content. A total of 33 students across three grades (fifth, eighth, and high school) participated in the think-aloud lab. Students were asked to talk out loud about their thinking as they completed science items. Students were also asked to complete Maze reading probes and a five-item survey after the test. Survey items focused on student opinions in regard to difficulty and usability. Results indicated that the TNReady science test items evoked the intended cognitive processes. Participants interacted with the tests as the developers intended. The cognitive lab procedures are outlined, and key findings are highlighted. Theoretical influences, such as accessibility theory, cognitive load theory, and universal design standards, as well as previous think aloud studies, are reviewed. Limitations of the current study and implications for practice are also discussed.