Layers of competence: analyzing 8th graders’ perceptions of visual and verbal assessment questions on chemical processes vis-à-vis their performance

Abstract

The design of assessments shapes what we learn about students’ conceptual understanding. In the context of chemistry education, visual representations are important components of learning and assessment. To examine the role of representations in assessing students’ reasoning about chemical and physical changes, we developed two multiple choice questionnaires – one that represent the choice options in verbal form, and one that represent isomorphic options - in pictures. The questionnaires included a second-tier rating scale of the questions’ general comprehensibility and the clarity of the pictures. The questionnaires were distributed among 374 eight graders in two phases. In the first phase we found that on average, students performed slightly better on the verbal version, and some verbal items were significantly easier than their visual counterparts, but one item showed the opposite trend. Interviews revealed that visual representations exposed a wider range of ideas among respondents, and in some cases, created confusion. The second phase focused on the visual version, and revealed significant positive correlations between clarity judgements and performance in questions with technical pictures of graphs and molecular structures, and no correlations on others. The analysis of the interviews, together with the clarity ratings, indicates that some visuals are conceived as a layer of challenge in the questions while others entail conceptual misunderstandings that are unrelated to the surface layer of the visuals.