Towards a theoretically sound measure of chemistry students’ motivation; investigating rank-sort survey methodology to reduce response style bias

Abstract

Prior research has demonstrated the important role of chemistry students’ affect in academic performance. Likert-scale surveys are the most prevalent tools to measure students’ affect within chemistry education research, however, data collected through a Likert-scale survey may exhibit response style bias which can hinder accurately measuring students’ affect. This study investigates the utility of a novel survey methodology, termed rank-sort survey, in understanding students’ academic motivation in a general chemistry course. Informed by Q methodology, in a rank-sort survey participants rank a set of statements in terms of level of agreement with limits in place on how many items can be assigned a particular rank. In this investigation, a rank-sort survey was developed by using statements from an existing Likert-scale instrument, the Academic Motivation Survey in Chemistry. Data collected from the rank-sort surveys, compared to Likert-scale surveys, showed a better alignment with self-determination theory, the underlying theoretical framework, and a better ability to predict students’ academic performance in chemistry. The study also discusses which surveys in chemistry education research are likely to benefit from adopting a rank-sort approach.