Jump to main content
Jump to site search
Access to RSC content Close the message box

Continue to access RSC content when you are not at your institution. Follow our step-by-step guide.

Issue 2, 2019
Previous Article Next Article

Development and use of a multiple-choice item writing flaws evaluation instrument in the context of general chemistry

Author affiliations


Multiple-choice (MC) exams are common in undergraduate general chemistry courses in the United States and are known for being difficult to construct. With their extensive use in the general chemistry classroom, it is important to ensure that these exams are valid measures of what chemistry students know and can do. One threat to MC exam validity is the presence of flaws, known as item writing flaws, that can falsely inflate or deflate a student's performance on an exam, independent of their chemistry knowledge. Such flaws can disadvantage (or falsely advantage) students in their exam performance. Additionally, these flaws can introduce unwanted noise into exam data. With the numerous possible flaws that can be made during MC exam creation, it can be difficult to recognize (and avoid) these flaws when creating MC general chemistry exams. In this study a rubric, known as the Item Writing Flaws Evaluation Instrument (IWFEI), has been created that can be used to identify item writing flaws in MC exams. The instrument was developed based on a review of the item writing literature and was tested for inter-rater reliability using general chemistry exam items. The instrument was found to have a high degree of inter-rater reliability with an overall percent agreement of 91.8% and a Krippendorff Alpha of 0.836. Using the IWFEI in an analysis of 1019 general chemistry MC exam items, it was found that 83% of items contained at least one item writing flaw with the most common flaw being the inclusion of implausible distractors. From the results of this study, an instrument has been developed that can be used in both research and teaching settings. As the IWFEI is used in these settings we envision an improvement in MC exam development practice and quality.

Back to tab navigation

Supplementary files

Article information

26 Oct 2018
22 Jan 2019
First published
28 Jan 2019

Chem. Educ. Res. Pract., 2019,20, 369-382
Article type

Development and use of a multiple-choice item writing flaws evaluation instrument in the context of general chemistry

J. Breakall, C. Randles and R. Tasker, Chem. Educ. Res. Pract., 2019, 20, 369
DOI: 10.1039/C8RP00262B

Social activity

Search articles by author