Reliable multi method assessment of metacognition use in chemistry problem solving

Abstract

Metacognition is fundamental in achieving understanding of chemistry and developing of problem solving skills. This paper describes an across-method-and-time instrument designed to assess the use of metacognition in chemistry problem solving. This multi method instrument combines a self report, namely the Metacognitive Activities Inventory (MCA-I), with a concurrent automated online instrument, Interactive MultiMedia Exercises (IMMEX). IMMEX presents participants with ill defined problems and collects students actions as they navigate the problem space. Artificial neural networks and hidden Markov modeling applied to the data collected with IMMEX produce two assessment parameters: the strategy state, which is related to the metacognitive qualities of the solution path employed, and the ability which is a measure of the problem difficulty students can properly handle. The ability values are significantly correlated with the MCA-I scores, and groups of students who performed using more metacognitive state strategies had significantly higher mean MCA-I values than those using fewer metacognitive strategies. This evidence is indicative of convergence between the methods. This instrument can be used diagnostically to guide the implementation of interventions to promote the use of metacognition; it takes little instructional time, is readily available and allows for the assessment of large cohorts.