Metacognitive problem solving: exploration of students’ perspectives through the lens of multi-dimensional engagement

Abstract

Solving chemical problems entails content knowledge and mastery of problem-solving processes. However, students sometimes lack metacognitive processes required for problem solving in chemistry. This study investigated how first-year chemistry students engaged with the metacognitive problem-solving scaffold Goldilocks Help. Data was collected from an activity, which involved students reflectively comparing their problem-solving attempts to an expert solution. These comparative reflections (N = 373) were thematically analysed to investigate scaffold engagement in three dimensions: cognitive, emotional, and behavioural. Findings showed that scaffold use, coupled with self-reflection, allowed students to identify flaws in their solutions that were either problem specific or related to their problem-solving skills. Students were able to propose improvement strategies, such as posing prompting questions to themselves and finding multiple alternatives for evaluating an answer. Students, who initially lacked structured problem-solving skills, found that scaffolding helped them to slow down metacognitive processes that would otherwise be rushed through or engaged with on a surface level. Students’ resistance to the scaffold was due to fear of making a mistake or viewing the scaffold as requiring extra time and effort. Within a semester, many students demonstrated an improvement in successful and structured problem solving but some required more practice to internalise the scaffold. Our findings also indicated that students’ reflections on problem solving became more sophisticated as a result of continued exposure to the scaffold and iterative opportunities to compare their work to expert solutions, to self-assess, and to reflect. Further research on reflective writing in chemistry education should focus on the ipsative nature of such assessments, i.e. processes focusing on students’ own progress, growth, and improvement, compared to their previous performance, while recognising the power relations operationalised in course-embedded reflections. From the teaching practice perspective, having an awareness of students’ thoughts, emotions, and actions can help instructors differentiate between levels of student capabilities, mindsets, and needs for extra support, allowing teaching efforts to be directed at promoting metacognitive and structured problem solving.

---

Introduction

In the realm of chemistry, the process of problem solving can often pose significant challenges for students (Bodner and Herron, 2002). These challenges can arise from a lack of metacognitive engagement (Yuriev et al., 2019; Vo et al., 2022). A complex interplay of actions and influences, both from students and instructors, can collectively contribute to the hurdles encountered by learners (Yuriev et al., 2017).

Role of metacognition and metacognitive training in problem solving

Some students struggle to solve chemistry problems due to unproductive beliefs and actions. In the absence of metacognitive awareness, students’ perceptions of their abilities in comparison to their true competences can be misaligned (Boud, 2001). Negative attitudes and past negative experiences in problem solving can erode students' self-confidence in their problem-solving abilities (Harper, 2006; Drummond and Selvaratnam, 2008), while limited problem-solving experience can lead to a tendency to overestimate such abilities (Schlösser et al., 2013). Additionally, the development of effective problem-solving strategies can be overshadowed if students are focussed on seeking the “right answer” (Cohen et al., 2000; Bodner and Herron, 2002; Harper, 2006) or on hastily completing the task at the expense of accuracy and the productivity of their problem-solving processes (Rodriguez et al., 2019).

While metacognition is an important problem-solving skill, there is a lack of metacognitive strategies incorporated into instructional approaches (Mutambuki et al., 2020; Heidbrink and Weinrich, 2021). For example, instructional materials requiring students to follow procedures without fostering their capacity to articulate their reasoning can often hinder the cultivation of essential problem-solving abilities. To address these multifaceted challenges faced by students and instructors, it is imperative to incorporate metacognitive and reflective strategies into both learning activities and teaching practices.

To motivate student engagement with metacognitively-aware problem solving, instructors must actively promote student buy-in (White et al., 2015; Tharayil et al., 2018). One such activity involves incorporating worked solutions/examples into the learning process (Mayer, 2020). Worked solutions provide students with a comprehensive understanding of the steps required to solve a problem, going beyond mere numerical values to demonstrate the underlying processes (Kalyuga et al., 2001) and engaging them in ‘cognitive apprenticeship’ (Collins et al., 1991; Kirschner and Hendrick, 2024). Worked examples have been used in chemistry courses in a variety of ways: in lectures (Smith, 2013), tutorials (Rubin and Abrams, 2015), videos (Koretsky, 2020), and as pre-class materials (Fitzgerald and Li, 2015). Exposure to worked solutions allows students to understand how to apply problem-solving strategies and to appreciate the impact of their problem-solving actions. Structuring information in the form of a problem-solving example minimises the demands on working memory, making it less mentally taxing for students to integrate metacognition during problem solving (Sweller, 1994). External references, such as worked solutions, could serve as calibration tools, allowing students to reflect on their own solutions, thereby fostering greater self-awareness and leading to improved problem solving (van Harsel et al., 2020).

Metacognitive scaffolding for problem solving

Scaffolding is an effective approach to cultivate metacognitive processes in student problem solving (Belland, 2011; Graulich et al., 2021). Our previous work involved the design of the Goldilocks Help (GH) scaffold (Fig. 1), to provide metacognitive support to students during problem solving (Yuriev et al., 2017). GH scaffolds problem solving by prompting students: to consider the problem statement and explicit and implicit information embedded in it (‘understanding’), to analyse relationships between the variables and plan mathematical re-arrangements to express for the unknown, to implement the solution, both quantitatively and dimensionally, and finally to evaluate the answer and the solution process. Based on prior research, students who achieved academic success consistently demonstrated scaffold elements and a deeper understanding of the problem statement (Vo et al., 2024a). However, while the GH workflow played a vital part in fostering a structured and metacognitively aware problem-solving approach, certain students still displayed resistance to the scaffold (Vo et al., 2022).

Fig. 1 Goldilocks Help, a problem-solving scaffold with metacognitive prompts. Reproduced from reference by Yuriev et al. (2017) with permission from The Royal Society of Chemistry.

Student problem-solving profiles

Previously, based on teaching associates’ perspectives (Vo et al., 2022), we have proposed that student solutions can be classified into one of four profiles, defined by problem-solving success and engagement with the scaffold (Fig. 2). In this representation, each quadrant corresponds to a separate student type, or profile.

Fig. 2 Four student profiles of problem-solving. Reproduced from reference by Vo et al. (2022) with permission from The Royal Society of Chemistry, with minor modifications.

Students that are unsuccessful in problem solving and do not engage with the scaffold are located in the bottom-left quadrant. In TAs’ observations, these students “rush” through problems (hence named Sprinters) and consider the scaffold unnecessary or time-consuming. The bottom-right quadrant corresponds to student types who use the scaffold but often unsuccessfully. TAs noted that these students seem to be motivated by external factors like grades or use the scaffold “out of obligation,” and thus these performance-driven (Ames, 1992; Elliot and Hulleman, 2017) students were called Collectors. Students who are successful in their problem-solving attempts are categorised into two top profiles. Those who actively utilise the scaffold fall in the top-right quadrant. From TAs’ perspectives, these students are likely to be intrinsically motivated by self-improvement and invest effort into mastering the tool, therefore these mastery-driven students (Ames, 1992; Elliot and Hulleman, 2017) were designated as Explorers. Successful students that do not use the scaffold fall into the top-left quadrant. TAs suggested that these students already have successful problem-solving approaches and perceive little value in using the scaffold. Therefore, these students were referred to as Settlers.

This classification of student problem-solving types assumes a hierarchical relationship, whereby students ideally progress from Sprinters and/or Collectors to Explorers, and eventually Settlers. The transition from a Collector to an Explorer (a ‘vertical’ shift) may occur due to the shift in the nature of motivation to engage with the scaffold. Extrinsically-motivated students practising a performative ‘tick box’ approach to problem solving could eventually transition to mastery motivation. Continued engagement with the scaffold would expose Collectors to its benefits and thereby intrinsically motivate them to develop structured problem-solving skills. The shift from an Explorer to a Settler (a ‘horizontal’ shift) is related to the internalisation of the scaffold. Once structured problem-solving skills are developed and naturally incorporated into their thought processes, the Explorers become Settlers. The scaffold is a temporary tool that is introduced to assist students in the development of cognitive processes. Therefore, once the student is able to handle challenging chemical problems due to their newly developed problem-solving skills, the explicit scaffold is no longer required and can be phased out (Belland, 2011).

Using student written work, we have investigated the classification of student solutions, based on the demonstration of structured problem solving and correctness of the solution (Vo et al., 2024a). In the present paper, we further explore these profiles by examining students’ reflections and gaining insight into their perceptions, values, and motivations.

Reflective practice as a critical component of learning and skill development

Reflective practice is a process of engaging with one's experiences to make sense of what has transpired (Boud, 2001; Jay and Johnson, 2002; Silver, 2010). Such practice acknowledges that learning is built upon prior experience and that any attempt to promote meaningful learning must take that experience into account. As a result, being reflective improves an understanding of oneself, which in turn improves cognitive abilities and self-regulation. Reflective practice can be integrated into student learning via comparative reflection, a process that involves analysing similarities and differences between concepts and ideas (Nicol, 2021). Silver (2010) outlines four phases to help educators incorporate comparative reflection in student learning (Fig. 3): setting up criteria, comparison, refinement, and transfer. Comparison could be non-cognate (to a peer's or an expert's work) or cognate (to one's own prior work). Employing teaching strategies such as using criteria to help students create a detailed comparison, prompt discussions between students to refine their comparisons, and applying learnings to a similar task enhances student understanding and increases knowledge transfer.

Fig. 3 Silver's (2010) four phases of comparative reflection in student learning.

The ability to productively utilise comparative reflection in problem solving depends on an individual's level of expertise (Yan, 2020). When solving problems, seasoned problem solvers use fast thinking, benefiting from the automaticity of their well-constructed schemas (Kahneman, 2011). Conversely, for inexperienced problem solvers, such as students, intentionally building these mental structures is challenging. Therefore, they often resort to means-end strategies, resulting in dead-ends and false starts that impede their progress (Yuriev et al., 2017). Engaging in a deliberate reflective comparison of different problem-solving approaches takes time and effort (Varga and Hamburger, 2014) and needs to be purposefully encouraged. Thus, integration of reflection into problem-solving instruction discourages the use of hasty, externally-motivated problem-solving strategies (Vo et al., 2022).

Theoretical framework

Engagement is a multifaceted phenomenon. Interpretation of engagement depends on the study context, object (individual students or a cohort), and learning activities (Lawrie, 2023). Different perspectives on engagement exist, which reflect the diversity of possible interpretations of this construct (Wong and Liem, 2022). This study was underpinned by the prevailing theory that conceptualises engagement as a phenomenon existing in three main dimensions: cognitive, behavioural, and emotional (Fredricks et al., 2004). These researchers later elaborated and updated their theoretical engagement framework by adding the social dimension (Fredricks et al., 2016; Wang et al., 2016). Naibert et al. (2022) explored students’ perceptions of the three original dimensions of engagement, in the context of worksheet activities in a general chemistry course. They found that students tended to conflate the behavioural and cognitive dimensions. And they have also identified social themes in students’ perceptions of engagement, reflecting the social dimension.

Cognitive engagement is related to students’ investment of time and effort into learning or mastering a skill (Fredricks, et al., 2004). Students judge the engagement effort in relation to the perceived benefits. Engagement to achieve mastery requires deep-learning strategies (Marton and Säljö, 1976), self-regulation, and metacognition, to promote understanding and expertise (Cleary and Zimmerman, 2012). Conversely, performative engagement relies on surface effort-avoidant learning strategies, such as rote memorisation. Therefore, cognitive engagement reflects students’ stance on problem solving and ability to cope with perceived failures. Emotional engagement encompasses students’ reactions to the instructor or peers, and the classroom, in a general sense of the word. Such engagement can span from feeling frustrated or anxious to happy and interested. A positive or negative emotion can influence the activation or deactivation of engagement. For example, excitement can result in high engagement and prolonged attention (Pekrun, 2006). Behavioural engagement involves participatory actions ranging from appearing withdrawn, through cooperative participation, to contributing to class discussions (Chi and Wylie, 2014). The fourth pillar of student engagement is social engagement, which reflects students’ willingness to collaborate, give and receive feedback, and assist peers (Bowden et al., 2021). This multi-dimensional approach to engagement recognises the complexity of student interactions with instructors and each other.

Engagement researchers point out that the three dimensions, as well as additional social and agentic dimensions, can often overlap and rarely exist in isolation (Lo, 2024). Fredricks et al. (2004) themselves acknowledged the interconnectivity of these dimensions, by noting that they are “dynamically interrelated within the individual; they are not isolated processes.” Similarly, Skinner et al. (2009) emphasised the importance of understanding engagement as a multifaceted construct, where each dimension contributes to the overall experience of engagement in learning contexts. Furthermore, they have investigated disaffection, the opposite of engagement, exemplified by giving up as a behavioural disaffection and frustration as an emotional disaffection.

Rationale for study and research questions

In this study, we focus on students' development of structured problem solving with the aid of the metacognitive scaffold Goldilocks Help. We explore how students interact with the scaffold through the three dimensions of engagement: what they think, feel, and how they act while utilising metacognitive processes to solve chemistry problems. To investigate the cognitive engagement, we studied how students use various problem-solving strategies, ranging from surface (rote memorisation) to deep (self-regulation). From the emotional perspective, we explored the students’ attitudes towards problem solving and metacognitive processes. The behavioural component related to the students’ problem-solving actions to complete a problem, or the lack thereof. Due to remote teaching and learning in 2020, students had reduced opportunity to engage in face-to-face collaborative problem solving. Therefore, the social aspect of engagement was not expected to be readily manifesting itself in the reflections to the extent that would have been expected otherwise. In this study, the research questions we addressed through analysing students’ reflections on problem solving are:

RQ1: How do students perceive their engagement with metacognitively scaffolded problem solving?

RQ2: What barriers/resistance do students perceive to encounter when taught with the metacognitive scaffold?

RQ3: In what ways do student perceptions of problem solving reflect the manifestation of the four profiles?

Methodology

This study represents the third part of the investigation of student engagement with metacognitive scaffolding. Previously, we analysed the teaching associates’ perceptions on and practices of teaching with metacognitive scaffolding (Vo et al., 2022). The data collected from students include their written problem-solving work (Vo et al., 2024a) and associated reflective writing. In this study we used a constructionist view (Burr, 2015) to analyse the reflections in order to understand students’ perceptions and experiences of using metacognitive scaffolding in solving problems. Aligning with this worldview, we used a phenomenological approach (Casey, 2007) to interpret students’ comparative reflections.

Context: setting and participants

In this study, student development of problem-solving skills and metacognitive awareness is supported by the problem-solving scaffold, Goldilocks Help (Fig. 1) (Yuriev et al., 2017). This scaffold provides specific prompts for five main problem-solving elements: understand the problem statement, analyse the relationships between knowns and unknowns, plan and implement the solution, and evaluate the answer and/or the problem-solving strategy. The scaffold was introduced to students through a combination of lectures and workshops. In an induction workshop, students were introduced to structured problem solving and metacognition. In lectures, the scaffold was used to guide students through worked examples. In workshops, students actively worked on problems using the Goldilocks Help as a guide. Teaching Associates (TAs) running the workshops were equipped with model solutions with explicit workflow prompts for all problem-solving elements. To assist students in monitoring their problem-solving processes, TAs encouraged students to explicitly demonstrate their reasoning and to make their thinking ‘visible’ (Vo et al., 2022). In 2020, problem-solving workshops were held using a combination of Zoom meetings, live (synchronous) discussion forums, and Google Docs (Yuriev et al., 2021). These online tools provided real-time, i.e. synchronous, audio-visual and written communication between students and TAs, closely mimicking the structure of a classroom interaction.

In order to study student engagement with the scaffold, we recruited first-year undergraduate students taking a physical chemistry course in the Bachelor of Pharmaceutical science program at Monash University in 2020 (BPS1031). There were 178 students enrolled in the course who completed the course activities.

Ethical considerations

This study was approved by Monash University Human Research Ethics Committee (Project ID: 23346). The objectives of the study were explained to participants and their participation was optional. Responses were de-identified, with names changed to research names for anonymity. All students completed an online consent form, and 145 students provided permission to use their reflections in the study.

Data collection

Following Silver's framework (2010), we have designed a two-part problem-solving activity (Fig. 4). This activity also follows the principles of case-based tasks, enhanced with expert feedback and reflection opportunities, used in medical education practice (Williams et al., 2017). In Part 1, students submitted a solution to an allocated problem, hand-written or typed, and a self-assessment of their problem solving. Analysis of student submissions forms the basis of a separate manuscript (Vo et al., 2024a).

Fig. 4 Two-part problem-solving activity with a focus on Task 2 (comparative reflections).

After the deadline for Part 1, students received the expert solution to the problem. The expert solution included all the Goldilocks Help components and illustrated the required level of reasoning. In Part 2, students compared their work to the expert solution (Phases 1 and 2 of Silver's framework), wrote a reflection on their solution (Phase 3), and again self-assessed their problem solving, now informed by the exposure to the expert solution. When writing the reflection, students had access to a rubric, as required by Silver's Phase 1 (Table 1). To close the loop, cohort-level feedback was provided to students whereby common approaches and errors were discussed. This activity was repeated four times for all topics taught throughout the first semester: unit conversions, acids and bases, thermodynamics, and chemical kinetics. Silver's phase 4 (transfer) was operationalised via applying the improved skills to the next round of activity.

Table 1 Rubric for the reflection on problem-solving activity

Level
Criteria	Insufficient	Partial	Full
Reflection on the productivity of own solution (correct/incorrect)	Did not reflect on the correctness of own answer	Implicitly reflected on the correctness of own answer e.g. stated their answer was ‘different’ to the expert answer	Explicitly reflected on the correctness of own answer
Identification of problem-solving elements that are similar to or differ with the expert model answer	Did not identify similarities and differences of own solution to the expert model answer	Identified some similarities and differences of own solution to the expert model answer	Identified all similarities and differences of own solution to the expert model answer
Reflection on the quality of own solution	Did not reflect on the quality of own solution	Implicitly reflected on the quality of own solution e.g. broad or inaccurate reflection of quality	Explicitly reflected on the quality of own solution
Identification of flaws or gaps in own solution	Did not identify flaws or gaps in own solution to the expert model answer	Identified some flaws or gaps in own solution to the expert model answer	Identified all flaws or gaps in own solution to the expert model answer
Identification of improvement strategies	Did not state improvement strategies to apply in future problem solving	Implicitly stated improvement strategies to apply in future problem solving e.g. referenced areas of concern to be addressed	Explicitly stated improvement strategies to apply in future problem solving

---

This study explores student perceptions of problem solving through the analysis of comparative reflections. Students were directed to reflect on the quality of their original solution, determine differences and similarities in the problem-solving approaches used in their solution in comparison to the expert solution, identify flaws and gaps, and suggest improvement strategies to incorporate into future problem-solving tasks (Table 1). Some students added annotations to their original solutions and included these in their reflections. A total of 373 student reflections (1–2 pages each) were analysed (Table 2). Analysis of the self-assessment component will be published separately (Vo et al., 2024b).

Table 2 The number of collected comparative reflections

	Number of submissions
Topic 1: unit conversions	93
Topic 2: acids and bases	97
Topic 3: thermodynamics	96
Topic 4: chemical kinetics	87
Total	373

---

Data analysis

To investigate student metacognitive awareness during problem solving and the ability to use problem-solving processes, a qualitative approach was used to analyse the student reflections. In this study we took an interpretivist stance in order to understand how and why students chose to engage (or not to engage) with structured problem solving.

Reflections were analysed abductively, to construct themes that are grounded in the data (inductive coding) but also informed by existing theories (deductive coding) (Thompson, 2022). We have applied the existing framework of multidimensional engagement (Fredricks et al., 2004; Naibert et al., 2022) and the inductive coding focused on student perceptions of structured problem solving. Students were considered to have demonstrated the types of cognitive, emotional, and behavioural engagement or non-engagement (disaffection), if they specified what they thought, felt, and did, respectively, in the problem-solving context. For example, for cognitive engagement, we looked for comments about mental strategies, for emotional engagement – comments about feelings of stress, and for behavioural engagement – comments about performing specific problem-solving actions. The behaviour was conceptualised as an ‘action’, in a manner similar to the one described by Sinatra, et al. (2015). In this conceptualisation, the action is defined as something done by a student that is not necessarily observed by a teacher or researcher, but could be self-reported by the student. The detailed description of all codes is available in the codebook (ESI). Abductive coding allowed the development of new insights and explanations from the data, while also acknowledging the influence of the existing framework and assumptions.

The reflexive thematic analysis of comparative reflections followed the guidelines of Braun and Clarke (2006; 2021) and was conducted in NVivo software, version 20.3 (QSR International). The initial analysis involved familiarisation with the data (phase 1), using the entire dataset. The next phase involved generating initial codes and capturing student perceptions on the problem-solving elements, including scaffold receptivity (engagement) and resistance (non-engagement, disaffection) (phase 2). To ensure reliability in coding, the constant comparison method (Glaser, 1965) was implemented which involved: (i) comparing new codes to formerly developed codes to ensure new data was consistent with existing data and (ii) comparing excerpts of codes between reflection/students/topics.

Codes and their definitions were then reviewed to refine and ensure internal consistency, to create a clear distinction between codes, and to combine closely related codes into sub-themes and themes. At this stage, comparative reflections were re-examined to identify additional codes and to produce the initial codebook (phase 3). The above analysis was performed by the first author (KV). In the next stage of analysis, two researchers (KV and MS) reviewed the data and the codebook and discussed the codes to ensure that the meanings and definitions were clear and representative of the data (phase 4). As a result of the review, some codes were removed, some were divided to create specificity, and some were merged. Finally, two researchers (KV and EY) performed additional analysis to refine the themes and sub-themes, generating clear names and definitions, and to reorganise them into the final codebook (phase 5, Table 3).

Table 3 Engagement and non-engagement themes and dimension sub-themes

Theme 1: engagement with structured problem solving

Cognitive sub-themes

Assessed quality and depth of solution

Identified cognitive processes to use in the future

Identified flaws in solution

Recognised progress in problem-solving skills

Identified specific mistake that led to the wrong answer

Identified strategies for future improvements

Identified benefits of using the scaffold (clearer communication, methodical and consistent approach, starting point)

Emotional sub-themes

Felt confident

Felt positive about their performance

Helps to deal with stress

Behavioural sub-themes

Demonstrated problem-solving elements

Evaluated their work

Slowed down problem-solving process for deliberate and methodical action

Used expert solution as a checklist to confirm correctness

Theme 2: non-engagement (disaffection) with structured problem solving

Cognitive sub-themes

Cognitive challenge

Struggled to use the scaffold

Unfamiliar with the scaffold

Surface level use of the scaffold

Own preferred or established method

Scaffold use was not necessary

Structured problem solving was assumed knowledge

Scaffold use required extra time and/or effort required

Did not want to make mistakes

Emotional sub-themes

Lack of self-confidence

Felt stressed

Felt over-critical of their performance

Behavioural sub-themes

Skipped problem-solving steps

Completed problem-solving steps but did not show them

Forgot

Rushed problem solving

---

At this point, the researchers looked at the reflections of individual students longitudinally, to identify their level of either receptivity or resistance to the scaffold as well as the dynamics in perceptions developed across the semester. The level of engagement was established on balance, taking all three dimensions into account. Namely, if a student expressed receptivity/engagement for two or three dimensions (e.g., cognitively and behaviourally, but not emotionally), they were considered to have high engagement. Alternatively, if a student expressed receptivity/engagement only for one dimension (e.g., only behaviourally) or not at all, they were considered to have low engagement. Based on this level (low or high) and the correctness or incorrectness of the student's answer (Vo et al., 2024a), each student was classified into one of the four profiles for each reflection cycle (Fig. 4 and Table 2).

The negotiated agreement process was used to establish the reliability of data analysis (Watts and Finkenstaedt-Quinn, 2021). The detailed documentation of the thematic process was conducted to demonstrate trustworthiness of the research (Lincoln and Guba, 1985; Nowell et al., 2017). The study examined how students engage with the scaffold over the course of a semester. This extended period allowed the researchers to avoid preconceptions, enhancing credibility. A detailed description of the data collection and analysis process was recorded to ensure other researchers can apply the findings in other contexts, promoting transferability. The research process was rational and well-documented to be easily traceable for readers to critique the research procedure's dependability. The reasoning behind the theoretical and methodological decisions was discussed throughout to ensure the interpretations were grounded in the data, ensuring confirmability.

In addition, all researchers participated in a reflexivity exercise (Barry et al., 1999), where we identified our diverse backgrounds. KV is a science graduate and a teaching associate, who completed this project as part of her doctoral studies, MS is an education researcher with a background in science education, PW is a pharmacology educator and pharmaceutical science education researcher, and EY is a chemistry educator and chemistry education researcher. This exercise allowed the team to acknowledge and address factors impacting data analysis, for example, the dual roles of academic team members as researchers and instructors. Furthermore, research meetings were conducted where researchers challenged each other's beliefs and values, primarily positivist vs. interpretivist epistemologies. These discussions enabled the researchers to reflexively engage with the data, aiding the rigour of the research.

The researchers acknowledge that, due to the complexity of the engagement construct and given the potential overlap and inter-relations between the engagement dimensions, the interpretations of students’ reflections and specific references to thoughts, emotions, and actions within the reflections, are subject to interpretation. This work presents how the research team interpreted the reality of student perceptions. The rich description of the coding process, the interpretivist stance, and the reflexive nature of the thematic analysis are re-iterated to substantiate the trustworthiness of the data analysis. Finally, having described the data collection and analysis, the researchers conclude that the study design and execution allow to claim sufficient information power. Information power requires that the study is guided by an established theory, its aim is sufficiently narrow, and the data is rich and diverse (LaDonna et al., 2021; Malterud et al., 2021). The present study used the established theory of multidimensional engagement, which we enriched by investigating students’ motivations to engage and reasons to not engage in several dimensions. The specific aim of the study is reflected in the explicit research questions, and the data is very rich and diverse due to the composition of the sample and the multiple iterations of data collection.

Results and discussion

Through qualitative analysis of the comparative reflections, an in-depth understanding of student engagement or non-engagement with structured problem solving was captured, addressing the Research Questions 1 and 2, respectively (Table 3 and Fig. 5). Sub-themes related to the cognitive, emotional, and behavioural dimensions were identified. The sub-themes are indicated by italics in the text below.

Fig. 5 Three dimensions of engagement and non-engagement with the problem-solving scaffold: cognitive, emotional, and behavioural.

Theme 1: engagement with structured problem solving

To answer the Research Question 1 (How do students perceive their engagement with metacognitively scaffolded problem solving?), sub-themes were identified that reflected how students engaged with structured problem solving cognitively, emotionally, and behaviourally.

Cognitive engagement. Students made comparisons between their work and the model answer, in terms of the quality and depth of their solutions. The work was defined by students as high quality, when it was described as ‘organised’ or when it was done ‘in the stepwise manner in a coherent and functional manner.’ Students related poor quality to a lack of clarity in their written solutions, which impacted the effectiveness of solving a problem:

‘The expert model answer was similar to mine in its initial stages, but its planning was clearer and more concise than mine, and the implementation was executed in a more methodical and effective way’. – Pham

Drawing a comparison with an expert solution enabled students to gain a fresh perspective on their work, leading to novel, not previously considered, ideas. Namely, students identified the benefits of using the scaffold. Scaffold use was reflected to help students apply a methodical and consistent approach when solving a problem. Some stated that applying a structured approach across all problems was effective, particularly for ‘longer, more difficult questions.’ Explicitly demonstrating structured problem solving was claimed to reduce students’ confusion when reviewing their own work and when communicating with other students and instructors:

‘I did so that it easier for the person that is reading my work to follow my method.’ – Andrea

The communication-related results were also observed in our earlier study that examined the perspectives of teaching associates (TAs) and the benefits of teaching with the scaffold (Vo et al., 2022). In both cases, students and instructors regarded the scaffold as a valuable tool, not only for doing problem solving but also for communicating problem solving. The scaffold offered well-defined metacognitive terminology that served as a shared language for students and instructors to discuss their problem-solving methods and challenges. This interaction between students themselves and between instructors and students facilitated the process of giving and receiving feedback and enabled students to elaborate on their comprehension of their learning and metacognitive processes (Vo et al., 2022).

For students that struggled to solve problems, the prompts within the scaffold facilitated the ability to identify a starting point and break down the problem. For example, students were asked to assess their understanding of the content, and organise data. In doing so, students were able to answer the questions of ‘where am I now?’, ‘where do I need to be next?’. The ability to answer such questions allowed students to gain a deeper metacognitive awareness and control over their thinking processes. A stronger understanding of their metacognitive resources prevented students from relying on surface strategies such as memorising algorithms to solve problems.

Students identified flaws in their solutions that were either problem-specific or related to their problem-solving processes:

‘My answer was completely off – in terms of both the final solution and the problem-solving steps in my solution.’ – Oscar

Students identified specific mistake(s) leading to the wrong answer. Importantly, they linked such mistakes to specific problem-solving element(s), for example, by ascribing the incorrect answer to the lack of correctly understanding the problem statement:

‘My answer…[was] incorrect because I did not understand at the beginning that this was a partial neutralisation buffer preparation question.’ – Seb

Similarly, their incorrect solutions were attributed to the lack of planning due to choosing the ‘wrong formula’ or making a mistake ‘in rearranging the formula’.

Clearly identifying and analysing their mistakes enabled students to identify strategies for future improvement such as ‘finding different ways to solve the problem and to reinforce the answer’ or using ‘higher-level questions’. Other improvement strategies clearly related to the cognitive processes captured in the Goldilocks Help scaffold were: ‘evaluating…the numerical value’, being ‘more organised and complete by writing down the formulas required and used’, ‘clarifying with myself and justifying why the answer is the value it is’.

By comparing and reflecting, students monitored their own performance, thereby generating internal feedback (Nicol, 2021). Instead of simply focusing on getting the correct numerical value, they reflected mindfully, became conscious of their cognitive processes, and were able to pinpoint any learning gaps and appropriate strategies. Consistent with Nicol's research (2021), making mindful comparison a formal act allowed students to develop the capacity to identify areas for improvement, discern patterns, principles, and relationships within their work, ultimately enhancing their self-regulated learning skills.

The iterative cycle of solving a problem and reflecting helped students monitor their learning based on their prior attempts and recognise the development of their problem-solving skills:

‘It took a while for me to develop the skills for this new way of problem solving. This led to several challenges in previous allocated problems I attempted, however, I am happy with how this one went and I hope I can continue to devise answers that I am happy with.’ – Youssef

Some students commented that, with practice, structured problem solving becomes ‘a natural habit’. Our previous work (Vo et al., 2022) demonstrated that, from the instructor perspective, continued exposure to the scaffold increases student engagement with the scaffold.

Readily cross-referencing multiple sources of a single student's work may be challenging for instructors providing feedback (Nicol and McCallum, 2022). On the other hand, students possess unique insights into their own thought processes and learning experiences. As a result, students are better equipped to generate self-regulatory comments compared to instructors. Being encouraged to compare and reflect, students were able to comment on their work's quality in relation to their previous efforts, thus self-generating more specific internal feedback.

‘…the quality of my solution is getting better compared to the previous … assignments … as I’m getting more used to using the Goldilocks method.’ – Sofia

Traditional methods of feedback, such as grades, may place emphasis on comparison between peers. Furthermore, for lower performing students, poor grades and a lower ranking amongst peers can demotivate learning (Hughes, 2011). Conversely, implementing ipsative assessments shifts students’ attention from competing with others to competing with themselves (Malecka and Boud, 2023). The problem-solving activity, used in this study, placed an emphasis on personal growth to ensure students continued progress and lasting motivation.

Emotional engagement. Students expressed positive feelings about problem solving, when they performed well, and were ‘proud of’ or even ‘delighted with’ their performance. The positive self-assessment occurred when students felt they had adequately executed components of the scaffold into their solution. Also, the use of structured problem solving was reflected to instil a sense of confidence in the quality of their solution:

‘The evaluation at the end was also useful…and therefore am confident that I arrived at the correct answer.’ – Luc

Many students noted that Goldilocks Help is an aid they can use to deal with stress when solving problems:

‘When I first read the problem, it dawned to me that I do not know how to solve it, thus it had made me more stressed out … This means that I have to … put more brain power in understanding, analysing, and planning the solution instead of stressing…’ – Leila

Engagement with the scaffold resulted in resource activation (Hammer et al., 2005). Scaffold guided students to overcome obstacles, ‘dead ends and false starts’ (Yuriev et al., 2017). For example, the scaffold provided students with strategies such as ‘Is the meaning of all the terms clear?’ when they do not know how to start to solve a problem. Providing metacognitive prompts helped reduce the emotional strain by alleviating the cognitive load. These findings are aligned with the control-value theory of achievement emotions (Pekrun, 2006), which postulates that students’ values, positive or negative, placed on an activity and/or outcome are related to the control they exercise and the emotions they experience.

Behavioural engagement. Due to the nature of the data (written reflections), the behavioural aspect of engagement was not as readily observed as cognitive and emotional aspects. The reflections were interpreted to refer to the behavioural scaffold engagement when students explicitly described their actions, that is when they described what they did or did not do and when they stated that they did or did not execute a specific problem-solving element.

Students talked about a need to slow down their problem-solving process in order to focus on it and execute ‘careful, methodical movement through the UAPIE template’:

‘…force myself to slow down the pace, spend more time to read and understand the problem first.’ – Aashna

‘Overtime, this process of slowing down should become natural and should be highly beneficial.’ – Reese

The scaffold prompted students to incorporate deep-level strategies to solve problems. Identifying core concepts underpinning the problem and evaluating their actions helped to inform students of their next step. The act of slowing down and taking deliberate actions are characteristic of Kahneman's System 2 mode of thinking (2011). This system of thinking requires problem solvers to refrain from acting in a rash manner, and instead to be reflective, carry out higher-order reasoning, and make logical decisions (Chi and Wylie, 2014).

Students referred to executing specific problem-solving elements, often noting the importance of evaluation:

‘I utilised the evaluation stage to implement my value into the Henderson-Hasselbach equation to determine if my values were right.’ – Elliot

Another form of behavioural engagement was recorded when students used the expert solution as a checklist. The expert solution was formatted similarly to the scaffold, illustrating to students how to integrate the metacognitive processes of understanding, analysis, planning, and evaluation. Owing to the comprehensive and scaffold-congruent nature of the expert solution, students frequently utilised it to verify the accuracy of their solutions:

‘By analysing the expert model solution and my own solution, it can be concluded that my answer of $76 was correct.’ – Sage

‘My problem-solving elements are similar to the expert model answer in terms of Understanding, Analysis, Planning, and Implementation. I have defined the terms and relevant principles such as number of moles, mass, molar mass, molar concentration, volume. I have analysed the correlation between variables and established the relationships between the data and the unknowns. From that, I have implemented to the calculation step.’ – Hitomi

Theme 2: non-engagement (disaffection) with structured problem solving

To answer the Research Question 2 (What barriers/resistance do students perceive to encounter when taught with the metacognitive scaffold?), sub-themes were constructed that reflected students’ perceptions of the disaffection with structured problem solving and the barriers that prevented them to engage with it, in the cognitive, emotional, and behavioural dimensions.

Cognitive non-engagement. Students on the ‘disaffected’ (Skinner, et al., 2009) end of the engagement spectrum commented that they did not use the scaffold due to it being unfamiliar and/or struggling to explicitly demonstrate structured problem solving in their worked solutions:

‘My though[t] process was not dealing well with the set out… I did not particularly cope with the writing out of understanding, analysis etc. … I lost my plan while doing so.’ – Akiko

‘Not understanding and familiar with the way to use the scientific problem solving, and missing plenty of the elements in the problem solving’ – Jamie

Simultaneously mastering the content and approaching problem solving structurally and metacognitively proved to be too cognitively challenging for these students. In an attempt to alleviate some cognitive load (Sweller, 1994), these students ignored the scaffold:

‘still follow the calculation method that used in the high school’ – Shannon

Cognitive ease can occur when information is perceived as familiar (Kahneman, 2011). For students who have not previously seen structured problem solving, such as Goldilocks Help, the new approach can be cognitively demanding. The initial process of engaging with the scaffold was described by students as requiring extra effort or time as it ‘slow[s] down’ the process of completing a problem.

A common reason for not engaging with the scaffold was the belief that students’ own established or preferred problem-solving processes were sufficient, often with an emphasis on the speed of arriving at an answer:

‘My solution…was productive and correct, it was a fast method’ – Damien

These students reasoned they did not use the scaffold as it ‘wasn’t necessary’ or ‘assumed this knowledge’.

Other students used the scaffold unproductively, in a surface level manner (Marton and Säljö, 1976). These students focused on assigning parts of their worked solution to specific problem-solving elements:

‘All relevant information is present, however placed in wrong sections of Understanding, Analysis, Planning, Implementation and Evaluation.’ – Mei-long

Finally, fear of making a mistake was also mentioned as a justification for not following the scaffold. For example, Kyle reasoned that ‘I did not rearrange the equation to isolate the required data because this was such a long and complex equation that if I made a mistake in this rearranging step, I will carry out this error in further calculation.’

The above categories of students that did not engage with the scaffold in the cognitive dimensions are representative of novice problem solvers who often resort to means-end analysis (Bodner and Herron, 2002). Such means-end approaches are not productive since they can overwhelm limited working memory and hinder the learning process.

Emotional non-engagement. Students who stated that they ‘lacked self-confidence’ or were ‘stressed’ used that as a reason for why they were reluctant to use the scaffold. Experiencing negative emotions, such as stress and self-doubt, can contribute to disengagement (Pekrun and Linnenbrink-Garcia, 2012). Therefore, these students did not engage with structured problem solving, based on their emotions, despite the cognitive benefits of the scaffold.

For some students, exposure to the expert solution led them to become overly critical of their own work. They became unsatisfied with the quality or depth of their solution due to high self-set standards in academic performance:

‘My solution to the assignment was not close to be as good as his/hers, it was a complete loss of time and had no focus.’ – Remi

Students' perceptions of their learning abilities can impact their ability to adapt to learning new concepts and metacognitive processes (Wright et al., 2022). Perhaps from past negative experiences with problem solving, students who lacked self-confidence became stressed and overly critical when exposed to the scaffold. Here again students’ perceptions reflected the connections between values, exercised control, and aroused emotions (Pekrun, 2006) – in this case, emotions of stress and frustration.

Behavioural non-engagement. Some students stated they rushed through the problem-solving process:

‘…I hastily jumped into the question without expanding on the given information to formulate an ICE table which would have shown better understanding of the question and lead me to different working out.’ – Anika

They acknowledged that skipping problem-solving steps or not showing them by doing them ‘in [their] head’ made their problem solving ‘harder’ or lead to a wrong answer:

‘I rushed a quick alternate solution using trial and error with different values … and thus reached an incorrect answer.’ – Miranda

These students relied on Kahneman's System 1 model of thinking, where decisions are made quickly and impulsively (Kahneman, 2011). Due to the ‘rush’ nature of System 1, these students gravitated towards surface-level and effort-avoidant strategies. As a result, they did not allow themselves time to account for facts and prior experiences to inform their decisions.

Several students admitted that they forgot to apply structured problem solving to their worked solutions: ‘forgot… to show which variables are known and which are unknown’ and ‘forgot to do the check unit’.

Problem-solving profiles manifestations in student perceptions of problem solving

Previously, we have analysed students’ solutions for the four problem-solving cycles (Vo et al., 2024a). Based on their problem-solving outcome (successful or unsuccessful) and the demonstration of problem-solving elements (none/limited or medium/substantial), as demonstrated in their written work, each student solution was assigned to one of the four problem-solving groups. That study demonstrated progress in successful and structured problem solving, with a growing fraction of students demonstrating an exploratory approach over the course of a semester. However, student written work only allows one to analyse what students do when solving problems, not why they do it. In order to investigate students’ perceptions of problem solving and whether they align to the four proposed profiles (Fig. 2, Research Question 3), we analysed the reflections for common features that could indicate the manifestations of the four problem-solving profiles.

Students who solved problems adeptly and expressed confidence in their abilities, along with a preference for their own metacognitive processes, were classified as Settlers. For example:

‘I understand there is another method to solve this problem…but I think the method I used logically makes more sense to me.’ – Alex

Unsuccessful problem solvers who frequently expressed a desire to be ‘fast’ and a great urgency to save time despite being error prone were classified as Sprinters. For example:

‘I did not particularly cope with the writing out of understanding, analysis etc. as it slowed me down and I lost my plan while doing so.’ – Louise

Novice problem solvers are likely to prioritise speed over vital problem-solving processes. In this study, opportunities to compare and reflect prompted Sprinters to assess their thought processes and redirect in order to avoid dead-ends and false starts. For example:

‘Very poor analysis. Rushed over this step and as a consequence was detrimental to the rest of the question.’ – Carlos

Students who often engaged with the scaffold at a surface level, resulting in incorrect answers, were classified as Collectors. For example, they typically used the expert solution as a checklist to show their scaffold engagement. Employing the scaffold in a performative fashion prevented them from using metacognitive processes productively. Their external motivations to engage with the scaffold, such as receiving marks, were seen in their vague reflections and unsuccessful problem-solving attempts. For example, Morgan's reflection on her problem-solving attempt was generic, lacking identification of the specific mistake that led to the incorrect answer, or any reference to the conceptual or metacognitive processes involved:

‘My method matched that of the expert answer. My analysis was very similar, and my dimensional analysis was also similar. My understanding of the question lacked quite a bit, while I got the plan and analysis correct, my actual understanding was not broad enough which may have resulted in an incorrect answer in terms of range. My layout could also be approved on and my evaluation needs to be expanded.’ – Morgan

Students who were successful in problem solving and were driven by the need to improve as learners were classified as Explorers. These students were intrinsically motivated to engage with the scaffold. Their reflections characteristically contained comments relating to their growth as problem solvers and featured remarks regarding the benefits of structured problem solving. For example:

‘I am proud of how I tackled this problem and feel as though I have developed my problem solving skills over the course of the semester….it is still important that I carefully think about the way I approach a given problem, and as always, there are still improvements to be made.’ – Priya

Analysis of students’ written solutions carried out previously demonstrated a dynamic shift of students’ problem-solving success and quality of their work from (i) not engaging with the scaffold and not being successful in problem solving to (ii) engaging with the scaffold and being first unsuccessful and then successful in problem solving, and to being successful without engaging with the scaffold (Vo et al., 2024a). The analysis of comparative reflections presented here allowed to gain insight into students’ values and motivations for their problem-solving efforts. The results explain what drives the evolution of student perceptions of problem solving that is metacognitively scaffolded and supported by opportunities to compare and reflect. Below we present two examples of students’ progressive development of such perceptions.

Student transitions between profiles: example 1 (Darcy). At the start of the semester, in Topic 1 (Unit Conversions), the content was relatively familiar to most students. As a result, many students treated the problem in Topic 1 as an exercise rather than a problem (Bodner, 1987; Randles and Overton, 2015), reflecting their expertise in this area of knowledge and placing them into either an Explorer or a Settler profile. For example, Darcy correctly solved the problem in Topic 1 without the use of the scaffold, as aptly stated in his reflection:

‘The way I answered my question was correct … I did not use dimensional analysis and separate my working into categories (understanding, analysis, planning, implementation, evaluation) like shown in the expert answer.’

When prompted to reflect on potential areas of improvement, Darcy noted that ‘the way in which I present my working out would be professionally concise if it were like the expert model.’

In Topic 2, which introduced novel content (acid–base equilibria), Darcy faced difficulties and was unable to solve the problem. He reflected:

‘I have not separated my working out into the goldilocks’ method, which I’m finding difficult. The way I also did my working out is evidently wrong. I have not considered the OH in the base and there is inconsistency in my working…’

Darcy struggled to apply structured problem solving in his work. After comparing it with the model solution, he deemed his attempt ‘evidently wrong’. Like many other students who regarded Topic 1 as more of an exercise, when confronted with unfamiliar content he struggled to solve problems without the scaffold. In reflecting on improvement areas for Topic 2, Darcy stated goals to ‘work…on my understanding of acid and bases I can develop the skills needed…’ and ‘to develop my presentation of my solutions through using the goldilocks’ method.’

The scaffold functions as a guiding tool that employs metacognitive prompts to help students conceptually break down problems. However, at this stage, Darcy perceived the process of understanding fundamental concepts as distinct from engaging with the scaffold to solve problems. He engaged with the scaffold performatively and focused on ‘improving presentation’, i.e. the problem-solving output rather than his problem-solving process. This performative attempt at engaging with the scaffold signified his backward transition from a Settler to a Collector between Topics 1 and 2.

In Topic 3 (Thermodynamics), Darcy ‘found [the problem-solving elements of] understanding and analyzing easy to grasp.’ Meaningfully engaging with the scaffold through demonstrating the underlying concepts and organising data into what is known and unknown, Darcy successfully solved the problem, which tested novel content and concluded:

‘when comparing my solution with the expert model, most of the aspect in the model was covered within my own solutions.’

Additionally, when reflecting on areas of improvement, he stated ‘I should read the question more clearly to eliminate any potential misconceptions such as the use of the SLC temperature and pressure instead of the normal boiling point of water.’ This marks a significant contrast to his initial reflection, where suggested improvements were rather vague, with mentions of progress in terms of ‘neatness’ and ‘professionalism’. In Topic 3, he identified specific and actionable improvement strategies. From Topic 2 to 3, Darcy transitioned from a Collector to an Explorer.

At the end of the semester, Topic 4 (Chemical Kinetics), Darcy noted the time taken to understand the problem.

‘Understanding this question took some time…Since the process was a second order process, the second-order rate law needed to be used along with the second-order half-life formula.’

He was able to recognise that he slowed down to identify and process the concepts being explored in the problem. Explicitly demonstrating his understanding helped to discern the next appropriate steps to solve the problem. In line with Kahneman's work on System 2 mode of thinking (2011), deliberate and reflective actions, such as demonstrating ones understanding, is particularly vital when dealing with novel tasks.

In the final topic, he declared, ‘All the components of my solution are the same as the expert model answers.’ This signified a notable difference from the beginning of the semester when he faced challenges in demonstrating structured problem solving. By the end of the semester, Darcy reliably established himself as a confident Explorer, a pattern commonly observed among students throughout the four topics.

Student transitions between profiles: example 2 (Nadia). Since Topic 1 consisted of familiar content to most students, Nadia assumed the profile of a Settler, akin to Darcy. When exposed to unfamiliar content of Topic 2, she faced difficulties in solving the problem. Her reflective analysis accurately attributed her lack of success to the absence of structured problem solving:

‘Due to my lack of understanding of the problem, my failed attempted at analysis, and no planning of the strategy, I was not able to arrive at an answer… When comparing my solution to the expert answer, my solution visibly lacks the majority of the steps taken in order to solve this problem.’

Nadia further detailed, ‘I rushed through the problem was because of the time constraint. When I first read the problem, it dawned to me that I do not know how to solve it, thus it had made me more stressed out with the time count down.’ This response, characteristic of a Sprinter, explains why this student type does not engage with the scaffold. It stems from viewing the scaffold as requiring extra effort or time and the perceived need to be fast, even at the cost of productivity and accuracy. These perceptions were associated with cognitive and behavioural constraints such as insufficient chemical knowledge or lacking self-confidence in problem solving. When prompted to discuss areas of improvement, similarly to Darcy, Nadia came up with generic suggestions to ‘pay less attention to the time and put more brain power in understanding, analysing, and planning the solution’.

In Topic 3, Nadia did engage with the scaffold, however, her engagement remained superficial, leading to her inability to solve the problem, on which she reflected:

‘…due to insufficient understanding and evaluation, I was not able to identify the mistake I made when identifying the values needed for calculation as well as the incorrect answer I have calculated.’

Her strategies to improve involved ‘spend[ing] more time thinking, processing, and comprehending problems to ensure that I have a complete understanding before moving on to solving the problem… more time evaluating and checking back on my understanding and analysis of the problem to identify all my mistakes in time to correct my answer if needed.’

Similar to Darcy, her improvement strategies became more specific and actionable after three iterations of the activity. For example, time was initially viewed as a stress factor, but by Topic 3, time was viewed as a resource to be allocated towards problem-solving elements. The transition from Topic 2 to 3 reflects a change from no engagement with the scaffold to partial engagement, marking Nadia's shift from a Sprinter to a Collector.

By Topic 4, Nadia demonstrated meaningful engagement with the scaffold, successfully solved the problem, and transitioned to an Explorer:

‘I have correctly identified the known variables and the missing variables in both attempts at solving the problem. From that I was able to determine the steps need to acquire all the variables needed in order to reach the final answer… I was able to successfully perform all sections of the goldilocks workflow…my solution is similar to the expert solution.’

The journey from a Sprinter to Collector to Explorer was influenced by shifts in student motivations. Being a Sprinter, the hesitation to participate in structured problem solving typically arose from the desire to be quick and save mental effort. To encourage these resistant students to engage, earning marks act as an incentive, which shifts them to Collectors. As a Collector, engaging in a performative manner gives students the opportunity to experience the advantages of integrating metacognitive processes, thus motivating them to remain engaged. Over time, motivations for engagement evolved from initial resistance and extrinsic incentives to intrinsic motivation as students start to aspire to mastering problem-solving skills.

Conclusion and implications

Findings indicated that students’ problem-solving perceptions became more sophisticated as a result of continued exposure to the scaffold and iterative opportunities to compare their work to expert solutions, to self-assess, and to reflect. Scaffold use coupled with self-reflection provided students with ability to identify flaws in their solutions that were either problem specific or related to their problem-solving processes, and identify the specific mistakes that led to the wrong answer. Students also identified improvement strategies, for example, asking prompting questions to self, finding multiple alternative approaches to evaluate an answer, and practise problem solving under timed conditions.

The application of structured problem solving was reflected to reduce confusion in thought and allowed others, for example instructors, to readily understand student work, identify mistakes, and pressure points. These perceptions were mirrored in our paper exploring how teaching associates taught structured problem solving with Goldilocks Help (Vo et al., 2022). For students who lacked structured problem-solving skills, scaffold engagement helped to slow down metacognitive processes that would otherwise be rushed or engaged with on a surface level. For students who were initially resistant to the scaffold, this resistance was due to the belief that their own established problem-solving process was sufficient or preferred, fear of making mistakes, or viewing the process as time- and/or effort-consuming.

The combination of metacognitive scaffolding and reflective practice provided students with a concrete framework for self-assessment of their problem-solving abilities and limitations for self-regulation and intrinsic motivation, and for deeper and more meaningful engagement with problem solving.

Implications for teaching practice

In our research, we observed different types and degrees of student engagement with metacognitive scaffolding. Having an awareness of students’ thoughts, emotions, and actions can help instructors differentiate between levels of student capabilities, mindsets, and needs for extra support. This is particularly useful for early career instructors, such as teaching associates, who lack the experience to readily recognise different student populations. Furthermore, teaching efforts should be directed at promoting metacognitive and structured problem solving. For students resistant to scaffolding due to negative emotions, instructors should cultivate a non-judgmental learning environment (Wright et al., 2022) and encourage students to recall past successes and provide affirmations of their capabilities. To assist with effective communication of abstract concepts, instructors should promote the use of scaffold terminology during instructor-student and student-student discourse to help convey their problem-solving processes clearly and efficiently.

One of the important observations of this work is the positive progression of student perceptions of problem solving and own problem-solving abilities, captured in their reflections. Iterating opportunities for students to reflect and self-assess initiate ipsative processes, i.e. processes focusing on students’ own progress and improvement compared to their previous performance (Malecka and Boud, 2023). While the implementation described here did not involve individual feedback for students on their progress beyond using the rubric, it encompassed many features of ipsative assessments: iterative task design, traceability to previous performance, reflective tasks for self-awareness, action planning for goal setting and improvement, and fostering of intrinsic motivation to improve and excel (Hughes, 2011). These motivational benefits can be especially impactful for learners with lower self-esteem, as it encourages their confidence and fosters a growth mindset (Malecka and Boud, 2023).

The opportunity, and the responsibility, to reflect and self-assess enhances students' perception of autonomy as it emphasises their active role in the learning process, whereby they become better at planning, evaluating, and regulating their own learning. This capacity to self-regulate positions students to become independent and lifelong learners (Zimmerman, 1990; Yan, 2020). Moreover, multiple opportunities to reflect provide a longitudinal overview of learning by focusing on progress between tasks, resulting in more realistic perceptions.

Instructors should also provide multiple opportunities for students to compare their work to the work of others (Silver, 2010), peers or experts (Graulich et al., 2021), and reflect on the comparison. Comparative reflections offer a multitude of cognitive benefits. They act as a cognitive tool for relational reasoning and help students establish connections between concepts (Gentner, 2016), linking new knowledge to existing knowledge, and improving retention and comprehension. Furthermore, comparing their work to the work of others allows students to view their solution from another perspective, to re-evaluate their conceptual understanding and problem-solving procedures, and to produce new ideas. All these processes foster the generation of internal feedback (Nicol and Selvaretnam, 2022). Encouraging students to reflect also reveals the potential misalignment between what students think vs. their actual performance and the disconnect in-between. Informing students of this disparity can drive intrinsic motivation to become more proficient problem solvers. Besides, identifying students' internal thought processes and areas of difficulty also enables instructors to deliver more targeted feedback that aligns with students' specific needs. Conversely, and in keeping with the ‘new feedback paradigm’ (Winstone et al., 2021), the feedback agency could be shifted to students by having them write their own feedback comments, in addition to or as part of comparative reflection (Nicol and Kushwah, 2023). Finally, explicit guidance for comparative analysis supports students in harnessing process skills of organising information, linking underlying concepts, and defending decisions, often overlooked by students (Silver et al., 2007). Such guidance affords students the opportunity to articulate their thought processes tangibly. Increased awareness of their cognitive processes enables them to externalise their thinking and exercise control over these processes, such as identifying gaps in their understanding (Pintrich, 2002).

Limitations and research implications

Comparative reflections analysed here were submitted for a summative assessment, albeit a low-stakes one. Therefore, it must be acknowledged that some students may be performative in their writing and ‘matching’ it to the assessment rubric, thus potentially skewing the data. This phenomenon has been described as ‘addressivity’, implying that students may address their reflection specifically at the instructor, since they recognise that the instructor may “wield the power of assessment” (Ross, 2014). However, while the rubric provided to students in this study guided them to specify improvement strategies, it did so without stating what the strategies should be. Our findings showed that students were able to identify specific strategies useful to them, such as ‘finding different ways to solve the problem and to reinforce the answer’, being ‘more organised and complete by writing down the formulas required and used’, and ‘clarifying with myself and justifying why the answer is the value it is’. Furthermore, given the ubiquitous nature of reflection in higher education, it is important to recognise the potential for performativity rather than see it as a disqualifying factor (Ross, 2014). Further research on reflective writing in chemistry education should therefore focus on the ipsative nature of such assessments, with the emphasis on personal progress and growth, while recognising the addressivity and power relations operationalised in course-embedded reflections.

In this study, the reflections were analysed as a total body of work to construct the engagement and non-engagement themes and to allocate individual students to problem-solving profiles. In the reflections, students primarily exhibited cognitive engagement, followed by emotional engagement. The substantial level of cognition was in harmony with the task's demands, as students were solving problems and reflecting on their problem-solving approaches. Consequently, discussions regarding behavioural engagement, which typically entail observable actions, were relatively limited. Furthermore, traditional social engagement was impacted by COVID-19 when all teaching shifted online where strategies to replicate social engagement involved discussion forums and conducting workshops over Zoom. Since students had fewer chances to participate in collaborative problem-solving activities, the social dimension did not manifest in their writing as would have been anticipated in a different context. Further research could explore the social and behavioural dimensions of metacognitive scaffolding. These specific forms of scaffold engagement could be investigated through classroom observations and group think-aloud interviews. We also plan to return to this dataset to carry out a relationship analysis, to study the connections between the various dimensions of engagement. Recent developments in applying machine learning and artificial intelligence to chemistry education research (e.g., Martin et al., 2023) should make such analysis more effective.

The four-profile student typology itself requires further elaboration. Firstly, there are students who are skill motivated, like Explorers, but have not yet developed advanced problem-solving mastery to be successful. There are also students who are confident in their perceived abilities, like Settlers, yet that over-confidence is unwarranted. It would be valuable to explore how these two combinations of student motivations and mastery levels fit into a more nuanced relationship between the level of scaffold engagement and structured problem solving. These points of interest could be explored through classroom observations of both instructors and students, and through individual and focus-group interviews and think-aloud sessions.

Finally, the wider motivational aspects of student engagement with structured problem solving could be investigated using the recently developed person-centred approach that captures students' motivational profiles (Kubsch et al., 2023).

Data availability

Data collected from human participants (comparative reflections) are not available for ethical confidentiality reasons.

Conflicts of interest

There are no conflicts of interest to declare.

Acknowledgements

KV acknowledges the receipt of RTP (Research Training Program) Postgraduate Scholarship. We are also grateful to participants for providing access to their reflections analysed in this study.

References

1. Ames C., (1992), Classrooms: goals, structures, and student motivation, J. Educ. Psychol., 84, 261–271.
2. Barry C. A., Britten N., Barber N., Bradley C. and Stevenson F., (1999), Using reflexivity to optimize teamwork in qualitative research, Qual. Health Res., 9, 26–44.
3. Belland B. R., (2011), Distributed cognition as a lens to understand the effects of scaffolds: the role of transfer of responsibility, Educ. Psychol. Rev., 23, 577–600.
4. Bodner G. M., (1987), The role of algorithms in teaching problem solving, J. Chem. Educ., 64, 513–514.
5. Bodner G. M. and Herron J. D., (2002), Problem-solving in chemistry, in Chemistry education: towards research-based practice, J. K. Gilbert, O. DeJong, R. T. Justi, D. Treagust and J. Driel (ed.), Kluwer Academic Publishers, ch. 11, pp. 235–266.
6. Boud D., (2001), Using journal writing to enhance reflective practice, New Dir. Adult Contin. Educ., 2001, 9–17.
7. Bowden J. L.-H., Tickle L. and Naumann K., (2021), The four pillars of tertiary student engagement and success: a holistic measurement approach, Stud. High. Educ., 46, 1207–1224.
8. Braun V. and Clarke V., (2006), Using thematic analysis in psychology, Qual. Res. Psychol., 3, 77–101.
9. Braun V. and Clarke V., (2021), One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qual. Res. Psychol., 18, 328–352.
10. Burr V., (2015), Social constructionism, London, UK and New York, NY: Routledge.
11. Casey K., (2007), Phenomenology, in Theoretical frameworks for research in chemistry/science education, G. Bodner and M. Orgill (ed.), Pearson Prentice Hall, ch. 7, pp. 117–126.
12. Chi M. and Wylie R., (2014), The ICAP framework: linking cognitive engagement to active learning outcomes, Educ. Psychol., 49, 219–243.
13. Cleary T. J. and Zimmerman B. J., (2012), in Handbook of research on student engagement, S. L. Christenson, A. L. Reschly and C. Wylie,(ed.) New York: Springer, ch. 12, pp. 237–257.
14. Cohen J., Kennedy-Justice M., DePierro E., Garafalo F., Pai S., Torres C. and Toomey R., (2000), Encouraging meaningful quantitative problem solving, J. Chem. Educ., 77, 1166.
15. Collins A., Brown J. S. and Holum A., (1991). Cognitive apprenticeship: making thinking visible, Am. Educ., 15, 6–11, 38–46. https://www.aft.org/ae/winter1991/collins_brown_holum.
16. Drummond H. P. and Selvaratnam M., (2008), Students’ competence in intellectual strategies needed for solving chemistry problems, S. Afr. J. Chem., 61, 56–62.
17. Elliot A. J. and Hulleman C. S., (2017), Achievement goals, in Handbook of competence and motivation: Theory and application, A. J. Elliot, C. S. Dweck and D. S. Yeager (ed.), (2nd edn), The Guilford Press, pp. 43–60.
18. Fitzgerald N. and Li L., (2015), Using presentation software to flip an undergraduate analytical chemistry course, J. Chem. Educ., 92, 1559–1563.
19. Fredricks J. A., Blumenfeld P. C. and Paris A. H., (2004), School engagement: potential of the concept, state of the evidence, Rev. Educ. Res., 74, 59–109.
20. Fredricks J. A., Wang M.-T., Schall Linn J., Hofkens T. L., Sung H., Parr A., Allerton J., (2016) Using qualitative methods to develop a survey measure of math and science engagement, Learn. Instr., 43, 5–15.
21. Gentner D., (2016), Language as cognitive tool kit: how language supports relational thought, Am. Psychol., 71, 650.
22. Glaser B. G., (1965), The constant comparative method of qualitative analysis, Soc. Probl., 12, 436–445.
23. Graulich N., Langner A., Vo K. and Yuriev E., (2021), Scaffolding metacognition and resource activation during problem solving: a continuum perspective, in Problems and problem solving in chemistry education, G. Tsaparlis (ed.), Royal Society of Chemistry, pp. 38–67.
24. Hammer D., Elby A., Scherr R. E. and Redish E. F., (2005), Resources, framing, and transfer, in Transfer of learning from a modern multidisciplinary perspective, J. P. Mestre (ed.), Greenwich, CT: Information Age Publishing, ch. 3, pp. 89–119.
25. Harper K. A., (2006), Student problem-solving behaviors, Phys. Teach., 44, 250–251.
26. Heidbrink A. and Weinrich M., (2021), Encouraging biochemistry students’ metacognition: reflecting on how another student might not carefully reflect, J. Chem. Educ., 98, 2765–2774.
27. Hughes G., (2011), Towards a personal best: a case for introducing ipsative assessment in higher education, Stud. High. Educ., 36, 353–367.
28. Jay J. K. and Johnson K. L., (2002), Capturing complexity: a typology of reflective practice for teacher education, Teach. Teach. Educ., 18, 73–85.
29. Kahneman D., (2011), Thinking, fast and slow, New York: Farrar, Straus and Giroux.
30. Kalyuga S., Chandler P., Tuovinen J. and Sweller J., (2001), When problem solving is superior to studying worked examples, J. Educ. Psychol., 93, 579.
31. Kirschner P. A. and Hendrick C., (2024), How Learning Happens: Seminal Works in Educational Psychology and What They Mean in Practice, (2nd edn), Routledge, ch. 29, ‘Making thinking visible’, pp. 310–318.
32. Koretsky M. D., (2020), Re-flipping in the remote classroom: the surprising uptake of video-recorded worked examples, J. Chem. Educ., 97, 2754–2759.
33. Kubsch M., Fortus D., Neumann K., Nordine, J., Krajcik J., (2023), The interplay between students' motivational profiles and science learning, J. Res. Sci. Teach., 60, 3–25.
34. LaDonna K. A., Artino Jr A. R., Balmer D. F., (2021), Beyond the guise of saturation: rigor and qualitative interview data, J. Grad. Med. Educ., 13, 607–611.
35. Lawrie G., (2023), Engaging with engagement in learning chemistry: a guiding note about a multidimensional construct, Chem. Educ. Res. Pract., 24, 791–792.
36. Lincoln Y. S. and Guba E. G., (1985), Naturalistic inquiry, Newbury Park: Sage Publications.
37. Lo, M.-T., (2024), Understanding momentary engagement in university students: exploring the interaction between competence and value beliefs on emotions and cognitive engagement – A multilevel investigation, Learn. Individ. Differ., 111, art.102435.
38. Malecka B. and Boud D., (2023), Fostering student motivation and engagement with feedback through ipsative processes, Teach. High. Educ., 28, 1761–1776.
39. Malterud K., Siersma V. and Guassora A. D., (2021), Information power: sample content and size in qualitative studies, in Qualitative research in psychology: Expanding perspectives in methodology and design, P. M. Camic (ed.), American Psychological Association, pp. 67–81.
40. Martin P. P., Kranz D., Wulff P. and Graulich, N., (2023), Exploring new depths: applying machine learning for the analysis of student argumentation in chemistry, J. Res. Sci. Teach. DOI:10.1002/tea.21903.
41. Marton F. and Säljö R., (1976), On qualitative differences in learning: I—Outcome and process, Br. J. Educ. Psychol., 46, 4–11.
42. Mayer R. E., (2020), Advances in designing instruction based on examples, Appl. Cogn. Psychol., 34, 912–915.
43. Mutambuki J. M., Mwavita M., Muteti C. Z., Jacob B. I. and Mohanty S., (2020), Metacognition and active learning combination reveals better performance on cognitively demanding general chemistry concepts than active learning alone, J. Chem. Educ., 97, 1832–1840.
44. Naibert N., Vaughan E. B., Brevick K. and Barbera J., (2022), Exploring student perceptions of behavioral, cognitive, and emotional engagement at the activity level in general chemistry, J. Chem. Educ., 99, 1358–1367.
45. Nicol D., (2021), The power of internal feedback: exploiting natural comparison processes, Assess. Eval. High. Educ., 46, 756–778.
46. Nicol D. and Kushwah L., (2023), Shifting feedback agency to students by having them write their own feedback comments, Assess. Eval. High. Educ., 49, 419–439.
47. Nicol D. and McCallum S., (2022), Making internal feedback explicit: exploiting the multiple comparisons that occur during peer review, Assess. Eval. High. Educ., 47, 424–443.
48. Nicol D. and Selvaretnam G., (2022), Making internal feedback explicit: harnessing the comparisons students make during two-stage exams, Assess. Eval. High. Educ., 47, 507–522.
49. Nowell L. S., Norris J. M., White D. E. and Moules N. J., (2017), Thematic analysis: striving to meet the trustworthiness criteria, Int. J. Qual. Methods., 16(1) DOI:10.1177/1609406917733847.
50. Pekrun R., (2006), The control-value theory of achievement emotions: assumptions, corollaries, and implications for educational research and practice, Educ. Psychol. Rev., 18, 315–341.
51. Pekrun R. and Linnenbrink-Garcia L., (2012), Academic emotions and student engagement, in Handbook of research on student engagement, Boston, MA: Springer, pp. 259–282.
52. Pintrich P. R., (2002), The role of metacognitive knowledge in learning, teaching, and assessing, Theory Pract., 41, 219–225.
53. QSR International. NVivo qualitative data analysis software (Version 20.3).
54. Randles C. A. and Overton T. L., (2015), Expert vs. novice: approaches used by chemists when solving open-ended problems, Chem. Educ. Res. Pract., 16, 811–823.
55. Rodriguez J. G., Bain K., Hux N. P. and Towns M. H., (2019), Productive features of problem solving in chemical kinetics: more than just algorithmic manipulation of variables, Chem. Educ. Res. Pract., 20, 175–186.
56. Ross J., (2014), Performing the reflective self: audience awareness in high-stakes reflection, Stud. High. Educ., 39, 219–232.
57. Rubin S. J. and Abrams B., (2015), Teaching fundamental skills in Microsoft Excel to first-year students in quantitative analysis, J. Chem. Educ., 92, 1840–1845.
58. Schlösser T., Dunning D., Johnson K. L. and Kruger J., (2013), How unaware are the unskilled? Empirical tests of the “signal extraction” counterexplanation for the Dunning–Kruger effect in self-evaluation of performance, J. Econ. Psychol., 39, 85–100.
59. Silver H. F., (2010), Compare & contrast: Teaching comparative thinking to strengthen student learning, Alexandria, Virginia USA: ASCD.
60. Silver H. F., Strong R. W. and Perini M. J., (2007), The strategic teacher: Selecting the right research-based strategy for every lesson, Alexandria, Virginia USA: ASCD.
61. Sinatra G. M., Heddy B. C. and Lombardi D., (2015), The challenges of defining and measuring student engagement in science, Educ. Psych., 50, 1–13.
62. Skinner E. A., Kindermann T. A. and Furrer C. J., (2009), A motivational perspective on engagement and disaffection: conceptualization and assessment of children's behavioral and emotional participation in academic activities in the classroom, Educ. Psychol. Meas., 69, 493–525.
63. Smith J. D., (2013), Student attitudes toward flipping the general chemistry classroom, Chem. Educ. Res. Pract., 14, 607–614.
64. Sweller J., (1994), Cognitive load theory, learning difficulty, and instructional design, Learn. Instr., 4, 295–312.
65. Tharayil S., Borrego M., Prince M., Nguyen K. A., Shekhar P., Finelli C. J. and Waters C., (2018), Strategies to mitigate student resistance to active learning, Int. J. STEM Educ., 5, 1–16.
66. Thompson J., (2022), A guide to abductive thematic analysis, Qual. Rep., 27, 1410–1421.
67. van Harsel M., Hoogerheide V., Verkoeijen P. and van Gog T., (2020), Examples, practice problems, or both? effects on motivation and learning in shorter and longer sequences, Appl. Cogn. Psychol., 34, 793–812.
68. Varga A. L. and Hamburger K., (2014), Beyond type 1 vs. type 2 processing: the tri-dimensional way, Front. Psychol., 5, art.993.
69. Vo K., Sarkar M., White P. J. and Yuriev E., (2022), Problem solving in chemistry supported by metacognitive scaffolding: teaching associates’ perspectives and practices, Chem. Educ. Res. Pract., 23, 436–451.
70. Vo K., Sarkar M., White P. J. and Yuriev E., (2024a), Development of problem-solving skills supported by metacognitive scaffolding: insights from students’ written work (manuscript submitted to Chem. Educ. Res. Pract.).
71. Vo K., Sarkar M., White P. J. and Yuriev E., (2024b), Integrating metacognitive training, reflective comparative benchmarking, self-assessment, and internal feedback generation for problem-solving skill development (manuscript submitted to J. Chem. Educ.).
72. Wang M.-T., Fredricks J. A., Ye F., Hofkens T. L., Linn J. S., (2016), The Math and Science Engagement Scales: scale development, validation, and psychometric properties, Learn. Instr., 43, 16–26.
73. Watts F. M. and Finkenstaedt-Quinn S. A., (2021), The current state of methods for establishing reliability in qualitative chemistry education research articles, Chem. Educ. Res. Pract., 22, 565–578.
74. White P. J., Larson I., Styles K., Yuriev E., Evans D. R., Short J. L., Rangachari P. K., Malone D. T., Davie B. and Naidu S., (2015), Using active learning strategies to shift student attitudes and behaviours about learning and teaching in a research intensive educational context, Pharm. Educ., 15, 116–126.
75. Williams I., Schliephake K., Heinrich L. and Baird M., (2017), Integrating science and practice (iSAP): an interactive case-based clinical decision-making radiography training program, MedEdPublish, 6, 1–10.
76. Winstone N., Boud D., Dawson P. and Heron, M., (2021), From feedback-as-information to feedback-as-process: a linguistic analysis of the feedback literature, Assess. Eval. High. Educ., 47, 213–230.
77. Wong Z. Y. and Liem G. A. D., (2022), Student engagement: current state of the construct, conceptual refinement, and future research directions, Educ. Psych. Rev., 34, 107–138.
78. Wright S., Dean K. L. and Forray J. M., (2022), Negative student emotions and educator skill in experiential education: a taxonomy of classroom activities, High. Educ., 83, 987–1002.
79. Yan Z., (2020), Self-assessment in the process of self-regulated learning and its relationship with academic achievement, Assess. Eval. High. Educ., 45, 224–238.
80. Yuriev E., Basal S. and Vo K., (2019), Developing problem-solving skills in physical chemistry, in Teaching chemistry in higher education: A festschrift in honour of Professor Tina Overton, M. K. Seery and C. McDonnell (ed.), Dublin: Creathach Press, pp. 55–76.
81. Yuriev E., Clulow A. J. and Short J. L., (2021), “Teaching Chemistry Down Under in an “Upside Down” World: Lessons Learned and Stakeholder Perspectives”, ACS Symp. Ser., 1389, 105–122.
82. Yuriev E., Naidu S., Schembri L. and Short J., (2017), Scaffolding the development of problem-solving skills in chemistry: guiding novice students out of dead ends and false starts, Chem. Educ. Res. Pract., 18, 486–504.
83. Zimmerman B. J., (1990), Self-regulated learning and academic achievement: an overview, Educ. Psychol., 25, 3–17.

---

Footnote

† Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d4rp00096j

---