Eliciting student thinking about acid–base reactions via app and paper–pencil based problem solving

Abstract

An understanding of acid–base reactions is necessary for success in chemistry courses and relevant to careers outside of chemistry, yet research has demonstrated that students often struggle with learning acid–base reaction mechanisms in organic chemistry. One response to this challenge is the development of educational applications to support instruction and learning. The development of these supports also creates an opportunity to probe students’ thinking about organic chemistry reaction mechanisms using multiple modalities—i.e., using an app interface or the traditional paper–pencil. This study used think-aloud interviews conducted with undergraduate students in their first semester of organic chemistry to understand how they worked through two acid–base reactions using either paper–pencil or an app. Analysis of the interviews indicates that students from both groups recognize the steps of acid–base reactions, but do not always apply the underlying concepts, such as assessment of pK_a values or resonance, when determining how a reaction will proceed. The modality seemed to somewhat influence students’ thinking, as the app prevented students from making chemically unreasonable mistakes. However, some students relied on the cues it provided, which could potentially be problematic when they are required to respond to assessments that do not provide these cues. Our results suggest that instructors should emphasize the conceptual grounding for the steps that govern acid–base reactions to promote chemical thinking about the relationships between the reaction components and how those influence reaction outcomes, as well as support students to think critically about the chemical information contained within the modalities they are using.

---

Introduction

Acid–base chemistry is a fundamental topic in organic chemistry that guides our understanding of chemical reactivity and reaction pathways. Acid–base reactions frequently appear as steps within other reaction mechanisms students learn in introductory organic chemistry (Stoyanovich et al., 2015). Furthermore, acid–base chemistry was consistently identified as one of the top three most important topics in a study of professors’ beliefs about fundamental concepts in organic chemistry (Duis, 2011). Not only must students have a conceptual understanding of the topic, but they must also be able to apply that conceptual knowledge when reasoning through reaction mechanisms to be successful in organic chemistry (Grove et al., 2012; Stoyanovich et al., 2015). Beyond the importance of acid–base chemistry in organic chemistry, an understanding of the topic is also necessary because acid–base reactions commonly appear in other settings such as biochemistry (Stoyanovich et al., 2015; Bell et al., 2019) and materials chemistry (Cowie and Arrighi, 2007). Reactions mediated by acid–base chemistry are one of the first reaction types covered in the organic chemistry curriculum, and it is within this context that students begin developing the ability to apply conceptual reasoning to reaction mechanisms. Therefore, it is valuable to specifically study how students think about acid–base organic reaction mechanisms.

For research that explores students’ thinking about a particular topic, it can be valuable to probe student reasoning using multiple modalities, as the modality may elicit or influence certain thought processes. In particular, with the increase in touch-screen educational software to support students’ learning of organic chemistry (Cooper et al., 2009, 2010; Grove et al., 2012; Larson, 2012; Libman and Huang, 2013; McCollum et al., 2014; Mechanisms, 2018; Duffy et al., 2019), it is of interest to explore student's thinking about acid–base reactions when working with representations of reaction mechanisms on touch-screen devices as compared to their thinking when working acid–base mechanisms with the conventional paper and pencil. Prior studies have shown how the nature of the task—e.g., the type of problem posed or the way a question is asked—can influence students’ reasoning about acids and bases (McClary and Talanquer, 2011; Cooper et al., 2016). McClary and Talanquer (2011) identified that some students use different mental models of acids when performing different tasks related to ranking relative acid strength, and, in a separate study, Cooper et al. (2016) demonstrated that the structure of an assessment task influenced the quality of students’ reasoning about acid–base reaction mechanisms. While these studies have shown that the way a problem is posed can influence students’ thinking about acid–base chemistry concepts, there has been little research into how the modality of a task itself might similarly affect students’ thinking due to inherent differences in prompting and structure depiction.

Student understanding of acid–base reaction mechanisms

Organic chemistry typically begins with a re-introduction to the acid–base concepts taught in high school and undergraduate general chemistry courses. Studies have documented common alternative conceptions about acid–base chemistry at these introductory levels (Garnett Patrick et al., 1995), which students might bring into organic chemistry. In addition, the reasoning skills students develop in general chemistry do not necessarily transfer to successful reasoning about acids and bases in organic chemistry (Anderson and Bodner, 2008; Cartrette and Mayo, 2011). For example, Anderson and Bodner (2008) identified that while some students can successfully transfer their notions of periodic trends to understand that acids such as HBr and HCl react similarly, their reliance on the location of elements on the periodic table can lead them to classify H₃O⁺ as reacting differently than HBr and HCl. Additionally, Cartrette and Mayo (2011) identified that students often rely on the Brønsted–Lowry definitions of acids as proton donors and bases as proton acceptors in the context of organic reaction mechanisms, perhaps due to the focus on the Brønsted–Lowry theory during general chemistry instruction. These studies suggest that students are able to transfer knowledge from general to organic chemistry, but they do not always successfully use this knowledge to reason through acid–base reaction mechanisms. This may be exacerbated by the difficulties that students have using pK_a values in the context of organic chemistry reactions (Flynn and Amellal, 2016). Beyond the lack of successful transfer from general to organic chemistry, the challenges students face with learning acid–base chemistry can persist into graduate school (Bhattacharyya, 2006). Hence, it is necessary to support students’ understanding of the different acid–base theories and how to successfully use them for problem solving early in the undergraduate curriculum (Shaffer, 2006; Cartrette and Mayo, 2011).

Lewis acid–base theory has been found to be particularly important for students’ learning of organic reaction mechanisms involving acids and bases because of the theory's focus on electron transfer (Cooper et al., 2016; Dood et al., 2018). Corroborating these findings, studies of faculty members’ perceptions have identified that understanding Lewis acid–base theory is critical for successful mechanistic reasoning (Bhattacharyya, 2013). However, students are often not able to accurately identify Lewis acids and bases, though they are able to correctly identify Brønsted–Lowry acids and bases (Cartrette and Mayo, 2011). Other research has revealed that students have difficulties understanding, applying, and describing reactions in terms of the electronics inherent to Lewis acid–base theory (Cartrette and Mayo, 2011; Schmidt-McCormack et al., 2019). Furthermore, students have many mental models of acids and bases and they often struggle to switch between models (McClary and Talanquer, 2011). In particular, when considering acid strength, students tend to focus primarily on surface features related to the Arrhenius and Brønsted–Lowry acid–base theories—such as the presence of dissociable protons—rather than the implicit electronics of Lewis acid–base theory, and only invoke the Lewis theory in conjunction with mental models related to the other two theories (McClary and Talanquer, 2011; Dood et al., 2018).

Taken together, the prior research on students’ conceptions of acids and bases suggests that students struggle to apply Lewis acid–base theory in comparison to other theories. This is potentially troubling in the context of organic reaction mechanisms, as both Lewis acid–base theory and organic reaction mechanisms involve explaining reactions based on the movement or transfer of electron pairs. The focus on electron transfer in the Lewis acid–base theory leads into an understanding of mechanisms more generally, as the Lewis theory allows for an electronic explanation of how proton transfers occur (Cooper et al., 2016). Electronic explanations of mechanisms are necessary for mechanistic reasoning in organic chemistry (Bhattacharyya, 2013), and it is therefore valuable to understand if and how students are using the Lewis theory to think about acid–base reaction mechanisms. This foundation is particularly important because conceptual understanding of acid–base reaction mechanisms lends itself to better understanding of other reaction mechanisms, such as nucleophilic additions (Shaffer, 2006; Cartrette and Mayo, 2011; Stoyanovich et al., 2015; Cooper et al., 2016).

Conventional versus touch-screen interfaces in organic chemistry

Line-angle structures are the conventional method for presenting organic molecules. Students often work mechanism problems by drawing arrows from nucleophilic to electrophilic sites represented in the line-angle structures. In addition to line-angle structures, interfaces on touch-screen devices also exist that allow students to construct and manipulate organic structures (Cooper et al., 2009; Larson, 2012). One such application, “OrganicPad,” allows students to construct Lewis structures and place arrows to illustrate one-step reaction mechanisms (Cooper et al., 2009). After drawing Lewis structures, students can direct the application to check for possible mistakes or convert their two-dimensional representations into three dimensions (Cooper et al., 2009). “OrganicPad” has been used in research settings to identify challenges students face with drawing Lewis structures (Cooper et al., 2010) and with drawing static reaction mechanisms (Grove et al., 2012). A similar application, “Molecules,” allows users to manipulate two-dimensional projections of three-dimensional ball-and-stick and space-filling representations of organic structures using a touch screen (Larson, 2012). This application has been shown to improve students’ representational competence skills (McCollum et al., 2014). While these applications have been shown to support students’ learning of organic representations, there has not been research focused on applications that specifically target the process of organic reaction mechanisms.

A recently-developed app, “Mechanisms,” can act as a tool for students studying organic reaction mechanisms (Mechanisms, 2018; Winter et al., 2019). It encompasses a comprehensive range of mechanisms including acid–base, addition, substitution, elimination, and electrophilic aromatic substitution reactions. The app models atoms, bonds, and electrons in a way that allows the user to dynamically manipulate chemical structures over the course of a mechanism. This interactive interface allows users to tap on carbon atoms to reveal implicit hydrogen atoms and to tap on heteroatoms or carbanions to reveal non-bonding electron pairs. Students are able to form bonds by dragging electron pairs from bonds or atoms to another atom, and the app shows users the chemical feasibility of the electron movements in real-time by either allowing the new bonds to form or by rejecting the electron movements and returning the electrons to their source. The app also provides students with guidance towards correct product formation through task cards, goals, and hints, which give information about the reaction. Since the app offers a different modality for students to work through reaction mechanisms—a modality which inherently presents reactions differently and provides additional prompting compared to the traditional paper–pencil modality—it is valuable to explore students’ thinking when using this modality as it may elicit a greater range or different types of conceptions. The app's interactive interface could be of particular interest in light of the Bongers et al. (2020) finding that students developed more dynamic mental models of reaction mechanisms following a learning activity that incorporated animated, as opposed to static, representations of a reaction mechanism. As such, the present study focuses on exploring students’ thinking—the chemical features and concepts they consider—when working through acid–base organic reaction mechanisms using either the “Mechanisms” app interface or the traditional paper–pencil.

Theoretical framework

This research is guided by the models and modelling framework originally derived from the Lesh et al. (2000) formulation of mental models and adapted by Briggs and Bodner (2005) and Briggs (2007). This framework separates mental models into five components: (1) referents, (2) relationships, (3) rules/syntax, (4) operations, and (5) results (Briggs and Bodner, 2005; Briggs, 2007). Referents are specific representations or symbols, such as atoms or molecules. Relationships are how referents relate to one another, either within molecules (e.g., atoms within a molecule relate to one another through bonds) or between molecules (e.g., the relative acidity or basicity of two molecules). The relationships are dictated by rules and syntax, where rules are defined as concepts and syntax as how rules are utilized in a task (Briggs and Bodner, 2005; Briggs, 2007). In our context, an example of a rule is the concept that bases donate electron pairs to acids, and syntax would be knowing to consider the relative acidity and basicity of sites on a molecule—using other concepts such as pK_a values and resonance—when determining which atom will donate or accept electron pairs. Operations are how referents are manipulated by applying relationships and rules to produce new representations. For example, an operation would be the action of applying the rules and syntax related to acidity and basicity to protonate the base present in the reaction. Lastly, results are the outcomes of the operation which can be used as a source of new knowledge that may inform future steps (e.g., the result of a reaction intermediate with a new set of properties that can be used to guide decisions about the next step of a reaction). Operations are unique in that they are a dynamic component whereas the other components are static.

The models and modelling framework provides a lens for examining the chemical features that students consider and apply when working through organic reaction mechanisms. The ability to identify the key referents and the relationships between them and then apply the appropriate rules and syntax allows students to proceed through a reaction mechanism as a series of chemically correct and favoured operations. With each new result, students have to take into account how the components may have changed to determine the next operation to perform and to know when they have reached the final result or product. Not only may there be variation across reactions in how students use the components of mental models, but the way information is presented may also elicit different modes of thinking or influence how students utilize the components of mental models. For example, students may engage differently with the representation of referents in the modalities explored herein, as lone electron pairs that are drawn explicitly on paper are hidden in the app unless students tap on atoms to reveal them. Additionally, the two modalities contain specific prompts that are inherent to them which may influence which components of the framework students use as well as how they use them. For example, in the app, the results of some incorrect operations are either not allowed or lead to hints that act as cues to the relationships, rules, and syntax important to the reaction. Thus, probing and analysing student thinking via multiple modalities, and situating this analysis in the models and modelling framework, provides a better understanding about how students think about reaction mechanisms.

Research questions

This study investigated how first semester organic chemistry students reason through acid–base reaction mechanisms when completing tasks via different modalities. To do this, we had students think aloud while working through two acid–base reaction mechanisms. Students were assigned to one of two groups, where one group worked through the reactions on paper and the other group worked reactions with the “Mechanisms” app. The following research questions guided our investigation:

1. How are students in organic chemistry reasoning when using either a touch-screen application or the traditional paper–pencil method when working acid–base reaction mechanisms?

2. What components of mental models do students focus on when reasoning through acid–base reaction mechanisms?

Methods

Context and participants

The study was conducted at a large, Midwestern research university. Students were recruited using a mix of purposeful and convenience sampling (Cohen et al., 2011) across three semesters from the first of a two-course, lecture-based introductory organic chemistry sequence. Brønsted–Lowry acid–base reactions are the first reaction types covered in the course, following a review of relevant general chemistry content and an introduction to resonance, VSEPR and MO theory, and the curved arrow notation. Students were recruited prior to the first exam, which also covered electrophilic addition reactions. Students were expected to be able to identify strong versus weak acids and bases, identify the most acidic proton or basic atom in a structure, use the pK_a table to determine approximate pK_a values and to identify whether structures are protonated or deprotonated given the pH of a solution, and draw mechanisms for acid–base reactions. During the first semester of data collection, students were recruited using a list provided by the instructor of the course which contained the names of the students from the top and bottom pools of scores from the first exam. This allowed for purposeful selection so that participants would have a range of abilities and conceptions. During the second and third semester of data collection, students were recruited by a course announcement for convenience sampling to increase the number of participants in the study. During the recruitment process, students were told that participating in the study would provide them with practice on organic chemistry mechanisms and, following working through the reactions, that they would be able to ask the interviewer any organic chemistry related questions they had. No additional incentives were provided. In total, thirteen students were recruited to participate in think-aloud interviews. Six of the students worked through the reaction mechanisms using the conventional paper–pencil method, denoted as paper–pencil students, and seven worked through mechanisms using the “Mechanisms” app, denoted as app students. Students were randomly assigned pseudonyms that are not representative of their ethnicity, gender, or other identities (Table 1). The research team received Institutional Review Board approval (HUM00156602) for the data collection and analysis in this study. Students consented to be part of the study at the beginning of the think-aloud interviews.

Table 1 Student participants by think-aloud interview group type

Reaction modality groups	Participants
Paper–pencil	Ana, Aurora, Daisy, Francis, Mary, Perdita
App	Angela, Belle, Flynn, Jasmine, Pepper, Peter, Tiana

---

Reaction selection

We selected reactions from the app based on the reactions covered in the course. The app presents students with the reactants (Appendix 1 – Fig. 3) but does not show the target products; however, each puzzle starts with a task card that shows mechanistic arrows indicating moves students will have to make or intermediates of the reaction. Additionally, the app may present students with hints and goals during the puzzles to direct students toward the desired products (Appendix 1 – Fig. 3). To mirror the level of information that students received from the app, we depicted the reactions for the paper–pencil students by presenting the line-angle representation of the organic reactants and the molecular formula of the major product, with the additional reagents depicted above the reaction arrow. To assess the content validity of the chosen reactions, we discussed them with three instructors for the course, one who was teaching the course during the first semester of data collection and two who had previously taught the course at the study institution. They felt the chosen reactions were similar to those students would be expected to solve and were at an appropriate difficulty level. Additionally, input from expert organic chemistry instructors guided the translation of presenting the problems within the app to the presentation on paper, to ensure students’ responses were reflective of how students would be thinking when working with these different modalities in authentic settings (e.g., while studying for an exam). We discussed the presentation with one instructor, made adjustments, and confirmed with the other instructors that the approach would not cause students undue difficulty in interpreting the questions and that they were similar in terms of the initial information provided by the app. For example, the molecular formulas of the major products, but not the minor products, were provided to the paper–pencil students in an effort to mitigate the advantage tendered to the app students via the provided hints and goals. Additionally, the reactions were unbalanced due to similar reasoning. The instructors verified that students should be familiar with reactions presented in this form, with both the lack of minor products and balancing mimicking how reactions are sometimes presented in organic chemistry lectures and textbooks. The final selected reactions are depicted in Fig. 1 and 2.

Fig. 1 Reaction schemes for the deprotonation of a 1,3-dicarbonyl by a strong base as presented during the think-aloud interviews to (A) paper–pencil students and (B) app students in the task card prior to beginning the reaction.

Fig. 2 Reaction schemes for the protonation of imidazole by a strong acid as presented during the think-aloud interviews to (A) paper–pencil students and (B) app students in the task card prior to beginning the reaction.

Think-aloud interviews

Interviews followed a think-aloud procedure, where students were prompted to verbalize their thinking as they worked through the series of reactions (Ericsson and Simon, 1980; Herrington and Daubenmire, 2014). Each think-aloud interview consisted of students working through four organic chemistry reaction mechanisms, either on paper or using the app. Results from the two acid–base reaction mechanisms are presented herein. At the beginning of the interview, students did a practice think-aloud to acclimate them to verbalizing their thoughts. During the think-aloud interviews, interviewers used probes such as “Why did you make that move?” or “What are you thinking about right now?” to prompt students to explain their reasoning. Additionally, all students were provided with the pK_a table used in their organic chemistry course for reference as, in this institutional context, it is a resource they receive at the beginning of the semester and during course assessments. The pK_a values from the table relevant to the two reactions discussed herein are presented in Appendix 2 – Fig. 4. For each student, order of the reactions was randomized. All of the interviews were video and audio recorded.

In the paper–pencil think-aloud interviews, students used a Livescribe™ pen and notebook, which recorded their writing in real time. Data collected with the Livescribe™ supplemented the audio and visual data. Prior to each interview, the interviewer wrote the reactions on separate pages in the Livescribe™ notebook in random order. Students were prompted to write all their work in the notebook and could use additional pages if necessary. To align how the reactions were presented to the app and paper–pencil students, the paper–pencil students were told the type of reaction they were doing prior to starting each reaction, as the reaction type was given in the task card presented by the app. Additionally, paper–pencil students were asked at the end of the reaction whether there were any resonance structures relevant to the reaction, as the app prompted students to show all resonance structures. We did not provide explicit cues to students to parallel the other prompts that were provided by the app (e.g., hints).

Interviews with the app students were conducted similarly to paper–pencil interviews with the addition that students were given an abbreviated version of the tutorial provided by the app before starting the think-aloud interview. The tutorial was adapted by one member of the research team (ESW) and refined by independently piloting it with two other members of the research team (SFQ and MP) who had not yet used the app. The tutorial instructed students on how to reveal implicit lone pairs and hydrogen atoms, how to create and break bonds, and how to move and rotate molecules. This ensured that unfamiliarity with the app's functions did not inhibit students’ abilities to work through the reactions. Two of the app students had used the app previously and the remaining app students did not exhibit undue difficulty. An occasional difficulty students encountered when using the interface was getting the app to register their intended movements of electron pairs. When a student made a correct move that the app did not register as such, the interviewer suggested they try again as the difficulty was not related to the student's thinking about the chemistry.

Development and application of the coding scheme

The coding scheme was developed through open coding and constant comparison of the think-aloud interviews (Corbin and Strauss, 1990). Four of the researchers (SFQ, MP, ESW, SA) reviewed the transcripts and audio/visual data produced from the think-aloud interviews, noting observations related to students’ thinking and identifying initial codes. The research team discussed the codes and grouped them into parent codes of chemical considerations, reaction step, participant usage, justification, student actions, and app-specific. Two of the four researchers (SFQ and MP) then finalized the coding scheme and trained a fifth member of the research team (FW) to use the coding scheme. The coding scheme is presented in Appendix 3 – Table 3.

To establish what sections of each transcript should be coded, all transcripts were divided into units of analysis corresponding to thinking stages, where students verbalized their ideas about steps in the reaction, and action/operation stages, where students performed the electron movements to break and form bonds. The two members of the research team who finalized the coding scheme (SFQ and MP) identified and agreed upon the units of analysis for all transcripts before coding. One of those researchers (MP) and the trained fifth member (FW), who was not involved in the development of the coding scheme, then independently coded both reactions from four participants (30% of the data), met to clarify the coding definitions, and came to a consensus on the application of the coding scheme for these reactions. Afterwards, the same two researchers (MP and FW) independently coded both reactions from the remaining nine participants (70% of the data). During this process, the researchers met to discuss the application of the coding scheme, assess agreement using the fuzzy kappa statistic (Kirilenko and Stepchenkova, 2016), and come to a consensus for coding. The initial fuzzy kappa value for the 70% of the data coded after clarifying the coding scheme was 0.82, within the range indicating near-perfect agreement (McHugh, 2012). Furthermore, as consensus was reached for each transcript, the researchers overcame initial coding disagreements to achieve complete agreement.

Results

The results are drawn from the qualitative analysis of students’ think-aloud interviews in which they attempted to produce the mechanisms for two acid–base reactions using one of the two modalities. This analysis was guided by the models and modelling framework, and thus we refer to atoms and molecules as referents, the concepts students draw upon as rules, and the way students apply concepts as syntax. By examining the rules/concepts students referred to and the syntax with which they applied these rules, we are able to identify the reasoning students exhibited when considering the mechanisms. Analysing the interviews through the lens of the models and modelling framework additionally allows us to begin differentiating whether students’ difficulties arise from their conceptual knowledge or their ability to apply that knowledge. Furthermore, we examine how the two modalities, and the prompts inherent to each, may influence student reasoning. We first present students’ responses when producing a mechanism for the deprotonation of a 1,3-dicarbonyl, followed by students’ responses when producing a mechanism for the protonation of imidazole.

Deprotonation of a 1,3-dicarbonyl by a strong base

In this reaction, students first needed to assign the roles each molecule would play (i.e., acid or base) by determining the relationship between the referents. Then, considering the rules and syntax associated with acid–base chemistry, they needed to identify the most acidic site for deprotonation on the dicarbonyl (Fig. 1). The pK_a table all students were given included, among pK_a values for other structures, a dicarbonyl similar to that in the reaction and the pK_a value for water which they could use to identify relative acidity and basicity should they need it as a resource (Appendix 2 – Fig. 4A). Following their decisions about acidity and basicity, students could then perform the associated operations, where the result should lead to a consideration of resonance stabilization of the product. The students in each group tended to approach each step of the mechanism using distinct reasoning, potentially due to differences in prompting by the modalities, and thus they will be discussed separately.

Most paper–pencil students started the reaction by attempting to determine the acid–base relationships between the molecules in the reaction. One student, Ana, used an atom-counting strategy to determine that the dicarbonyl compound would lose a proton and then identified that hydroxide would remove the proton. All other paper–pencil students who completed the reaction used the rules of the pK_a table to determine the acid–base relationship between the molecules, where only one student, Francis, first correctly identified the acid and the base using chemical thinking and then confirmed their decision with the pK_a table. Of the students who went directly to the pK_a table to identify each species, Mary correctly identified the role of each species. Daisy and Aurora, however, had some difficulties identifying the acid–base relationship and exhibited an incomplete knowledge of the syntax for using pK_a values in doing so. Aurora incorrectly identified the dicarbonyl as a base and hydroxide as an acid when first looking at the structures, and then turned to the pK_a table to identify the relevant pK_a values. Aurora then started to doubt their original assignment of acid and base, but resorted to using the formula of the major product to determine that the dicarbonyl was losing a proton and must be the acid in the reaction rather than basing their reassignment on the pK_a values. Daisy correctly identified the acid and base using values from the pK_a table, but then revealed incorrect understanding of the underlying concepts when considering how the species would react:

So, since it's an acid, that means it gets protonated. So, this bond between the OH would break. And then the lone pairs go on the oxygen… And this hydrogen would now be added to one of these. One of the oxygens with the lone pair.

After completing these steps, Daisy counted atoms and identified a discrepancy between the product they had drawn and the given condensed formula, but did not know how to address this discrepancy and stopped working on the reaction. While for Aurora the pK_a values cued a discrepancy with their original assignment of acid and base, Daisy was not able to move from the pK_a values to what they indicated about which species was donating or accepting a proton.

One paper–pencil student, Perdita, did not attempt the problem, initially approaching the reaction similarly to Aurora by first considering the carbonyl oxygen atom as a base and then using an atom-counting strategy. However, as side-products were not shown and the presented reaction was not balanced, Perdita did not know how to account for the apparent loss of an oxygen atom:

Well, I guess I’m confused in general, because there's three oxygens over here, and then over here there's only two. So I’m like, where does this third oxygen go? Which I’m confused about. So… I don’t know, an oxygen just vanishes.

Although Perdita did not complete the reaction, they did initially attempt to identify the acid–base relationship. Perdita recognized their initial assignment of acid and base to be incorrect, but then did not attempt the reaction further after not knowing how to navigate the unbalanced reaction. Perdita's difficulty with how the paper–pencil representation was presented is important to note, as instructors and textbooks do not always provide students with balanced reactions.

The app students were more varied in how they began the reaction. Few students began by attempting to determine the acid–base relationship and only one student, Belle, correctly identified the acid and the base, noting the charge on the hydroxide and using the pK_a table to guide their thinking. Tiana immediately looked at the reacting species and the pK_a table and incorrectly identified the hydroxide hydrogen atom as the most acidic proton. However, after attempting an electron movement the app did not allow, Tiana examined the task card and immediately realized the appropriate mechanistic step. Angela also struggled to identify the acid and base, recognizing both the hydroxide and the carbonyl oxygen atoms as having lone electron pairs and capable of being protonated. Notably, Angela did not use the pK_a table to guide their thinking, instead attempting to protonate one of the carbonyls—a move the app would not allow—before turning to the goals within the app to help guide their thinking. The remaining app students immediately relied on the task card that was presented to them at the beginning of the reaction to guide their first steps, effectively skipping the step of identifying the relationship between the molecules as the task card indicates which molecule gains and which loses the proton that is transferred during the reaction (Fig. 1B).

For the paper–pencil students who identified the acid and the base in the reaction, the next step was to use the rules and syntax of acid–base reactions to determine the operation of which proton would be removed from the dicarbonyl compound. They primarily used the pK_a table, with some also considering the rules and syntax associated with resonance to make this decision. Mary and Francis used the pK_a table to identify the appropriate proton to be removed. Mary commented on the difference in pK_a values between the acid and the conjugate acid of the base to confirm their choice and, while they did deprotonate at the correct site, did not consider which protons adjacent to the carbonyls were the most acidic. Francis considered other protons that could be removed from the dicarbonyl, but justified that one of the protons in between the two carbonyls would be removed because they recognized that deprotonation between the two carbonyls would result in a product that could be stabilized by resonance. Aurora and Ana, also paper–pencil students, recognized the need to consider which of the protons adjacent to the carbonyls would be removed and considered resonance to guide the decisions they made. However, both neglected to consider the protons in between the two carbonyls. Aurora started to consider the correct protons following probing about why they had considered the protons they initially focused on. After this probing, they identified the oxygen atoms in the carbonyls as allowing the potential for resonance stabilization in the deprotonated product, and then used the pK_a table to confirm which were the most acidic, ultimately deprotonating the correct carbon atom:

Yeah, I guess it could also come off here, that might actually be more stable. I don’t know if there [is] an exact pK _a —oh wait, this is kind of… this is 9.2, this is the one for the hydrogen right there, so that would probably be it because that's more stable because there's more resonance coming from both these O's.

Despite also consulting the pK_a table and considering the possibility for resonance structures in the deprotonated product, Ana ultimately did not use the appropriate syntax for these concepts and chose to deprotonate the incorrect carbon atom.

The majority of the app students who relied on the task or goal cards did not consider which proton to remove when performing their first operation. The task card showed an intermediate step rather than the first step of the reaction, presenting a molecule of water next to the dicarbonyl with a lone electron pair and negative charge at the central carbon atom (Fig. 1B). Jasmine, Pepper, Angela, and Tiana used the task card to guide their reasoning to deprotonate at the appropriate location without vocalizing any chemical thinking about the rules or syntax of acid–base reactions. In addition to using the task card to guide their initial steps, Flynn and Peter used some chemical thinking to identify the most acidic proton. Both recognized from the task card that the reaction used hydroxide to form water, after which Flynn used the pK_a table to correctly identify the most acidic proton while Peter identified that forming a carbanion adjacent to one of the carbonyls would result in a lone pair that could be delocalized. However, Peter made the same mistake as Aurora and Ana in the paper–pencil group and initially tried to remove a proton that would result in a structure with less resonance stabilization. Since the app did not allow Peter to make this move, Peter then consulted the pK_a table and used the information provided to identify which proton to remove.

After the operation of deprotonation, the final step of the reaction was to use the rules and syntax affiliated with resonance to identify the two primary resonance contributors for the product. All three of the paper–pencil students who deprotonated at the appropriate carbon atom on the dicarbonyl were able to complete this task without difficulty, and most described their reasoning in terms of electronegativity. Following deprotonation, Francis and Mary both drew one of the resonance contributors to show stabilization of the negative charge on the carbon atom. Aurora provided similar reasoning following a post-reaction interview question about the potential for resonance structures. In their discussions, both Francis and Aurora expressed incorrect understanding about resonance. Aurora considered drawing both resonance contributors, but felt that one structure was more stable than the other, conflating stability with degree of contribution to the resonance hybrid. When considering the possibility of the second resonance contributor with the negative charge on an oxygen atom, Francis revealed a misconception regarding resonance structures:

Oh you would have a mixture, because you would always have a mixture… like all three of these could still exist in solution.

Only one app student, Belle, showed the resonance structures without being prompted by the app. Belle realized that the carbanion produced was not very stable and was able to depict the two resonance contributors where the negative charge was on one of the carbonyl oxygen atoms which stabilized the structure. The remaining app students required prompting from either the task or goal cards before showing both resonance structures. Only Jasmine and Tiana explicitly expressed that the presence of resonance contributors would stabilize the product, as it places a partial negative charge on the more electronegative oxygen atom. Angela had some difficulties showing the resonance structures, struggling to identify the correct place to start the movement of electrons, first using the lone pairs on the carbonyl oxygen atom before realizing that they needed to start drawing the resonance structures from the lone pair on the negatively charged carbon atom.

In all, students exhibited differences in approach to this reaction depending on whether they were working with the app or with paper-and-pencil. The paper–pencil students tended to begin by trying to identify the acid–base relationship, while app students often skipped this step due to the intermediate structure being provided in the task card for the reaction. Similarly, students from the app group were able to determine the site of deprotonation using the app's guidance, a task which proved challenging for many paper–pencil students. Students across both groups tended to use the rules and syntax of resonance to identify the resonance structures for the product without difficulty, though some did exhibit problematic thinking.

Protonation of imidazole by a strong acid

In the strong acid protonation of imidazole (Fig. 2) students had to identify the most basic nitrogen atom in the ring by applying the rules and syntax associated with acid–base chemistry and resonance. The key to this reaction was for students to recognize that, after the first operation of protonation, the positive charge on one of the nitrogen atoms would be stabilized through resonance whereas the other would not, indicating the preferred product. The pK_a table that students received had two potential structures they could identify as structurally similar to the two nitrogen atoms in the ring and use to guide their thinking (Appendix 2 – Fig. 4B). Unlike in the dicarbonyl reaction mechanism, where the paper–pencil and app students appeared to make relatively distinct moves, the students approached the imidazole reaction more similarly across the groups and thus will be discussed together.

Most students from both groups began this reaction by recognizing HCl as a strong acid and using their knowledge of the acid–base relationship to identify that one of the nitrogen atoms in the imidazole ring would be protonated. While most students did not provide a thorough explanation for why a particular nitrogen atom would be protonated, a few students cited reasons for why nitrogen rather than one of the carbon atoms would be protonated. Tiana considered the relationships between the two types of atoms by comparing their basicity, mentioning that nitrogen is more basic than carbon. Aurora reasoned that carbon should not receive a charge and Jasmine identified that the carbon atoms were closed shell, leading both to conclude that a carbon would not be protonated. This indicates that students have some ability to correctly identify basic sites, but it is unclear whether this is from recognizing atoms they are familiar with from other acid–base reactions or if they are actually thinking about chemical properties.

The majority of students generally struggled with the rules and syntax when determining which nitrogen atom to protonate during the first operation. Overall, students in both groups showed a heavy reliance on the pK_a table to determine the correct site for protonation (Appendix 2 – Fig. 4B). Aurora, Daisy, Belle, and Flynn, two students from each group, each only identified one relevant pK_a value on the table and chose to protonate at the corresponding nitrogen atom in imidazole. The thinking behind this was verbalized by Aurora and Flynn, who reasoned that the relevant pK_a values are either given in the table or provided in the reaction. Aurora said:

Yeah, I mean, I feel like a lot of times if they don't have it on the pK _a table and it's really important then they give you that value in the question, since the value's not in the question it makes me think that maybe it's not it. Which probably isn't a very good answer, but in a test situation that's probably would I would do.

While three of the four identified the correct nitrogen atom to protonate and were able to proceed, Flynn identified the conjugate acids of ammonia and methylamine in the pK_a table and determined that the pK_a of the secondary amine in the ring would fall between the affiliated pK_a values. Flynn tried to protonate at that nitrogen atom but was prevented by the app. Mary did identify two nitrogen-containing structures in the pK_a table; however, the more basic structure they identified was not a good approximation for the protonated nitrogen atom in imidazole that they related it to. This led Mary to protonate the incorrect nitrogen atom and form the incorrect product. Both Angela and Pepper, app students, did not rely on the pK_a table or initially exhibit chemical reasoning. Angela chose the incorrect nitrogen atom without verbalizing their reasoning before being cued by the app to consider which nitrogen atom was the most basic; Pepper based their decision on the task card for the reaction which showed the lone pairs on the most basic nitrogen atom (Fig. 2B). After a probing question by the interviewer, both students discussed how they thought the nitrogen atom they did not protonate would be less basic because it already had a hydrogen atom attached.

The remaining students, three from each group, thought about how the rules and syntax of resonance would impact which nitrogen atom was favoured for protonation. However, only Francis and Ana, paper–pencil students, recognized that for this reaction they should be considering the potential for resonance in the products and drew potential resonance contributors. Ana said:

So now I have to see which of these structures is better, or which N can hold the positive better.

Peter, Tiana, Jasmine, and Perdita all focused on resonance stabilization of the reactant rather than the possible products, incorrectly applying the syntax of resonance structures and ultimately selecting the incorrect nitrogen atom to protonate. Of the four, only Perdita was a paper–pencil student and proceeded to form the incorrect product. Peter, Tiana, and Jasmine received a hint from the app that they should use the most basic lone pair and show delocalization of the positive charge through resonance. While this did not lead them to reason through why their original thinking was incorrect, they did subsequently protonate the correct nitrogen atom. The focus on resonance stabilization of the reactant indicates a gap in students’ understanding of how to appropriately apply the syntax of resonance when considering acid–base reaction mechanisms.

Following the operation of protonating the nitrogen atom, students were prompted to draw resonance structures for the resulting product molecule either by the app or, in the case of the paper–pencil students, as part of post-reaction interview questions. Students from both groups had difficulty with this. All of the students except Mary recognized that the product would be stabilized by the presence of resonance contributors, but most students had some difficulty identifying what source of electrons to use when performing the operation to depict the resonance structures. All of the app students, except Flynn, and three of the paper–pencil students tried to start depicting resonance structures from one of the carbon–carbon double bonds in the imidazole rather than using the available lone pairs on the nitrogen atom. Two of the remaining paper–pencil students, Aurora and Mary, did not draw resonance structures; for Mary, this was because they had drawn an incorrect product that did not have the potential for resonance. Francis, the last paper–pencil student, did use the lone pair electrons on the neutral nitrogen atom to start their resonance structures. For the app students, the focus on the double bonds may have been exacerbated by the fact that the lone pairs are not automatically visible in the app and students first had to select the nitrogen atom to reveal them. This is especially interesting as all the paper–pencil students had drawn in the lone pairs present in their final products. This could indicate a focus on the explicit features, such as double bonds, present in the referents and that the app students had difficulty in readily identifying the implicit lone pair electrons on the neutral nitrogen atom.

Overall, this reaction was potentially more difficult for students. Many struggled to apply the rules and syntax of acid–base chemistry and resonance, which led them to protonate the incorrect nitrogen atom during the first operation, or exhibited minimal reasoning when they chose the correct one. The potential for resonance in the product also caused difficulties, where some students recognized the rules of resonance stabilization but they struggled to apply the syntax in predicting the reaction outcome and when depicting the resonance structures of the product.

Discussion

This research used two modalities, paper–pencil and app, to elicit student reasoning about acid–base organic chemistry reactions. By describing the results through the lens of the models and modelling framework we can characterize what chemical features and concepts students identified as important for reaction progress and how those informed the mechanistic steps they made. This framework also allows for an initial understanding of whether the different representations, or modalities, resulted in different use of the models, which is worth investigating further. We present differences and similarities between students’ responses when using the two modalities, and we emphasize that these differences may also stem from differences between the modalities in both how the reactions are presented and how different levels of feedback or prompting are provided. Generally, the students using the app and paper–pencil modalities exhibited commonalities in the chemical features they focused on but appeared to have differences in their approaches, in particular for the dicarbonyl reaction. This may be due to the fact that the presentation of the reaction, which is inherently connected to the modality, may have guided students’ thinking. Beyond differences in how the reactions are presented between modalities, differences in students’ thinking may also stem from the level of feedback provided within the app compared to the minimal level of feedback when working with paper and pencil. Hence, we consider how the modalities as a whole influence students’ reasoning. The common problematic thinking that students demonstrated across both groups and for both reactions are summarized in Table 2.

Table 2 Common student difficulties across modalities

Problematic student thinking	Reaction	Level(s) in the models and modelling framework
When identifying acids and bases, limiting considerations to surface features and/or Brønsted–Lowry definitions	Dicarbonyl	Relationship, rules and syntax
Not considering the relative acidity of hydrogen atoms	Dicarbonyl	Syntax
Identifying resonance structures as a mixture rather than contributing to a resonance hybrid	Dicarbonyl	Rules
Overreliance on the pKa table	Imidazole	Rules and syntax
Inability to generalize from the structures provided in the pKa table	Imidazole	Rules
Focusing on resonance in the reactant rather than the potential product	Imidazole	Syntax
Difficulty drawing resonance structures	Imidazole	Syntax, operations

---

Students generally focused on explicit, rather than implicit, referents and relationships

Generally, students discussed the reactions in terms of the molecules and atoms involved, using minimal language to describe the breaking and forming of bonds or the movement of electrons. The lack of students using language to describe electron movement to break and form bonds is in contrast to other studies (Galloway et al., 2017; Bhattacharyya and Harris, 2018), though it does support the finding that students often devalue the physical meaning behind the electron-pushing formalism (Bhattacharyya and Bodner, 2005). When students did talk about electrons, they were often referring to lone pairs available to participate in reaction steps. This supports previous research that indicates students focus on the explicit referents in reactions rather than more implicit features (Domin et al., 2008; Anzovino and Lowery Bretz, 2015; Galloway et al., 2017; Graulich and Bhattacharyya, 2017; Caspari et al., 2018; Graulich et al., 2019).

When solving either acid–base reaction, students generally began their thinking by identifying the relationships between referents in the reaction by assessing relative acidity and basicity. Students had more difficulties identifying the acid and base for the dicarbonyl reaction. This could be due to the fact that the acid in the reaction—the dicarbonyl—did not have explicit hydrogen atoms to signal students toward thinking about its relative acidity when combined with hydroxide in the reaction. Similarly, although the hydroxide presented to students in the dicarbonyl reaction had a negative charge, many students did not immediately recognize it as a base and some students mislabelled it as an acid. That students mislabelled hydroxide as an acid is similar to Anderson and Bodner's (2008) finding that students incorrectly transfer knowledge of periodic trends when identifying acidic species. Furthermore, the difficulties students had identifying the base despite the presence of a negative charge is suggestive that students were not considering the ability of the reactant to donate electron pairs, aligning with the finding of Cartrette and Mayo (2011) that students focus on the Brønsted–Lowry definitions of acids as proton donors and bases as proton acceptors.

Similarly, for the imidazole reaction, students tended to determine the acid–base relationship using surface features of the molecules given: the presence of HCl and of nitrogen atoms in the ring. Hydrochloric acid is likely one of the first strong acids that students learn in general chemistry, and many students immediately recognized it as an acid. Similarly, many students explained that they knew nitrogen atoms in molecules tended to act as basic sites. Students’ thinking appeared to be guided by the surface features of these molecules, and as a result they tended to not explicitly consider any specific theory of acids and bases. This is similar to prior findings in the literature in which students were found to make decisions about organic reaction mechanisms by focusing on the surface features of the reactants rather than the chemical information communicated by the structure (McClary and Talanquer, 2011; Anzovino and Lowery Bretz, 2015). Students in particular were not considering the Lewis acid–base theory, focusing on the atoms and molecules themselves rather than the ability of reactive species to accept or donate electrons, a finding similar to those in prior research (Cartrette and Mayo, 2011; Dood et al., 2018; Schmidt-McCormack et al., 2019).

The different levels of ease with which students were able to determine the acid and base between the two reactions may explain why the groups of students were similar in their responses to the imidazole reaction but dissimilar in their responses to the dicarbonyl reaction. Specifically, most students automatically identified HCl as the acid in the imidazole reaction but they had difficulty assigning acid–base character in the dicarbonyl reaction and so relied more heavily on the supports available to them. For the paper–pencil students this was the pK_a table, but the app students were also able to rely on the modality itself as a source of information.

Students recognized the rules related to the reactions, but could not always successfully apply the affiliated syntax

For both reactions, students generally recognized the rules, or pertinent concepts, for the reaction—knowledge of pK_a values, resonance, and that the reaction would involve one species deprotonating another. However, students’ recognition of the syntax—of the need to use knowledge of pK_a values and resonance to make a decision about reactivity—differed between reactions. It is important for students to know both the rules and the syntax affiliated with acid–base reactions, as acid–base concepts are frequently utilized in more complex organic chemistry reactions (Stoyanovich et al., 2015). For the dicarbonyl mechanism, most paper–pencil students knew to use the pK_a table but not without difficulty—and ultimately some students relied on alternative strategies to make a decision with respect to the rule, such as counting atoms which was similar to the mapping strategy identified previously (Ferguson and Bodner, 2008; Bhattacharyya, 2014; Flynn and Featherstone, 2017; Galloway et al., 2017; Webber and Flynn, 2018). With the app, however, students appeared to not consider pK_a or resonance. Many of these students began with simply trying mechanistic steps, using the app-directed tasks to guide their thinking. On the other hand, for the imidazole reaction, students in both groups knew to use the pK_a table to identify the specific site on the molecule where the reaction would occur, though they had difficulty utilizing the pK_a table as none of the exact structures from the reactions were present. This indicates that while students generally knew that they could use the pK_a table, they may not know how to effectively apply the information the pK_a table contains and may preferentially use it in lieu of chemical thinking. These findings align with the research by Flynn and Amellal (2016) who identified that students had difficulties using the pK_a table when given more complex molecules and when they needed to approximate pK_a values.

Students from both groups frequently referred to resonance, aligning with findings by Ferguson and Bodner (2008). They demonstrated a range of thinking with respect to the resonance concept, many exhibiting learning difficulties similar to those described by Taber (2002) and Kim et al. (2019). In the dicarbonyl reaction, students exhibited an understanding of the concepts, or rules, relating to resonance stabilization when determining the site where the reaction would occur. However, students’ approach to the imidazole reaction revealed some difficulties with the syntax of resonance, where a number of students focused on resonance stabilization in the reactant rather than the product when determining the relative acidity of the two nitrogen atoms. This is similar to work by Cartrette and Mayo (2011) which indicates that students can identify the importance of resonance for assessing acidity or basicity, but may struggle to apply it successfully. Furthermore, this ability to determine relative acidity is one of the ten necessary learning outcomes for the resonance concept as identified by Carle and Flynn (2020). Thus, it is valuable to recognize that not all students are meeting this learning outcome. A few students verbalized incorrect thinking about the relationships between resonance structures, specifically by expressing that various resonance structures are present as a mixture rather than contributing to the resonance hybrid. This incorrect understanding aligns with the previously reported findings that students consider resonance structures as distinct entities or as representations that denote rapid interconversion between double and single bonds (Taber, 2002; Kim et al., 2019). As considering resonance structures can be important when determining how a reaction will proceed for many types of reactions (Carle and Flynn, 2020), it is key to build students’ understanding of this concept and how to apply it in different contexts.

Students often considered one possible operation (i.e., mechanistic pathway), unless otherwise prompted

Our analysis indicates that there may be a difference between app and paper–pencil students in the extent to which they consider multiple mechanistic pathways. The paper–pencil students did not as often consider different possibilities in order to select the most likely mechanistic pathway and, for these students, incorrect decisions were often carried throughout the remainder of the reaction without notice or led to frustration later in the mechanism when they identified that something was not correct. This frustration compelled students to simply stop working on the reaction. On the other hand, students using the app were able to try different electron movements to see what the app would allow. The app students were able to get feedback from the app and could use this to guide their decision-making. This is not without drawback, as students tended to try things before considering the chemical feasibility of different possible mechanistic steps. However, some students did apply chemical reasoning after determining the mechanistic steps to explain why a particular step was correct once the app accepted the electron movements they tried. The app also prevented students from making and justifying incorrect mechanistic steps, providing targeted hints that could guide their thinking and constraining students from making chemically incorrect moves. This is particularly valuable in that it prevents students from the frustration caused by carrying through chemically infeasible steps that might lead students to stop thinking about the reaction altogether.

Limitations

There are a few limitations to this study inherent to the methodology used. This study was small and qualitative in nature and so the claims are limited in that we may not have captured the full range of students’ thinking regarding acid–base reaction mechanisms and cannot make claims as to the relative prevalence of conceptions discussed herein. This study also only included students from a single institution and thus the results may not broadly apply across institutions. A larger sample size across a range of institutions may have revealed a greater range of conceptions and indicated differences in conceptions due to students’ prior chemistry knowledge, the order in which the material is taught, and instructor methods. Specifically, most of the students at the study institution bypass general chemistry at the undergraduate level and go directly into first semester organic chemistry. Additionally, we might expect different reasoning by students who went through a revised curriculum such as that described by Flynn and Ogilvie (2015). While a quantitative study using survey methodology could provide information about the relative prevalence of students’ conceptions, our study design was able to capture individualized conceptions. Additionally, while utilizing the two modalities allowed us to elicit a range of thinking across the students, there were inherent differences in the think-aloud procedures for the two groups of students that may have led to differences in student responses. However, in developing the interview protocol, and during the expert validation of the chosen reaction mechanisms, we attempted to ensure that the problem representation and prompting most aligned with how students would authentically engage with the different modalities, while mitigating differences from features other than the modalities and their inherent differences in prompting (e.g., providing both groups of students with pK_a tables).

Conclusions and implications

This study captured how students thought through acid–base reaction mechanisms by using two different modalities—i.e., paper–pencil and app based—and applied a models and modelling framework to examine the chemical features and concepts that students used to inform the mechanistic steps they made. Students’ thinking was elicited through think-aloud interviews in which students worked through two acid–base reaction mechanisms either on paper or using the “Mechanisms” app. In general, students from both groups focused on the explicit features present in the modality they were using with minimal consideration of implicit electronics. They were familiar with the pertinent steps and rules for acid–base reactions, such as needing to determine the acidic and basic sites in a given reaction, and were familiar with the syntax used to make judgments about such rules, such as considering pK_a values or resonance. However, they often exhibited difficulty in applying the syntax to make decisions about the rules for the given reactions, indicating a poor conceptual grounding. Additionally, students showed reliance on explicit features, supports, and prompting—the nature of which differed between modalities—and did not always exhibit chemical thinking. For example, students resorted to strategies such as counting atoms to determine the acidity or basicity of a molecule, identifying similar structures on a pK_a table without thinking about implicit structural features, or using the app for guidance before using their own content knowledge. While resources such as the pK_a table or prompts provided by the app can be useful and support learning, it is important to train students to use these resources to support their critical thinking.

The results of this study have implications for both research and practice. Utilizing both the app and paper–pencil modalities for the think-aloud interviews elicited a greater range of student thinking. Therefore, this interview methodology has potential for future research focused on student thinking about reaction mechanisms and supports using multiple modalities to probe different thinking strategies that students may utilize. Our findings indicate that future research expanding this work to different reaction types or institutions may be merited. In particular, it would be valuable to compare students’ thinking across institutions that use different instructional methods to teach the organic chemistry curriculum, such as that described by Flynn and Ogilvie (2015). Additionally, with the increased prevalence of app-based instructional tools, it is important to understand how these tools do or do not impact student thinking. Our results indicate that the app can be helpful for guiding student thinking and providing beneficial feedback to prevent students from performing chemically infeasible steps or obtaining incorrect products. However, additional scaffolding by instructors to promote reflective thinking may be necessary to mitigate rote use of the app. Promoting this type of reflective thinking would also benefit students working through reaction mechanisms in the traditional mode on paper, by helping them consider multiple reaction pathways and the chemical feasibility of proposed mechanistic steps.

Conflicts of interest

This research was funded in part by Alchemie, the company that created the Mechanisms app. Alchemie did not play any role in research design, data collection, or data analysis.

Appendices

Appendix 1 – App goal cards and initial reaction screens

Fig. 3 Goal cards (A and C) and initial reaction screens (B and D) seen by the app students as they worked through the 1,3-dicarbonyl and imidazole reactions, respectively.

Appendix 2 – Excerpts from pK_a table

Fig. 4 The structures that students referenced from the pK_a table they received during the think-aloud interviews: (A) pK_a values relevant to the 1,3-dicarbonyl reaction and (B) pK_a values relevant to the imidazole reaction. Students were provided with the complete pK_a table they use in the organic chemistry courses at the study institution.

Appendix 3 – Coding scheme

Table 3 Coding scheme

Parent code	Subcode	Definition	Exemplars
Chemical considerations	Protonation/deprotonation	Student discusses where protonation or deprotonation will occur or talks about protonating/deprotonating during a step of the reaction.	“This one's been protonated, it's going to take hydrogen from somewhere…”
	Acid–base	Student identifies the acid, base, or the acidic/basic site on a molecule or in the reaction.	“That's a strong acid that will dissociate. HCl…”
	Charge	Student thinks about the role charged atoms play in directing the reaction steps or discusses charge on atom/molecule. Charge can be implicitly mentioned (i.e., talking about further reaction at carbocation because it is unstable).	“I'm looking at this and I don't think carbon wants to have that negative charge very much.”
	Carbocation	Student explicitly mentions a carbocation. This could be the presence of, formation of, or stabilization of a carbocation.	“Yes. Actually no. Because you can't really move the double bond around too much because then the carbon will become a carbocation.”
	Resonance	Student talks about the presence of resonance structures or resonance stabilization.	“I know, in this, the resonances look different to me.”
	Electronegativity	Student considers the electronegativity of various atoms to help determine reactivity.	“The oxygen's more electronegative, so that's going to be more likely to have that negative charge.”
Reaction step	Bond breaking/forming	Student explicitly talks about breaking or forming a bond during the reaction step.	“I'll drag one of the electron pair to the hydrogen and break the hydrogen bond to form the water, and now we have a negatively charged carbon atom.”
	Electrons	Student explicitly talks about electrons or lone pairs that are present or moving during the reaction step.	“…so this is allowed to move the electrons.”
	Molecule/atom-focused	Student talks about a molecule or atom reacting during the reaction step.	“Alright. I know HCl is a really good acid, which means that it likes to give its hydrogen away.”
Justification	Recognizes reaction component or step	Student recognizes a step/component of a reaction because they know it is a step/component of the type/classification of reaction they are doing. Often they explicitly identify some surface features to identify the step or type of reaction; this can be species in the reaction, functional groups, individual atoms, bonds, etc. (not just stating reaction type because this is told to them).	“so that tells me that this is a proton addition, or proton transfer, reaction.”
	App hint/goal/task card directed action	Student explicitly verbalizes that a hint, goal, or task card directed their action.	“and then the arrows also showed the electrons that are this double bond over here to get the oxygen lone pairs.”
Student actions	Incorrect	Student makes a move that is incorrect. Co-coded with the chemical feature/move that is incorrect.	“So, I'll drive one of the hydrogens to the oxygen. Not gonna work.”
	Draw or pop out implicit protons or lone pairs	Student draws out the protons or lone pairs on a line-angle notation molecule; also code if they redraw molecules as Lewis structures.	“…okay. I'm gonna say it keeps this lone pair. Just… all right. And then you have 1, 2, 3 C's an five Hs.”
	Counting atoms	Student counts atoms at the beginning to identify what changes or at the end to make sure all atoms are accounted for.	“So this one, isopropyl formula, this one is two, three, four, five, six, C6 with two O's”
	pKa table	Student references the pKa table provided or verbalizes memorized pKa values.	“To see if, well I know this is a strong acid but I see it's pKa and see if it can protonate one of the two nitrogens”
App-specific	Hint	Student gets a hint during the puzzle.	“not the most basic lone pair… positive charges… resonance structures. Right, so. Yeah. I'm going to just restart.”
	Goals	Student looks at the goals during the puzzle.	“it told me that wasn't the…”
	Trying random things	Student starts trying random actions to find something that will work.	“I don't even know what I'm trying to do at this point.”
	Restarted puzzle	Student restarts the puzzle mid-reaction.	“And so, restart that.”

---

Acknowledgements

This work has been supported, in part, through a grant to Alchemie from the National Science Foundation Small Business Innovation Research program, #1659983, and the Michigan Corporate Relations Network (MCRN), funded by the Michigan Economic Development Corporation (MEDC) and administered by the University of Michigan Business Engagement Center and the U-M Economic Growth Institute's Small Company Innovation Program (SCIP), #AWD006745. This work was also supported by the National Science Foundation Graduate Research Fellowship Program, #DGE1256260. Additionally, we would like to thank the students who participated in our study.

References

1. Anderson T. L. and Bodner G. M., (2008), What can we do about “Parker”? A case study of a good student who didn’t “get” organic chemistry, Chem. Educ. Res. Pract., 9(2), 93–101.
2. Anzovino M. E. and Lowery Bretz S., (2015), Organic chemistry students’ ideas about nucleophiles and electrophiles: the role of charges and mechanisms, Chem. Educ. Res. Pract., 16(4), 797–810.
3. Bell E., Provost J. and Bell J. K., (2019), Skills and foundational concepts for biochemistry students, ACS Symp. Ser., 1337, 65–109.
4. Bhattacharyya G., (2006), Practitioner development in organic chemistry: how graduate students conceptualize organic acids, Chem. Educ. Res. Pract., 7(4), 240–247.
5. Bhattacharyya G., (2013), From source to sink: mechanistic reasoning using the electron-pushing formalism, J. Chem. Educ., 90(10), 1282–1289.
6. Bhattacharyya G., (2014), Trials and tribulations: student approaches and difficulties with proposing mechanisms using the electron-pushing formalism, Chem. Educ. Res. Pract., 15(4), 594–609.
7. Bhattacharyya G. and Bodner G. M., (2005), “It Gets Me to the Product”: how students propose organic mechanisms, J. Chem. Educ., 82(9), 1402.
8. Bhattacharyya G. and Harris M. S., (2018), Compromised structures: verbal descriptions of mechanism diagrams, J. Chem. Educ., 95(3), 366–375.
9. Bongers A., Beauvoir B., Streja N., Northoff G. and Flynn A. B., (2020), Building mental models of a reaction mechanism: the influence of static and animated representations, prior knowledge, and spatial ability, Chem. Educ. Res. Pract.,  10.1039/c9rp00198k.
10. Briggs M. W., (2007), Models and modeling: a theory of learning, in Bodner G. M. and Orgill M. (ed.), Theoretical Frameworks for Research in Chemistry/Science Education, Pearson Prentice Hall, pp. 69–82.
11. Briggs M. and Bodner G., (2005), A model of molecular visualization, in Gilbert J. K. (ed.), Visualization in Science Education, Netherlands: Springer, pp. 61–72.
12. Carle M. S. and Flynn A. B., (2020), Essential learning outcomes for delocalization (resonance) concepts: how are they taught, practiced and assessed in organic chemistry? Chem. Educ. Res. Pract.,  10.1039/C9RP00203K.
13. Cartrette D. P. and Mayo P. M., (2011), Students’ understanding of acids/bases in organic chemistry contexts. Chem. Educ. Res. Pract., 12(1), 29–39.
14. Caspari I., Kranz D., and Graulich N., (2018), Resolving the complexity of organic chemistry students’ reasoning through the lens of a mechanistic framework. Chem. Educ. Res. Pract., 19(4), 1117–1141.
15. Cohen L., Manion L. and Morrison K., (2011), Sampling, Research Methods in Education, Routledge, pp. 143–164.
16. Cooper M. M., Grove N. P., Pargas R., Bryfczynski S. P. and Gatlin T., (2009), OrganicPad: an interactive freehand drawing application for drawing Lewis structures and the development of skills in organic chemistry, Chem. Educ. Res. Pract., 10(4), 296–301.
17. Cooper M. M., Grove N., Underwood S. M. and Klymkowsky M. W., (2010), Lost in Lewis structures: an investigation of student difficulties in developing representational competence, J. Chem. Educ., 87(8), 869–874.
18. Cooper M. M., Kouyoumdjian H. and Underwood S. M., (2016), Investigating students’ reasoning about acid-base reactions, J. Chem. Educ., 93(10), 1703–1712.
19. Corbin J. and Strauss A., (1990), Grounded theory research: procedures, canons, and evaluative criteria, Qual. Sociol., 13(1), 3–21.
20. Cowie J. M. G. and Arrighi V., (2007), Polymers: Chemistry and Physics of Modern Materials, 3rd edn, Boca Raton, FL: CRC Press.
21. Domin D. S., Al-Masum M. and Mensah J., (2008), Students’ categorizations of organic compounds, Chem. Educ. Res. Pract., 9(2), 114–121.
22. Dood A. J., Fields K. B. and Raker J. R., (2018), Using lexical analysis to predict Lewis acid-base model use in responses to an acid-base proton-transfer reaction, J. Chem. Educ., 95(8), 1267–1275.
23. Duffy P. L., Enneking K. M., Gampp T. W., Amir Hakim K., Coleman A. F. and Laforest K. V., et al., (2019), Form versus function: a comparison of Lewis structure drawing tools and the extraneous cognitive load they induce, J. Chem. Educ., 96(2), 238–247.
24. Duis J. M., (2011), Organic chemistry educators’ perspectives on fundamental concepts and misconceptions: an exploratory study, J. Chem. Educ., 88(3), 346–350.
25. Ericsson K. A. and Simon H. A., (1980), Verbal reports as data, Psychol. Rev., 87(3), 215–251.
26. Ferguson R. and Bodner G. M., (2008), Making sense of the arrow-pushing formalism among chemistry majors enrolled in organic chemistry, Chem. Educ. Res. Pract., 9(2), 102–113.
27. Flynn A. B. and Amellal D. G., (2016), Chemical information literacy: pK_a values-where do students go wrong? J. Chem. Educ., 93(1), 39–45.
28. Flynn A. B. and Featherstone R. B., (2017), Language of mechanisms: exam analysis reveals students’ strengths, strategies, and errors when using the electron-pushing formalism (curved arrows) in new reactions, Chem. Educ. Res. Pract., 18(1), 64–77.
29. Flynn A. B. and Ogilvie W. W., (2015), Mechanisms before reactions: a mechanistic approach to the organic chemistry curriculum based on patterns of electron flow, J. Chem. Educ., 92(5), 803–810.
30. Galloway K. R., Stoyanovich C. and Flynn A. B., (2017), Students’ interpretations of mechanistic language in organic chemistry before learning reactions, Chem. Educ. Res. Pract., 18(2), 353–374.
31. Garnett Patrick J., Garnett Pamela J. and Hackling M. W., (1995), Students’ alternative conceptions in chemistry: a review of research and implications for teaching and learning, Stud. Sci. Educ., 25(1), 69–96.
32. Graulich N. and Bhattacharyya G., (2017), Investigating students’ similarity judgments in organic chemistry, Chem. Educ. Res. Pract., 18(4), 774–784.
33. Graulich N., Hedtrich S. and Harzenetter R., (2019), Explicit versus implicit similarity – exploring relational conceptual understanding in organic chemistry, Chem. Educ. Res. Pract., 2015, 924–936.
34. Grove N. P., Cooper M. M. and Cox E. L., (2012), Does mechanistic thinking improve student success in organic chemistry? J. Chem. Educ., 89(7), 850–853.
35. Grove N. P., Cooper M. M. and Rush K. M., (2012), Decorating with arrows: toward the development of representational competence in organic chemistry, J. Chem. Educ., 89(7), 844–849.
36. Herrington D. G. and Daubenmire P. L., (2014), Using interviews in CER projects: options, considerations, and limitations, in Bunce D. and Cole R. (ed.), Tools of Chemistry Education Research, Washington, DC: American Chemical Society, pp. 31–59.
37. Kim T., Wright L. K. and Miller K., (2019), An examination of students’ perceptions of the Kekulé resonance representation using a perceptual learning theory lens, Chem. Educ. Res. Pract., 20(4), 659–666.
38. Kirilenko A. P. and Stepchenkova S., (2016), Inter-coder agreement in one-to-many classification: fuzzy kappa, PLoS One, 11(3), e0149787.
39. Larson B., (2012), Sunset Lake Software, Molecules, [Mobile app].
40. Lesh R. A., Hoover M., Hole B., Kelly A. and Post T., (2000), Principles for developing thought-revealing activities for students and teachers, in Kelly A. and Lesh R. A. (ed.), Handbook of Research Design in Mathematics and Science Education, Routledge, pp. 591–645.
41. Libman D. and Huang L., (2013), Chemistry on the go: review of chemistry apps on smartphones, J. Chem. Educ., 90(3), 320–325.
42. McClary L. and Talanquer V., (2011), College chemistry students’ mental models of acids and acid strength, J. Res. Sci. Teach., 48(4), 396–413.
43. McCollum B. M., Regier L., Leong J., Simpson S. and Sterner S., (2014), The effects of using touch-screen devices on students’ molecular visualization and representational competence skills, J. Chem. Educ., 91(11), 1810–1817.
44. McHugh M. L., (2012), Interrater reliability: the kappa statistic, Biochem. Medica, 22(3), 276–282.
45. Mechanisms, (2018), Alchemie Solutions, Inc., [Mobile app].
46. Schmidt-McCormack J. A., Judge J. A., Spahr K., Yang E., Pugh R. and Karlin A., et al., (2019), Analysis of the role of a writing-to-learn assignment in student understanding of organic acid-base concepts, Chem. Educ. Res. Pract., 20(2), 383–398.
47. Shaffer A. A., (2006), Let us give Lewis acid-base theory the priority it deserves, J. Chem. Educ., 83(12), 1746–1750.
48. Stoyanovich C., Gandhi A. and Flynn A. B., (2015), Acid-base learning outcomes for students in an introductory organic chemistry course, J. Chem. Educ., 92(2), 220–229.
49. Taber K. S., (2002), Compounding quanta: probing the Frontiers of student understanding of molecular orbitals, Chem. Educ. Res. Pract., 3(2), 159–173.
50. Webber D. M. and Flynn A. B., (2018), How are students solving familiar and unfamiliar organic chemistry mechanism questions in a new curriculum? J. Chem. Educ., 95(9), 1451–1467.
51. Winter J. E., Wegwerth S. E., DeKorver B. K., Morsch L. A., DeSutter D., Goldman L. M. and Reutenauer L. M., (2019), The mechanisms app and platform: a new game-based product for learning curved arrow notation. in Houseknecht J. B., Leontyev A., Maloney V. M. and Welder C. O. (ed.), Active Learning in Organic Chemistry: Implementation and Analysis, American Chemical Society, pp. 99–115.

---

Footnote

† M. N. P. and F. M. W. contributed equally to this work.

---