How do students work through organic synthesis learning activities?

Abstract

Organic chemistry has the long-standing reputation as a challenging course, and organic synthesis is an aspect of organic chemistry that requires students to make the most links between concepts and requires the highest order of thinking. One-on-one interviews were conducted with students from a second undergraduate organic chemistry course in which participants solved synthesis problems using a think aloud protocol. Those problems had been previously designed to scaffold students' acquisition of synthesis problem-solving skills. The research question for this study asked whether students worked through the synthesis learning activities as designed, toward the intended learning outcomes. The results show that in some questions, students used or tried to use desirable problem solving skills, such as using reaction mechanisms and chemical principles to explore possible solutions. However, with other question types, students (i) relied on familiarity with the reactions in question and lacked a problem-solving strategy when they could not recall the answer or (ii) avoided the purpose of the question and attempted to provide an answer that the professor “wanted.” Strategies for promoting desired synthesis skills and addressing other issues are discussed.

---

Introduction

Organic chemistry has a long-standing reputation as a very difficult subject (Grove and Bretz, 2012). Possibly the most difficult and involved aspect of organic chemistry is designing a synthesis and retrosynthesis. To solve a synthesis problem, a student must, on paper, design a series of reactions in which they combine molecules to give a specific, more complex molecule. Students must choose from the dozens of reactions they have learned in current and previous courses, ensuring their strategy accounts for regiochemistry, stereochemistry, and that it maximizes chemoselectivity, safety, and yield, while minimizing cost and waste. Thus, they integrate several aspects of knowledge (Biggs and Collis, 1982; Biggs and Tang, 2007), propose one or more solutions to a new problem, and evaluate the solution(s). Some of the skills required to successfully plan a synthesis can be classed at the lower end of the revised Bloom taxonomy (i.e., at the “remember” and “understand” levels) while other skills require higher order thinking skills (e.g., at the “evaluate” and “synthesize” levels) (Krathwohl, 2002).

There is a gap in the teaching of organic synthesis in that students are taught basic reactions and then they are typically expected to propose a full synthesis without having been taught to integrate their knowledge and skills. Previously, synthesis learning activities were designed to scaffold students' acquisition of synthesis skills (Flynn, 2011). The development of the learning activities was based on the author's experience and the existing synthesis education and problem-solving research. The present study asks: How do students solve existing organic synthesis learning activities? Do they do so as intended on their way to achieving the intended learning outcomes (ILOs) and overcoming the barriers to learning? The little research that exists in organic synthesis education is summarized below, followed by a description of the synthesis learning activities that were the focus of the study, then the present study's theoretical framework, methodology, and results.

To begin the discussion on problem-solving, a problem is differentiated from an exercise as follows: “the difference between an exercise and a problem is the result of differences in the level of familiarity with similar tasks the individual brings to a given task” (Bodner and Domin, 2000). Wheatley's problem-solving model (Bodner, 2009) stressed the non-linear nature of the problem-solving process—and it's a messy model!

A key phase in the problem solving process is the very beginning. In those early stages, the relevant information is disembedded from the question and the problem is restructured (Larkin et al., 1980; Bodner and McMillen, 1986). Experts rely on underlying principles and simplify the task at hand (Larkin et al., 1980). Successful chemistry problem-solvers draw preliminary structures to explore solutions (Bodner and Domin, 2000).

A number of commonalities exist between successful problem solvers. Successful problem solvers: switch from one representation to another (Bowen, 1990; Kozma, 2003), move away from verbal/linguistic representations (Bodner and Domin, 2000; Bhattacharyya, 2008), interpret symbols as having physical representations (not just meaningless letters or numbers on a page as do unsuccessful problem solvers), construct representations that can contain elements of more than one representational system, and use symbolic representations most commonly (rather than writing sentences or phrases) (Bodner and Domin, 2000). Heiser and Tversky (2006) found that experts have expressed mental models that emphasized function rather than form, which is more commonly expressed by novices. Similarly, Kraft et al. (2010) found that the most successful problem-solvers tended to use models-based reasoning.

Kraft et al. (2010) also found that successful problem-solvers broke down the problem into steps (rather than making ‘leaps’ like unsuccessful problem-solvers) and were more metacognitive, which is becoming a larger focus in chemistry (Urena and Cooper, 2011). Few participants did much ‘scratch work’ on paper, possibly because the representations would have little to no meaning to students, they didn't recognize the value of writing things down, or they thought it was important to be able to work things out in their heads.

Students' understanding of chemical principles and processes also impact their problem-solving abilities. Strickland et al. (2010) investigated organic chemistry graduate students' conceptions of the terms used to describe chemical reactivity (such as functional group, acid, base, nucleophile, and electrophile) and expressed mental models of the images used to depict organic reactions and mechanisms. Participants' conceptualizations contained very few mechanistic, or process-oriented, attributes. Participants' verbalizations revealed a surface-level understanding of the representations used in the diagrams, i.e., they were unable to ‘see’ beyond the representations. While this work did not focus on synthesis, the tasks students were asked to perform (e.g., define common terms and verbally describe a mechanism and overall transformation) are components of synthesis problem-solving, and so student difficulties in these tasks would inevitably lead to difficulties in more complex, synthesis problem-solving.

Reaction mechanisms, including the electron-pushing formalism, are integral to the experts' problem solving process in organic chemistry. Anderson (2009) reported that by undertaking useful problem solving activities, graduate students began to find mechanisms more useful than they did initially in their studies and that using mechanisms allowed them to troubleshoot unexpected problems. The electron-pushing formalism is a major focus in the organic chemistry course related to the present study and its relevance to synthesis is further discussed later in this manuscript.

The knowledge and skills needed for successful synthetic analysis were identified from the above research and the author's experience. These attributes were used to create the associated specific learning activities that are described below. Understanding students' approaches to these learning activities were the focus of this study.

In-class learning activities

Previously, this author created a number of new types of synthesis questions as in-class learning activities (Flynn, 2011) and modified others to large class settings (Sauers and Morrison, 2007; Straumanis and Ruder, 2009) to help scaffold student learning of organic synthesis (Fig. 1). Scaffolding, as defined by Wood et al. (1976), is a “… process that enables a […] novice to solve a problem, carry out a task or achieve a goal which would be beyond his unassisted efforts.” (p. 90). Scaffolds, or learning supports, are provided to the students to help them solve a problem and then are gradually removed, thus moving the learner from assisted to independent levels of learning. The learning activities are aimed at students' zone of proximal development, the “distance between the actual developmental level as determined by independent problem solving and the level of potential development [through guidance]” (Vygotsky, 1978).

Fig. 1 In-class synthesis questions.

Most of the question types used in this study were of the synthetic planning type; most have previously been reported (Flynn, 2011). They do not address project-level problems (such as deciding on a synthetic target) and few addressed day-to-day problems, being the ‘hurdles’ (such as the formation of side products) faced in laboratories while carrying out a synthesis (Raker and Towns, 2012b). The activities, to the extent possible, reflect the experiences of practicing organic chemists (Raker and Towns, 2012a).

In the first question type, students were asked to identify the site of a specific reaction, given a complex final product (Fig. 1A) (Flynn, 2011). A numeric question was initiated with the classroom response system (CRS). Students typed in the numeric response by listing the three numbers of the carbon atoms involved, in order and without punctuation or spaces (e.g., 567). The histogram of results was analyzed, which led to further discussion in which students compared their answers with those of their classmates. This question type encouraged students to find patterns within a product's structure, such as the 1,3 relationship in the aldol product. They practiced finding those patterns in bonds or functional groups that appear “upside down” compared to how students are used to seeing them (e.g., 123) and that have been derivatized after the initial aldol reaction (e.g., 171819).

In the second question type, students were asked to identify the reaction required to make an indicated bond in a complex product, given a list of possibilities (Fig. 1B) (Flynn, 2011). Multiple questions could be asked at once by asking students to identify the reaction required to make bonds a, b, and c and listing them in a single answer, or by opening multiple CRS questions simultaneously (preferred). Here, students had to consider all the reactions they learned in the current course and all previous organic chemistry courses. They were also encouraged to find multiple answers or strategies.

For the complex substrates used in question types A and B, the students were instructed to initially disregard the order of reactions and competing reactions. Follow-up questions were used to further class discussion, such as asking students to compare synthetic routes proposed by their peers or explaining why using one reagent combination might be superior to another.

In the third question type (Fig. 1C), developed by Sauers and Morrison (2007) and described by Straumanis and Ruder (2009), students were asked to identify the best starting material(s) needed to generate a specified product. Students had to first identify an appropriate reaction type to generate the product, count atoms to identify which starting material is most appropriate, and use the mechanism in combination with the functional groups in the starting materials and products to determine which pairings are appropriate. They entered their answers using the CRS.

In question types D & E (Fig. 1), students were asked to propose a full synthesis of a product; these questions are the culmination of the previous question types. In question type D, students were given tables of reactions and reagents for specific transformations. Question type E was more difficult, in that students were only given the starting material and a list of reagents (Fig. 1E). The students typed in the synthetic route on their device of choice (e.g., phone, tablet, computer) using the CRS (“Top Hat,” 2014). They could combine multiple reactions or reagents in each step and they typed a period (.) between each step. These full synthesis questions required students to consider the order of the reactions, the roles of the reagents, etc.

All the questions can be asked initially without showing the list of options, to encourage students to generate their own answer first. When first asking questions of this type, the list of options is shown immediately. Later in the course, only the question is shown initially, giving the list of options after students have had a few minutes (3–10 min) to work without cues, thus scaffolding student learning. These questions could be asked in a think-pair-share format (Allen and Tanner, 2002), in which students are first asked to work out the answer individually, then discuss, and finally submit their answer using the CRS; optionally, students could vote after each think-pair-share stage. Using the histogram of responses, class discussion and further questioning could be guided based on the students' answers. This type of follow-up question gave the instructor the opportunity to re-direct common questions and errors back to students for them to work through. For the many questions for which students proposed multiple synthetic routes, an in-class analysis of two to three of those options ensued. For example, students would discuss the options then vote on the best, or would comment on the strengths and flaws of various routes. Doing so in class was meant to demonstrate the validity of multiple synthetic routes. Synthesis questions on exams were marked with a similar philosophy, giving credit for the quality of the proposed route, not for the degree to which it matched the answer key.

The knowledge and skills students will ideally acquire are summarized in Table 1; they were identified by the author's experience and by the existing synthesis research described above. The associated Bloom levels (revised version) (Krathwohl, 2002) were assigned by the author based on her experiences with the specific learning activities; the Bloom level could vary depending on how a given activity is conducted in class. The present study was designed to understand whether students were working through those synthesis questions as desired (i.e., to gain specific skills targeted by the questions).

Table 1 Requisite synthesis skills and associated revised Bloom levels (Krathwohl, 2002)

Fig. 1	Learning activity	Skills targeted (Bloom level)
A	Identify the site of a specified reaction, given a complex molecule	• Use the reaction mechanism to predict the product pattern (apply)
		• State or draw the patterns of the reaction products (analyze)
		• Recognize different oxidation levels and functional group transformations (FGTs) (analyze)
		• Describe features of a reaction (e.g., steric, stereochemical, or electronic considerations) (analyze)
B	Identify the reaction required, given a complex molecule, an indicated bond, and a list of reaction choices (and draw the starting materials)	• Simplify the structure (understand)
		• Analyze bonds for possible reaction types (apply)
		• Recognize patterns in products (apply)
		• Associate a bond type with an appropriate reaction (apply)
C	Identify the starting materials (SMs), given the product of a reaction	• Count carbons (analyze)
		• Identify the site of bond formation (analyze)
		• Consider the mechanism to predict sites of bond formation (apply)
D	Propose a synthesis (type 1), given a table of reactions	• Decide on sets of reagents to accomplish the transformations (apply)
		• Decide the appropriate order of reactions (evaluate)
E	Propose a synthesis (type 2), given the SMs, products, and a list of reagent choices	• Interpret the function of each reagent in the list (understand)
		• Analyze synthetic disconnections to identify the best synthetic strategy (evaluate)
		• Draw appropriate reagents to effect the desired transformations (synthesize)

---

Theoretical framework and research question

The goal of this work was to help students become more successful at complex problem-solving, specifically in the context of organic synthesis. As one step toward this goal, our group has undertaken the evaluation of the synthesis question types (used for in-class learning activities) within Guskey's evaluation framework (Guskey, 2000; 2002). The Guskey evaluative model is similar to the Kirkpatrick model (Kirkpatrick, 1996) but also addresses organizational support and change (level 3).

Student satisfaction (level 1) was evaluated using surveys and interviews with students; the results revealed that students were of the opinion that the learning activities were useful, improved their confidence, and should be used again (Syoufi and Flynn, 2012). Their main criticism had been of minor technical difficulties. The second phase of the evaluation, reported here, investigated whether students worked through the learning activities in the intended fashion. The next phase of the evaluation—which is in progress and will be reported in due course—seeks to determine the impact of the new synthesis questions on students' learning outcomes.

In this phase of the evaluation, the following research question (RQ) was investigated, which evaluated the organic synthesis learning activities at Guskey's level 2: How do second semester organic chemistry students solve the synthesis learning activities described in Fig. 1? Do they do so as intended on their way to achieving the ILOs and overcoming the barriers to learning?

The theoretical framework used for these specific research questions is that of personal constructivism, that is, that the learner constructs his or her own knowledge and understanding. This form of constructivism is tied to the zone of proximal development (Vygotsky, 1978), in that learners link the new information to what they already know (Bodner and McMillen, 1986); in other words, “knowledge is constructed in the mind of the learner” (p. 873). This study was not designed to assess whether students were answering the questions correctly or incorrectly, because they could hypothetically do either with or without using the skills or processes that the learning activities were meant to scaffold. Rather, this study focussed on students' problem-solving process and skill acquisition.

Methodology

Students and course

This research was conducted at a large, research-intensive Canadian university with student participants who were enrolled in a second year level organic chemistry course, Organic Chemistry II, in 2013. The course was their second in organic chemistry (Organic Chemistry I is taught in the winter of students' first year). The class enrolment was 398 (∼75% Faculty of Science, ∼17% Faculty of Health Sciences, ∼8% other Faculties).

The course was taught in a flipped format (Flynn, 2014). Students in the course attended two classes (1.5 hours each) and one discussion group session (optional, also called a DGD, tutorial, or recitation) each week, for twelve weeks. Assessment in the course was comprised of pre-class tests (5%), assignments (10%), two midterms (10–20% each), participation in classroom responses system (CRS) questions (5%), and a final exam (40–60%). For components with a range, the weighting that gave the students the best final mark was used. Learning activities related to synthesis and retrosynthesis were conducted at the conclusion of each section, i.e., approx. once every 3 weeks. The course curriculum had a mechanistic organization rather than a functional group one, in which students learn the electron-pushing formalism before any actual reaction (Flynn and Ogilvie, Submitted and in revision). In the first year organic course, students learned carbonyl addition reactions (simple π bond electrophiles), reactions with π bond nucleophiles, and electrophilic aromatic substitutions (stabilized π bond nucleophiles), in addition to basic structure and bonding concepts. In the second year course, students learned E1, E1, S_N1, S_N2, spectroscopy, and more complex carbonyl reactions (e.g., aldol, Fischer esterification).

Data collection and analysis

Students from Organic Chemistry II in 2013 (71% class average) were invited to participate in one hour, one-on-one audio-recorded interviews in which they were asked to solve synthesis problems of the types done in classes using a think aloud protocol (Patton, 2002). Thirteen students participated—5 male and 8 female. Their average course grade was 82%.

All phases of the research were reviewed and approved by the University of Ottawa's Office of Research Ethics and Integrity. All research participants were provided with information detailing their rights as human subjects; informed consent was obtained from all of the participants and they could withdraw from the study at any point. Throughout this report, pseudonyms are used in place of the students' real names to protect their identities.

The interviews took place in February 2014, that is, approximately two months after the completion of the course. The participants were provided paper, a pencil and a pen to make any notes. They also had access to the course textbooks (Smith, 2011; Klein, 2012; Wade, 2013) and a summary of the reactions they learned in the course (i.e., a reference page). Participants were asked to elaborate, clarify, and provide more detail about their answers. The recordings were transcribed verbatim and the participants' notes were incorporated into the transcripts. Next, the transcripts were coded using NVivo (QSR International, 2013) and analyzed using a constant comparison method (Patton, 2002; Kolb, 2012). First, the data were openly coded and comparisons were made within all the answers for each question type and within one participant's answers, moving back and forth between the data, the codes, the research question, and the intended learning outcomes. Next, axial coding involved making comparisons and connections between the codes and finally, selective coding involved identifying the core categories that related back to the research question and therefore to the intended learning outcomes. Member checking was done to increase the validity of this study (Creswell, 2011) by asking some of the participants to read the sections that referred to them (e.g., in which they were quoted) and having them comment on the accuracy and completeness of the analysis.

Limitations

A fairly small number of students participated in the interviews, and they had higher grades, on average, than the class average. Therefore, a potential limitation of this study was that the results might not have adequately captured the problem solving strategies used by the lower-achieving students. The interviews were conducted two months after the completion of the course, which may have affected their responses. A few of the problems in the first version of the interview guide were not new to the students (i.e., identifying the site of an aldol reaction in discodermolide and identifying the starting materials of an aldol reaction); because of this, the problems might not have been “authentic” ones (Bodner and Domin, 2000) for the student. Because the participants were interviewed by their former professor and they knew the overall goal of the study was to help improve synthesis learning activities, the results of the study could be biased (Rosenthal and Jacobson, 1968). For example, students may have solved problems differently during the interview than in the actual class or exam setting.

Results and discussion

Each type of synthesis question asked in class had its own intended learning outcome. That is, each question was designed to help students acquire and demonstrate specific skills or knowledge. As a first step, we wanted to know if students were working through the problems as designed. For example, a student might use heuristics to obtain the correct answer without using the process skills that each question was designed or “intended” to target. Using heuristics—i.e., simple reasoning processes to reduce the effort in a task (Shah and Oppenheimer, 2008) or rules of thumb—can help students simplify a task and solve a problem, but have been shown to often lead students astray (Taber, 2009; Maeyer and Talanquer, 2010; McClary and Talanquer, 2011). If students did not use the intended strategies and skills, they would not be able to achieve the intended learning outcome, even though they might be able to answer a given question. If they were not working through the problems as intended, students' learning and the results of measurement of their learning could be strongly affected.

Table 1 describes the skills each question type is intended to develop. For most of the synthesis questions, multiple answers were possible and it was the problem-solving strategy and not the answer itself that was emphasized in class and in this study.

Question type: identify the site of a given reaction (Fig. 1A)

The first question type is shown in Fig. 1A. Participants were asked to identify the product of 1–2 of the following reaction types: aldol, Baeyer–Villiger, Friedel–Crafts acylation, epoxide opening, or S_N2; discodermolide and taxol were the products provided. The ways in which students who were interviewed worked through question type A—identify the site of a specified reaction, given a complex molecule (Fig. 1A)—were analyzed.

Students worked through the problems as intended (targeted skills are summarized in Table 1). Some struggled with the question, had to look up answers or be guided, but all the skills identified in Table 1 eventually emerged or were clearly developing.

The five students who answered this question type looking for an aldol product seemed quite familiar with the 1,3-oxygenation pattern indicative of an aldol reaction. It seemed that they had memorized and could easily recall that pattern, likely because it was emphasized in their organic course and was one of the last reactions learned. All the students were able to identify the aldol pattern, with two of them being able to do so immediately, the others did so after looking up the pattern. For example, Louis described how he recognized the aldol product:

I was just looking for 3 carbons and then oxygens, because I remember that's what aldol does. And I remembered you can take off one of the OHs [points to 9], by, uh, I think it's call condensation. Aldol condensation.

Three of the students (Georgia, Avery, and Abigail) drew the aldol pattern of the product (Fig. 2). They tended not to use R groups (only one student did so in a single structure) or draw out the mechanisms; all used highly simplified structures to represent the aldol and condensation patterns.

Fig. 2 Students' notes to themselves when identifying the site of an aldol reaction in discodermolide.

This tendency to not draw out the mechanism or even structures recurred frequently in other variants of this question type, except in the case of the epoxide opening variant. In that variant, students turned to the mechanism of the reaction. Three of the five students who answered this question drew partial mechanisms as part of their rough work (Fig. 3); the other two discussed the mechanism, pointing to bonds in the structure as they did so. It seemed that students relied on the reaction mechanism to solve problems involving less familiar reactions (the epoxide opening had been less-studied in a synthetic context in the course). This outcome seems to reflect the conclusions drawn by Heiser and Tversky (2006)—that experts expressed mental models that emphasized function rather than form. While the students who used mechanisms to solve problems in this study could not yet be considered experts, they were certainly demonstrating behaviours consistent with successful problem-solving strategies used by experts.

Fig. 3 Three of the five students' rough work when answering the “identify the site of an epoxide opening reaction in taxol” included partial mechanisms and structures.

The complexity of the compounds was considered as a factor that could affect how students worked through this type of problem. All students except one—Avery—identified many possible sites of the aldol reaction, including those in which further functional group transformations (FGTs) would have occurred; they worked systematically from one side of the molecule to the other. Avery, on the other hand, pointed out two possible sites of an aldol reaction then seemed to give up, saying: “…that's really bulky and awful to look at.” Other students also reported thinking the size of the molecules was daunting. Abigail commented:

Abigail: I always found them like, just such a size, I was like, oh my God, I don't know what to look for. Like I'd always be super overwhelmed by the size of it.

Interviewer: Ok

Abigail: And then like, you know, I'd kind of get over it, but like initially, like, that was super scary for me.

The idea that students can get used to the size and complexity of the molecule and look for patterns is one that recurred throughout the interviews. Avery did not identify many sites of reaction in the aldol question, but later was able to with a different reaction (Friedel–Crafts acylation) in the same question type. He seemed to get used to working with complex molecules in the short time frame of the interview. The importance of working with molecules with some degree of complexity returns later (see below) when questions related to competing reactions are discussed.

Even when asked to identify many possible sites of reaction (i.e., when the question was extended even farther), students still worked through the question systematically. Some students did not initially recognize that an aldol product could be further derivatized (e.g., the alcohol in the product could be oxidized, alkylated, etc.), but once this was pointed out to them, they began to identify additional reaction sites.

For the same question type but looking for the product of a Baeyer–Villiger reaction, one student (Ryan) who began to identify the possible products immediately did not consult any other aids and did not draw the mechanism or other structures. His familiarity with the reaction and its products seemed to make the use of additional resources of processes unnecessary. In contrast, Madelaine was able to identify many potential products of the Baeyer–Villiger reaction once she had looked up the reaction and drawn a partial mechanism. In doing so, she also discussed some of the key features of the reaction mechanism. For example:

And, this one [indicated the benzoate], uh, mm, I feel like the oxygen would of been on this side [the side with the benzene ring] cause the benzene ring is more able, or more capable of stabilizing the positive charge.

When asked about the class atmosphere, a few different ideas recurred. For some, it was “more competition to see who could find more [sites of reaction].” Others described how this question type helped them see patterns they wouldn't otherwise have recognized. When asked to describe her opinion on this question type, Madelaine said:

Madelaine: I, I like these kinds of questions because it, it like, I like that you did stuff like this in class because it gave me the opportunity to see kinda how other people were thinking too.

Interviewer: Mm. Ok.

Madelaine: And realize like, other points of views but like it just made me, I don't know, it's kinda hard to explain. Um. It, it kinda widened my perspective on, yeah.

Interviewer: Ok.

Madelaine: So I, I like doing these kinds of questions in class cause say for example like I would of missed one, like that, that would really stand out to me like, oh ok I missed this one but like everybody else got it so like what am I not doing that their doing and it helped me along…

Two concerns that came up with this question type. First, students were concerned with the parameters or “rules” of the question. For example, were students allowed to imagine that a functional group transformation (FGT) had taken place after the reaction of interest? Louis said: “I don't know if I can do a reaction after the reaction…” They also wondered whether they were allowed to use any reaction conditions. In the epoxide question, for example, Madelaine wondered if only one of acidic or basic conditions could be used, and was greatly reassured when either was “allowed”:

Madelaine: Does like, does it matter, like what we're using for, like for the ring opening, like it's a strong base or a… like a… reaction… conditions?

Interviewer: It can be under any conditions.

Madelaine: Ok. [looked at the reference sheet]

Interviewer: Yeah.

Madelaine: So that one here. [pointed to the oxetane. She continued answering the question giving increasingly advanced answers]

Second, many students reported that when answering this type of question on a midterm or final exam, they did not know exactly how many points were allocated to each identified answer (e.g., was it 1 point for each) and so they felt there was ambiguity knowing when to stop answering. Madelaine stated: “I'm fairly hesitant when it comes to this kind of stuff cause I'm always paranoid that I'm like missing something.” This is in contrast to most other question types, such as “draw the product,” in which students can be more certain when they have completed answering the question.

Eight of the participants answered this type of question. Once it became evident that every student was working through this question type as designed, i.e., that saturation had been reached (Patton, 2002), this question was removed from further interviews. The research question for this type of learning activity had been satisfactorily answered and students demonstrated the skills they were developing.

Summary of this section

• Students worked through the “identify the site of a given reaction in a complex product” question as intended.

• Students drew all or part of the reaction mechanism when the reaction was less familiar.

• The complex molecules were daunting to some students at first, but they overcame that and all students eventually (within the timeframe of the interview) worked systematically through the questions.

• Students reported working collaboratively in class to find more answers; others competed to find the most answers. Both methods reinforced the goals of the question.

• Clarifying the expectations on a final exam would alleviate the ambiguity surrounding the question. For example, students should be told whether FGTs are possible and the number of sites of a given reaction to identify.

Question type: identify the reaction required to make the indicated bond (Fig. 1B)

In this question, students were presented with a complex target molecule and were asked how they would make a specific bond (e.g., Fig. 1B). Students were able to identify the reaction required to make the indicated bond only if it was already familiar to them. If they did not recognize the required reaction, they did not have the skills to figure it out or to look it up, either in the reference sheet or in the textbook. For the students who identified the required reaction, they had difficulty in drawing the required starting materials, that is, in translating the reaction name into actual starting materials. Hannah gave one example of the search for familiar reactions when she said: “I look at what's being bonded to what usually and then I go through like the possibilities of like how you can get there, I guess.”

When Hannah could not remember the reaction required for the question or she did not recognize the pattern (e.g., due to the molecule's complexity), she did not have a strategy to successfully solve the question. When other students, such as Julie (who drew the starting material fragments, unlike many others), could not recognize the reaction required to make a bond, tried to look it up, but none managed to look up a pattern when they did not know the reaction name they were looking for (e.g., none looked for a bond type or functional group as a starting point). That was Julie's experience when trying to propose the reagents that made bond 2:

Julie: Like… [drew the fragments shown in Fig. 4 , ∼20 s pause] An oxidation just makes… [mumbling, trailed off, ∼15 s pause]… I want to say there's somehow, like a… substitution… onto that one [pointed to B, Fig. 4 ] to make it able to take on the oxygen, er, to make – switch from an alcohol to make the bond? [looked briefly at the reaction reference sheet in the derivatives of the CO ₂ H section, then turned away].

Fig. 4 Julie disconnected bond 2, but did not draw appropriate SMs and could not find a reference reaction (blue arrow and circled “B” were added after the interview by AF).

For two students, the mechanism was an important step in answering the question. Louis drew out the mechanism, then talked it through, then corrected himself in the steps of the mechanism, all even though he had already correctly answered the question. The issue of ambiguity again came up with Louis, who was concerned about how the question would be marked (e.g., whether it was acceptable to ignore possible side reactions).

Ryan's strategy to a solution stood out in particular. He did not resort to naming reaction types, but rather, drew out and analyzed various mechanistic possibilities (Fig. 5). His answer was a logical one that was based on chemical principles—closely resembling expert-like thinking. It was Ryan—interviewed 7th—who provided the inspiration to use a synthon approach with later participants (see below).

Fig. 5 Ryan drew out partial mechanisms while solving the “identify the reaction required to make the bond” question.

Julie struggled with this question type from the start and did not recognize the reaction type that could have made that bond (i.e., it was not familiar to her). Even though she drew the fragments from which bond 2 was formed, those fragments could not have combined in a reaction to make the desired bond 2 (Fig. 4). Furthermore, she tried unsuccessfully to look up a suitable reaction type in the reference sheet. At this point, the interviewer described the synthon approach (Corey, 1967; Corey and Cheng, 1989) to her (Fig. 6), and she logically and successfully solved both parts of the question using that approach, drawing starting materials for bonds 2 and 6 shown in the diterpene structure in Fig. 5.

Fig. 6 Written portion of the explanation of the synthon approach described to Julie (for bond #2 of the question).

Julie immediately identified pair “A” (Fig. 6), in which the oxygen has a negative charge, as being most plausible. When asked to draw the actual reagents, Julie considered two possibilities, but settled on the one that avoided a possible competing reaction:

Julie: Okay. [∼70 s pause] So the C-H-2 would have to be attached to something that would have – make the… this one be delta negative I guess?

Interviewer: Yes, exactly.

Julie: Could it be attached to like an alcohol? [5 s pause] But then the H could react with that…

Interviewer: Mhm. [∼15 s pause] What other kind of negative, delta negative groups… have we seen?

Julie: Mm… [∼10 s pause, looked at cheat sheet] Chlorine… delta negative groups… [mumbling, looked at the reference sheet, ∼15 s pause] Cause if you, mm… [∼15 s pause] Cause if you make the chlorine leave… [drew chloromethane].

Interviewer: Mhm.

Julie: It takes the electrons, so it'd make it uh… positive… and then the C-H-2 could react with the oxygen.

Not only did Julie approach the retrosynthetic analysis while guided through the synthon approach for bond 2 of that question, she also figured out most of a solution for bond 6 without any prompting. She immediately drew a plausible pair of synthons then talked through and dismissed the other possible synthon pair. She readily drew the reagent electrophile although she struggled to draw the reagent required to generate the nucleophile. This difficulty was understandable because metal counterions or use of conjugate acids had not yet been discussed within the context of synthons.

With the next student, Aaron, the synthon approach was explained using an unrelated example before he saw the actual question. He then used the synthon approach to identify the possible synthons, drawing the pair he thought was most probable and explaining why he thought the other pair was less so (Fig. 7). From the synthons, he quickly drew the actual reagents. He repeated the approach for the second bond. He approached the question in a logical, systematic way, and clearly used chemical principles and reaction mechanisms rather than familiarity with reaction types:

Fig. 7 Aaron's drawings when answering the “Draw the SMs” question using the synthon approach (note: he did not erase the + and − signs after adding the Na, Br, and Cl atoms).

Aaron: C-H-3 plus [CH ₃ ⁺ ] sitting by itself wouldn't happen. So you would have to have something attached to it that was more negative or, uh, delta negative or something [drew bromomethane].

Notably, Aaron had written nothing to this point in the interview but quickly and comfortably drew synthons and their related mechanisms, then drew only a partial structure in a later question. Although there were errors in his final reagents because he didn't adjust the formal charges to account for the added bonds/atoms, he took a big step toward solving a difficult question using a chemical strategy and not simply relying on familiarity.

Marta showed an even higher level of mechanistic competence with the clear drawing of the synthons, verbal descriptions of her choices of synthon pairs, and how she determined what reagents to use (e.g., methyl lithium in one situation and the carboxylic acid in the other) as synthon pairs in each case (example in Fig. 8). Marta went further in her analysis to determine (using acid/base principles) which base would be appropriate to deprotonate the carboxylic acid and generate the requisite nucleophile. Like Aaron, she also started writing for the first time during this question. There seems to be something about the synthon approach that made students think/believe they did not have to solve the question in their heads.

Fig. 8 Marta used the synthon approach to determine the starting materials for each of the bonds indicated above.

It was exciting to see that when the synthon approach was explained to the four students, they solved the problem using a mechanistic thought process and relying on chemical principles. They did so even though the synthon approach had not previously been taught in the course. All the students who worked through the synthon approach started writing structures and mechanisms without prompting, a habit and skill encouraged for all students in the course. They worked through those question types systematically, even though they involved the complex type of structure that students report finding daunting. Even Marta started her problem solving—before she seamlessly solved the question to the question—with: “Uh oh… it looks so big… Even though, when I think about it, I'm just dealing with individual bonds, so it doesn't really matter how big it is.” Because of the success with this approach in helping students design a retrosynthetic analysis, the synthon approach will unquestionably form a key synthesis questions in the future.

Summary of this section

• If students did not already recognize the reaction type, they did not know how to solve the problem.

• Students struggled to look up reaction types; they seemed to lack that basic research skill.

• Students who were able to name the requisite reaction to form a specified bond still often struggled drawing the actual starting materials.

• Introducing a synthon approach (with four students) based on mechanistic considerations and chemical principles provided a strategy for students to systematically and successfully work through the problems.

Question type: identify the starting material(s) (Fig. 1C)

Students were presented with a simpler target molecule and a selection of possible starting materials; they were asked to select the appropriate starting material(s) to generate the desired target (Fig. 1). The two students who were given the first version of this question easily eliminated half the options (the alcohols) based on familiarity with the required starting materials. They eliminated the other incorrect options because they looked wrong (i.e., by roughly estimating the number of atoms). For example, Louis described how he arrived at a solution (Fig. 9):

Fig. 9 In this first version of the “identify the starting materials” question, students easily eliminated most of the options based on familiarity with reaction type (correct answer: 1 and 6).

Oh, yeah. Well the thing is I counted the carbons after. When I was comparing these two [1 & 6, the correct answer], that's only when I started counting them. With those other ones [the alcohols], I just started eliminating them. So it could only be one reaction because those couldn't work that start with OH. Well, they could, but not in this case [i.e., when you don't have the option to oxidize first]. If I'm doing an aldol then you can't start with those [alcohols] so I just eliminated those guys. And I knew that it was either one of these two combinations [1 & 6 or 3 & 8] because the number of carbons… you don't even really have to count them.

As with any multiple choice type question, the distractors should be plausible (DiBattista, 2008a, 2008b). Version 2 of the question was created (Fig. 10) in which the target and possible answers were changed slightly, so that the molecule seemed to be symmetrical. That apparent symmetry led many students to use #3 twice (once as the nucleophile in the enolate form, the other equivalent as the electrophile). Ryan, however, drew out the mechanism of the reaction and counted the carbons. He worked through the question systematically, without relying on familiarity.

Fig. 10 Version two of the “identify the starting materials” question (correct answer: 1 and 4).

Ryan initially considered using molecule #3 twice (Fig. 10), but eventually settled on 1 and 4 (the correct answers). He explained his reasoning as follows:

Umm… I think it's because I thought that… uh… I could… make an, uh, make an enolate with that [#3]… uh, and then… use that to attack uh, you know, the, the same, same one… But, I dunno just kinda going through it in my head it didn't really make sense because then… cause then this [the α-carbon of 3 that he drew] would be attached to the carbon [indicated the mechanism that he drew]… yeah the carbon wouldn't be the, uh… wouldn't match up.

Julie also drew out mechanisms (Fig. 11) and when trying a mechanism between reactants 2 and 4, had to ask herself: “Where did that methyl group [α carbon in 2] go?” She realized by drawing the mechanism that molecules 2 and 4 were not the starting materials. She proceeded to explore mechanisms for other options and count carbons more carefully to eventually arrive at the correct answer.

Fig. 11 Julie drew out the mechanism between 2 and 4 and in doing so realized that this reaction would not give the desired product (correct answer: 1 and 4).

The clear advantages to this new version of the question were that students could not quickly eliminate options on the basis of functional groups and they had to both consider the mechanism and count carbons. Although the question was presented as a retrosynthesis, students worked through it in the forward direction.

Yet another instance of a student not being able to look up a reaction and recognize a pattern was evident as Marta solved this problem. She found the aldol reaction in the reaction reference sheet, but did not believe that variation in the carbon structure was possible, as revealed in the following quote:

The aldol is looking similar to what this [question] is… But in the product that's drawn on the [reference] sheet there's this other methyl group that comes off where the hydroxyl is, and I'm not seeing that here, so… I'm trying to remember how…

These student weaknesses in looking up reactions, which will likely ultimately affect their research abilities and their ability to learn reactions beyond those explicitly taught in the course, suggest a new question type that should be developed (described below). Another issue is suggested by this result: students can probably not take a specific reaction and generalize it—it is not possible to look up a reaction when it is not known what to look for.

Although not a large sample, all the students who drew mechanisms and counted carbons were successful at solving version 2 of the problem. Only Marta did not draw mechanisms for the reagents in this question (she only did so on the reaction reference sheet), and she did not find a correct answer.

Summary of this section

• Students considered the mechanism of the reaction and counted carbons—both desirable skills for students to gain—when the question was well designed with plausible distractors.

• Students struggled to look up the reaction (either in the reference sheet or in the textbook) when it was not familiar to them, suggesting a deficit in research skills.

Question type: propose a synthesis (Fig. 1D and E)

The simplest “propose a synthesis” questions such as the one involving electrophilic aromatic substitution (see Fig. 1D) were quite straightforward and asked students to think especially through the order of their reactions (i.e., synthetic strategy). They needed to think less about how to achieve each transformation, because the sets of reagents were provided to them and were fairly standard, particularly for electrophilic aromatic substitution reactions and derivatizations.

Although aromatic chemistry was covered in the first year, this topic seems to have “stuck” with students. They easily recalled the existence of directing groups, although often had to look up the specifics (which most students had difficulty doing). Once they reviewed the directing group principles, solving the problem became straightforward.

The second type of “propose a synthesis” question (see Fig. 1E) was more complex because students had to identify which reactions were required to generate various bond types of fragments of the molecule, decide on the appropriate reagent(s), and then choose appropriate reagents from the list of reagents provide. The list of reagents provided sometimes matched up well with a student's proposed synthesis, but often the student either had to make a minor modification of their synthesis (such as replacing the H₂SO₄ they proposed using with the HCl that was the strong acid available in the reagent list) or had to reconsider their entire synthesis because it was not achievable with the list of reagents provided.

The interviews revealed a key issue with this question type. Students who were able to propose their own synthesis often abandoned their own synthesis when the list of reagents did not fit their proposal. That was Avery's experience:

I'm kind of stuck in my ways, but the first thing I want to do is I want to find my route to fit these new reagents… Cause that's generally the easiest way to do it instead of coming up with a new one but mine was totally different from what you… from any option here so I'm just re-doing it now… Cause this [not being able to accomplish his synthesis with the list of reagents provided] happened a lot. I have a very… um… odd or different way of looking at things. I have no idea what it is about it. So the combination or options it [the reagent list] would show was 90% of the time it was nowhere near what I had.

Similarly, Madelaine explained the mismatch between her answer and the one provided: “Especially synthesis questions, I tend to do things like the, the longer way, well like not so much like the longer way but um, I I [sic] find I rarely get like the answer that's in the textbook.”

During the interviews, seven of the nine students who answered this question type abandoned their own route once provided the list of reagents, even the four students whose routes were completely correct. Of the two students who did not abandon their proposed syntheses, David only needed a hint from the list to complete his own (i.e., his synthesis essentially fit the list) and Louis' synthesis could be accomplished using reagents from the list. When Louis was asked what he would do if his synthesis could not be accomplished using a list of reagents provided, he responded in a way that added evidence that the purpose of this question type was not being achieved: “Ah! I would assume that you wanted a different pathway [laughed] so I would just look for another pathway.”

None of the students who successfully proposed a different synthesis than the one in the list made a case for their synthesis being equally valid. This was a concern, because there are usually multiple routes to a given target molecule, and an important value of synthetic analysis (both for chemistry and in developing transferable skills) is analyzing and testing (in the laboratory) the viability of those different routes.

In addition, the four students who were unsuccessful at proposing their own synthesis, i.e., who got stuck trying to propose their own synthesis and/or could not use the list to generate a proposal, did not have an alternate strategy.

Summary of this section

• When the synthetic strategy (e.g., the order of the reactions needed for regiocontrol) was the focus because the sets of reagents were provided to the students, they worked through the questions as intended (question type in Fig. 1D).

• When students proposed their own synthesis first, then tried to match their synthesis to a list of reagents, they tended to completely abandon their own synthetic route if it did not fit the list. This defeated the purpose of synthetic analysis, which should embrace the possibility of different routes and seek to evaluate those alternatives.

• This question format seemed to “force” a specific answer rather than allowing for multiple routes to a product, which would enrich the learning experience.

Research skill: finding analogous reactions and appropriate references

A theme that emerged across all the question types was the gap in students' abilities to look up answers (e.g., reaction types and reagents) when necessary. This issue emerged initially with the questions intended as warm-up. In these questions, students were presented with a starting material and product and were asked to match the reaction to the one in the reaction reference sheet. This was the first time they were seeing this sheet, which summarized all the reactions from their first and second year organic courses.

If the reaction in question was familiar to the student, then they could usually match it up with a reaction in the sheet. However, if it was not familiar, they were not able to find it. They did not look up reactions on the basis of their functional group or governing mechanism or use the textbook's index efficiently.

For example, Louis and Aaron immediately recognized the required transformation (reduction and acetalization, respectively), then matched the reaction to the appropriate one in the reference sheet:

Louis: So the first one I see is just a reduction, right? Cause it just changes the carbonyl O to an OH… I'm just trying to remember what does that [tentatively started to look at sheet]… There, a Grignard… [found the reaction at the right spot in the sheet] I mean, not a Grignard, I mean NaBH ₄ [said the letters and number].

Aaron: “So it's making an acetal… Like a ringed acetal. [looked at sheet, found the acetal]… Or a… so you just have to add… So this is like a diol so you'll add a diol and then, um, you would have to… so you go through all the steps where you hydrolyze all the different things, like the five step [mechanism].

While David recognized the acetal functional group easily (as did the other students), he struggled to find it in the reference sheet:

David: And this one I see that we've um… we've added a protecting group—is that what you call it? And that is just… if I can remember properly [looks at sheet]… it's adding in basic conditions so you want a base and also um… I think water as well… [still looking at the sheet]

Interviewer: What made you recognize a protecting group?

David: Well, it's just, uh, I guess… the two Os and they're bonded by the uh ethane group… I guess and uh it's not really a protecting group unless you're going back and you reconvert it into a… Well I guess protecting group always makes me think of the two oxygens bonded in that cyclic way.

Interviewer: Ok

David: Alright, so, you need a base first… a strong base… NaOH and then you also need to you should be able to add the ethane in… Ok, perfect, ok… [struggling to look up reactions] ok, so 2 times … [found the correct reference reaction] ok, sorry, so we'd just use ethanediol [drew it incorrectly] but first I would need to add… um… [looking back and forth between the structure and the reference sheet]… um… water… you're getting water out… base or acid… ok so I would add an acid first because that would protonate that [drew the mechanism of carbonyl protonation] and then that would attack the C of the carbonyl… there.

Aaron also had difficulty finding the alcohol oxidation reaction; he only did so successfully once prompted, and still struggled:

Aaron: So we have to, uh, remove the hydrogen and then we can get a negative charge on that [pointed to the O] which we can bring the electrons down [did not write anything for this question].

Interviewer: Mhm.

Aaron: Um. So you could do that with like a stronger acid. Or yeah.

Interviewer: Ok. And can you find it? See if you can find the similar reaction in the sheet.

Aaron: uh… Ok. It'd be… like an oxidation. So I guess the big thing is you didn't want to go too far with it.

A weakness in students' abilities to conduct research (e.g., find an appropriate experimental procedure to conduct a specific reaction) was previously noted by this author to be weak in the fourth year students (Flynn and Biggs, 2011), but this study revealed an even more fundamental gap in students' skillsets.

Summary of this section

• Constructivism is based on ascertaining what the learner knows/can do and teaching accordingly (Ausubel, 1968). The assumption that students could already look up reactions was clearly incorrect and students need to be explicitly taught this skill.

• These results suggest another learning deficit: that students are not able to generalize a specific reaction, which likely lead to difficulties in applying their specific learning to new situations.

Conclusions

The present study gave insight into how students answered many different question types that are designed to help students learn the skills required for synthetic planning. Many themes that were observed were consistent with previous research, such as the importance of redrawing structures (Bodner and Domin, 2000), using the reaction mechanism (Anderson, 2009) and interpreting symbols meaningfully (Kozma, 2003).

As summarized in Table 2, students worked through some question types as intended (i.e., Fig. 1A, C and D), but did not do so with others (i.e., Fig. 1B and E). The implication for practice is that question types A, C, and D seem to be most effective for developing the targeted skills, while types B and E should be redesigned if they are to target the skills described above (Table 1). Students were not using or attempting to use the skills targeted in the latter sections. For example, in question type B, students did not attempt to simplify the structure, analyze bonds for a possible reaction type, or associate bond type with an appropriate reaction, unless the bond type immediately linked to a memory of the required reaction (i.e., if they recognized the required reaction).

Table 2 Summary of results

Question type	Learning activity	Key results
A	Identify the site of a specified reaction, given a complex molecule	• Students worked through the question as intended (e.g., looked for patterns).
		• Students drew all or part of the reaction mechanism when the reaction was less familiar.
		• The complex molecules were daunting to some students, but only at first.
		• The expectations for the questions must be clearly communicated.
B	Identify the reaction required, given a complex molecule, an indicated bond, and a list of reaction choices (and draw the starting materials)	• If students did not already recognize the reaction type, they did not know how to solve the problem (e.g., they did not simplify the structure or use the mechanism).
		• Students struggled to look up reaction types; that research skill was lacking.
		• Students who were able to name the requisite reaction to form a specified bond still often struggled to draw the starting materials.
		• The synthon approach (with four students)—based on mechanistic considerations and chemical principles—provided a strategy for students to systematically and successfully work through the problems.
C	Identify the starting materials, given the product of a reaction	• With well-designed questions (bearing plausible distractors), students analyzed the mechanism of the reaction and counted carbons—both desirable skills for students to gain.
		• Students struggled to look up the reaction when it was not familiar to them, again demonstrating the deficit in research skills.
D	Propose a synthesis (type 1), given a table of reactions	• Students worked through the questions as intended and focused on the synthetic strategy (e.g., regiocontrol).
E	Propose a synthesis (type 2), given the SMs, products, and a list of reagent choices	• When students proposed their own synthesis first, then tried to find appropriate reagents from the list provided, they tended to completely abandon their own synthetic route if it did not fit the list. This defeated the purpose of synthetic analysis, which should embrace the possibility of different routes and seek to evaluate those alternatives.
		• This question format seemed to “force” a specific answer rather than allowing for multiple routes to a product, which would enrich the learning experience.

---

While it seems the synthon approach will be a very powerful one for students, more research is needed to understand the strengths and implications of using that approach in the classroom. For synthesis questions—as with any type of question that has multiple possible answers—the existence and value of the multiple answers should be made explicit through the learning activities and intended learning outcomes. This is especially true for students who have been conditioned to believe that science is dichotomous (right/wrong) rather than the highly variable nature of science (Van Dijk, 2014).

For the problematic question types, students resorted to familiarity with reactions, they lacked a defined strategy when their rote memory failed, and they struggled to look up reactions. A major modification of type B—asking some students to adopt a synthon approach—had a notable benefit for those students. They began using mechanistic thinking and drawing out their ideas—components of expert-like thinking. Type E was quite problematic in that students abandoned their own synthetic route if they could not make it match the list of reagents provided. Students did not attempt to evaluate their synthetic designs—a key skill for students to acquire, and one that is applicable across disciplines. Students also relied on familiarity and did not have a problem-solving strategy to fall back on if they came upon an unfamiliar reaction. This tendency of students in organic chemistry to memorize a set of rules and reactions has been previously reported to hinder student learning (Anderson and Bodner, 2008; Grove and Bretz, 2012). Other question types will be designed to replace this question to help students learn to plan syntheses systematically and using mechanistic thinking, as well as evaluate synthetic routes.

When the synthon approach was explained to four of the students, all of them began to draw molecular structures and reaction mechanisms; all could compare the various synthon pairs to choose the most plausible one and were successful at solving the related synthesis problems.

The gap in research skills will be addressed in part with new learning activities that help students learn to identify general features of reactions and then to research them in a resource such as a textbook. This skill deficit (i.e., the ability to generalize a reaction or make connections between reactions) may be related to students' unawareness that such connections exist; they are not aware that another way of learning organic chemistry exists, i.e., meaningful learning (Grove and Bretz, 2012).

These synthesis questions, in principle, are closing the gap between students' learning of the basic organic chemistry concepts and learning the knowledge and skills to accomplish full retrosynthetic and synthetic analyses. Because only a small number of participants (13) were involved in the study, they were interviewed by their former professor and they knew the overall goal of the study was to help improve synthesis learning activities, the results of the study could be biased (Rosenthal and Jacobson, 1968). In addition, the mechanistic curriculum that we use might be inextricably linked to students' problem-solving process in some questions. For example, in the “draw the starting materials” question, students employed the synthon approach (which required keeping track of electrons) with ease once they were exposed to it.

The next phase in this research project will be to design and evaluate new question types that target the students' weaknesses identified in this study. The effect of these types of questions on student learning in the larger course setting and other educational settings (e.g., with teaching assistants in tutorial/recitation settings, at other institutions, other levels of instruction, etc.) will be evaluated, as will the synthon approach. The intent is to promote students' development of higher order thinking skills (Krathwohl, 2002), to encourage students to move their learning from rote to meaningful (Grove and Bretz, 2012), and in doing so, to achieve higher level learning outcomes (Biggs and Collis, 1982).

Acknowledgements

I thank Gautam Bhattacharyya for helpful discussions and gratefully acknowledge Nicholas Bode's and Delphine Amellal's contributions.

References

1. Allen D. and Tanner K., (2002), Approaches in cell biology teaching, Cell Biol Educ., 1(1), 3–5, DOI: http://10.1187/cbe.02-04-0430.
2. Anderson J. P., (2009), Learning the language of organic chemistry: How do students develop reaction mechanism problem-solving skills? Doctoral dissertation, Retrieved from ProQuest, UMI Dissertations Publishing (3379296).
3. Anderson T. L. and Bodner G. M., (2008), What can we do about ‘Parker’? A case study of a good student who didn't ‘get’ organic chemistry, Chem. Educ. Res. Pract., 9(2), 93–101, DOI: 10.1039/b806223b.
4. Ausubel D. P., (1968), Educational Psychology; A Cognitive View NY: Holt, Rinehart and Winston, New York, NY: Hold, Rinehart and Winston.
5. Bhattacharyya G., (2008), Who am I? What am I doing here? Professional identity and the epistemic development of organic chemists, Chem. Educ. Res. Pract., 9(2), 84–92, DOI: 10.1039/b806222f.
6. Biggs J. B. and Collis K., (1982), Evaluating the quality of learning: the SOLO taxonomy (Structure of the Observed Learning Outcome), New York, NY: Academic Press.
7. Biggs J. B. and Tang C. S.-K., (2007), Teaching for quality learning at university: what the student does, Maidenhead: Society for Research into Higher Education and Open University Press.
8. Bodner G. M., (2009), A View from Chemistry, in Smith M. U. (ed.), Toward a Unified Theory of Problem Solving Views from the Content Domains, New York, NY: Routledge, p. 21.
9. Bodner G. M. and Domin D. S., (2000), Mental Models: The Role of Representations in Problem Solving in Chemistry, Univ. Chem. Educ., 4(1), 24–30.
10. Bodner G. M. and McMillen T. L. B., (1986), Cognitive restructuring as an early stage in problem solving, J. Res. Sci. Teach., 23(8), 727–737, DOI: http://10.1002/tea.3660230807.
11. Bowen C. W., (1990), Representational systems used by graduate students while problem solving in organic synthesis, J. Res. Sci. Teach., 27(4), 351–370, DOI: http://10.1002/tea.3660270406.
12. Corey E. J., (1967), General methods for the construction of complex molecules, Pure Appl. Chem., 14(1), DOI: http://10.1351/pac196714010019.
13. Corey E. J. and Cheng, X.-M., (1989), The Logic of Chemical Synthesis. New York, NY: John Wiley & Sons, Inc.
14. Creswell J. W., (2011), Educational research: Planning, conducting, and evaluating quantitative and qualitative research, 4 edn, Boston, MA: Pearson Education.
15. DiBattista D., (2008a), Getting the Most Out of Multiple-choice Questions. St. Catharines, ON. Retrieved from http://www.uoguelph.ca/tss/pdfs/Handout_Guelph-MC%20Jan%2028-08.pdf.
16. DiBattista D., (2008b), Making the Most of Multiple-Choice Questions: Getting Beyond Remembering, Collected Essays on Learning and Teaching, 1, 119–122.
17. Flynn A. B., (2011), Developing problem-solving skills through retrosynthetic analysis and clickers in organic chemistry, J. Chem. Educ., 88(11), 1496–1500, DOI: http://10.1021/ed200143k.
18. Flynn A. B., (2014), Large courses, large flips. Presented at the Canadian Society for Chemistry Conference, Vancouver. Retrieved from http://abstracts.csc2014.ca/00001236.htm.
19. Flynn A. B. and Biggs R., (2011), The Development and Implementation of a Problem-Based Learning Format in a Fourth-Year Undergraduate Synthetic Organic and Medicinal Chemistry Laboratory Course, J. Chem. Educ., 89(1), 52–57, DOI: http://10.1021/ed101041n.
20. Flynn A. B. and Ogilvie W. W., (Submitted & in revision), Mechanisms before reactions: a mechanistic approach to the organic chemistry curriculum based on patterns of electron flow, J. Chem. Educ..
21. Grove N. P. and Bretz S. L., (2012), A Continuum of Learning: From Rote Memorization to Meaningful Learning in Organic Chemistry, Chem. Educ. Res. Pract., 13(3), 201–208.
22. Guskey T. R., (2000), Evaluating Professional Development, Thousand Oaks, CA: Corwin.
23. Guskey T. R., (2002), Does It Make a Difference? Evaluating Professional Development, Educ. Leadership, 59(6), 45–51.
24. Heiser J. and Tversky B., (2006), Arrows in Comprehending and Producing Mechanical Diagrams, Cognitive Sci., 30(3), 581–592, DOI: http://10.1207/s15516709cog0000_70.
25. Kirkpatrick D., (1996), Great ideas revisited, Training and Development, 50(1), 54–59.
26. Klein D., (2012), Organic Chemistry, 1st edn, Hoboken, NJ: John Wiley & Sons, Inc.
27. Kolb S. M., (2012), Grounded theory and the constant comparative method: valid research strategies for educators, Journal of Emerging Trends in Educational Research and Policy Studies, 3(1), 83–86.
28. Kozma R., (2003), The material features of multiple representations and their cognitive and social affordances for science understanding, Learn. Instr., 13(2), 205–226, DOI: http://10.1016/S0959-4752(02)00021-X.
29. Kraft A., Strickland A. M. and Bhattacharyya G., (2010), Reasonable reasoning: multi-variate problem-solving in organic chemistry, Chem. Educ. Res. Pract., 11(4), 281–292, DOI: 10.1039/c0rp90003f.
30. Krathwohl D. R., (2002), A revision of Bloom's taxonomy: an overview, Theory Into Practice, 41(4), 212–218, DOI: http://10.1207/s15430421tip4104_2.
31. Larkin, J., McDermott, J., Simon D. P. and Simon H. A., (1980), Expert and novice performance in solving physics problems, Science, 208, 1335–1342, DOI: http://10.1184/pmc/simon/box00013/fld00909/bdl0001/doc0001.
32. Maeyer J. and Talanquer V., (2010), The role of intuitive heuristics in students' thinking: ranking chemical substances, Sci. Educ., 94(6), 963–984, DOI: http://10.1002/sce.20397.
33. McClary L. and Talanquer V., (2011), Heuristic Reasoning in Chemistry: Making decisions about acid strength, Int. J. Sci.Educ., 33(10), 1433–1454, DOI: http://10.1080/09500693.2010.528463.
34. Patton M. Q., (2002), Qualitative Research & Evaluation Methods. Retrieved from http://www.sagepub.com/booksProdDesc.nav?prodId=Book9906.
35. QSR International, (2013). NVivo10.
36. Raker J. R. and Towns M. H., (2012a), Designing Undergraduate-Level Organic Chemistry Instructional Problems: Seven Ideas from a Problem-Solving Study of Practicing Synthetic Organic Chemists, Chem. Educ. Res. Pract., 13(3), 277–285, DOI: 10.1039/C1RP90073K.
37. Raker J. R. and Towns M. H., (2012b), Problem types in synthetic organic chemistry research: implications for the development of curricular problems for second-year level organic chemistry instruction, Chem. Educ. Res. Pract., 13(3), 179–185, DOI: 10.1039/C2RP90001G.
38. Rosenthal R. and Jacobson L., (1968), Pygmalion in the classroom: teachers' expectations and pupils' intellectual development, New York, NY: Holt, Rinehart and Winston.
39. Sauers A. L. and Morrison R. W., (2007), In-lecture guided inquiry for large organic chemistry classes (pp. 838–CHED). Presented at the Abstracts of Papers, 233rd National Meeting of the American Chemical Society, Chicago, IL: American Chemical Society.
40. Shah A. K. and Oppenheimer D. M., (2008), Heuristics made easy: an effort-reduction framework, Psychol. Bull., 134(2), 207–222, DOI: http://10.1037/0033-2909.134.2.207.
41. Smith J. G., (2011), Organic Chemistry, 4 edn, New York, NY: McGraw-Hill.
42. Straumanis A. R. and Ruder S. M., (2009), A Method for Writing Open-Ended Curved Arrow Notation Questions for Multiple-Choice Exams and Electronic-Response Systems, J. Chem. Educ., 86(12), 1392, DOI: http://10.1021/ed086p1392.
43. Strickland A. M., Kraft A. and Bhattacharyya G., (2010), What Happens when Representations Fail to Represent? Graduate Students' Mental Models of Organic Chemistry Diagrams, Chem. Educ. Res. Pract., 11(4), 293–301, DOI: 10.1039/C0RP90009E.
44. Syoufi M. and Flynn A. B., (2012), Learning synthesis: how useful has THM been? Presented at the Undergraduate Research Opportunities Program Symposium, Ottawa: University of Ottawa.
45. Taber K. S., (2009), College students' conceptions of chemical stability: the widespread adoption of a heuristic rule out of context and beyond its range of application, Int. J. Sci.Educ., 31(10), 1333–1358, DOI: http://10.1080/09500690801975594.
46. Top Hat, (2014), Top Hat. Tophat.com. Retrieved June 2014, from http://https://tophat.com/.
47. Urena S. S. and Cooper M. M., (2011), Enhancement of metacognition use and awareness by means of a collaborative intervention, Int. J. Sci.Educ., 33(3), 323–340, DOI: http://10.1080/09500690903452922.
48. Van Dijk E. M., (2014), Understanding the Heterogeneous Nature of Science: A Comprehensive Notion of PCK for Scientific Literacy, Sci. Educ., 98(3), 397–411, DOI: http://10.1002/sce.21110.
49. Vygotsky L. S., (1978), Mind in Society, Cambridge, MA: Harvard University Press, pp. 79–91, DOI: http://10.1525/aa.1979.81.4.02a00580.
50. Wade L. G. Jr., (2013), Organic Chemistry, 8 edn, Upper Saddle River, NJ: Pearson Prentice Hall.
51. Wood D., Bruner J. S. and Ross G., (1976), The role of tutoring in problem solving, J. Child Psychol. Psyc., 17, 89–100.

---