College chemistry students' use of memorized algorithms in chemical reactions

Abstract

This study sought to uncover memorized algorithms and procedures that students relied on in responding to questions based on the particulate nature of matter (PNM). We describe various memorized algorithms or processes used by students. In the study, students were asked to balance three equations of chemical reaction and then draw particulate representations of the compounds in the reactions. Students were then interviewed to uncover their understanding of underlying chemistry, taking note of any memorized algorithms that students were using. In addition to specific algorithms that students used, two trends were apparent from our analysis: (1) students successfully applied algorithms (in operations such as equation balancing) without necessarily understanding why they used the particular operations or processes. (2) Students have memorized processes and ideas which they incorrectly applied. Implications for assessment, research and instruction are also suggested.

---

I just memorize like the rules like whether it is a single replacement or a double replacement, a combustion reaction I look for oxygen, and in acid carbonate reactions, they give a salt, water and carbon dioxide. (Franklin, a participant in this study)

Introduction

A key goal of science education is to equip students with knowledge to enable them to live and function as informed citizens who can make decisions about important contemporary scientific issues, or as professionals in scientific fields (Price and McNeill, 2013). Specifically, chemistry education seeks to help students develop problem solving competence (Gabel and Bunce, 1994; Tsaparlis, 2005; Taasoobshirazi and Glyn, 2009), gain conceptual chemical understanding (Cros et al., 1987; Nakhleh, 1993) and equip students with science-process skills (Heeren, 1990) among other competences. However, instruction that enables students to be scientifically literate requires more than facilitating and encouraging memorization of facts (Gotwals and Songer, 2013). Meaningful learning in science requires conceptual understanding as opposed to rote memorization and application of algorithms to solve simple problems (Adadan et al., 2010). Research in chemistry education has unfortunately shown that students leave chemistry courses lacking in problem solving skills and adequate conceptual understanding of requisite chemistry content (Teichert and Stacy, 2002; Bodner, 2003).

In our past research (see Kern et al., 2010 and Nyachwaya et al., 2011), we uncovered gaps in students' conceptual understanding of the particulate nature of matter (PNM) related to chemical bonding. This research revealed a gap between students' understanding of chemistry at the symbolic level and their understanding at the particulate level (see Nyachwaya et al., 2011). While 99% of students in our study could correctly balance given equations of chemical reactions, only 30% drew appropriate particulate representations of the covalent compounds in these reactions. Ionic compounds were particularly problematic to represent at the particulate level, with less than 5% of students drawing appropriate particulate representations of the ionic compounds involved. The work presented in this paper is an extension of this previous study.

While our open-ended drawing tool elicits rich information about students' particulate knowledge of bonding, this paper and pencil assessment does not reveal student thinking in developing their particulate drawings. Thus, as a next step in our research, we conducted interviews to uncover students' conceptual understanding of underlying concepts represented in the different chemical equations. Specifically, we sought to understand students' thinking and reasoning processes during the equation balancing and drawing tasks. In this paper, we describe specific algorithms or memorized ideas and processes that students used related to the particulate nature of matter that were apparent from our interviews. We also describe the ways in which students used the algorithms or memorized processes.

Prior research has looked at the algorithmic learning in the context of problem solving, with a number of studies categorizing students into conceptual and algorithmic learners based on the types of problems they successfully solved (e.g.Sawrey, 1990; Nakhleh, 1993; Zoller et al., 1995; Zoller, 2000; Cartrette and Bodner, 2010). Our study was not conducted in the context of problem solving, nor did it involve quantitative problems. While confirming students' use of algorithms, this qualitative study documents algorithms students use, specifically related to PNM. We add to the existing knowledge in the field by reporting specific algorithms students used that relate to very fundamental aspects of PNM.

The overarching question guiding our work was:

What are college students' conceptual understandings of the particulate nature of matter (PNM)?

This specific paper was guided by the research question:

What specific algorithms and memorized processes do students use when responding to questions related to chemical reactions?

Understanding in chemistry

According to Nieswandt (2007), conceptual understanding in chemistry encompasses three ‘types of knowledge’: declarative knowledge (knowledge of facts), procedural knowledge (encompassing rules, algorithms, and concepts) and conditional knowledge – when to use particular information, and why a piece of information is appropriate in a given situation (Paris et al., 1984). Students' ability to recognize and organize pieces of information constitutes conceptual understanding (Nieswandt, 2007).

Another important construct for chemical understanding considers the different levels of representation necessary for complete, conceptual understanding. Chemistry is commonly represented at three levels: macroscopic, symbolic and submicroscopic or particulate levels (Johnstone, 1991). Students should understand chemistry at the three levels, recognize the level they are operating in, navigate between and within the levels fluently, and understand how the three levels contribute to understanding of chemical phenomena (Treagust et al., 2003). Explaining chemical phenomena requires the use of particulate and sometimes symbolic levels of representation (Treagust et al., 2003). This implies therefore that unless students understand the particulate nature of matter, they will not be able to make sense of and explain chemical phenomena (Dori and Hameiri, 2003). Unfortunately, research has shown that students struggle with representing chemistry at all three levels of representation, explaining chemistry at the three levels and the relationship between the levels (Yarroch, 1985; Bodner, 1991; Nakhleh, 1992, 1993; Bunce and Gabel, 2002; Treagust et al., 2003; Nyachwaya et al., 2011; Naah and Sanger, 2012).

Two types of learners have been identified, based on their learning approaches and success with different types of questions: algorithmic learners and conceptual learners. According to Pushkin (1998), “algorithmic learners can master assessment items requiring mimicking, regurgitation and short-term memorization, while conceptual learners can master assessment items requiring evaluation, comparison, and attribution skills” (p. 809). More recently, Grove and Bretz (2012), more recently suggested that students fall into four categories: (1) indifferent learners, (2) unaware learners, (3) transitional learners and (4) meaningful learners. This new scheme challenges the notion of a dichotomy that previously existed, instead placing learners on a continuum from rote memorization to meaningful learning. According to Grove and Bretz (2012), while indifferent learners resort to rote memorization, meaningful learners recognize a need to develop sound understanding of concepts and meaningful approaches to problem solving.

Students' use of algorithms in solving chemistry problems

Despite the long held belief that understanding science requires conceptual understanding, students resort to memorizing formulas and algorithms for solving problems (Hammer, 1994). A number of studies in chemistry education have found and uncovered students' preference of rote learning and algorithmic problem solving (e.g.Yarroch, 1985; Nakhleh, 1993; Bodner, 2003; Pappa and Tsaparlis, 2011). However, memorized algorithms interfere with students' ability to understand chemistry at the conceptual level, as well as developing higher order thinking skills (Nakhleh and Mitchell, 1993; Dori and Hameiri, 2003). Zoller et al. (1995), describe the application of memorized algorithms to solve problems as lower order cognitive skills. Research has shown that success in solving algorithmic problems does not necessarily mean that students understand the relevant or underlying concepts (Nakhleh, 1993; Zoller et al., 1995; Papaphotis and Tsaparlis, 2008).

That students have been found to use formulas algorithmically without an understanding of the concepts underlying the problems (Bergquist and Heikkinen, 1990; Bodner, 2003), can be traced to some classroom practices and the way textbooks are written, where students are required to apply formulas without explaining what they are doing. Indeed, as Gabel (2003) warned, a lack of conceptual understanding by students leads them to carry out manipulations of mathematical equations which they have not thought about. For example, in a study involving college students, Zoller (2000), found that the students had a recipe they used to solve problems, where they applied memorized algorithms to solve problems, which they believed would always get them the ‘right answer’. The author concluded that, “students have been conditioned to think algorithmically” (p. 198).

Even when students have memorized algorithms, sometimes they may not apply them correctly. For example, in a study looking at high school students' quantum chemical concepts, Papaphotis and Tsaparlis (2008), found that students applied incorrect algorithms in arranging sub-shells in terms of energy demonstrating a lack of conditional knowledge-when to apply a specific algorithm (Nieswandt, 2007). In this study, the researchers found that student performance on questions requiring a combination of knowledge and critical thinking to be very low compared to those that required the application of algorithms. This finding agrees with others that have found students are more successful with tasks that can be solved with algorithmic processes (Nakhleh, 1993; Bodner, 2003).

Why do students resort to use algorithms?

One of the factors associated with students' preference for algorithmic learning is their conception and perception of the nature of chemistry as a collection of facts and formulas that they can memorize and use in examinations (Beall and Prescott, 1994) and a belief that memorization of facts, formulas and algorithms demonstrates understanding of the material (Hammer, 1994). Thus, there is no motivation to seek deeper understanding of the subject. Students' conceptions of the nature of chemistry may be due to the fact that instructors put more value on algorithmic learning than on conceptual learning (Pushkin, 1998), giving students the false impression that they can succeed in science by relying on the algorithms.

The fact that some concepts and processes in chemistry are abstract and complex makes the subject challenging for many students. When students cannot understand these concepts and processes at a conceptual level, they resort to rote memorization (Stefani and Tsaparlis, 2009). Students revert to rote memorization when they cannot keep up with the pace of a course, and earn a good grade (Hammer, 1989). Students who memorize to get by on tests do not end up learning the intended material, or being able to use the memorized material to solve problems (Weissglass, 1993).

Assessments that emphasize or promote algorithmic learning have been associated with students' preference to memorize and attempt to use algorithms instead of aiming for conceptual understanding of underlying chemistry concepts (Zoller, 2002). In fact, some questions used in chemistry do not necessarily require any level of conceptual understanding. Instead, they require a mere recall of information or the application of algorithmic processes through specific steps or procedures for their solution (Zoller et al., 1995). Research has repeatedly shown that students use algorithmic strategies to solve problems, with no understanding of underlying concepts (Zoller et al., 1995, Nakhleh, 1993; Papaphotis and Tsaparlis, 2008). Contrary to conventional thought by teachers, students' ability to solve quantitative problems is not a satisfactory indicator for conceptual understanding, (Sawrey, 1990; Zoller et al., 1995). Assessments should require higher order cognitive skills (Zoller, 1993; Zoller et al., 1995; Tsaparlis and Zoller, 2003) and provide evidence for meaningful learning, which according to Novak (2002), occurs when tests do not exactly mirror what students saw during instruction. Teaching that only emphasizes solving problems without requiring demonstration of understanding of underlying concepts is not helpful especially in enabling students apply knowledge to novel situations (Dahsah and Coll, 2007).

Why we used chemical reactions

The concept of chemical reactions (formulae and equations of chemical reactions) is central to the discipline of chemistry (Van Driel et al., 1998). An equation of a chemical reaction is a representation of chemistry at the symbolic level (Johnstone, 1991). Reading a chemical equation requires an understanding of a number of concepts, such as reactants, products, the relationships between atoms in compounds, the physical states of reactants and products (state symbols), and the related quantitative aspects (Ben-Zvi et al., 1987). Asking students to draw particulate representations of chemical equations therefore assesses their understanding of all the information carried in an equation of a chemical reaction. This research study uses equations of chemical reactions because they help reveal the extent of students' knowledge of all the information entailed in an equation of a chemical reaction.

Specific to chemical reactions, a number of studies have identified student struggles with the fundamental ideas and concepts that draw on chemical bonding and equations. Student struggles include confusion with subscripts and coefficients (Mulford and Robinson, 2002; Sanger, 2005), differentiating between atoms and molecules (Wood and Breyfogle, 2006), interpreting the information carried in chemical equations, such as the meaning of symbols (Wood and Breyfogle, 2006; Kern et al., 2010; Kelly et al., 2010). It has also been shown that while many students can balance equations of chemical reactions, they are for example unable to predict products of a reaction or draw appropriate particulate representations of the reactions (Davidowitz et al., 2010; Nyachwaya et al., 2011).

Methods

Context of the study

The data for this study was collected in a first semester general chemistry class at a small mid-western, public comprehensive university in the U.S. A pre-requisite for the course was one year of high school chemistry. While some of the students enrolled in the course planned to go into a scientific field, other students took the class to meet a graduation and/or pre-professional requirement. The class mainly has freshman and sophomore students with fewer Junior and Senior students. Each student was required to attend three 50-minute lecture periods, one 50 minute discussion (1/3 of the students met at a time), and one three hour lab. The course instructor taught both the lecture and discussions along with several of the lab sections.

The instructor of the class is dedicated to students' conceptual understanding, explicitly addressing the particulate nature of matter throughout the course utilizing the three representations of chemistry, macroscopic, symbolic, and particulate. Active learning instructional strategies were used throughout the course, including interactive lectures with embedded clicker questions, offering think-pair-share opportunities for students, and the use of POGIL (Process-oriented Guided Inquiry Learning) activities, allowing students to explore and develop concepts.

Participants

The data presented here came from 10 students. These 10 students were part of a larger group (n = 71) that took part in our study looking at students understanding of PNM across different types of reactions. An announcement was made in class to recruit interview participants at the end of a class period. 18 students initially consented but due to scheduling difficulties, 8 students could not be interviewed. Of the 10 interviewees, 8 were female. Demographic details of the 10 participants are provided in Table 1.

Table 1 Summary of participants' demographic information

Name	Year	Gender	Major	Semesters of HS chemistry
Note: all names used here are pseudonyms.
Mary	Freshman	F	Chemistry	Four
Maggie	Freshman	F	Undecided	Two
Julie	Freshman	F	Pre-med	Four
Irene	Freshman	F	Biotechnology	Two
Alex	Freshman	F	Biochemistry	Four
Alexus	Sophomore	F	Food science	Two
Frankline	Freshman	M	Conservation	Two
Jayne	Senior	F	Biomedical sciences	Two
Antonio	Freshman	F	Health/human performance	One
Randy	Senior	M	Psychology	One

---

Data collection and sources

As a characteristic of a grounded theory study, initial analysis of students' drawings pointed to the need to collect more data to help answer questions that the study sought to answer, which could not be satisfactorily answered by only looking at students' drawings. Interview data was therefore collected as complementary data. The questions probed for students thinking and processes during equation balancing, uncovering their understanding of subscripts, chemical reactions, types of chemical reactions represented by the chemical equations, physical states, and other topics related to their individual drawings which demonstrate their understanding of PNM.

Two forms of student data were collected: (1) students' written responses to each of the three open-ended drawing questions and (2) student interviews. Two of the open response questions were part of two different mid-term exams and the third open response item was part of the final course examination. The interviews took place a week before the course final examination. In this study, students were asked to balance three equations of chemical reactions and draw particulate level representations of the three reactions (Nyachwaya et al., 2011) (see Fig. 1). As the 10 participants completed these assessment items, they were asked semi-structured interview questions (Rubin and Rubin, 2005) to help us understand their thinking and decision making processes. Follow-up questions were asked to access students' reasoning for their drawing decisions. While many of the questions were generic, each student was asked questions specific to features of their drawings.

Fig. 1 Sample interview questions.

Data analysis

The data in this study were analyzed using grounded theory (Creswell, 2007). As per grounded theory, the analysis took an inductive stance, striving to derive meaning from the data (Creswell, 2007). As a characteristic of a grounded theory study, initial analysis of students' drawings to see their understanding of PNM pointed to the need to collect more data to help answer questions that the study sought to answer, which could not be satisfactorily answered by only looking at students' drawings. A decision on whether a student understands the particulate nature of matter was therefore informed by data from their particulate drawings and response to interview questions seeking to determine their ability to explain the chemistry underlying the equations. Consistent with grounded theory design, one researcher was the main instrument of data collection. Data analysis took an inductive stance, striving to derive meaning from the data (Creswell, 2007). The resulting theory on students' conceptual understanding of the particulate nature of matter emerged from the data (Charmaz, 2000). Two researchers coded 10 student drawings independently for correct equation balancing and appropriate particulate drawings and compared results; inter-rater reliability was 91%. One researcher then coded the entire data set. Two other researchers verified the coding. Differences were resolved through discussion. Two researchers coded three interview transcripts, looking for instances where students were relying on algorithms, and then compared results; inter-rater reliability was 89%. One researcher coded the other 7 transcripts. The codes were verified by the group. Consensus was reached through discussion.

Results

Students were asked to balance the three given equations, and then draw particulate representations of the three equations. Table 2 provides a summary of the whole class' performance on the two initial tasks, while Table 3 provides a summary of the 10 students' performance on these two initial tasks during the interview. Data in the tables compares students' understanding at the symbolic and particulate levels of representation. Appropriate particulate representations required that students indicate correct relative atomic sizes, recognition of the nature of particles (ion or molecule) and correct molecular geometry for molecular compounds among other attributes. For detailed criteria, see Nyachwaya et al., 2011.

Table 2 Summary of student performance at symbolic and particulate levels for the whole class

Question/equation	CH4 + O2	AgNO3 + CaCl2	HCl + CaCO3
% correct balancing	96	100	98
% appropriate particulate representation	21.6	7.8	3.9

---

Table 3 Summary of the 10 students' performance at symbolic and particulate levels

Question/equation	CH4 + O2	AgNO3 + CaCl2	HCl + CaCO3
% correct balancing	100	100	100
% appropriate particulate representation	20	10	10

---

Of our 10 participants, only Randy and Franklin drew appropriate representations for reaction 1, while Antonio and Tanya drew appropriate particulate representations for reactions 2 and 3 respectively. None of the participants demonstrated particulate level understanding of the three reactions through their drawings. Tables 2 and 3 indicate student struggles with the particulate level of representation, confirming results from our earlier studies focused on covalent reactions (Kern et al., 2010 and Nyachwaya et al., 2011). This current study also indicates students' struggles with particulate representations of ionic reactions. It can also be seen from the tables that the 10 students who were interviewed are a representative of the class as a whole.

In the following sections, we identify and provide examples of how students used algorithms, instances where students successfully use memorized information without understanding the underlying chemistry, and cases of students misapplying memorized algorithms to answer questions. We found that the algorithms students used fell into two broad categories: symbolic level algorithms and particulate level algorithms. Our findings will be presented and discussed under these two broad categories.

a. Symbolic level algorithms

The use of symbolic level algorithms was apparent for the following topics: (i) equation balancing, (ii) understanding of subscripts, (iii) switching partners (the meaning and process of a chemical reaction), and (iv) the classification of reactions. Each of these issues are described in detail below.

(i) Equation balancing. During the equation balancing activity three of the participants applied a process they had memorized without knowing why the algorithm worked. For example, in balancing the three chemical equations, Mary methodically started with the first element in the first compound, followed by the second element, and then the first element in the second compound. Asked why she followed this approach, she attributed this process to how she was taught in high school, saying:

The first thing I was taught to do back in high school is to start with the ‘C’ (referring to carbon in methane). Then I would look at the H's (meaning hydrogen) next, and then I would go to oxygen. I was taught to do H's and O's last.

Asked why she followed that procedure, she responded:

I don't know. My high school chemistry teacher taught us that you do H's and O's last. That's how I was taught.

While this algorithm or memorized procedure worked, students did not understand the process they were using. Our goal is to highlight the fact that the participant followed an algorithm (which worked) without an understanding of why it worked. He reason for its use was simply that it was how she was taught. This student was successful in balancing the given equation. However, following a heuristic without understanding why it works can be problematic especially in novel situations, where some judgment is needed. Also, it is possible that this tendency encourages rote memorization as opposed to learning for conceptual understanding. Equation balancing is usually taught early on in chemistry, and compared to other topics is not to be considered a difficult idea. Starting to memorize sets a bad precedence, which can lead to problems with conceptual understanding if a student chooses to focus on memorizing facts.

(ii) Understanding of subscripts. When asked about the significance of subscript two in calcium nitrate, Maggie, one of the participants, said that ‘two’ is the charge on calcium adding that “since charges always travel, the charge transfers to the other part, from calcium to the nitrate”. She illustrated the process of “charge transfer” as shown in Fig. 2 below:

Fig. 2 Students' illustration of the process of ‘charge transfer’.

Asked why this process of converting charges into subscripts worked, Maggie replied:

I really don't know the reason behind it but just it is something I know that when you have a charge here or when it is something like three plus, it goes down here, to the other to whatever element is here.

Despite the fact that she knew that ‘the process of charge transfer occurs’ Maggie could not identify the ‘two’ as a subscript, instead calling it a coefficient. It is clear that she has memorized a process, which is charge transfer in this case, without knowing what a subscript is.

Similarly, Alex explained the significance of the subscript ‘two’ in the formula of calcium nitrate, stating,

the two comes from calcium because calcium has a two-plus and nitrate has minus one charge”[…] when they combine, you have to make sure that they are balanced together so that like one of them is not as overwhelming the other. You switch the charges on them so that they can combine. So this entire thing (referring to nitrate) gets a two and calcium gets a minus one.

Alex first referred to the subscript two as a di-nitrogen, later calling it a coefficient before eventually correctly identifying it as a subscript. Like Maggie, Alex could describe the ‘switching processes without knowing the name given to the number ‘two’ and ‘minus one’ that she used in describing the switching process.

(iii) “Switching Partners” (understanding of the meaning of a chemical reaction). During, the interviews, participants were asked to explain what happens in a chemical reaction. When asked what happens in a chemical reaction, Alex, said that

It is kind of when you learn about this, you are just sort of told that they switch partners, or they switch which ones pair with which, but I guess I don't really know why they would decide to switch you know but I know it will depend on the amount of energy that is present, like some reactions will require energy for the switch to happen, others will release energy, but I don't know why they switch partners to begin with.

The misapplication of this algorithm became clear when students were asked to write an equation for dissolving sodium chloride in water. For example, Antonio wrote the following equation:

When asked to explain her choice of products, she replied:

When you write the equation, sodium will combine with oxygen while chlorine will combine with hydrogen to form the products.

The choice to use the approach of ‘switching partners’ follows given Antonio's description or understanding of what happens in a chemical reaction. From the equation that she wrote, it is evident that Antonio applied the algorithm of ‘switching partners’ in predicting the products and ultimately writing the equation.

In (ii) and (iii) above, the participant describes the process of ‘switching’ as occurring to achieve a certain purpose- for example, in (ii) above, the students described switching occurring “so that one is not overwhelmed”. It is interesting that the student is anthropomorphizing the reaction and also using a teleological explanation – to avoid overwhelming one of the species and a decision or need to switch partners. According to Talanquer (2007), teleological explanations describe an event as occurring to achieve a certain purpose. For example, students think of reactions taking place in order for atoms to achieve full shells (Taber, 2002; Taber and Adbo, 2013). In the results reported here, the student assigns ‘human’ qualities to particulate species-where they ‘decide to switch’.

(iv) Classification of reactions. One notable trend in classifying reactions was that participants made decisions based on memorized rules, such as the identity of the products. As an example, while four of the ten participants correctly classified the third reaction, none of them could identify the base in the reaction. They all pointed to the salt, carbon dioxide and water as evidence that it was an acid–base reaction. When was asked to explain her choice, Alexus stated ‘because this is hydrochloric acid and I want to say that calcium chloride is a base’ even though calcium chloride is a product in this reaction. While the classification of acid–base is correct, the fact that Alexus and others could not identify the base in the reaction is of concern.

Similarly, Mary identified reaction 3 as an acid–base reaction, but was not sure whether calcium carbonate was a base. Her criterion for classifying reactions was based on identifying some compounds in the reaction. When asked to use these criteria, she identified reaction 2 as a ‘salt reaction’, and reaction 1 as “one that gives off a gas”.

Like Mary, when asked to classify the three reactions, Julie's response was

I think equation one is oxidation reduction… through the process of elimination I guess. I think they are all oxidation–reduction reactions. For equation 3, HCl is an acid but it is not an acid–base because there is no base in the reaction… when I think of a base, I look for an OH as has been emphasized to us. In an acid, hydrogen is generally present.

Franklin was very clear about his use of memorized rules, stating that

I just memorize like the rules like whether it is a single replacement or a double replacement, a combustion reaction I look for oxygen, and in acid carbonate reactions, they give a salt, water and carbon dioxide.

These memorized criteria became a source of confusion some students. For example, For example, when Antonio was asked to classify the three reactions, she responded,

This one (referring to equation 1) I would look at like how there is water produced and so I kind of wonder if it is an acid base reaction kind of thing… based on the fact that there is a water produced. For equation two I see that the products are switching who they are with as ionic equations that we have been dealing with. For equation 3, there is water produced, so automatically my mind goes to acid base.

Our participants' reliance on algorithms is apparent in their classification of the three reactions. Student struggles with classification of reactions has been previously documented (Stains and Talanquer, 2008). While the algorithms help most students correctly classify the reactions, they hide the lack of understanding of other aspects of chemical reactions, such as identifying the reactant that is a base in reaction three. When an aspect of a students' algorithm does not fit with the information given, such as the absence of a compound with an OH, the algorithm does not work.

b. Particulate level algorithms

The use of particulate level algorithms was apparent in our participants' explanation of (i) periodic trends, (ii) accounting for molecular geometry and (iii) breaking apart of salts and (iv) ions in solid ionic compounds. Each of these issues is described below.

(i) Periodic trends: atomic size. Six of the ten participants could state the correct atomic size trends they applied to determine the relative size of the particles in their pictures but could not explain the trends. For example, Mary drew the following particulate representation of reaction 1 (see Fig. 3).

Fig. 3 Particulate drawing of the combustion of methane.

When asked why she had drawn the carbon atom in methane larger than the hydrogen, Mary responded,

Carbon would be bigger than hydrogen because it is the central atom

While the correct trend is shown in the drawing, Mary does not exhibit an understanding of why her drawing is appropriate. This lack of understanding of relative atomic sizes can also be seen in how she drew the water molecule in Fig. 3 above, where the hydrogen atoms do not appear to be of the same size, and are larger than the oxygen atom.

Julie's particulate drawing of reaction 1 (see Fig. 4) also shows correct relative sizes for C and H atoms in methane. When asked to explain her decision about these relative sizes she responded,

Fig. 4 Julia's particulate drawing of the reaction of methane with oxygen.

because carbon has six electrons I believe while hydrogen has one, and so technically it will be six times larger than hydrogen”.

This participant has clearly portrayed the correct relative sizes of atoms in Fig. 4 without a clear knowledge or understanding of the correct reason for the trend as shown. While the number of electrons in an atom bears some relationship to the size of an atom, the idea of direct proportionality inferred here is inaccurate.

Prior research has indicated student struggles with atomic and ionic size (Coll and Treagust, 2003; Nyachwaya et al., 2011). With the data presented here, we see that students' depiction of size is informed by an algorithm as opposed to conceptual knowledge and understanding. It is possible that some of the drawings that we considered appropriate may not have necessarily been informed by a good understanding of the underlying principles.

(ii) Molecular geometry. Only two students drew appropriate particulate representations of the first reaction (see Table 3). Looking at the ‘appropriate drawings, one would assume that students understood the molecular geometry of water and carbon dioxide molecules and relative atomic sizes in the water molecule. It was however evident during the interview that their choices were not necessarily guided by an understanding of molecular geometry or atomic size. Fig. 5 below is a portion of Alexus' drawing for reaction one, illustrating the ‘correct’ molecular geometry and relative atomic sizes of water and carbon dioxide.

Fig. 5 Alexus' particulate drawing of water and carbon dioxide molecules.

Note that the water molecule has the correct relative atomic sizes and molecular geometry as drawn. However, her reason for the drawing was not based on the knowledge of either the molecular geometry or relative atomic sizes:

I just drew it as a mickey mouse I don't know why. I don't know it just makes sense to me. It just makes sense to draw the center one bigger and the side ones smaller

It seems that the student knows the ‘mickey mouse’ shape of a water molecule (when drawn). Some probing however revealed that the student has been driven by the ‘mickey mouse’ idea more than correct conceptions of molecular geometry. Past research has shown that students struggle with the concept of molecular geometry (Nyachwaya et al., 2011). In our work specifically, we considered drawings like the one in Fig. 5 above to be appropriate. The interview data however leads us to speculate that some students used algorithms instead of conceptual understanding in their drawings.

(iii) ‘Breaking apart’ of salts. This theme, while illustrating algorithms that our participants used, also points to a difficulty with polyatomic ions. In both of the cases described below, participants used a covalent bonding model to represent ionic compounds (Kelly et al., 2010; Nyachwaya et al., 2011; Naah and Sanger, 2012). Two of our participants used an algorithm based on the idea that ‘salts break apart’. The first excerpt is part of the interview transcript with Alex.

I: from your drawings, you have used the same model to represent all compounds. Earlier, you defined aqueous as meaning ‘dissolved in water’. Describe to me what happens when you dissolve common salt, sodium chloride in water.

Alex: the salt dissolves in the water and you can barely see the salt particles. The salt breaks apart to form ions (see Fig. 6)

Fig. 6 Alex's illustration of sodium chloride breaking apart to form ions.

I: given this idea, (referring to the step above) how would you represent aqueous silver nitrate and aqueous calcium nitrate in equation 2?

Alex: we'll have…. Silver will be two plus, everything else will be negative. For calcium nitrate, we'll have… (See Fig. 7)

Fig. 7 Alex's particulate drawings of silver nitrate and calcium nitrate.

A second participant, Randy, used the idea of ‘breaking apart’ during the interview to explain his particulate drawings. The following excerpt shows his choices and reasoning:

I: could you name one property of ionic compounds

Randy: they are easily broken apart into ions. They are found in most salts.

I: what happens to a salt like calcium chloride when it is dissolved in water?

Randy: the salt breaks apart

I: could you then draw here what happens to calcium chloride in equation 2 when it is dissolved in water?

Randy: yes. I believe it will be (see Fig. 8)

Fig. 8 Randy's illustration of calcium chloride breaking apart.

I: you mentioned that ionic compounds are easily broken apart into ions. Could you draw here the ions you would see in calcium carbonate?

Randy: yes. I believe it will be like this (see Fig. 9)

Fig. 9 Randy's illustration of calcium carbonate breaking apart.

These drawings are guided by the broad idea of ‘breaking apart’ mentioned by the participants in the excerpts above. In both cases, while the algorithm of breaking apart worked for compounds such as CaCl₂, it did not work for AgNO₃ and CaCO₃, compounds with polyatomic ions. In the two instances, the participants indiscriminately applied the rule (algorithm) of ionic compounds breaking into constituent ions. While the algorithm works for ionic compounds composed of monoatomic ions, it clearly fails when applied to polyatomic ions. The students failed to realize that polyatomic ions react as a single unit (Smith and Metz, 1996; Nyachwaya et al., 2011; Naah and Sanger, 2012). These drawings point to a lack of understanding of the nature of polyatomic ions and also an overly generalized application of the algorithm of ‘splitting’ of ionic compounds.

(iv) Ions in solid ionic compounds. In this theme, students represent solid ionic compounds at the particulate level without charges on the constituent species. For example, in Fig. 10 the student drew both sold silver chloride in equation 2 and solid calcium carbonate in equation 3 as covalently bonded, as shown in Fig. 10 below.

Fig. 10 Alexus' particulate drawing of silver chloride (a) and calcium carbonate (b).

The transcript below shows the student's account of her drawings.

I: looking at reaction 2 here, in silver nitrate you have silver ions and nitrate ions. How come you did not indicate ions in silver chloride here?

Alexus: because they are kind of like there one group so there is no like positive or negative charge on them because they are like a group together.

I: why will they not have charges on them?

Alexus: because we were just told they are not supposed to have ions. The ions are stripped from each other, but they like trade in so they don't form separate positive and negative.

I: so that would be the same case for calcium carbonate here?

Alexus: yes, so there is no positive and negative charge since they are grouped together they are one group where they share electrons and this is particularly the case for solids. They trade their ions to not have a positive or negative ion.

In this case, the student consistently uses the ‘rule’ that in solid ionic compounds, there are no ions at the particulate level – she uses the rule in equation silver chloride and calcium carbonate. As seen in Fig. 10, the student clearly differentiated how she drew aqueous ionic compounds at the particulate level. In our past work, we found that our most of our students (up to 95%) used a covalent bonding model to represent ionic compounds at the particulate level (Nyachwaya et al., 2011). In our subsequent work, we found that students particularly struggled to represent solid ionic compounds. This interview data gives us a glimpse into why students particularly struggle to represent solid ionic compounds at the particulate level.

Discussion

This study further explores students' reliance on algorithmic thinking and uncovers specific algorithms, especially related to PNM. Students' use of “black-box” algorithms to solve problems has been documented in other research studies (e.g.Sawrey, 1990; Nakhleh, 1993; Zoller et al., 1995; Zoller, 2000; Stains and Talanquer, 2008; Cartrette and Bodner, 2010; Pappa and Tsaparlis, 2011). Our research uncovered different specific ways in which students rely on algorithms, and demonstrate a lack an understanding of basic and fundamental ideas in chemistry (such as balancing equations, classification of reactions or understanding subscripts).

Two themes emerged from our data: (1) successful use of algorithms without conceptual knowledge of why the algorithm works and (2) inappropriate use of algorithms. In the first case, students successfully applied algorithms in responding to interview questions. As evident from our results, despite the fact that students could perform routines such as ‘equation balancing’ following certain procedures, some could not explain why it would be necessary to start by balancing certain elements first. Indeed, a good number of chemistry students solve problems without understanding the concepts underlying the problems (Nakhleh and Mitchell, 1993; Chei, 2001; Tsaparlis and Zoller, 2003; Papaphotis and Tsaparlis, 2008). As Nakhleh (1993), and Robinson (2003) note, algorithms seem helpful, but reliance on them can hinder meaningful learning and conceptual understanding.

Also evident from our data are instances where students apply the wrong algorithm, or incorrectly use algorithms in answering particular questions. It is known that even when students have memorized algorithms, they may not apply them correctly (Papaphotis and Tsaparlis, 2008). From our results, students who relied on the ‘switching partners’ heuristic in double replacement reactions predicted the wrong products for dissolution of sodium chloride in water. Naah and Sanger (2012) cited a similar finding in their research on student misconceptions of dissolving ionic compounds in water. We suggest that a student can be successful when problem solving with algorithmic approaches – under some circumstances. The caveat is that algorithms are models, with assumptions and limitations and thus do not work under all scenarios. The expert knows this and acts accordingly; the novice is more haphazard in their approach. Participants in our study also seem to be lacking ‘conditional knowledge’ (Nieswandt, 2007), which would help them discern when particular information is useful for solving a given problem.

In order to successfully solve a problem, one has to understand what the problem is asking, what one is provided with, underlying, requisite concepts (Gabel, 1986) and knowledge of how to apply the concepts in solving the problem. Our results indicate that students rely on some of the information, such as some features of given equations in responding to questions. For example, a student took products of the combustion reaction (CO₂ and H₂O) as evidence for an acid–base reaction. In order to successfully solve a problem, one has to understand what the problem is asking, what one is provided with, underlying, requisite concepts (Gabel, 1986) and knowledge of how to apply the concepts in solving the problem. If we are to think of these as conditions necessary for successful problem solving, one needs all or a combination of them. For example, students correctly classified chemical reactions using surface features, such as the presence of carbon dioxide gas and water, but could not identify and acid or base in the reaction. This particular finding, of relying on surface features to classify reactions was noted by Stains and Talanquer (2008) in a study looking at classification of chemical reactions.

Implications

Our findings have implications for assessment practices in chemistry education. Most assessments used in college chemistry are algorithmic (Pappa and Tsaparlis, 2011) and as such can at best promote algorithmic learning. According to Nakhleh (1993), if we want our students to demonstrate conceptual understanding, we have to give them an opportunity to demonstrate it. As long as we continue to reward rote memorization through assessments, students will not have the motivation to seek conceptual understanding. It is clear from our study that students have been asked questions that require them to only state the ‘what’, without asking the ‘how and ‘why’ of phenomena.

Traditional forms of assessment in chemistry are algorithmic and tend to focus on students' ability to solve problems and get the ‘correct answer, with teachers equating success at solving quantitative problems to conceptual understanding (Bennett, 2008; Pappa and Tsaparlis, 2011). Many of the traditional forms of assessment tend to focus more on students' ability to recall definitions and facts, and apply known formulas and algorithms to solve problems, and less on conceptual understanding (Gabel, 2003). Quantitative problems in particular often have steps that students can memorize and blindly apply-they promote algorithmic learning, at the expense of conceptual understanding and development of higher order cognitive skills (Zoller, 2002).

Designing effective assessments that will uncover students' reliance on algorithms or a lack of understanding of underlying chemistry is a challenge. In our past work (Nyachwaya et al., 2011), we developed and used an open-ended drawing tool to uncover students' conceptual understanding of the particulate nature of matter. We used students' particulate drawings to make judgments on their conceptual understanding. Our results in the present study reveal that this may not necessarily be the case. The open ended drawing tool helped us uncover students understanding of at the symbolic and particulate levels. Hidden behind the demonstrated competency is the reliance on algorithms as demonstrated by the results reported here. In saying this, we acknowledge the challenges of conducting interviews as a way of assessing students.

One of the biggest challenges that a college instructor faces in relation to rote learning is that their students have experienced many years of instruction and evaluation where rote learning has been encouraged. This experience makes it difficult for students to change their learning practices (Novak, 2002). This is especially true if the rote memorization has worked for them in the past. It is evident from the data above that tasks that students are used to solving did not go the extent of asking ‘why’. The student who balances equations using a memorized and practiced procedure successfully did not know why, beyond the fact that that is how they were taught. The student who has predicted products of reactions using the process of ‘switching partners’ has probably seen that work, until they encountered a case like dissolving sodium chloride in water, where the algorithm of switching partners' does not work. Changes to traditional teaching approaches that promote rote learning have been slow at best (Novak, 2002).

While heuristics sometimes make teaching of some ideas easy, it is important to point out to students that they do not always work. For example, while ‘switching partners’ can be a useful heuristic in predicting products of reactions in double replacement reactions, it does not work for dissolving salts in water (Naah and Sanger, 2012). A good possible approach to expose students to shortcomings of heuristics is to provide students with situations where the particular heuristics do not work and explicitly discuss the conditional thinking that underlies the use of a heuristic or algorithm.

The results reported here point to situations where instruction can lead to unintended consequences. While the rationale for ‘balancing some elements first’ when learning to balance chemical equations takes care of fractional coefficients, some of our participants seem to have missed the reasoning behind the move. Also, while it is true that water and carbon dioxide gas are products of an acid–base reaction, the two products are not necessarily an indication of an acid base reaction. With our results, instructors can anticipate algorithms that students are likely to pick up during instruction, especially during topics or concepts related to PNM and address them as appropriate. In the case where students are using teleological explanations, which are promoted by text books, while the intention is to help develop conceptual understanding (Taber and Watts, 1996), students will not necessarily take away the same message.

Limitations of this study

Our interview participants had varied experiences in high school chemistry, ranging from one to four semesters of high school chemistry. We therefore do not know what effect this may have had on our findings since some of our participants referred to their high school chemistry during interviews. This is something we hope to explore in our future work.

References

1. Adadan E., Trundle K. C. and Irving K. E., (2010), Exploring grade 11 students' conceptual pathways of the particulate nature of matter in the context of multirepresentational instruction, J. Res. Sci. Teach., 47(8), 1004–1035.
2. Beall H. and Prescott S., (1994), Concepts and Calculations in Chemistry Teaching and Learning, J. Chem. Educ., 71, 111–112.
3. Bennett S. W., (2008), Problem solving: can anybody do it? Chem. Educ. Res. Pract., 9, 60–64.
4. Ben-Zvi R., Eylon B. and Silberstein J., (1987), Students' visualization of a chemical reaction, Educ. Chem., 24, 117–120.
5. Bergquist, W. and Heikkinen, H. (1990), Student ideas regarding chemical equilibrium: What written test answers do not reveal, J. Chem. Educ., 67, 1000–1003.
6. Bodner G. M., (1991), I have found you an argument: the conceptual knowledge of beginning Chemistry Graduate Students, J. Chem. Educ., 68, 653–655.
7. Bodner G. M., (2003), Problem solving: the difference between what we do and what we tell students to do, Univ. Chem. Educ., 7, 37–45.
8. Bunce D. M. and Gabel D. L., (2002), Differential effects on the achievement of males and females of teaching the particulate nature of chemistry, J. Res. Sci. Teach., 39(10), 911–927.
9. Cartrette P. D. and Bodner G. M., (2010), Non-mathematical problem solving in organic chemistry. J. Res. Sci. Teach., 47(6), 643-660.
10. Charmaz K., (2000), Grounded Theory: Objectivist and Constructivist Methods, in Denzin N. K. and Lincoln Y. S. (ed.), Handbook of Qualitative Research, 2nd edn, Thousand Oaks, CA: Sage, pp. 509–535.
11. Chei M. H., (2001), Algorithmic problem solving and conceptual understanding of chemistry by students at a local high school in Taiwan. Proc. Natl. Sci. Counc., Repub. China, Part D, 11, 20–38.
12. Coll R. K. and Treagust D. F., (2003), Investigation of secondary school, undergraduate, and graduate learners’ mental models of ionic bonding, J. Res. Sci. Teach., 40, 464–486.
13. Creswell J., (2007), Qualitative inquiry and research design: choosing among 5 approaches, 2nd edn, Thousand Oaks, CA: Sage.
14. Cros D., Chastrette M. and Fayol M., (1987), Conceptions of second year university students of some fundamental notions of chemistry, Int. J. Sci. Ed., 10, 331–336.
15. Dahsah C. and Coll R. K., (2007), Thai grade 10 and 11 students' conceptual understanding and ability to solve stoichiometry problems, Res. Sci. Tech. Educ., 25(2), 227–241.
16. Davidowitz B., Chittleborough C. and Murray E., (2010), Student-generated submicro diagrams: a useful tool for teaching and learning chemical equations and stoichiometry, Chem. Educ. Res. Pract., 11, 154–164.
17. Dori Y. J. and Hameiri M., (2003), Multidimensional analysis system for quantitative chemistry problems - symbol, macro, micro and process aspects, J. Res. Sci. Teach., 40(3), 278–302.
18. Gabel D. L., (1986). Problem solving in chemistry. In research matters—To the science teacher. NARST Occasional Publication, ERIC Document Reproduction Service No. ED266957.
19. Gabel D. L., (2003), Enhancing conceptual understanding of Science, Educ. Horizons, 81, 70–76.
20. Gabel D. L. and Bunce D. M., (1994), Research on problem solving: Chemistry. in Gabel D. L. (ed.) Handbook of Research on Science Teaching and Learning, New York, Macmillan, pp. 310–326.
21. Gotwals A. W. and Songer N. B., (2013), Using assessments to gather validity evidence for a learning progression on evidence-based explanations with core ecological content, J. Res. Sci. Teach., 40(5), 597–626.
22. Grove N. P. and Bretz S. L., (2012), A Continuum of Learning: From Rote Memorization to Meaningful Learning in Organic Chemistry, Chem. Educ. Res. Pract., 13, 201–208.
23. Hammer D., (1989), Two approaches to learning physics. Phys. Teach., 27(12), 664–670.
24. Hammer D., (1994), Epistemological beliefs in introductory physics, Cog. Inst., 12(2), 151–183.
25. Heeren J. K., (1990), Teaching chemistry by the Socratic Method, J. Chem. Educ., 67(4), 330–331.
26. Johnstone A. H., (1991), Why is science difficult to learn? Things are seldom what they seem, J. Comp. Ass. Learn., 7, 75–83.
27. Kelly R. M., Barrera J. H. and Mohamed S. C., (2010), An analysis of undergraduate general chemistry students' misconceptions of the submicroscopic level of precipitation reactions, J. Chem. Educ., 87(1), 113–118.
28. Kern A. L., Wood N. B., Roehrig G. H. and Nyachwaya J. M. , (2010), Chem. Educ. Res. Pract., 11, 165–172.
29. Mulford D. R. and Robinson W. R., (2002), An inventory for alternative conceptions among first-semester general chemistry students, J. Chem. Educ., 79, 739–744.
30. Naah B. M. and Sanger M. J., (2012), Student misconceptions in writing balanced equations for dissolving ionic compounds in water, Chem. Educ. Res. Pract., 13, 186–194.
31. Nakhleh M., (1992), Why some students don’t learn chemistry, J. Chem. Educ., 69, 191–196.
32. Nakhleh M. B., (1993), Are our students conceptual thinkers or algorithmic problem solvers? J. Chem. Educ., 70, 52–55.
33. Nakhleh M. B. and Mitchell R. C., (1993), Concept learning versus problem-solving: there is a difference, J. Chem. Educ., 70, 190–192.
34. Nieswandt M., (2007), Student effect and conceptual understanding in learning chemistry, J. Res. Sci. Teach., 44, 908–937.
35. Novak J. D., (2002), Meaningful learning: the essential factor for conceptual change in limited or appropriate propositional hierarchies (liphs) leading to empowerment of learners, Sci. Educ., 86(4), 548–571.
36. Nyachwaya J. M., Mohamed A-R., Roehrig G. H., Wood N. B., Kern A. L. and Schneider J. L. , (2011), Chem. Educ. Res. Pract., 12, 121–132.
37. Papaphotis G. and Tsaparlis G., (2008), Conceptual versus algorithmic learning in high school chemistry: the case of basic quantum chemical concepts, Part 2. Students' common errors, misconceptions and difficulties in understanding, Chem. Educ. Res. Pract., 9, 332–340.
38. Pappa E. T. and Tsaparlis G., (2011), Evaluation of questions in general chemistry textbooks according to the form of the questions and the Question-Answer Relationship (QUA): the case of intra-and intermolecular chemical bonding, Chem. Educ. Res. Pract., 12, 262–270.
39. Paris S. G., Cross D. R. and Lipson M. Y., (1984), Informed strategies for learning: A program to improve children's reading awareness and comprehension, J. Ed. Psych., 76, 1239–1252.
40. Price J. F. and McNeill K. L., (2013), Toward a lived science curriculum in intersecting figured worlds: an exploration of individual meanings in science education, J. Res. Sci. Teach., 50(5), 501–529.
41. Pushkin D. B., (1998), Introductory Students, Conceptual Understanding, and Algorithmic Success, J. Chem. Educ., 75(7), 809–810.
42. Robinson W. R., (2003), Chemistry problem-solving: symbol, macro, micro, and process aspects, J. Chem. Educ., 80, 978–982.
43. Rubin H. and Rubin I., (2005), Qualitative interviewing the art of hearing data, Thousand Oaks, CA: Sage.
44. Sanger M. J., (2005), Evaluating students' conceptual understanding of balanced equations and stoichiometric ratios using a particulate drawing, J. Chem. Educ., 82, 131–134.
45. Sawrey B. A., (1990), Concept learning vs. problem solving: Revisited, J. Chem. Educ., 67, 253–264.
46. Smith K. J. and Metz P. A., (1996), Evaluating student understanding of solution chemistry through microscopic representations, J. Chem. Educ., 73, 233–235.
47. Stains M. and Talanquer V., (2008), Classification of chemical substances using particulate representations of matter: An analysis of student thinking, Int. J. Sci. Educ., 29, 643–661.
48. Stefani C. and Tsaparlis G., (2009), Students’ levels of explanations, models, and misconceptions in basic quantum chemistry: a phenomenographic study, J. Res. Sci. Teach., 46, 520–536.
49. Taasoobshirazi G. and Glyn S. M., (2009), College students solving chemistry problems: a theoretical model of expertise, J. Res. Sci. Teach., 46(10), 1070–1089.
50. Taber K., (2002), Chemical misconceptions - prevention, diagnosis and cure: vol. 1: theoretical background, London: Royal Society of Chemistry.
51. Taber K. S. and Adbo K., (2013), Developing chemical understanding in the explanatory vacuum: Swedish high school students' use of an anthropomorphic conceptual framework to make sense of chemical phenomena, in Tsaparlis G. and Sevian H., (ed.) Concepts of Matter in Science Education., Springer, pp. 347–370.
52. Taber K. S. and Watts M., (1996), The secret life of the chemical bond: students' anthropomorphic and animistic references to bonding. Int. J. Sci. Educ., 18(5), 557–568.
53. Talanquer V., (2007), Explanations and teleology in chemistry education. Int. J. Sci. Educ., 29(7), 853–870.
54. Teichert M. A. and Stacy A. M., (2002). Promoting understanding of chemical bonding and spontaneity through student explanation and integration of ideas, J. Res. Sci. Teach., 39, 464–496.
55. Treagust D. F., Chittleborough G. and Mamiala T. L., (2003), The role of submicroscopic and symbolic representations in chemical explanations, Int. J. Sci. Educ., 25(11), 1353–1368.
56. Tsaparlis G., (2005), Non-algorithmic quantitative problem solving in university physical chemistry: a correlation study of the role of selective cognitive factors, Res. Sci. Tech. Educ., 23, 125–148.
57. Tsaparlis G. and Zoller U., (2003), Evaluation of higher- versus lower-order cognitive skills-type examinations in chemistry: implications for university in-class assessment and examinations, Univ. Chem. Educ., 7, 50–57.
58. Van Driel J. H., Verloop N. and de Vos W., (1998), Developing science teachers' pedagogical content knowledge, J. Res. Sci. Teach., 35(6), 673–695.
59. Weissglass J., (1993), Small-Group Learning, Am. Math. Mon., 100(7), 662–668.
60. Wood C. and Breyfogle B., (2006), Interactive Demonstrations for Mole Ratios and Limiting Reagents, J. Chem. Educ., 83(5), 741.
61. Yarroch W. L. (1985), Student understanding of chemical equation balancing, J. Res. Sci. Teach., 22, 449–459.
62. Zoller U., (1993), Lecture and learning: are they compatible? Maybe for LCOS; unlikely for HOCS, J. Chem. Educ., 70(3), 195–197.
63. Zoller U., (2000), Teaching tomorrow's college science courses – are we getting it right? J. Col. Sci. Teach., 29, 409–414.
64. Zoller U. (2002), Algorithmic, LOCS and HOCS (chemistry) exam questions: performance and attitudes of college students, Int. J. Sci. Educ., 24(2), 185–203.
65. Zoller U., Lubezky A., Nakhleh M. B., Tessier B. and Dori J., (1995), Success on algorithmic and LOCS vs. conceptual chemistry exam questions, J. Chem. Educ., 72, 987–989.

---