Reasoning about benefits, costs, and risks of chemical substances: mapping different levels of sophistication

Abstract

The ability to evaluate options and make informed decisions about problems in relevant contexts is a core competency in science education that requires the use of both domain-general and discipline-specific knowledge and reasoning strategies. In this study we investigated the implicit assumptions and modes of reasoning applied by individuals with different levels of training in chemistry when engaged in a task that demanded the evaluation of the benefits, costs, and risks (BCR) of using different chemical substances. We were interested in identifying and characterizing different levels of sophistication in the use of chemistry concepts and ideas in BCR reasoning. Our qualitative study elicited reasoning patterns that ranged from intuitive to mixed to normative, with students mostly in mid-undergraduate years demonstrating reasoning that was a mixture of intuitive and chemical ways of thinking. Intuitive reasoning was governed primarily by affective impressions about the substances under evaluation. Consideration of compositional, structural, and energetic features of substances was observed with increased training in chemistry, with a tendency to mix particle-level explanations with intuitive assumptions. Normative thinking shifted toward proactive use of appropriate disciplinary knowledge, recognition of a need for more data about bulk properties particularly on large scales, and consideration of pros, cons, and trade-offs. Implications are discussed for ways to improve the undergraduate chemistry curriculum so that students gain proficiency in making productive judgments and informed decisions.

---

Introduction

Standards and policy documents in science education emphasize the need to develop students' abilities to use scientific knowledge and practices to make informed decisions in realistic contexts (AAAS, 1993; NRC, 1996, 2011, 2013). Nevertheless, making these types of decisions often involves the integration of scientific understandings and social, economic, and environmental considerations not often analyzed in conventional science classrooms (Feinstein, 2011). Dominant chemistry curricula at all educational levels focus on the presentation of central concepts and ideas in the discipline, without much substantive analysis of the benefits, costs, and risks of using chemical products or engaging in chemical practices (Eilks et al., 2013). Despite the central role that chemical knowledge plays in addressing major problems confronting modern societies, from global warming to availability of alternative energy sources, little class time is spent learning, debating, and reflecting about such topics. Moreover, little research has been done on how and to what extent students apply their chemistry knowledge in making decisions related to issues that, as those listed above, demand recognizing and weighing several competing factors.

Our understanding of how learners make use of their chemistry knowledge, together with other considerations (e.g., environmental, health), in judging costs and benefits and making decisions is limited. Although results from research in the field of socioscientific issues shed light on the factors that influence student decision making (Sadler and Zeidler, 2005; Evagorou et al., 2012), as well as on the challenges that students face in building arguments to justify their decisions (Zeidler et al., 2005), most of these studies have focused on the general characterization of the type and quality of the arguments built by students when debating highly complex issues involving moral and ethical considerations. Less attention has been paid to how specific scientific understandings are used in decision-making and to how to characterize student progress in this area.

Given the scarcity of research results that can inform the development of instructional models and practices to scaffold meaningful application of chemistry knowledge in decision making, the central goal of this research project was to investigate students' reasoning when engaged in a problem that demanded evaluation of the benefits, costs, and risks of using different chemical products for a specific purpose. In particular, we wanted to characterize, compare, and contrast the reasoning of individuals with different levels of training in chemistry, from undergraduate students to graduate students to practicing chemists. We were motivated by the belief that, in order to align science education with current educational standards in the US (NRC, 2011, 2013), as well as in other countries (Waddington et al., 2007; Osborne and Dillon, 2008), we need to better characterize progress in the ability of students to integrate different types of knowledge (Clark and Linn, 2003; Corcoran et al., 2009). In our study, we focused on characterizing different degrees of sophistication in the completion of a task that demanded the application of chemistry concepts, together with judgment and consideration of potential environmental, safety, and health benefits, costs, and risks of chemical products and activities. We used an analytical framework that, rather than focusing on the extent and appropriateness of the content knowledge demonstrated by our study participants, centered on the identification of underlying ways of thinking that seemed to guide decision making. Our approach relied on participants' choices and justifications to infer implicit ways of conceptualizing chemical entities and phenomena that may support or constrain student reasoning. Our findings provide insights into implicit assumptions and modes of reasoning that need to be effected to facilitate the meaningful use and integration of chemistry knowledge while making benefits-costs-risks decisions.

Benefits-cost-risks (BCR) decision making

Existing research on students' judgment and decision making regarding benefits, costs, and risks in chemistry is scarce. Science education researchers have studied how students understand chemical processes in the context of modern concerns that involve risks and benefits, including carbon cycling (Mohan et al., 2009), climate change (McNeill and Vaughn, 2012), hazardous waste management (Malandrakis, 2008), biotechnology (Dawson and Venville, 2009), and nuclear power (Kilinç et al., 2013). Risk psychology researchers have also studied how people consider risk, particularly expert-lay discrepancies, in the context of chemical problems, such as hazardous waste cleanup (Siegrist and Cvetkovich, 2000), health risks with chemical exposure (MacGregor et al., 1999), pesticide use (Williams and Hammitt, 2001), water quality (Dobbie and Brown, 2013), and nanotechnology (Becker, 2013). Risk perception and cost-benefit analysis are also areas of study within food science and nutrition, with studies that include considering chemical composition in food quality (Dickson-Spillmann et al., 2011) and additives and supplements (Devcich et al., 2007).

Results from the above research studies suggests that people exhibit strong preferences or biases in BCR decision making. For example, individuals are known to prefer products and processes considered to be “natural” over those judged to be artificial (Rozin, 2005). Brun (1992) found that people classify hazards according to this scheme, and ascribe less risk to natural hazards than to those that are manmade. People tend to perceive “chemicals” as artificial or manmade, and often attribute a negative connotation to them. In an interview study of over 26 000 European citizens across all 27 European member states, the Eurobarometer project assessed people's perceptions of chemical products (Joint Research Centre, 2011). Respondents generally considered chemicals to be “dangerous or harmful to the environment, rather than useful or innovative” (p. 11). Dickson-Spillmann et al. (2011) found that people often assume that when chemicals are added to food, the food has greater potential for detrimental health effects. Individuals who have a greater affinity for “natural” food are more likely to hold negative attitudes toward “chemicals.” In general, natural substances and processes are often linked to a subjective impression of goodness, while the products of human intervention are frequently judged more negatively (Rozin, 2005). Such beliefs influence people's arguments and decisions in many areas of current interest, such as bioethics and gene therapy (Nielsen, 2012).

Research in BCR decision making has revealed that laypeople's judgments are influenced not only by the knowledge or the information they have, but also by the feelings evoked by what they perceive. The positive or negative emotions prompted by words, images, objects, or events affect judgments regarding benefits, costs, and risks, influencing people's preferences and choices (Slovic, 1987; Finucane et al., 2000). The use of readily available affective impressions to make decisions (usually referred to as “affect heuristic”) can be easier and more efficient than weighing multiple pros and cons, but may also lead to irrational choices (Slovic et al., 2003). In the area of risk perception, two primary factors are thought to influence laypeople's affective impressions (Slovic, 2010): “dread risks” which are characterized by how much a person perceives there to be a lack of control, dread, catastrophic potential, fatal consequences, and the inequitable distribution of risks and benefits; and “unknown risks” which are characterized in terms of a person's assessment of how unobservable, unknown, new, and delayed the risk is in its manifestation of harm. In general, perceived benefit and perceived risk are inversely correlated in people's minds. In contrast, experts' perceptions of risk are more closely related to objective evaluations of probability of harm.

Recent studies in the area of argumentation of socio-scientific issues also provide important insights into students' BCR reasoning. Research by Kahan et al. (2011) suggests that individuals often selectively credit or dismiss evidence of benefits, costs, and risks based on personal values that they share with others rather than on scientific knowledge. In the context of science education, science learners have been found to rely on emotive, intuitive, and rationalistic resources when analyzing socio-scientific issues, independently of their level of content knowledge about a subject (Sadler and Zeidler, 2005; Sadler and Donnelly, 2006). Students' abilities to generate high-quality BCR analyses seem to vary in a non-linear fashion with content knowledge acquisition (Sadler and Fowler, 2006). Comparative analysis of decision-making skills between novice students and experts suggests that students' decisions tend to be less integrative, and focused more narrowly on particular themes (Hogan, 2002). Novices' decision-making is affected by the use of cognitive heuristics known to bias judgment under conditions of uncertainty, limited time and knowledge, or low motivation to complete a task (Acar et al., 2010).

Levels of sophistication

In recent years there has been a surge of interest in the development of frameworks or approaches to characterize different levels of sophistication and complexity in student knowledge and reasoning in a given domain. Such is the case of research studies in the area of learning progressions (LPs) (Corcoran et al., 2009; Alonzo and Gotwals, 2012). These LPs describe successively more sophisticated ways of thinking about a topic and are based on educational research about how people learn and existing pedagogical content knowledge in the area of interest, as well as on the critical analysis of the structure of the associated disciplinary knowledge (Duschl et al., 2011). Their development demands a solid understanding of students' ideas and their likely changes with instruction. LPs are expected to serve as curriculum models and assessment frameworks, guiding curriculum development as well as instructional and assessment practices to foment more meaningful learning, clearer standards of learning progress, and more useful formative feedback (Wilson, 2009).

Researchers have sought to characterize different aspects of students' knowledge in the development of LPs, from understanding of core ideas to ability to engage in science practices (Duschl et al., 2011). Existing learning progressions often describe sequences of ideas or practices representing successively higher levels of understanding (Mohan et al., 2009; Stevens et al., 2010). In some cases, authors have tried to characterize learning progressions not only in terms of the conceptual sophistication of students' knowledge, but also on the validity of students' reasoning (Brown et al., 2010). These more integrative approaches to the analysis of progression in student understanding can also be found beyond existing work on LPs. For example, in the SOLO taxonomy defined by Biggs and Collis (1982), student responses are allocated to a hierarchy of stages (e.g., prestructural, unistructural, multistructural) depending on the number and level of integration of knowledge elements involved in reasoning. This taxonomy has been used as a reference by Claesgens et al. (2009) to define and measure performance levels in students' understanding of chemistry, and by Bernholt and Parchmann (2011) to assess levels of achievement in science domains. Other scales have been proposed to differentiate how learners use knowledge of different complexity in various contexts (von Aufschnaiter and von Aufschnaiter, 2003), or the extent of knowledge integration as determined by the accuracy and cohesion of students' explanations (Clark and Linn, 2003).

Common approaches to mapping progression in student learning tend to focus on the elicitation of explicit understandings that students may demonstrate at different educational stages. For example, these types of studies reveal that when learning about structure of matter, students may first recognize that all matter is made up of atoms and later acknowledge that atoms have an internal structure (Stevens et al., 2010). We have proposed that the analyses of learning progressions could be enriched by also paying attention to changes in implicit assumptions about the nature of entities and processes that often constrain student reasoning in a domain (Talanquer, 2006, 2013a; Ngai et al., 2014; Sevian and Talanquer, 2014). Thus, for example, in reasoning about the structure of matter novice students may implicitly assume that matter is homogeneous at all scales, which will likely lead them to associate macroscopic properties to submicroscopic particles (Talanquer, 2009). Meaningful progress in this area can be achieved when individuals assume that properties vary at different scales and that new macroscopic properties may emerge from interactions between components at the submicroscopic level (Talanquer, 2008, 2015). A focus on “assumptions” directs attention to implicit ways of thinking that may either support or interfere with the types of understandings that would benefit students to develop. Different authors have highlighted the importance of better characterizing the implicit knowledge that guides student reasoning (Taber, 2014), from phenomenological primitives (diSessa, 1993) to ontological categories (Chi, 2008) and presuppositions (Vosniadou, 1994). It is also critical to analyze progress in terms of the implicit reasoning strategies, or modes of reasoning, that learners apply to build explanations and make decisions based on their prior knowledge and available contextual information (Sevian and Talanquer, 2014). Existing research suggests that novice learners, for example, tend to overuse information that is easily accessible, ignoring other relevant considerations and using fast and frugal heuristics to make decisions (Gigerenzer and Gaissmaier, 2011; Kahneman, 2011). Advanced learners, on the other hand, tend to apply more diverse modes of reasoning, from analogical reasoning (Goswami, 2013) to different models of causal reasoning (Perkins and Grotzer, 2005; Brown and Wilson, 2011). Mapping differences in assumptions and modes of reasoning expressed and applied by different learners enriches our understanding of learning progressions in any given domain.

Research goals

In this study we focused on the analysis of the assumptions and modes of reasoning expressed by individuals with different levels of training in chemistry when engaged in a task that demanded the evaluation of the benefits, costs, and risks of using different chemical substances. Chemists must be able to evaluate the consequences of both using and producing chemical substances, which encompasses consideration of both chemical entities and processes. To make this study more relevant to all participants, including both students majoring in chemistry and students taking chemistry but majoring in other sciences, we chose to focus data collection on the evaluation of using chemical substances rather than on producing them. We were interested in identifying implicit assumptions and modes of reasoning that can be used to characterize different levels of sophistication in BCR reasoning in contexts where the application of chemistry concepts and ideas is relevant. Analysis of student BCR thinking in chemistry is very limited and our study aims to provide insights into how to better scaffold student learning in this area.

Methodology

Setting and participants

Participants were recruited from a medium-sized, non-traditional university in the Northeastern US. They included: 11 college freshman students (9 female, 2 male) enrolled in an introductory General Chemistry course; 11 sophomore or junior students (6 female, 5 male) enrolled in Organic or Analytical Chemistry courses; 7 senior students (4 female, 3 male) in their last year of college studies; 5 chemistry graduate students (3 female, 2 male), and 5 chemistry professors (all male) from the same institution. In accordance with the institution's IRB approval, student participants were volunteers contacted via their research advisors or in class, with the instructor's consent, and were offered small denomination gift cards or course extra credit. Participants' racial ethnicity was representative of the university's population: 46% Caucasian, 7% African American, 11% Asian, and 36% from other ethnicities. For reference and privacy purposes, a label was assigned to each individual based on their level of training in the discipline (freshman, F; sophomore/junior, SJ; senior, S; graduate, G; professor, P) and their position on an interview list. For example, the second freshman on this list was assigned the label F2.

Data collection

Our exploratory study relied on individual semi-structured interviews as the main strategy for data collection. Interviewees were presented with a scenario in which they were asked to select one of four available fuels to power a GoKart for an amusement park.† We have reported elsewhere on the design of the instrument (Szteinberg et al., 2014), but summarize key features here for convenience. Instrument design went through several iterations of pilot testing and revisions. The instrument was designed to present a problem that involved consideration of core chemistry concepts, such as the relationship between composition and structure and physical and chemical properties, as well as other factors (e.g., environmental, health, safety). We chose to present a scenario that, although artificial, would create opportunities for all of our participants to make decisions based on different types of prior knowledge about the selected energetic resources (e.g., methane, ethanol, octane). We introduced some artificial constraints, like assuming similar costs for all available fuels, and made some simplifications, like indicating to participants that each fuel is primarily composed of a single substance, to reduce the number of factors that could be considered. Our goal was to use a task that was understandable and manageable for participants from all educational levels and that was representative of decision-making activities that instructors may potentially implement in different types of chemistry classrooms. We also expected individuals demonstrating a more sophisticated approach to the analysis of the task to spontaneously highlight and discuss the artificial elements in our scenario. A key feature of the instrument is that there is no one right answer, given that relevant factors may be weighed in different ways depending on personal values and context.

The interview protocol was designed to first freely explore the factors that influenced participants' judgments and decisions without explicit information about the chemical nature and properties of the available fuels. Then, the interviewer gradually provided data about the physical and chemical properties of the fuels (e.g., state of matter, chemical composition, and molecular structure) to investigate the extent to which interviewees' judgments and decision making were influenced by chemical information that they might have not considered in their initial evaluations. A total of 39 participants were interviewed over the course of six months. Interviews lasted between 15 and 30 minutes.

Data analysis

Individual interviews were audio-recorded and then transcribed. We applied an iterative, non-linear constant comparison method of analysis (Charmaz, 2006), using qualitative analysis software to facilitate the analytical process. Interview transcripts were first analyzed to identify the features noticed and used by different participants in making their decisions. These features were grouped into different categories such as “Common Use” (paying attention to the typical use of a substance in daily life), “Origin” (paying attention to the source of the fuel), “Molecular Size” (referring to the length of fuel molecules), “Bonding” (referring to the number or types of chemical bonds present in molecules). We used these noticed features to identify implicit assumptions that guided participants while making decisions and justifying their choices. These implicit assumptions were inferred from students' justifications and explanations by analyzing the types of cues participants paid attention to, the verbal predicates that they used, and the nature of the claims that they made. For example, if a student paid attention to the presence of oxygen in the ethanol molecule and claimed that ethanol was more flammable because it had oxygen and oxygen was needed for combustion, we inferred that this student assumed that a substance could be expected to exhibit similar properties as its components. This approach to inferring implicit knowledge elements has been used by a variety of authors (Keil, 1979; diSessa, 1993; Slotta et al., 1995). Inferred assumptions were grouped into various categories such as: assuming that the origin of a substance affects its properties; assuming that the properties of a substance are directly determined by the properties of its components; assuming that bigger entities contain more energy.

We also sought to characterize the modes of reasoning used by study participants to make decisions. To this end:

(a) We paid close attention to how participants used prior knowledge and contextual cues to select one fuel over another and to justify their choices. We used this information to infer the extent to which participants relied on heuristic reasoning in decision-making. For example, some study participants quickly made their selection based on the recognition of one of the substances as a fuel while the other substances were unknown or thought to have other uses (e.g., ethanol is used for sanitizing wounds). No other justification besides recollection of or familiarity with one of the substances was offered to justify the choice. This pattern of reasoning suggested the application of a “recognition heuristic” commonly stated as follows: If one of two objects is recognized and the other is not, then infer that the recognized object has the higher value with respect to the criterion (Gigerenzer and Gaissmaier, 2011).

(b) We carefully analyzed the types of relationships that participants built between concepts, seeking to characterize the complexity of their reasoning. We considered the types of entities our participants paid attention to, the types of properties they assigned to these entities, and the types of relationships or interactions they assumed between them. This analysis allowed us to determine whether justifications and explanations were based on the construction of simple associations between two features (e.g., the larger the molecule the more energy it contains), simple causal chains (e.g., a larger molecule contains more carbon atoms that can react with oxygen to produce CO₂ upon combustion), or multicomponent causal relationships (e.g., a larger molecule contains more carbon atoms that can react with oxygen to produce CO₂, which will result in the production of more greenhouse gases but also more energy per molecule). This type of analysis is similar to that used by other authors to characterize the extent to which students engage in mechanistic reasoning in which specific entities and their interactions and activities are used to build explanatory accounts (Perkins and Grotzer, 2005; Russ et al., 2008; Brown et al., 2010; Brown and Wilson, 2011).

All of the different elements used to characterize participants' BCR reasoning were ordered from least to most sophisticated, looking to identify and characterize different levels of sophistication in making decisions about what fuel to use. Given our research goals, we paid particular attention to the extent and quality of the chemistry concepts and ideas expressed and applied by study participants. The initial set was generated by organizing conceptual sophistication into three levels according to Sevian and Talanquer (2014): (a) intuitive (making judgments based on everyday experiences and intuition), (b) hybrid (relying on a combination of intuitive judgments and academic knowledge often used inappropriately), and (c) academic (using appropriate and relevant academic knowledge to make judgments). Once an initial set of levels was generated, we proceeded to assign study participants to the level that best represented their expressed BCR reasoning. This categorization effort led us to refine the identification and description of characteristic patterns of reasoning at each level of sophistication.

To ensure inter-rater reliability, all transcripts were coded by at least two of the authors, and half were coded by all three authors. In this manner, codes generated by an individual researcher in any given category (e.g., features, assumptions, modes of reasoning) were reviewed by at least one other author. All discrepancies were discussed and resolved satisfactorily. Once a coding system was generated that was considered sufficiently comprehensive, it was consistently applied to all the transcripts. A similar procedure was used to order different features from least to most sophisticated, and in assigning participants to different levels of sophistication.

Findings

Analysis of the types and number of features noticed and used to make decisions, together with the implicit assumptions and reasoning strategies applied by the participants when selecting the best fuel for a GoKart, allowed us to identify different levels of sophistication in BCR reasoning. The general characterizations of the three levels (Intuitive, Hybrid, Academic) by Sevian and Talanquer (2014), as described above, were found to be sufficient. However, we have modified the names of these levels to Intuitive, Mixed and Normative, to correspond to vocabulary already present in the literature, as we found that the three general levels of sophistication share similar characteristics with levels of knowledge integration identified by Clark and Linn (2003). The distribution of participants into these three major levels of sophistication is shown in Table 1, where F – freshman, SJ – sophomore and junior, S – senior, G – graduate student, and P – professor.

Table 1 Distribution of study participants among different levels of sophistication in BCR reasoning

Educational level	Number of participants
	Intuitive	Mixed	Normative
Freshman (F; N = 11)	8	3	—
Sophomore/junior (SJ; N = 12)	2	8	1
Senior (S; N = 6)	2	3	2
Graduate (G; N = 5)	—	—	5
Professor (P; N = 5)	—	—	5

---

In general, participants with little or considerable training in chemistry tended to demonstrate, respectively, low or high levels of sophistication in BCR reasoning. The distribution of students in the intermediate stages of training was somewhat broader, although many of them demonstrated a “Mixed” level of sophistication. The boundaries between the different levels in BCR reasoning identified in our study were not sharp. Within any given level, we found participants who exhibited different degrees of sophistication. One could thus expect to find individuals whose reasoning falls near the boundary between any two levels. Overall, our categories highlight three approaches, from least to most sophisticated, in BCR thinking that provide insights into how knowledge and reasoning progress in this area.

Intuitive BCR reasoning

Close to one third of our study participants (12 of 39) exhibited an intuitive approach to BCR reasoning. Two thirds of these individuals were freshman chemistry students; no graduate students or chemistry professors fell within this category (see Table 1). Study participants placed at this level mostly relied on intuitive knowledge and ways of reasoning, rather than on chemical concepts and ideas to make judgments and decisions. Common patterns of reasoning applied by individuals in this group are described in the following paragraphs. These patterns of reasoning are conceived as emerging from the interaction of implicit assumptions about relevant substances and processes and particular modes of reasoning about them.

Recognition of substances. The decisions made by students at the intuitive level of BCR reasoning were strongly influenced by their experiences with the different fuels included in the GoKarts instrument. Recollection of or familiarity with the name, use, or effects of a substance were frequently used as main criteria to select or exclude options. In particular, recognition of the uses of a fuel was used by several of these study participants (4 of 12) as a heuristic to discriminate between options and make a first choice during the interview. Consider, for example, the following interview excerpt:

I: Which one would you choose?

F7: (pause) The gasoline from petroleum.

I: Okay. And why?

F7: Because that's what I put in my car I think.

In this case, the first spontaneous choice of this participant was based on the recognition of gasoline as the fuel commonly used in cars. As illustrated by the excerpt below, known use of a substance not only influenced first choices but also led some students to assume that the known fuel was easier to obtain, had wider availability, or greater efficiency than other options:

I: Which one would you choose?

F5: I would choose octane.

I: Why would you choose octane?

F5: It's the most commonly used, so therefore it's easier to obtain.

I: Ok, and which octane would you choose? The one from wood pellets or petroleum?

F5: From petroleum.

I: Why is that?

F5: (pause) I guess ‘cause petroleum is more commonly known, so basing it on common knowledge I would probably go with petroleum, yeah.

Many participants at this level of BCR reasoning (8 of 12), used recognition of the known effects of a fuel to either select it or exclude it. Consider this excerpt:

I: Which fuel would you choose?

SJ10: Um… gas is used in like modern day vehicles, and machines and such, I would probably go with the first one [gasoline from petroleum].

I: Okay, gasoline from petroleum? Is there any other reason?

SJ10: Um, for one I don't know. I always associated this one, the methane, to be like harmful, so I kinda would avoid using that.

As was the case in the previous examples, the initial choice of this student was guided by recognition of common uses of gasoline, but the exclusion of methane was influenced by the association of this substance with perceived harmful effects. Other participants in this group related methane with “less pollution” and used this positive association to choose this substance as the best fuel option. In general, affective associations between fuels and their perceived effects played a major role in decision-making at this level.

Affective associations. The BCR reasoning of all of the students at the intuitive level was strongly influenced by their beliefs about the environmental impacts of the different fuels under analysis and, to a lesser extent (4 of 12), by their potential effects on human safety. However, the views of intuitive thinkers about environmental effects were often based on a simple association between fuel consumption and the production of entities (e.g., carbon, hydrogen, carbon dioxide, or pollutants in general) that, as illustrated by the following excerpt, were judged to be bad for the environment:

I: So, can you explain to me more why you would choose E85?

F3: Um it's because of how similar the first three are. They're all made from carbon and hydrogen, and um, I guess that would be really bad.

I: And why do you say it would be really bad?

F3: Just based off, like, because gasoline is made of carbon and hydrogen and we know that like, gasoline is bad for the environment. So if it produces carbon [and] hydrogen, then I guess anything else that produces carbon and hydrogen would also be bad for the environment.

I: Ok, so the E85… can you explain to me what about the E85… it also has carbon and hydrogen, so why won't it produce just as much emissions as the other three?

F3: I guess because of the oxygen in it. It kinda like mixes things up. It's like a different option I guess for me. I'd hope that it'd be better.

I: Uh huh. And can you explain to me what about the oxygen might make it better?

F3: Um, I wouldn't know really. It's just like… it's process of elimination. Like those three would be so bad I'll take any other fourth option.

In this example, the student thought of gasoline as something “bad” for the environment and associated similar negative effects to its components (i.e., carbon and hydrogen), choosing the option perceived to be less harmful based on similarity judgments. Intuitive BCR thinkers often used affective impressions triggered by the names or the representations of different substances to make decisions. Their reasoning seemed constrained by an “affect heuristic” (Slovic et al., 2003) in which positive or negative impressions prompted by words or images guided their judgments regarding benefits, costs, and risks. These participants' ideas about the properties of chemical components were mostly based on naïve affective associations, such as thinking that carbon was somehow “bad” because of hearing that CO₂ polluted the environment, or considering oxygen as “good” because oxygen was somehow pure, better for nature, or easier to burn (oxygen was also seen as “bad” by some, because it was more flammable or could lead to more CO₂ production). The following interview excerpt illustrates this type of reasoning:

SJ10: I mean, I guess with this one having the oxygen, still probably makes it a safer option.

I: When you say safer, what do you mean?

SJ10: Um, safer in terms of if it were to somehow come in contact with you… like it would be harmful… um… because obviously, just even in everyday uses like if you were to use gas for cars, it would, the fumes, the fumes it gives off are not safe. You know what I mean? I don't know [if] ethanol would make any difference in that, but just being that it contains oxygen makes me want to say that it is probably a little safer.

Knowledge or belief about the origin of the fuels was also an important influencing factor in the reasoning of individuals at the intuitive level (8 of 12). In general, the perception that a substance was “natural” triggered positive affective associations that led some students to favor that fuel over other options. As illustrated by the excerpt below, some students thought natural substances would produce less toxic pollutants:

F9: I don't really know about the E85, but I would just choose the natural gas. (pause) I remember in high school I knew that burning a whole bunch of stuff it releases all these toxins into the air and then it destroys our ozone layer and stuff like that. So if we were to use a natural resource instead of like something that would make our atmosphere like not good, it would be better to use.

In other cases, students simply referred to some sort of “gut feeling” about a choice without any further explanation:

I: Which fuel do you think would be the best fuel of the four?

F7: …(long pause)… Methane? I don't know.

I: Okay. Why methane?

F7: Because it says natural gas.

I: Okay. So because it's natural?

F7: Yeah. And I feel like if you have something like…in its natural state, that it …(long pause). I don't know how to like explain what's going on in my head. I don't know. It says natural. (laughing) It seems better for the environment.

Perception of the “naturalness” of the fuel's source had a similar effect on the choices made by some of these students. Preference for the “natural” was often justified with claims about lesser impact on the environment or on human health. This type of reasoning has been shown to influence BCR judgment in various areas (Slovic et al., 2003; Rozin, 2005).

Additive view of matter. During the interview, participants received information about the chemical composition and molecular structure of the substances under consideration. All of the students at the intuitive level of BCR reasoning acknowledged some of this information as relevant for their decisions, but used it in rather naïve ways. Most of these participants seemed to conceive chemical compounds as simple mixtures of elements, assuming that fuel properties would be determined by the inherent properties of individual components. This way of thinking is illustrated by the following interview excerpt:

I: You're gonna go with gasoline from petroleum?

SJ5: ‘Cause…just ‘cause oxygen sounds like it might be more flammable, so let's go with this one

I: So you don't want a fuel that's flammable?

SJ5: Well, I mean, all fuels are flammable, just ‘cause, I don't know, when I saw oxygen I felt like it sounds a little bit more dangerous, I guess, than regular fuel. Like something happens it might help be like more flammable than this one.

I: Okay. So ethanol would be more flammable than octane?

SJ5: Yes

This student thought of oxygen as a flammable substance and associated the same properties to ethanol based on the presence of oxygen in the formula and structure of the compound. The amounts of different components present in a chemical formula or a molecular structure were also used to make claims about the advantages of using one fuel over the other. Smaller molecular sizes were perceived by some as beneficial because they would lead to less CO₂ formed, or as less convenient by others because smaller particles would be consumed faster. Consider the following excerpt:

F4: Um…longer might mean that the fuel… lasts longer, perhaps, or it has a different, um, different way of, efficiency maybe. Maybe that's like um, maybe a car runs longer, a longer time with octane than methane.

The amount and diversity of components was also used as a cue to make judgments about how easy or difficult it would be to produce or process the fuel, as illustrated below:

F1: Well, I would say that less is generally better because I'm guessing there are different processes for isolating the carbon, hydrogen, and oxygen that would be used for each type of fuel. So I'm guessing this [ethanol] would need three processes and these [methane, octane] would need only two.

In general, students at this level of BCR reasoning seemed to implicitly assume that the properties of chemical compounds were similar to the properties of their components, and that the type, number, or size of the components in the molecules of a substance were indicative of how easy or difficult it was to make, process, or burn them. These study participants tended to “objectivize” chemical substances, thinking of them more as objects than as chemical entities (Krnel et al., 1998), and thus compared fuels using features or properties commonly applied to differentiate between objects, such as type and amount of components. This “additive” view of chemical substances has been elicited in other studies involving novice chemistry students (Talanquer, 2008, 2013, 2015).

Absence of mechanistic reasoning. In general, intuitive BCR thinkers expressed little knowledge about the causes and mechanisms underlying combustion or pollution processes. Their knowledge about the conditions, entities, interactions, and processes involved in the associated chemical processes was minimal. They often knew that the use of different fuels led to pollution, but did not know much about the mechanisms for either pollution generation or pollutants' action on the environment. Consider the following interview excerpt:

I: Can you explain to me what you think pollution is?

F3: Um. That's a great question. Wow. So, uh, pollution I think would be when there's any emission of carbon dioxide, I think, in the air that would damage the ozone layer, and that would be considered pollution.

I: And where does that carbon dioxide come from?

F3: Um… from the burning process of the octane and the methane I guess, like, carbon to oxygens? Wait. No… I don't know. That's a great question.

As this excerpt illustrates, most students at this level could not explain what happened when fuels were burned and expressed limited views about the generation and effects of pollutants in the environment. Their claims often focused on identifying the fuel that would generate more pollution, or more dangerous pollutants, based on the analysis of fuel components (e.g., the fuel with more carbons in its formula will produce more CO₂) and not on the analysis of relevant interactions, such as those between fuel and oxygen or between combustion products and other substances in the surroundings. As shown in several of the interview excerpts included in previous paragraphs, intuitive BCR thinkers often built justifications by assuming that substances, or their components, had some inherent property that made them appropriate or not for the targeted purpose (e.g., oxygen is flammable and thus it is not safe; carbon is toxic). This mode of reasoning is similar to the “inherence heuristic” described by Cimpian and Salomon (2014), who proposed that people tend to explain observed patterns in terms of the inherent features of their constituents.

In general, intuitive BCR thinkers tended to rely on relational reasoning rather than on mechanistic reasoning, using vague associations between the name or composition of entities and their expected properties (e.g., gases are dangerous, oxygen is good for us) to guide their thinking and justify their choices. The lack of mechanistic considerations was complemented by an absence of analysis of energy issues when making judgments and decisions. Participants at this level did not seem to have an understanding of how energy was generated through the combustion process, and their decisions did not involve energetic issues (e.g., energy costs of producing, processing, and transporting fuels; energy generated during combustion). The reasoning of these participants was highly sensitive to the information presented to them throughout the interview, as the recognition of some features triggered associations that led some students (5 of 12) to change their choices or question the appropriateness of their prior selections. In general, intuitive BCR reasoning was more reactive than proactive, and more hesitant than purposeful. Individuals at this level expressed few ideas of their own and mostly reacted to the information presented to them, expressing doubts about the validity or appropriateness of their judgments.

Mixed BCR reasoning

Over one third of our study participants (14 of 39) expressed a mix of intuitive and academic ideas, the latter often spurious, when engaged in BCR reasoning. Over half of these students (8 of 14) were at the sophomore or junior levels in their undergraduate chemistry studies; no graduate students or chemistry professors fell within this category (see Table 1). Individuals at the mixed level of sophistication frequently relied on ideas and ways of thinking similar to those characteristic of the intuitive BCR thinkers, but their reasoning was enriched by academic knowledge about chemical substances and reactions, and their ability to apply such knowledge in proper and productive ways was limited. Many of these students were also hesitant about their answers, were more reactive than proactive in the generation of ideas, and changed their fuel choice (7 of 14) as information was presented to them. Two major patterns of reasoning differentiated this group of students: (a) the presence of “hybrid” conceptions in which chemical concepts were conceptualized in intuitive ways; and (b) the construction of mechanistic links based on chemical knowledge often used in a spurious manner. These two patterns of reasoning were often related to each other making it difficult to describe them separately. Thus, in the following paragraphs we present examples of how both manifested in different contexts.

As with intuitive thinkers, familiarity with known uses and effects of the different fuels under consideration played an important role in students' BCR reasoning at the mixed level. Many students in this category also relied on vague recollections of environmental and safety issues to choose or discard a substance. However, as illustrated by the following interview excerpt, they often made attempts to build mechanistic links between expected properties of a substance and its chemical composition:

I: Okay so, you mention explosion from the octane and then combustion. Okay so talk to me a little more about that.

SJ1: Mmmm, I think they would just react quicker, I don't know why but I think with ethanol, it can sustain more than the rest of them. Well, I was kind of thinking of their chemical structures because octane is just carbon–hydrogen bonds and then methanol has the OH, I mean ethanol has the OH attached to it, so it can sustain more for boiling and melting than the other ones. So I think those ones might be like the first ones to blow up or something as opposing to the ethanol, which would take more to do.

I: So are you thinking of safety… because it would take longer to combust?

SJ1: Yeah, yeah. I think so.

In this case, the student was trying to relate the composition and functionality of ethanol to its potential reactivity (facility to explode). As was common among participants at the mixed level, this student was seeking to apply chemical knowledge to build mechanistic explanations (e.g., substances with OH groups can form hydrogen bonds and may thus have higher boiling points), but the arguments were frequently incomplete, as in the above example, incorrect, or somewhat irrelevant to the problem under consideration.

Attention to chemical composition was also substantial among participants at the mixed level, with arguments mostly focused on the nature and amount of different components. The assumption that inherent properties of the individual components determine the properties of the chemical compound (additive view of matter) was still pervasive, but properties discussed were linked to specific ideas about how those properties affected the combustion process. For example, some students thought that the presence of oxygen would make ethanol more combustible:

I: So you said the oxygen might make it more combustible?

F8: Yeah, ‘cause you don't, that's why you see O₂tanks like, don't go, don't put near flames, very flammable, because it's easily combustible.

Other participants recognized the presence of specific groups of atoms in molecules (5 of 14) and made claims about how these functional groups could alter the combustion process:

I: Mhm. Why would ethanol work best?

SJ4: Because it could react with other things that have OH groups and NH groups, and it could, I think it would have a cleaner, the reactions would probably be… they could potentially be cleaner than the reactions of octane or methane where you might have, in the other ones you might have harmful side um products as well as the energy, and the OH would probably yield less of those harmful, uh, byproducts.

In this case, the student seemed to claim that substances with OH are somehow “cleaner” or less harmful than others because they may generate less harmful byproducts. This last example illustrates how some students attempted to build mechanistic explanations and often “hybridized” their chemical knowledge with intuitive ideas about the nature of chemical substances (i.e., oxygen-containing entities seen as clean, not harmful chemical substances).

While intuitive BCR thinkers did not pay attention to energy issues in their selection of the best fuel, and most of them had little understanding of the burning process, participants at the mixed level often made many references to factors affecting energy production and the combustion process. They referred to factors such as energy costs (5 of 14), energy produced (6 of 14), and energy content (6 of 14) for the different fuels. Most individuals recognized that energy was generated as a result of a combustion reaction, although their understanding of such a process was, in most cases, incorrect or incomplete. Students in this category often linked physical or chemical features of the different substances with the amount of energy required or produced during combustion. However, student thinking about chemical energy was naïve, based on an assumption that chemical bonds contain energy that is released when the bonds are broken (Boo and Watson, 2001; Kind, 2004). Students again expressed “hybrid” conceptions in this area, conceptualizing relevant chemical concepts (chemical bond and bond energy) in rather intuitive ways (e.g., energy as fluid-like entity that can be contained and released). The following excerpt illustrates this type of reasoning:

S5: Maybe the smaller they are it's easier to burn them. It takes less time.

I: Why is that?

S5: ‘Cause it's easier to like break the bonds.

I: Of a smaller molecule?

S5: Yeah. But they can release less energy, so…

I: So breaking the bonds releases energy?

S5: Yeah.

In this example, the student struggled to decide between competing intuitive ideas about burning: smaller molecules are easier (faster) to burn, but they produce less energy when their bonds are broken. Other students struggled with other competing ideas, such as assuming that smaller molecules require less energy to break apart, or produce less CO₂ (less pollution), but they also generate less energy. Some of these participants also expressed misunderstandings about energy exchanges, such as believing that the more energy is invested in burning a fuel, more energy will be released upon combustion:

SJ7: I think given this information I might choose the natural gas…well, I think I would probably still stick with the gasoline from wood pellets because it would require more energy to combust it, the gas would be more easy to combust, so that might make it… less of an energy output.

Participants at the mixed level also relied on unproductive strategies to compare inputs and outputs in the combustion process (e.g., energy released versus amount of CO₂ produced), paying little attention to the specific constraints of the system under analysis (i.e., fuel tank with a fixed volume). They tended to compare one single molecule with another (as represented in the images presented during the interview), without ever questioning whether other approaches (e.g., comparing fuel samples of equal mass) would be more appropriate.

As with intuitive thinkers, BCR reasoning for most of the participants at the mixed level (12 of 14) was dominated by concerns about the environmental impact of using the different fuels. However, more than half of these students (8 of 14) also referred to human safety issues. Arguments about environmental and safety impacts tended to be weak, as students relied on generic associations (e.g., gases are more flammable and explosive than liquids, CO₂ is bad for the environment) to justify their choices. Considerations about the origin of the fuel were also important for this group (9 of 14), but affective associations were less prevalent than among intuitive thinkers.

Normative BCR reasoning

One third of our study participants (13 of 39) appropriately applied both their general academic knowledge in chemistry and their specific knowledge about the fuels under consideration to make judgments and decisions. Only three participants in this group were undergraduate juniors and seniors; the rest were graduate students and chemistry professors (see Table 1). There were major differences between the knowledge and ways of thinking expressed by individuals at the normative level and those at the intuitive and mixed levels of BCR reasoning:

(a) Participants at the normative level demonstrated a relatively broad knowledge base about fuels, their production, properties, and effects, and they proactively recalled information and generated ideas that allowed them to differentiate one fuel from another.

(b) They applied scientifically correct chemical knowledge that was relevant to the task at hand.

(c) They approached the decision-making process by weighing several factors before settling on a particular fuel option. They frequently evaluated pros and cons of different alternatives based on various criteria (e.g., energy vs. amount of CO₂ produced), and recognized that their choice could be different if they changed the weight given to some factors over others (e.g., safety over engine power) or had access to additional data.

(d) Normative BCR thinkers built one or more causal links between fuel characteristics and potential impacts, availability and management issues, and energy production.

(e) They paid attention to contextual factors in making judgments and reflected on the impact of “artificial” task elements in their decisions.

(f) They recognized the need for data or more information (e.g., heat of combustion values) to make more definitive decisions.

Specific examples of these major patterns of reasoning are presented below.

As in other levels, participants at the normative level of BCR reasoning paid attention to fuel characteristics related to known use and effects, origin, and chemical composition and structure. However, they expressed more extensive and sophisticated knowledge about relevant features than individuals at the mixed and intuitive levels. They often considered more than one factor at a time when making evaluations, as illustrated by the following interview excerpt:

G3: All these are gonna produce greenhouse gases so in some sense you are not eliminating that possibility. So gasoline whether you get it from petroleum or wood pellets, is still gonna be the same. The only difference between those two is where you're sourcing it from, so wood pellets you can say it's a sustainable resource, so gives it an edge over from petroleum but in the end you're not getting any benefits. Natural gas, it is still a non-renewable resource but on the exhaust side it's gonna be better than gasoline. Ethanol… I don't think it's very efficiently produced. If produced from corn, it's not economical or environmentally sound as much as we'd like to think. On the exhaust side I think you're still gonna produce CO₂. I think that on the exhaust side, the methane, natural gas will give you a better environmental footprint.

In this case, the graduate student was trying to weigh issues related to origin versus environmental effects of the fuels under consideration. This excerpt also illustrates the ability of participants at the normative level to recognize that judgments and decisions depended on a variety of factors that were not defined in the GoKarts probe, such as the source used to produce ethanol (e.g., corn vs. sugar cane) or the nature of the process needed to generate octane from wood pellets (which could be energetically and environmentally costly).

Normative BCR thinkers considered physical (e.g., states of matter) and chemical (e.g., chemical composition and structure) characteristics in ways that reflected a deeper understanding of the properties and transformations of matter. For example, several of these individuals (6 of 13) recognized that natural gas could be pressurized or liquefied (or that liquids may need to be vaporized to combust), and discussed the energy costs or safety issues that such processes could generate. Similarly, all of them noticed differences in chemical composition and structure, but most claims in this area focused on the effect of these factors on the nature of the products of the combustion reaction (e.g., long hydrocarbon chains may generate more diverse byproducts). None of these individuals looked at the properties of chemical substances as resulting from the average of the properties of their individual components (i.e., C, H, O). A few of them (4 of 13) paid attention to the number of bonds in a molecule to make predictions about energy production, although misunderstandings in this area (3 of 13) were still detected.

Most of the participants at the normative level (9 of 13) considered environmental impacts in making their decisions. Half of the people in this group referred to human safety issues, and two of them expressed economic and political considerations. Arguments about environmental issues were less definite than those generated by individuals at the intuitive or mixed levels, who tended to think of substances as either good or bad. Rather, normative BCR thinkers recognized that outcomes would depend on diverse factors, such as the nature of the source (e.g., corn versus biomass) and the process used to produce the fuel. The judgments and decisions of these individuals were also responsive to the particular context defined in the GoKarts task, as illustrated by the following interview excerpt:

S7: Because from burning ethanol it's going to be cleaner. Because it's an amusement park many of the players are children. So you don't want to burn petroleum which can contaminate the room. Ethanol, I think it's better because even though it costs the same, but it burn out like much cleaner. The methane is a gas, so it's harder to contain and fill. I would guess ethanol would cost more. But even when they cost the same I would choose ethanol because it's environmentally safe because you can drink it.

This student's evaluation of the potential impacts of the different substances was influenced by the recognition of the specific intended use of the available fuels. Some participants at the normative level also explicitly recognized the impact that artificial elements of the scenario presented to them had on their decisions. The following excerpt illustrates these types of reflections:

P4: Um, that I think corn is more valuable as corn than it is as ethanol.

I: Okay, can you tell me why you think that?

P4: I think it is my understanding from when I last look at the numbers that the conversion of corn into ethanol is expensive and inefficient and we are better off using it as cheap food than we are using it as expensive gasoline. Now, in your question you've said that this ethanol is magically free, so, so I suspect that it is the correct answer… so far. I think so. I think given what you have stated for this problem, I think ethanol is the answer that I would choose.

I: Okay, but only because we are ignoring cost?

P4: Yes.

I: So if we weren't ignoring cost, you would select one of the octanes? Sorry, I'm giving you many scenarios.

P4: No, I know, I know. So then I would need numbers because I don't know what the cost difference is between natural gas and octane. And I don't know if it makes up for the relative inefficiency of methane compared to octane.

This professor recognized that the assumption of equal costs for all of the fuels might be difficult to justify. This excerpt also illustrates other common patterns of reasoning of individuals at the normative level. All of them expressed a clear understanding of the combustion process and most of them (8 of 13) referred to differences in the amount of energy released upon combustion as a factor to consider in making decisions. However, many recognized that other competing factors needed to be taken into account, such as the energy invested in producing the fuel, the amount of CO₂ produced per unit of energy generated, or the efficiency of various types of engines. Although normative BCR thinkers recognized the role of chemical composition and structure in determining energy of reaction, only a few (3 of 13) attempted to make predictions based on these features. They were more likely to refer to the need for experimental data to make a decision. The following excerpt also illustrates this more complex multicomponent way of reasoning:

P5: We're getting into thermodynamics here. I'm thinking of… so yeah, I mean ethanol you can produce some water with the oxygen there, I guess. Well, could you? You've got carbon monoxide possibilities for each. Honestly it wouldn't be my primary concern for choice of a fuel. Not at all actually. I mean they're all going to produce CO₂. I guess some in different amounts, but at the same time, that's from an ideal perspective with respect to how much CO₂do you produce per kJ per mole. When I say that's from an ideal perspective. Not all fuels are burning with 100 percent efficiency. To make that decision you'd have to look further into the actual engine that's being used, and that's a case by case scenario with respect to which engines are burning fuels as efficiently as possible to purely CO₂rather than CO and other impurities. It wouldn't be a prime concern for me. If something is renewable, that would be… and is safe in the form of a liquid, that would be my prime concerns (sic).

This excerpt also illustrates nuances to the normative reasoning that tended to be introduced by some participants (mainly chemistry professors), when drawing upon their areas of expertise.

Discussion

Our analysis revealed substantial differences in the assumptions and modes of reasoning applied by our study participants to evaluate the benefits, costs, and risks of using different fuels in the GoKarts scenario. Major differences for individuals with different levels of sophistication in BCR reasoning are summarized in Table 2. Our findings elicit domain-general differences, which are likely to characterize the BCR reasoning of people in different contexts, and domain-specific differences, which are tightly linked to the actual focus of our research task (i.e., selection of the best fuel for a GoKart). At the domain-general level, our results highlight the central role that personal experiences and affective impressions play in the judgments and decision-making of novice learners or individuals with limited knowledge. Similar findings have been reported in the exploration of student reasoning in the context of complex socioscientific decision making (Slovic et al., 2003; Sadler and Zeidler, 2005). As shown in Table 2, intuitive BCR thinkers in our study often relied on recognition and affective associations that were triggered by level of familiarity with the entities under analysis, perceptions of risk or lack of control, and preference for what is natural. These intuitive responses were strongly influential in the decisions made by two thirds of our study participants (intuitive and mixed thinkers), including the majority of the undergraduate students who were interviewed.

Table 2 General and specific assumptions and modes of reasoning at different levels of sophistication of BCR reasoning elicited in our study

	Intuitive	Mixed	Normative
General reasoning patterns	Recognition and familiarity with a substance used as main criteria to make choices. Use of affective associations to make decisions (affect heuristic). Reliance on relational reasoning, using vague associations between the name of entities and their expected properties to guide thinking. Absence of mechanistic reasoning. Reactive and hesitant reasoning.	Reasoning enriched by academic knowledge about chemical substances and reactions. Knowledge applied in combination with intuitive ideas, or expressed as “hybrid” conceptions. Ability to generate simple causal links between properties and effects, but difficulty in applying academic knowledge in proper, targeted, and productive ways. Reactive and hesitant reasoning.	Application of normative and appropriate disciplinary knowledge and specific knowledge about substances to make decisions. Broad knowledge base about fuels, their production, properties, and effects. Proactive recall of relevant information. Attention to contextual factors in making judgments. Recognition of the need for more information to make better decisions. Consideration of pros, cons, and trade-offs.
Fuel characteristics	Familiarity with the use and effects of substances applied to discriminate between options. Intuitive preference for “natural” materials. Additive view of matter. Assume that the types and number of components in a substance determine its effects. Tendency to “objectivize” substances (i.e., think of them as objects).	Familiarity with the effects of substances used to discriminate between options. Recognition of compositional and structural features that affect properties. Assume that the types and number of components in a substance determine properties. Attention to structural factors related to energy production.	Extensive, accurate, and sophisticated knowledge about substance characteristics. Consideration of more than one characteristic at a time when making evaluations. Types and number of components in a substance mostly used to make claims about byproducts of combustion. Little attention to structural factors to make claims about energy production. Recognition of the need for experimental data about substance properties to make more definitive judgments.
Potential impacts	Decisions strongly influenced by the perceived environmental impact of substances. Little knowledge about pollutants (besides CO2) and their action mechanisms.	Decisions influenced mostly by perceived environmental impacts, but also by safety concerns. Limited or incorrect knowledge about pollutants and their action mechanisms.	Decisions influenced by environmental and safety concerns. Identification of political and economic issues that influence decisions. Recognition that impacts would depend on different factors that need to be weighed. Specific contextual issues taken into consideration when making decisions.
Fuel availability and management	Focus on perceived abundance and level of consumption of substances. Attention to issues of substance manipulation (storage and transportation).	Focus on abundance and renewability of fuels sources. Attention to issues of substance manipulation, combustion control, and fuel production.	Concerns about fuel availability mostly related to issues of renewability of fuel sources. Consideration of pros and cons in terms of fuel storage and, most distinctively, fuel processing. Recognition of the strong influence of methods of fuel production on decision making.
Energy production	No attention to energy production. Little understanding of combustion processes and their relation with energy production.	Consideration of various energy issues in making decisions. Recognition of combustion as an energy production process, but incorrect or incomplete understanding of the reaction. Assume that chemical energy is released when chemical bonds are broken.	Clear understanding of the combustion process and associated energy production. Attention to diverse energy costs that need to be considered, as well as to competing factors that should be analysed (e.g., energy produced vs. CO2 generated). Recognition of the need for experimental data given the limitations of making reliable inferences from available compositional and structural information for different substances.

---

At a general level, our results support the suggestion that the transition toward more expert knowledge and ways of reasoning often entails the development of hybrid or synthetic constructs, involving the merging of intuition and disciplinary concepts (Vosniadou, 1994). A large fraction of the students who had completed college chemistry courses beyond the introductory level expressed these types of ideas. Cheng and Brown (2010) have suggested that the integration of intuitive knowledge and abstract knowledge actually supports the construction of coherent and sophisticated explanatory models. Similarly, Claesgens et al. (2009) have shown that students who reason with hybrid constructs are more likely to generate reasonable answers than students who do not attempt to incorporate domain knowledge in their explanations. Our own results suggest that hybrid constructs aid in the construction of causal links and thus may support the transition from reasoning based on non-causal associations to mechanistic reasoning.

The knowledge base of study participants with higher levels of training in chemistry was certainly broader than that of freshmen, but the ability to apply such knowledge in proper and productive ways was mostly confined to those individuals with substantial chemistry training (i.e., graduate students and professors). The comparison of mixed and normative forms of BCR reasoning suggests that the transition from one level to the other demands considerable pruning and refinement of concepts and ideas, and significant reflection on the context of their application. Similarly to findings in other areas (Clark and Linn, 2003; Ericsson et al., 2006), higher levels of sophistication in BCR reasoning in our study were characterized by a wider and stronger integration of knowledge, a higher ability to recognize and weigh the effects of several variables, a greater attention to tradeoffs in decision making, and a more focused consideration of the specific goals and constraints of the task at hand. Intuitive and mixed thinkers in our sample were more likely to rely on non-compensatory decision-making strategies, in which options judged to be unacceptable under certain criterion were simply eliminated, while normative thinkers used compensatory approaches, in which benefits and drawbacks were more systematically weighed. These differences in decision-making reasoning between less and more advanced students have been observed in other scientific disciplines (Hong and Chang, 2004; Gresch et al., 2013).

Our study also revealed major domain-specific differences between study participants. Reliance on formal chemistry knowledge and ways of thinking was minimal among individuals at the intuitive level. On the other hand, students at the mixed level often tried to apply many chemistry concepts or ideas that were not necessarily relevant, appropriate, or productive for making the required judgments and decisions. Their expressed ideas revealed basic misunderstandings about the nature of chemical substances and processes. In particular, many of our study participants seemed to hold an “additive” view of matter (Talanquer, 2008, 2015), in which properties of substances were seen as the result of the average of the properties of their individual components (i.e., elements, atoms, bonds). Within this perspective, molecules were expected to behave like macroscopic objects whose properties were determined by the types, amounts, and sizes of their components. These components were thought to have inherent properties that were used to predict behaviors and justify choices, similarly to what Cimpian and Salomon (2014) characterize as reliance on an “inherence heuristic.” This overall conceptualization of substances had a strong influence on the decisions made by many of the undergraduate chemistry students who participated in our study, and remnants of this way of thinking were detected in the reasoning of several advanced students.

Participants at the normative level demonstrated a greater ability to recall and integrate different types of knowledge, as well as to recognize the limitations of making reliable inferences based on prior knowledge and available information. Consequently, they often referred to the need for additional experimental data, such as heats of combustion, to make more definitive claims in particular contexts. Normative BCR thinkers were also able to recognize how artificial elements in the scenario presented to them, such as assuming equal cost for all of the fuels or that fuels were composed of one pure substance (i.e., octane, methane, and ethanol), affected their judgments and decisions. The fact that individuals with advanced studies in chemistry were more likely to reason at a normative level may not be surprising. Nevertheless, our study provides clear insights into major differences in the patterns of BCR reasoning of individuals at different stages in their chemistry training and elicits key cognitive and affective elements that need to be targeted to foster and facilitate progression from intuitive to normative levels of reasoning.

Although our study involved only 39 participants and focused on a particular context, the nature of our findings, together with existing related research that is discussed above, allow us to speculate that similar results may be found when analysing students' decision making in other situations. In particular, if situations demand that students make decisions that involve chemical substances with which they have some familiarity, and if the decision requires reasoning based on relationships between the composition and structures of chemical compounds and the properties of those substances, then it is likely that decision making may be able to be interpreted through the lens of our findings.

Implications

Most undergraduate students in our sample, from freshmen to seniors, did not demonstrate normative levels of BCR reasoning during the interview. Their ability to make productive judgments and informed decisions in a context that demanded application of chemistry concepts and ideas was certainly limited. This may not be surprising given that dominant curricular approaches and teaching practices in chemistry at the undergraduate level offer few opportunities for students to apply and integrate their knowledge in tasks that demand evaluation of the benefits, costs, and risks of different alternatives. The undergraduate and graduate chemistry curricula are often aseptic in their approach to the discussion of chemical concepts, ideas, and practices. Additionally, despite well-substantiated educational benefits of activities that are more active, constructive, and interactive (Chi, 2009), college chemistry teaching is characterized by its reliance on passive forms of learning. Given this state of affairs, it is unlikely that isolated educational interventions in a few chemistry courses would have any major impact on building reasoning capacity which often demands concerted efforts over many years (Corcoran et al., 2009).

We suspect that significant improvement in BCR reasoning demands ambitious and coordinated changes in chemistry education (Eilks et al., 2013; Sevian and Talanquer, 2014). In particular, the results of our study suggest that progression in this area could be aided by deliberately planned learning activities that occur in a coherent manner across the curriculum. These activities should foster and facilitate:

(a) A shift from relying on recognition and affective associations in making judgments and decisions toward searching for physical and chemical features useful in predicting relevant properties and behaviors of the substances or processes under analysis.

(b) A shift from relying on non-causal associations between entities, events, and properties to make and justify decisions toward building mechanistic explanations to support prediction, explanation, and decision making.

(c) Recognition of strengths and limitations of both intuitive constructs and scientific models in explaining and predicting the properties of chemical substances and processes.

(d) Development of “design thinking” involving analysis of the context of a problem and relevant constraints, evaluation of trade-offs, and data-driven decision making.

We propose that it is possible to design learning activities that encompass all four of our proposed curricular improvements. One such activity might ask students to design a protocol for cleaning up a specific environmental pollutant. Such a scenario would require students to think about the nature of the pollutant itself (e.g., impacted biospheres, toxicology, and risk assessment) as well as the impacts of proposed clean-up techniques. The students would be guided by their instructor to focus on relevant physical and chemical features of the pollutant to develop mechanistic explanations. Learning would be scaffolded to support students developing justifications for claims based on evidence and observation, linking observations with their knowledge of chemistry and the behaviors of substances.

Existing research and development in the areas of model-based inquiry (Windschitl et al., 2008) and engineering design education (Crismond and Adams, 2012) provide insights into how the above outcomes could be achieved. Model-based approaches engage students in cycles of testing, evaluating, and refining models of systems of interest, assessing their adequacy against standards of evidence. The central goal is to support students' abilities to use observable evidence to generate mechanistic explanations of targeted phenomena. Reasoning is supported by prompting students to reflect on what counts as a good scientific explanation. Meta-conceptual awareness seems to be critical to help students who are “trapped” by their intuitions to build more coherent and consistent explanatory models (Cheng and Brown, 2010), as well as to monitor and control heuristic reasoning (Klaczynski, 2004; Böttcher and Meisert, 2013; Gresch et al., 2013).

The patterns of reasoning manifested by intuitive BCR thinkers in our study are similar to those identified in novice learners in engineering design (Crismond and Adams, 2012). These beginning designers tend to, for example, interpret design problems too simply and generate premature solutions impulsively and superficially, without recognizing the need for additional data or research. They base their design solution on a single idea and they pay little attention to design criteria and constraints. The teaching strategies devised to help novice designers progress in their thinking may thus be of great benefit in the development of students' BCR reasoning. These strategies include asking students to generate functional descriptions of what a viable and successful solution would be before making a decision, engaging them in the investigation of messy problems in which data from different areas must be considered and integrated, and allowing them to “mess about” with different ideas, analyzing their strengths and limitations. Implementing these types of educational strategies across the college chemistry curriculum demands substantive reform to more effectively prepare professionals that can make benefits-cost-risks decisions at a normative level. Successful models of reform could be drawn from existing work in the area of context-based chemistry education (Bulte et al., 2006; King, 2012).

Acknowledgements

The authors wish to acknowledge the funding sources, US National Science Foundation awards 1222624 and 1221494, that support our work. Any opinions, conclusions, or recommendations expressed in this article are those of the authors, and do not necessarily reflect the views of the funding sources.

Notes and references

1. Acar O., Turkmen L. and Roychoudhury A., (2010), Student difficulties in socio-scientific argumentation and decision-making research findings: crossing the borders of two research lines, Int. J. Sci. Educ., 32(9), 1191–1206.
2. Alonzo A. C. and Gotwals A. W., (2012), Learning progressions in science: current challenges and future directions, Rotterdam, The Netherlands: Sense Publishers.
3. American Association for the Advancement of Science (AAAS), (1993), Benchmarks for Science Literacy, Washington DC: Oxford University Press.
4. Becker S., (2013), Nanotechnology in the marketplace: how the nanotechnology industry views risk, J. Nanopart. Res., 15, 1426.
5. Bernholt S. and Parchmann I., (2011), Assessing the complexity of students' knowledge in chemistry, Chem. Educ. Res. Pract., 12, 167–173.
6. Biggs J. B. and Collis K. F., (1982), Evaluating the quality of learning: the SOLO Taxonomy (Structure of the Observed Learning Outcome), New York, NY: Academic Press.
7. Boo H. K. and Watson J. R., (2001), Progression in high school students' (aged 16–18) conceptualizations about chemical reactions in solution, Sci. Educ., 85(5), 568–585.
8. Böttcher F. and Meisert A., (2013), Effects of direct and indirect instruction on fostering decision-making competence in socioscientific issues, Res. Sci. Educ., 43, 479–506.
9. Brown N. J. S. and Wilson M., (2011), A model of cognition: the missing cornerstone of assessment, Educ. Psychol. Rev., 23(2), 221–234.
10. Brown N. J. S., Nagashima S. O., Fu A., Timms M. and Wilson M., (2010), A framework for analyzing scientific reasoning in assessments, Educ. Assess., 15(3), 142–174.
11. Brun W., (1992). Cognitive components in risk perception: natural versus manmade risks, J. Behav. Decis. Making, 5, 7–132.
12. Bulte A. M. W., Westbroek H. B., De Jong O. and Pilot A., (2006), A research approach to designing chemistry education using authentic practices as contexts, Int. J. Sci. Educ., 28(10), 1063–1086.
13. Charmaz K., (2006), Constructing grounded theory: a practical guide through qualitative analysis, Thousand Oaks, CA: Sage Publications.
14. Cheng M. and Brown D. E., (2010), Conceptual resources in self-developed explanatory models: the importance of integrating conscious and intuitive knowledge, Int. J. Sci. Educ., 32(17), 2367–2392.
15. Chi M. T. H., (2008), Three kinds of conceptual change: belief revision, mental model transformation, and ontological shift, in Vosniadou S. (ed.), International handbook of research on conceptual change, New York, NY: Routledge, pp. 61–82.
16. Chi M. T. H., (2009), Active-constructive-interactive: a conceptual framework for differentiating learning activities, Top. Cogn. Sci., 1,73–105.
17. Cimpian A. and Salomon E., (2014), The inherence heuristic: an intuitive means of making sense of the world, and a potential precursor to psychological essentialism, J. Behav. Brain. Sci., 37, 461–527.
18. Claesgens J., Scalise K., Wilson M. and Stacy A., (2009), Mapping student understanding in chemistry: the perspectives of chemists, Sci. Educ., 93(1), 56–85.
19. Clark D. B. and Linn M. C. (2003). Designing for knowledge integration: the impact of instructional time, J. Learn. Sci., 12(4), 451–494.
20. Corcoran T., Mosher F. A. and Rogat A. (2009). Learning progressions in science: an evidence-based approach to reform, Consortium for Policy Research in Education Report #RR-63, Philadelphia, PA: Consortium for Policy Research in Education.
21. Crismond D. P. and Adams R. S., (2012), The informed design teaching and learning matrix, J. Eng. Educ., 101(4), 738–797.
22. Dawson V. and Venville G. J., (2009), High school students' informal reasoning and argumentation about biotechnology: an indicator of scientific literacy? Int. J. Sci. Educ., 31(11), 1421–1445.
23. Devcich D. A., Pedersen I. K. and Petrie K. J., (2007), You eat what you are: modern health worries and the acceptance of natural and synthetic additives in functional foods, Appetite, 48(3), 333–337.
24. Dickson-Spillmann M., Siegrist M. and Keller C., (2011), Attitudes toward chemicals are associated with preference for natural food, Food Qual. Prefer., 22(1), 149–156.
25. diSessa A. A., (1993), Towards an epistemology of physics, Cognition Instruct., 10, 105–225.
26. Dobbie M. E. and Brown R. R., (2013), A framework for understanding risk perception explored from the perspective of the water practitioner, Risk Anal., 34(2), 294–308.
27. Duschl R., Maeng S. and Sezen A., (2011), Learning progressions and teaching sequences: a review and analysis, Stud. Sci. Educ., 47(2), 123–182.
28. Eilks E., Rauch F., Ralle B. and Hofstein A., (2013), How to allocate the chemistry curriculum between science and society, in Eilks I. and Hofstein A. (ed.) Teaching chemistry – a studybook, Rotterdam, The Netherlands: Sense Publishers, pp. 1–36.
29. Ericsson K. A., Charness N., Feltovich P. and Hoffman R. (ed.), (2006), Cambridge handbook of expertise and expert performance, Cambridge: Cambridge University Press.
30. Evagorou M., Jimenez-Aleixandre M. P. and Osborne J., (2012), Should we kill the grey squirrels? A study exploring students' justifications and decision-making, Int. J. Sci. Educ., 34(3), 401–428.
31. Feinstein N., (2011), Salvaging science literacy, Sci. Educ., 95(1), 168–185.
32. Finucane M. L., Alhakami A., Slovic P. and Johnson S. M., (2000), The affect heuristic in judgment of risks and benefits, J. Behav. Decis. Making, 13(1), 1–17.
33. Gigerenzer G. and Gaissmaier W., (2011), Heuristic decision-making, Ann. Rev. Psychol., 62, 451–482.
34. Goswami U., (2013), The development of reasoning by analogy. in Barrouillet P. and Gauffroy C. (ed.), The development of thinking and reasoning, New York, NY: Psychology Press, pp. 49–70.
35. Gresch H., Hasselhornb M. and Bögeholz S., (2013), Training in decision-making strategies: an approach to enhance students' competence to deal with socio-scientific issues, Int. J. Sci. Educ., 35(15), 2587–2607.
36. Hogan K., (2002), Small groups' ecological reasoning while making an environmental management decision, J. Res. Sci. Teach., 39(4), 341–368.
37. Hong J.-L. and Chang N. K. (2004). Analysis of Korean high school students' decision-making process in solving a problem involving biological knowledge, Res. Sci. Educ., 34, 97–111.
38. Joint Research Centre (2011), Special Eurobarometer 360: consumer understanding of labels and the safe use of chemicals, European Commission.
39. Kahan D. M., Jenkins-Smith H. and Braman D., (2011), Cultural cognition of scientific consensus, J. Risk Res., 14(2), 147–174.
40. Kahneman D., (2011), Thinking, fast and slow, New York: Farrar, Straus and Giroux.
41. Keil F. C., (1979), Semantic and conceptual development: an ontological perspective, Cambridge, MA: Harvard University Press.
42. Kilinç, A., Boyes E. and Stanisstreet M., (2013), Exploring students' ideas about risks and benefits of nuclear power using risk perception theories, J. Sci. Educ. Technol., 22(3), 252–266.
43. Kind V., (2004), Beyond appearances: students' misconceptions about basic chemical ideas, 2nd edn, London: Royal Society of Chemistry.
44. King D., (2012), New perspectives on context-based chemistry education: using a dialectical sociocultural approach to view teaching and learning, Stud. Sci. Educ., 48(1), 51–87.
45. Klaczynski P. A., (2004), A dual-process model of adolescent development: implications for decision-making, reasoning, and identity, in Kail R. V. (ed.), Advances in child development and behavior, San Diego, CA: Academic Press, pp. 73–123.
46. Krnel D., Watson R. and Glažar S. A., (1998), Survey of research related to the development of the concept of ‘matter’, Int. J. Sci. Educ., 20(3), 257–289.
47. MacGregor D. G., Slovic P. and Malmfors T. (1999), “How exposed is exposed enough?” Lay inferences about chemical exposure, Risk Anal., 19(4), 649–659.
48. Malandrakis G. N., (2008), Children's understandings related to hazardous household items and waste, Environ. Educ. Res., 14(5), 579–601.
49. McNeill K. and Vaughn M. H., (2012), Urban high school students' critical science agency: conceptual understandings and environmental actions around climate change, Res. Sci. Educ., 42(2), 373–399.
50. Mohan L., Chen J. and Anderson C. W., (2009), Developing a multi-year learning progression for carbon cycling in socio-ecological systems, J. Res. Sci. Teach., 46(6), 675–698.
51. National Research Council (NRC), (1996), National Science Education Standards, Washington DC: National Academy Press.
52. National Research Council (NRC), (2011), A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas, Committee on a Conceptual Framework for New K-12 Science Education Standards, Washington, DC: The National Academies Press.
53. National Research Council (NRC), (2013), The next generation science standards, Washington, D.C.: The National Academies Press.
54. Ngai C., Sevian H. and Talanquer V., (2014), What is this substance? What makes it different? Mapping progression in students' assumptions about chemical identity, Int. J. Sci. Educ., 36, 2438–2461.
55. Nielsen J. A. (2012). Arguing from nature: the role of ‘nature’ in students' argumentations on a socio-scientific issue, Int. J. Sci. Educ., 34(5), 5, 723–744.
56. Osborne J. F. and Dillon J., (2008), Science education in Europe, London: Nuffield Foundation.
57. Perkins D. N. and Grotzer T. A., (2005), Dimensions of causal understanding: the role of complex causal models in students' understanding of science. Stud. Sci. Educ., 41, 117–166.
58. Rozin P., (2005), The meaning of natural. Psychol. Sci., 16(8), 652–658.
59. Russ R. S., Scherr R. E., Hammer D. and Mikeska J., (2008), Recognizing mechanistic reasoning in student scientific inquiry: a framework for discourse analysis developed from philosophy of science, Sci. Educ., 92(3), 499–525.
60. Sadler T. D. and Donnelly L. A., (2006), Socioscientific argumentation: the effects of content knowledge and morality, Int. J. Sci. Educ., 28(12), 1463–1488.
61. Sadler T. D. and Fowler S., (2006), A threshold model of content knowledge transfer for socio-scientific argumentation, Sci. Educ., 90(6), 986–1004.
62. Sadler T. D. and Zeidler D. L., (2005), Patterns of informal reasoning in the context of socioscientific decision making, J. Res. Sci. Teach., 42(1), 112–138.
63. Sevian H. and Talanquer V., (2014), Rethinking chemistry: a learning progression on chemical thinking, Chem. Educ. Res. Pract., 15(1), 10–23.
64. Siegrist M. and Cvetkovich G., (2000), Perception of hazards: the role of social trust and knowledge, Risk Anal., 20(5), 713–719.
65. Slotta J. D., Chi M. T. H. and Joram E., (1995), Assessing students' misclassifications of physics concepts: an ontological basis for conceptual change, Cognition Instruct., 13, 373–400.
66. Slovic P., (1987), Perception of risk, Science, 236(4799), 280–285.
67. Slovic P., (2010), The psychology of risk, Saúde e Sociedade, 19(4), 731–747.
68. Slovic P., Finucane M., Peters E. and MacGregor D. G., (2003), The affect heuristic, in Gilovich T., Griffin D. and Kahneman D. (ed.) Intuitive judgment: heuristics and biases, New York: Cambridge University Press, pp. 397–420.
69. Stevens S. Y., Delgado C. and Krajcik J. S., (2010), Developing a hypothetical multi-dimensional learning progression for the nature of matter, J. Res. Sci. Teach., 47(6), 687–715.
70. Szteinberg G., Balicki S., Banks G., Clinchot M., Cullipher S., Huie R., Lambertz J., Lewis L., Ngai C., Weinrich M., Talanquer V. and Sevian H., (2014), Collaborative professional development in chemistry education research: bridging the gap between research and practice, J. Chem. Educ., 91, 1401–1408.
71. Taber K., (2014), The significance of implicit knowledge in teaching and learning chemistry, Chem. Educ. Res. Pract., 15, 447–461.
72. Talanquer V., (2006), Commonsense chemistry: a model for understanding students' alternative conceptions, J. Chem. Educ., 83(5), 811–816.
73. Talanquer V., (2008), Students' predictions about the sensory properties of chemical compounds: additive versus emergent frameworks, Sci. Educ., 92(1), 96–114.
74. Talanquer V. (2009), On cognitive constraints and learning progressions: the case of structure of matter, Int. J. Sci. Educ., 31(15) 2123–2136.
75. Talanquer V., (2013), How do students reason about chemical substances and reactions? in Tsaparlis G. and Sevian H. (ed.), Concepts of matter in science education, Dordrecht: Springer, pp. 331–346.
76. Talanquer V., (2015), Threshold concepts in chemistry: the critical role of implicit schemas, J. Chem. Educ., 92(1) 3–9.
77. von Aufschnaiter C. and von Aufschnaiter S., (2003), Theoretical framework and empirical evidence of students' cognitive processes in three dimensions of content, complexity, and time, J. Res. Sci. Teach., 40(7), 616–648.
78. Vosniadou S., (1994). Capturing and modeling the process of conceptual change, Learn. Instr., 4(1), 45–69.
79. Waddington D., Nentwig P. and Schanze S. (ed.), (2007), Standards in science education, Münster, Germany: Waxmann.
80. Williams P. R. D. and Hammitt J. K., (2001), Perceived risks of conventional and organic produce: pesticides, pathogens, and natural toxins, Risk Anal., 21(2), 319–330.
81. Wilson M., (2009), Measuring progressions: assessment structures underlying a learning progression, J. Res. Sci. Teach., 46(6), 716–730.
82. Windschitl M., Thompson J. and Braaten M., (2008), Beyond the scientific method: model based inquiry as a new paradigm of preference for school science investigations, Sci. Educ., 92(5), 941–967.
83. Zeidler D. L., Sadler T. D., Simmons M. L. and Howes E. V., (2005), Beyond STS: a research-based framework for socioscientific issues education, Sci. Educ., 89(3), 357–377.

---

Footnote

† The instrument that was used can be obtained by contacting the authors or is available in the online supporting information in Szteinberg et al. (2014) at http://pubs.acs.org/doi/abs/10.1021/ed5003042.

---