Ilse
Rootman-le Grange
*a and
Margaret A. L.
Blackie
b
aFaculty of Science, Private Bag X1, Matieland, South Africa. E-mail: ilser@sun.ac.za; Tel: +27-21-808-3535
bDepartment of Chemistry and Polymer Science, Private Bag X1, Matieland, South Africa. E-mail: mblackie@sun.ac.za; Tel: +27-21-808-3353
First published on 5th February 2018
The challenge of supporting the development of meaningful learning is prevalent in chemistry education research. One of the core activities used in the learning process is assessments. The aim of this paper is to illustrate how the semantics dimension of Legitimation Code Theory can be a helpful tool to critique the quality of assessments and reveal how this quality potentially contributes to meaningful learning. For this purpose we analysed an exam paper from an introductory chemistry module, using the semantics dimension as a framework. We discuss the tools that were designed for this analysis and how it was applied to reveal the weakness in this particular assessment. Suggestions for how this assessment can be improved is also discussed. This study illustrates how the semantics dimension can inform assessment practice and potentially contribute to the development of meaningful learning.
There are at least two facets that consistently appear in paper after paper, addressing the question of meaningful learning in chemistry. The first is the idea of conceptual understanding. Here George Bodner's classic study clearly, and depressingly, illustrates this is a major issue. In this study, he asked incoming graduate students (all of whom had graduated with a Bachelor's degree with a major in chemistry) what was in the bubbles that are created when water boils. The responses would send shivers down the spine of any chemistry educator: while 70% of the students did correctly answer that the water molecules were entering the gas phase, 20% of students claimed the bubbles were air or oxygen. Most disturbingly, a full 5% of chemistry graduates claimed the bubbles contained hydrogen and oxygen gas (Gabel, 1999a). The point here is simply to illustrate that it is indeed possible to pass through an entire undergraduate chemistry program carrying substantial misconceptions, which most chemistry educators would consider worrisome in a high school graduate. Indeed this was one of the questions on the Chemistry Concept Test developed by Potgieter and coworkers and used to measure conceptual understanding of incoming students to BSc degrees at several universities in South Africa (Potgieter et al., 2008; Potgieter, 2010; Potgieter and Davidowitz, 2010).
The second important facet drawing from mainstream education research and in particular constructivist conceptions of learning, points to the conditions required for learning to occur (Gabel, 1999b). There are various different articulations of this idea, but essentially, the key idea is that knowledge is constructed in the mind of the learner. The construct of the knowledge may be more or less accurate depending in part upon the foundation that the learner is building on, the way in which concepts are presented and the nature of the learning environment.
As an example of the nature of the learning environment, Bretz and coworkers have recently explored different aspects of the model of learning proposed by Novak. Novak claims that ‘Meaningful learning underlies the constructive integration of thinking, feeling, and acting, leading to human empowerment for commitment and responsibility’ (Novak, 2010). Much of the work carried out by Bretz and coworkers involves exploring one or other of these aspects. Their underlying presumption is ‘Opportunities must be provided for learners across all three domains in order to ensure successful integration and meaningful learning’ (Galloway et al., 2015). There has been some good work done on consciously attending to the cognitive, affective and psychomotor dimensions of the learning experience (Galloway et al., 2015).
This is resonant with work done across different disciplines in higher education. Barnett has published extensively on the importance of attending to the being of the student for the success of education in the development of adaptive and complex thinking capability (Barnett, 2004; Barnett, 2008; Barnett, 2009). Dall’Alba and Barnacle highlight the importance of attending to the ontology of the student in conjunction with more prosaic and utilitarian goals such as skills development or knowledge dissemination (Dall’Alba and Barnacle, 2007). Whilst such an explicit ontological focus is quite rare in the chemistry education literature, there are hints of it in the current research that goes beyond accurate transmission of chemistry concepts.
One such example is a recent paper building on Johnstone's triangle, in which Thomas calls for the explicit use of metacognition (Thomas, 2017). In this framework, evidence seems to suggest that meaningful learning will not occur in the absence of reflection. That is to say that, the new information will not be adequately assimilated and pinned to existing knowledge constructs. Thomas writes ‘It would mean that teachers also communicate explicitly with students about the levels of representations and explain how classroom activities and experiences relate to the possibility of their learning chemistry and coming to know about what it means to understand chemistry with respect to those representations. The goal of such communication would be to develop students’ metacognition’. This is in line with, albeit at a fairly low level, Barnett's aspirations for higher education (Barnett, 2008). That is creating a learning environment which facilitates accurate knowledge development but at the same time attempts to help the student develop their capacity to learn anything. In the case of Thomas’ work this is done by encouraging students to reflect on their process of learning, as well as attending to what they are learning (Thomas, 2017).
There appear to be two options in approaching chemistry education focused learning. The first one is by far the most common. The vast majority of chemistry education papers focus on conceptual understanding and meaningful learning either of a particular concept (does the student understand what an arrow push actually means) or within a particular learning context (for example, practicals or organic chemistry). The second one is far more rare. There are a few papers, which try to link conceptual development in a single context with the construction of knowledge, but this normally requires something of a leap from the ultra-specific grasp of this one concept to how this item is stored in long-term memory. Thomas's paper on triangulation does precisely this (Thomas, 2017).
Surely, there must be something in between? That is to say, these two options are not so much a binary as opposite ends of a spectrum. The question that therefore arises is: are there approaches which can be used across different disciplinary divisions in chemistry, across analytical and organic chemistry for example. There has been some work done to good effect to begin to tease out this ‘in between’ space in chemistry education. Taber has been a loud voice in this space (Taber, 2013). But there is still a long way to go.
This sparsely populated middle ground is perhaps the area of greatest challenge to the student, but also the place where chemistry education can fall short. This is precisely because the ability to move from the minutiae of the particular concept to the joining of this concept to the bigger picture of the domain to the broader self-understanding of what I need to do, in general terms, in order to ‘grasp’ a particular concept and satisfy myself that I have understood it, which qualifies as ‘powerful knowledge’ (Young, 2007) People negotiating new knowledge domains will however falter in different arenas.
Given that LCT is built on, among others, the foundation of Bernstein's work and uses the Bernsteinian ideas of horizontal and vertical knowledge structures, Maton uses the term ‘cumulative learning’ instead of the term ‘meaningful learning’ often used in the chemistry education literature. The term cumulative learning makes slightly more explicit the idea that one concept builds on a prior concept. We expect students to learn in a cumulative manner in chemistry. For example, although we teach stoichiometry formally only in first year, we expect a chemistry graduate to be able to know that they have to apply stoichiometry to setting up a reaction appropriately when they enter the laboratory as a graduate student.
We will restrict ourselves to the use of the semantics dimension, one of the tools of LCT, for this particular paper. This dimension provides a very useful distinction between abstraction and complexity. Abstraction here refers to the role context plays in defining meaning. In LCT terminology this is termed semantic gravity (SG). Complexity on the other hand refers to the meaning that is implicated by, or locked up in a specific word formula or symbol. The term used in LCT to express this is semantic density (SD). As with all of the LCT tools, the semantics dimension is plotted over a Cartesian plane. Semantic gravity (SG) varies along the y-axis from stronger semantic gravity (−y, SG+) to weaker semantic gravity (+y, SG−) and represents the variation in abstraction. Semantic density (SD) varies along the x-axis, from weaker semantic density (−x, SD−) to stronger semantic density (+x, SD+) and represents the variation in complexity (Fig. 1).
Fig. 1 The semantic plane, illustrating the relationship between semantic density and semantic gravity. |
Maton argues that cumulative learning will only occur if teaching models both the unpacking and repacking along both axes of abstraction and complexity. On the abstraction axis, we must move between the overarching principle (weaker semantic gravity, SG−) we are aiming towards and the specific examples (stronger semantic gravity, SG+) we are using to illustrate this point. On the complexity axis, we must likewise move between using more complex subject specific language (stronger semantic density, SD+) to simpler more explanatory language (weaker semantic density, SD−) and back again.
To exemplify this, consider organic reactions. One could describe a particular reaction – the reaction between concentrated hydrochloric acid and tert-butanol to form tert-butyl chloride. This is a specific example of a nucleophilic substitution reaction. We can weaken the semantic gravity of this specific question by using a generic leaving group (as opposed to the alcohol) and a nucleophile as opposed to the chloride. This is a small change in the semantic gravity but it is a weakening nonetheless. Similarly, we could weaken the semantic density by substituting the names with a structure. (Once a student has assimilated the meaning of line structures, they are far less obscure and require less interpretation than an IUPAC name.)
In her 2014 paper, Blackie gives a useful overview of the application of the semantic dimension to chemistry (Blackie, 2014). She argues that many chemistry educators are fairly adept at the variation of complexity. We all have a natural preference for either unpacking or repacking, while it requires some conscious effort to introduce the reverse action. This is not particularly challenging, it just requires practice. Likewise we are fairly good at testing students’ ability to both interpret and produce chemical complexity of the appropriate level. However, the contention was that we seem to presume that this capacity to engage with complexity (i.e. move to relatively stronger semantic density) is indicative of a depth of understanding, so the bulk of our assessment is aimed at testing the range of complexity (i.e. testing the range of semantic density) which students can navigate. At the same time, there is very little testing of the range of abstraction (i.e. the range of semantic gravity). To put this in terms of Johnstone (Johnstone, 1982) or Taber (Taber, 2013), we test well their interpretation of the symbolic representation, but we fail to test how students are making connections between sections of chemistry. However, it is the weakening of semantic gravity which is needed to make the connections between different sections in chemistry, and therefore to develop a real (and meaningful) chemical view of the world. It is this weakening of semantic gravity, which will allow a student to remember the key concept from the year before.
We have two aims for this this paper. Firstly, to interrogate Blackie's (2014) postulate that chemistry questions are more likely to favour a range of semantic density than semantic gravity. Secondly, to show how the semantics dimension of LCT can be used to critique existing assessments. In this paper then, we explore the variation in semantic gravity and semantic density in an introductory chemistry course. Since assessment is a crucial part of the learning process, significantly influencing students’ learning strategies, approaches and activities, we analyse a typical exam paper and we correlate that to the curriculum (Stallings and Leslie, 1970; Docan, 2006). We describe the method we have used to apply the semantics dimension of LCT to our context, which includes a description of the semantic gravity and semantic density translation devices. Finally, we discuss the possible implications this has on supporting cumulative learning in our particular context.
Students are thus expected to accumulate a base of knowledge during completion of this course upon which further learning in follow-up courses can be built. These higher-level courses in health sciences are strongly related to chemistry, but the links to the foundational chemistry course may not be made explicit in the teaching environment. Nonetheless, the expectation is that students should be able to transfer the newly attained chemistry related knowledge base to these new contexts as they encounter them.
The curriculum covered in the course is typical of most introductory chemistry courses, covering topics such as stoichiometry, molecular bonding and geometry, equilibria, acids and bases, kinetics and organic chemistry. The curriculum was designed in consultation with stakeholders in the Health Sciences faculty to ensure that the content covered is relevant to the respective programs. Furthermore, the study guide makes explicit mention of why students require each section in terms of both chemistry and their future studies in health sciences.
As assessment tends to profoundly shape learning, and based on the postulation made in the previous paper (Blackie, 2014) which suggests that we tend to lack in testing a range of semantic gravity, we decided to begin our analysis with the final exam paper for this course. We decided to analyse both the semantic density and semantic gravity of this paper. This was done in order to determine in the first instance whether Blackie's postulate was in fact true. And secondly, to offer a robust critique of the current method of assessment. As the overall structure of the exam had not changed substantially since the course was updated in 2011. Taking the final exam paper from one of the years was deemed sufficient to satisfy the two aims. The paper comprised two sections. The first section contained questions requiring written responses or calculations. The second section contained only multiple-choice questions. Each question and sub-question in the paper was coded separately. In total 44 questions were analysed and each was designated a code, based on the translation devices discussed below.
It is important to note that the semantics dimension can be applied in a wide variety of ways. And thus, is a useful and flexible tool. In order to use this tool well, the context in which it is being used must be clearly defined. The development of a robust translation device (i.e. one which can be used by more than one person and achieve consistent results) requires a clear definition of context. For the purposes of this study the variation of abstraction and complexity is bound by the textbook. The greatest degree of abstraction is therefore the greatest degree of abstraction that we would expect to see from a student who successfully completes this course with a good grade, rather than what we would expect from a professor of chemistry.
Each question was therefore analysed first to identify the concept or concepts to which it referred and second to identify the relationship between these concepts and the prescribed curriculum. If a question simply required the recall of a definition, or set of rules directly related to a specific theme in the curriculum, the semantic gravity would be indicated as very strong. Questions that required application of these concepts to curriculum specific examples (i.e. applications that are similar to the examples used in the textbook to introduce the concepts) were coded as having slightly weaker semantic gravity. The weakest level of semantic gravity that was defined in this translation device represents the level of abstraction that we believe should be displayed by these students on successful completion of this course. At this level students should be able to apply some of the concepts that are introduced in the curriculum, to related fields of study, specifically health science related contexts. This means that a student should be able to recognise a problem as being a chemical problem even if it is not explicitly presented as such. Table 1 presents the translation device that was used to code semantic gravity.
Allocated code | Criteria |
---|---|
SG−− | Concepts situated in the curriculum are integrated with general everyday knowledge to create meaning that is applicable in any type of context. |
SG− | The question requires concepts from different sections in the curriculum to be integrated to create a unified theory that is applicable to a broader context. |
SG+ | The question requires application of Chemical concept(s) from one section of the curriculum to a specific example. |
SG++ | The question is located in a specific section of the curriculum and only requires recall of the concepts, definitions or rules. |
Allocated code | Criteria |
---|---|
SD−− | No chemical terminology or concepts are required to answer the question. |
SD− | Only one term/structure/formula is given and needs to be interpreted in order to answer the question. |
SD+ | The given information needs to be manipulated – unpacked before it can be interpreted. |
SD++ | The chemical problem must first be identified before any interpretation or manipulation can be done in order to get to a solution/answer to the question (multiple steps required). |
The initial analysis was done by one researcher who started by comparing the different questions in an attempt to identify different levels of semantic gravity. This led to the development of a proposed translation device. The second researcher then coded the questions independently from the first, using the proposed translation device. Finally, the two analyses were crosschecked and the translation device finalised.
Fig. 2 A summary of the analysis of the 44 questions in the exam paper. The numbers represents the number of questions that has been designated the coding that is indicated in brackets below it. |
To better unpack the above results, let us discuss the following two questions from the analysed exam paper in more detail.
Question 1: Vitamin B5 (pantothenic acid) is shown below. This biologically active molecule is an optical isomer that behaves as a weak acid in water. Pantothenic acid partially dissociates to form pantothenate ions and hydrogen ions in aqueous solution. If the Ka value of pantothenic acid is 3.89 × 10−6, calculate the pH of a 0.100 M solution of this vitamin.
Question 2: If 7.24 g of sodium pantothenate (C9H16NO5Na) is added to 0.200 dm3 of a 0.100 M solution of pantothenic acid, calculate the pH of the resulting solution.
Question 1: When considering the semantic gravity, the crux of the question – to calculate pH – clearly locates it in the curriculum under the topic of acids and bases. The question is only located under this one topic and does not require any additional information in order to be solved.
Regarding the semantic density there are a number of chemical terms accumulated in this question namely, optical isomer, weak acid, partially dissociates, Ka, and pH. Furthermore, although the structure of pantothenic acid is given, the dissociation reaction is not given. Thus, manipulation of the data in order to write the equilibrium reaction equation is required, before the question can be attempted.
The semantic density of the question could be weakened by giving the equilibrium reaction equation, or the molecular formulas of pantothenic acid, pantothenate and hydrogen that is required to write the reaction. Therefore, it was coded as SG+ and SD+.
Question 2: Although this question also clearly asks for the calculation of pH, which would again locate it in a specific section of the curriculum, what is not made explicit is that this is a buffer solution. Thus, initially it appears that it is at the same level of semantic gravity as question 1, while in fact it requires application of principles from different section in the curriculum in order to be solved, thereby weakening the semantic gravity. The semantically dense terms in this question are the names of the chemicals as well as the term ‘pH’. Since the structure of pantothenic acid is already given in the previous question, and the molecular formula (C9H16NO5Na) of sodium pantothenate is given as part of this question, there are no terms or structures that needs to manipulated. The question is therefore semantically less dense than the previous question. Question 2 was therefore coded as SG− and SD−.
Furthermore, as this is a service course there is a strong expectation of knowledge transfer as students progress with their studies. To develop this knowledge transfer students need to be exposed to a range of semantic gravity. As we have seen from the results above, with the exception of two questions, all the questions are strongly context dependent. The students’ ability to abstract concepts from the context in which it is taught and display a more comprehensive understanding that would be applicable to a broader context, is not being assessed. The majority of questions focus on the application of one concept to a specific example. Since assessment has such a profound impact on what and how students learn it seems that this module does not effectively support cumulative knowledge building.
We would like to suggest that in order to assess students’ ability to decontextualize concepts, it is needed to ask more descriptive questions and questions that are not clearly located in a particular section of the textbook. In order to do this introducing the idea of ‘core concepts’, which different sections of the curricula draw on, would be one approach. An example of a core concept within the context of organic chemistry would be the recognition of the importance of bond polarity in creating nucleophilic and electrophilic centres. This use of core concepts requires the identification of the handful of essential abstract ideas that students must master, and then linking each section of the curriculum explicitly to one or more of these core concepts. This is the ‘semantic wave’ which Maton requires for cumulative learning (Maton, 2014).
Thus a small set of core concepts are used to scaffold the entire course, and these can then be tested in addition to the more specific, example based, context dependent questions we usually encounter. As soon as a question is located in a specific example it moves the focus away from a general understanding of the concept, and in this format where questions are structured generic to the text book, students become adept at recognising patterns rather than interpreting concepts. The result being that as soon as this pattern is disrupted or less explicit, the students do not recognise the concepts that are inherent to the situation. This is what happens when the context in which the concepts are used changes. The pattern changes and as a result students struggle to recognise these already familiar concepts when they are confronted with it in unfamiliar contexts, such as subsequent health science courses, during their studies. Thus in order to assess whether a student truly understands a core concept and will potentially be able to transfer the knowledge gained in this course to follow-up courses, it is necessary to assess these concepts outside of such a recognisable context or pattern.
To decontextualize questions it is thus necessary to remove it from specific examples. An example of such a question, related to the topic of acid–base equilibrium, might be the following:
An unknown acid reacts in a 1:1 molar ratio with sodium hydroxide. The pH of a 0.12 M solution of this acid at 25 °C is 2.62. What does this information tell you about the acid? Explain your reasoning.
In order to ask more descriptive questions lecturers will need to identify what the core principles are that students need to master in a particular module. And here it might be necessary to distinguish between core concepts and more general concepts. There are a great number of concepts that forms part of the first year chemistry curriculum but many of these concepts are built on a singular core principle. For example, the concepts around pH, pOH, neutralisation etc. all require a fundamental understanding of the core principle of chemical equilibrium. In far too many cases pH calculations are taught and assessed from a point of pure mathematical manipulation, with no understanding of the chemical significance. Manipulation of the equations is important but it is not the core concept which students need to understand if they need to apply the principle to other contexts.
This journal is © The Royal Society of Chemistry 2018 |