Stefanie
Schwedler
* and
Marvin
Kaldewey
Universität Bielefeld, Bielefeld, Nordrhein-Westfalen, Germany. E-mail: stefanie.schwedler@uni-bielefeld.de
First published on 15th June 2020
Research in the past decades repeatedly revealed university students’ struggles to properly understand physical chemistry concepts. In contrast to school, tertiary teaching relies heavily on the symbolic level, mainly applying abstract representations such as equations and diagrams. To follow the lessons and generate conceptual understanding, students need to connect those representations with macroscopic and submicroscopic aspects of the scientific concept depicted. For German first-year chemistry students, this increase in abstraction in a major subject of study contributes to excessive demand and demotivation (especially during out-of-class learning) and increases the risk of early dropouts. We designed a simulation-based learning environment (BIRC: Bridging Imagination and Representation in Chemistry) to suit the needs of first-year students and support them when learning physical chemistry at home. Our approach, featuring molecular dynamics simulations, requires students to assess their own mental models on the submicroscopic level and connect them to equations and diagrams on the symbolic level. Prior studies did already highlight the potential of individual BIRC learning units to foster conceptual understanding on specific topics. In this paper, we investigate if and how students of a broader sample use these learning activities as voluntary supplement beside regular coursework. During the term, we used think-aloud protocols, interviews and eight online questionnaires to analyse students’ mental and emotional interaction while working on BIRC, assessing whether students perceived BIRC as a suitable, enjoyable and supportive resource to enhance learning. Via two paper & pencil achievement tests we examined students’ retention concerning submicroscopic mental models and their ability to connect these mental models with symbolic representations 5–10 weeks later. Our findings indicate a cognitively engaging, comparably enjoyable learning process, which strengthens conceptual understanding and – despite the necessary time and effort – induces a broad number of students to voluntarily work on several units at home.
First year chemistry majors especially struggle to understand abstract concepts in mathematics and physical chemistry (Schwedler, 2017). Hence, feelings of excessive demand and stress occur frequently, especially during self-learning activities and homework. Many national projects aim to foster first-years’ abilities regarding mathematics (Dehling et al., 2014; Frettlöh and Hattermann, 2016), but similar strategies concerning physical chemistry have yet to be explored. Since physical chemistry as an important chemical sub-discipline seems to affect chemistry majors’ motivation stronger than minor subjects like mathematics and physics (Schwedler, 2017, p. 174), this task is of particular relevancy to improving chemistry students first-year experience.
In this paper, we report on the potential of a made-to-measure simulation-based teaching strategy (bridging imagination and representation in chemistry, BIRC, Schwedler, 2019) to support first-year chemistry students during self-learning activities in physical chemistry, aiming to foster their conceptual understanding, enable an enjoyable learning experience and possibly prevent feelings of excessive demand.
Conceptual understanding is a complex phenomenon. According to Nieswandt (2007), it includes not only the understanding of simple as well as complex scientific concepts, but also refers to students ability to apply, relate and connect these concepts to scientific phenomena in a meaningful way. Beside a lack of mathematic skills, lack of conceptual understanding contributes to students’ struggles concerning “numbers, […] letters and Greek symbols” (Schwedler, 2017, p. 175), since applying these models and solving problems requires students to understand the chemical conceptual background (Goldhausen, 2015). Accordingly, Nicoll and Francisco (2001) found conceptual understanding to be one of the best predictors for academic success in physical chemistry. Unfortunately, even students of advanced courses struggle to understand basic concepts (Bain et al., 2014; Bennett and Sözbilir, 2007; Nilsson and Niedderer, 2014). Many students who are able to solve problems correctly (usually by algorithmic means) and achieve high scores in exams, nevertheless nurture misconceptions (Becker and Towns, 2012; Hadfield and Wieman, 2010) and fail to understand the phenomenon in depth. According to Sözbilir (2004), students explicitly wish university teaching to focus more strongly on conceptual understanding.
One major obstacle, which hinders students from creating accurate mental models, is the abstract nature of physical chemistry concepts (Carson and Watson, 2002; Sözbilir, 2004, Goedhart and Kaper, 2002). Learners struggle to differentiate between system and surrounding, between state and process, between thermodynamic and kinetic aspects (Sözbilir et al., 2010) and to acknowledge the limits of validity for the laws and concepts in question. Moreover, misconceptions from everyday life interfere with scientifically accepted conceptions (e.g. treating heat like a substance, Bain et al., 2014; Bain and Towns, 2016). Although many of these difficulties are pronounced in physical chemistry, they can be found in other areas as well.
Relating to all areas of chemistry, Johnstone (2000) proposed that teaching strategies should holistically integrate the macroscopic, submicroscopic and symbolic level of chemical concepts in order to facilitate understanding. Accordingly, it is not enough to observe a chemical process in every-day-life or a laboratory environment (macroscopic) and depict measured data in plots and diagrams (symbolic). One also has to explain observations and symbols on the particle level (submicroscopic) and connect all three levels appropriately. In Johnstone's eyes, meaningful learning in chemistry requires exploring the “interior of the triangle” (Johnstone, 1993, p. 703), which most learners are so very unfamiliar with, that Johnstone compared it to a black hole (see Fig. 1).
His idea of three essential levels to learning chemistry has echoed greatly through science education literature, inducing many researchers to expand on it. For example, Devetak created the related Interdependence of the Three Levels of Science Concepts model (ITLS, Devetak et al., 2009) to emphasize the importance of connecting the levels for building appropriate mental models und develop a deeper understanding. This triangle also applies to physical chemistry concepts, which are usually based on complex and often dynamic interactions between huge entities of submicroscopic particles. Maxwell's speed distribution, for example, models a specific aspect of the dynamic interaction between trillions of gas particles. Concerning kinetics, the relation to particle entities is obvious. For thermodynamics however, the connection is subtle. When Boltzmann established his statistical definition of entropy, he successfully linked this originally purely macroscopic discipline for the first time to particle behaviour. In line with these historical origins, the submicroscopic level is often underrepresented in physical chemistry textbooks (Nyachwaya and Wood, 2014) as well as teaching strategies (Becker et al., 2015), whereas both put much emphasis on the mathematically-symbolic level.
In view of the difficulty to imagine the complex and dynamic behaviour of great particle entities, we believe many students to lack appropriate mental models on the submicroscopic level for physical chemistry concepts, restricting their ability to understand related equations and diagrams. This is in line with Becker et al. (2015) and Hernández et al. (2014), who stress the importance of successfully connecting the submicroscopic and symbolic level to gain conceptual understanding. Furthermore, they call for the development and evaluation of suitable instructional strategies to explicitly foster this connection. Most research done in this area focuses on in-course teaching strategies for comparatively small groups. We report on the development and evaluation of simulation-based self-learning activities (BIRC, Bridging Imagination and Representation in Chemistry) to individually support independent, out-of-class learning for a large and very heterogeneous group of first-year students. Moreover, this is the first time such a concept to facilitate the connection between Johnstone's levels in physical chemistry has been explored for German tertiary education.
1. What portion of the student body does voluntarily use BIRC activities as supplementary learning material, how many activities do students complete and what time does it take on average? Do the activities suit students’ rather heterogeneous abilities?
2. To what extent and for what reasons are students mentally activated as well as emotionally involved when working on BIRC and perceive these activities as a supportive resource for out-of-class learning?
3. How does working on BIRC impact students’ conceptual understanding, especially regarding submicroscopic conceptions and their link to abstract representations, in a broader student sample? Will the number of BIRC units completed correlate with students’ conceptual abilities at the end of the term?
Since this paper reports on German learning activities, which were evaluated in German, all materials, methods and data have been translated by the authors. Hence, nuances of the original may be lost in translation.
Field of study | Pre-test | Single-case | Year of study | Pre-test | Single-case |
---|---|---|---|---|---|
Chemistry, major | 168 (44.1%) | 38 (50.0%) | 1 | 290 (76.1%) | 68 (89.5%) |
Biochemistry, major | 84 (22.0%) | 24 (31.5%) | 2 | 61 (16.0%) | 8 (10.5%) |
Chemistry teaching | 32 (8.4%) | 4 (5.2%) | 3 and more | 30 (7.9%) | 0 |
Chemistry, minor | 97 (25.5%) | 10 (13.2%) | All | 381 | 76 |
All | 381 | 76 |
Since pre- and post-test were administered during class time, participants are limited to those choosing to attend lectures. Dropout rates from pre- to post-test (46%) are considerable and reflect course attendance. According to the corresponding lecturer, such rates are common due to high enrolment fluctuations as well as decreasing student presence during the term, since lecture attendance is not compulsory in Germany.
Molecular dynamics simulations are especially suitable to achieve the desired goals for several reasons. First, simulations are able to help students visualize the dynamic behaviour of large particle entities. Second, they can be designed as multiple representations, touching and connecting all three levels of Johnstone's triangle. Third, Landriscina (2009, p. 29) argues for computer simulations to be “the most suitable instructional method [to induce] a restructuring of the students’ individual mental models.” In order to achieve this, learners need to reflect on their own conceptions and assess them using simulations (Landriscina, 2013, p. 101). Especially mentally simulating the behaviour of a system in one's own head (mental simulation, Landriscina 2013, p. 135) has been shown to improve understanding and transfer knowledge to long-term-memory (imagination principle, Leahy and Sweller, 2004; Monaghan and Clement, 1999). And fourth, the emphasis on understanding and particle behaviour is in line with first-year chemistry students’ expressed interests (Sözbilir, 2004; Schwedler, 2017).
Since simulation learning is especially suited to restructure and reinforce pre-existing knowledge (Thomas and Hooper, 1991, p. 510), BIRC activities are designed to be worked upon at home after the corresponding lecture (see Fig. 2). Each unit takes roughly 15–30 minutes and is structured into three successive parts (Schwedler, 2019).
![]() | ||
Fig. 2 General structure of a BIRC unit and its intended schedule between lecture and regular homework. |
Part 1 begins describing and depicting a chemical system (also on the submicroscopic level) which relates to a major concept of physical chemistry. Students are required to answer an open question on the (future) behaviour of the system. They need to activate their own molecular mental models – in many cases to mentally simulate how the system evolves in time – in order to answer it. For example, part 1 of BIRC activity 3 (real gas law) features a depiction of two containers with two different gases, varying in molecular size and intermolecular attractions (see screenshot Fig. 3).
Students are instructed to consider how these particle properties are going to impact gas pressure inside the two containers. Therefore, they have to mentally simulate the impact of a change in both attractive forces and particle size on dynamic particle interactions. If doing so properly, learners might come to the conclusion that particle size decreases the effective volume within the container, forcing particles to bump into each other more readily, and consequently increases pressure. By contrast, intermolecular attraction slows particle movement down and hence decreases pressure. Subsequently, four different answers are presented for learners to choose from, supporting their reflections and forcing them to take up a position, since they can’t navigate to the next step without choosing one.
With their choices in mind (and also displayed on the screen), students work on interactive molecular dynamics simulations in part 2 (try). These made-to-measure simulations are based on the interface and algorithms of molecular workbench next generation (Tinker and Xie, 2008). Guided exploration and observation allows students to discover formerly hidden relationships, such as the dependency of pressure on particle size and intermolecular attractions in activity 3, assessing and refining their own mental models. During this part, students are provided with individual feedback and explanation, but not before working on the simulations and possibly reconsidering their choice from part 1 by themselves. After focusing on submicroscopic mental models, the emphasis of instruction shifts towards the symbolic level.
Part 3 focuses on equations, diagrams and different mathematical models connected to the scientific concept. Instruction requests students to relate these representations to their observations in part 2 in order to link submicroscopic and symbolic aspects of their conceptions. For example, BIRC activity 3 focuses on van der Waals’ equation and related p–V diagrams. Van der Waals expanded on the ideal gas law, which describes the dependencies between pressure p, temperature T, volume V and amount of substance n, as long as gas particles can be approximated as point masses without attractive forces. He introduced two coefficients a and b into the equation to account for particle size and intermolecular attraction (eqn (1) and Atkins et al., 2017, p. 45).
![]() | (1) |
Students – already familiar with the ideal gas law – are given the equation without specifying, which of the two coefficients a and b is related to particle size and which to intermolecular attractions. With help of an interactive p–V-diagram, learners explore the impact of a and b on the curve and compare this to the system behaviour observed during simulation. As a result, they can assign both coefficients to particle properties. Further tasks explore the behaviour and limits of van der Waals’ model at phase transition. Learning tasks and questions are, where appropriate, accompanied by optional support and explanatory feedback.
This instructional approach takes a middle ground between pure constructivist discovery learning and pure cognitivist reception learning, since simulation learning proves to be much more efficient in closely instructed settings than in settings with little guidance (Landriscina, 2013, p. 138; Mayer, 2004; Stieff and Wilensky, 2003). It encourages learners to actively explore and discover new relationships in order to change pre-existing mental models (diSessa and Sherin, 1998; Posner et al., 1982). But in contrast to more constructivist settings, students follow a well-orchestrated learning path comparable to the principles of programmed discovery. This path enables them to discover relationships and adjust the degree of difficulty (students can take their time, exploit optional support and further challenges, switch back and forth in the learning path at will), but stays focused on the core subject and refers to empirically proven principles of multimedia learning to enhance generative processing (such as imagination, worked examples, self-explanation, feedback, see Mayer, 2014). Furthermore, it is designed to take comparatively little time (20–30 minutes per unit on average). Considering first-year chemistry students’ notorious lack of time (Schwedler, 2017) this will probably enhance the acceptance and usability of BIRC as a voluntary learning tool considerably. As of yet, eight different BIRC activities have been developed according to this strategy (see Table 2).
No. | Physical chemistry concept | Mathematic tool |
---|---|---|
1 | Ideal gas laws | Partial differentials |
2 | Speed distribution in ideal gases | Distribution functions |
3 | Real gas law (van der Waals) | Functions and parameters |
4 | Change of inner energy | Increments and integrals |
5 | Volume-related work | Integrals |
6 | Energy distribution (entropy) | Logarithms |
7 | Temperature and heat (addition to 4) | Sums |
8 | Work and heat (addition to 5) | Vectors |
Concurrent think-aloud protocols (van Someren et al., 1994) consist of two parts: first, students verbalise their thoughts and emotions while working on the learning activity for 15–30 minutes. Participants are observed by a researcher, who encourages them to keep talking but is not allowed to provide help. Data collection include video-taping, recording desktop-streams and observation protocols. Second, in subsequent qualitative interviews (15–20 minutes, semi-structured) participants are questioned concerning their conceptions, learning outcomes, perceived support by the activity, emotional experience and individual attitude towards physical chemistry. The interview mainly consists of open questions to better access participants’ views and experiences. Additionally, closed questions (Likert, 4 and 5 steps, respectively) also assess students’ perceived level of competency, perceived support and emotional experience. Data collection focuses on audio-taping.
Retrospective interviews (30–45 minutes, semi-structured) take place a few days after students have been working on a BIRC unit at home. In the first part, students’ mental interaction with BIRC are reconstructed from memory. Students are prompted to relive their experience and narrate as much as they remember. The second part resembles the interview used in think-aloud protocols. Data collection focuses on audio-taping.
Both methods allow prompt, clarifying questions as well as deep insights into the conceptions and feelings of individual participants. The closed questions facilitate comparisons with findings from online-questionnaires and post-test.
Activity | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
N | 72 | 51 | 57 | 43 | 46 | 54 | 24 | 19 |
The pre-test (N = 381) questionnaire collects demographic data (e.g. subject of study, year of study, final school exam scores etc.) and investigates students’ conceptual abilities (skilltest 1), focusing on submicroscopic conceptions relating to subjects known from school or preceding introductory chemistry courses, such as speed distribution, inner energy, particle model of ideal gases and entropy.
The post-test questionnaire examines students’ use of, support by and emotional response to BIRC learning units. The second part on conceptual abilities (skilltest 2), focusing on submicroscopic conceptions and links to equations and diagrams, is used to evaluate the influence of BIRC on conceptual understanding. Since these advanced topics (van der Waals equation for real gases, path-dependency of volume-related work, boundary conditions of the ideal gas law) are very unlikely to be taught before the physical chemistry course, the influence of prior knowledge on students test results should be comparatively low. Moreover, the post-test takes place five to ten weeks after most students have been working on the corresponding BIRC units 1, 3 and 5, underlining its follow-up-like nature.
In the following section, the structure and design of skilltests 1 and 2 are discussed in more detail.
Skilltest 1 contains four questions: an open question on entropy, a multiple-choice-question on Maxwells’ speed distribution function and two two-tiers (multiple choice combined with open explanation, see Fig. 4) on inner energy and ideal gas particles. Skilltest 2 contains three questions in equal proportions, hence results can be correlated with the number of BIRC units worked on. Question 1 and 3 are two-tiers (ideal gas and volume-related work), requiring students to apply their knowledge on experimental findings or settings. In questions 2, students have to explain as detailed as possible, how van der Waals’ equation on real gases is connected to particle properties.
![]() | ||
Fig. 4 Question 3 of skilltest 1, translated from Schwedler, 2020, p. 258. |
All questions have been developed and qualitatively validated using prior single-case studies (between 7 and 20 single case studies per question) and a paper & pencil-questionnaire on second-year-students (N = 17). Skilltest 1 purposefully explores students’ (submicroscopic) conceptions on familiar subjects, while skilltest 2 touches advanced subjects, which are nearly unknown prior to university and related to, but not identical with those of skilltest 1. Using questions on unfamiliar topics in skilltest 1, students’ abilities concerning submicroscopic conceptions might be underestimated.
To thoroughly investigate students’ mental interaction with BIRC, their learning gains, emotional experience and perceived support, 76 qualitative case studies (think-aloud, N = 44 and interviews, N = 32) on six BIRC activities have been conducted. Qualitative studies to activities 1, 3, 5 and 6 were carried out in winter term 17/18, those to activities 7 and 8 in winter term 18/19, respectively. Students’ use, emotional acceptance, perceived support and conceptual understanding of a broader sample have been evaluated via paper & pencil questionnaires as well as skilltests during the first and last lecture of the term (N = 381 and N = 205), and via online-questionnaires (N = 366), one for each activity, directly after working on BIRC. Considering the timeline, these online questionnaires more closely resemble individual post-tests, whereas the second paper & pencil test mirrors a follow-up examination.
While pre- and post-test are designed to include as many active students as possible and hence will include participants of single-case-studies and online-questionnaires, students participating in single-case-studies will not fill in the online questionnaire for the respective BIRC-unit.
Multifactorial linear regression via SPSS was carried out to examine, whether BIRC activities do influence students’ conceptual understanding. This technique assumes a linear relationship between multiple independent variables xi and the dependent variable Y (see eqn (2)). To better compare the impact of independent variables, standardized coefficients i have been calculated.
Y = β0 + β1·x1 + … + βi·xi + ε | (2) |
The regression model features five variables: final school exam scores, skilltest 1 scores, students’ interest in physical chemistry, the number of voluntary weekly exercises completed and the number of completed BIRC activities. Our approach is based on a general model for academic success (Schiefele et al., 1992), assuming cognitive abilities, motivation and interest to be major contributors. Freyer et al., (2014) did successfully specify the model to predict first-year chemistry students’ success in an introductory general chemistry exam and identify relevant contributors. In the same style, final school exam scores and skilltest 1 scores model cognitive abilities in this study. Freyer et al. asked students, whether studying chemistry was their first choice, in order to account for motivation. We used a different approach, since the motivation to study chemistry as a whole might be very different from studying physical chemistry. What is more, measuring participants’ motivation at one point in time does not necessarily account for student behaviour during the term. Hence, motivation to study physical chemistry is operationalised by the amount of voluntary coursework (number of voluntary weekly exercise and the number of BIRC activities) participants completed during the term. Students’ interest in physical chemistry is surveyed by adapting two items from Busker and Parchmann (2010). Four of these five factors yield metric data, only student interest is measured via Likert-scale. Regression itself was performed via SPSS using the five factorial model described above. Since two factors did not yield significant correlations, a three-factorial regression after backwards elimination was also carried out.
At the end of term, 33% of all post-test participants had voluntarily completed all six regular activities (see Table 4a and b). 60% had completed at least 4 of the 6 units. 16 participants expressed the plan to work on more activities during exam preparation. Each BIRC activity was designed to take between 15 and 30 minutes. 56% of the participants estimated the time needed accordingly (Table 5). 36% reported durations of less than 15 min, 9% more than 30 min. The average student completed 4.0 activities and needed 19.0 minutes per unit.
(a) | |||||||
---|---|---|---|---|---|---|---|
Number of BIRC units completed | 0 | 1 | 2 | 3 | 4 | 5 | 6+ |
Student rate in % (N = 205) | 18 | 5 | 9 | 8 | 12 | 15 | 33 |
(b) | ||||||||
---|---|---|---|---|---|---|---|---|
BIRC unit | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
Student rate in % (N = 205) | 77 | 66 | 63 | 51 | 52 | 53 | 54 | 41 |
Duration for one unit | <10 min | 10–15 min | 15–20 min | 20–30 min | >30 min |
---|---|---|---|---|---|
Student rate in % (N = 205) | 4 | 32 | 35 | 21 | 9 |
Participants completing few or no units (usually the first ones) were asked to give their reasons. The question was answered by 79 of the 82 students completing 3 or less activities and nine students completing four activities.
Of the 88 answers given, 36 claimed lack of time, 23 stated to have other priorities coming first. In January, when excessive demand is particularly high (Schwedler, 2017), user rates dropped, since students focussed on preparations for other exams and laboratory work besides having to do regular classwork. 19 students expressed lack of discipline and/or ability to organize themselves, 16 students claimed they would work on more units during the upcoming exam preparation. 12 students professed a lack of motivation and/or interest concerning physical chemistry in general. Other factors, such as BIRC not being relevant to pass exams, BIRC being incompatible with smartphones§ and BIRC not matching students’ needs or preferred style to learn, were rare and mentioned by one to five students each.
164 of all post-test participants have completed at least one unit and can hence report on the level of difficulty. 74% of post-test students’ consider the degree of difficulty to be exactly appropriate (see Fig. 6); the remaining quarter distributes fairly evenly between (rather) easy (15%) and (rather) difficult (11%). Findings of other questioning methods (single-case studies, online questionnaires) are quite similar.
![]() | ||
Fig. 6 Perceived degree of difficulty, translated from Schwedler, 2020, p. 315. |
Participants predominantly regard BIRC as matching their capabilities. Considering students’ heterogeneity (Schwedler, 2017) this highlights BIRCs ability to account for several levels of difficulty. Reasons for not (often) using BIRC are mainly related to a lack of time as well as different priorities, and much less frequent with the concept itself. Although students completing more than three activities were not specifically targeted, nine students completing four units felt the need to give their reasons for not completing all units anyway. Since their answers matched those of students completing two or three activities, they were kept in the dataset.
![]() | ||
Fig. 7 Results on perceived enjoyment by post-test, including a cross-check to single-case-studies (sc), translated from Schwedler, 2020, p. 324. |
“It was fun to use the simulations, and I liked the challenge to find the right answers […]. But working on equations is rather boring.”
14 students (8%) stated reasons for BIRC being boring, but named no positive aspects. The other 34% gave no answer.
Table 6 lists all student responses. In order to gain a better overview over the reasons for positive and negative emotions as well as their frequency, ambivalent feedback, containing separate reasons for enjoyment and boredom, was split and added to the categories reasons for enjoying BIRC and reasons for BIRC being boring respectively. Students liked to interact with simulations and diagrams, they express having fun with these tools (n = 30) as well as enjoying the interactivity (n = 29). 27 students enjoyed its quiz-like structure: being made to think and choose, check their answer themselves and get feedback, especially when doing so successfully. 19 students expressed a general pleasure to understand and learn, describing their enjoyment in having a light bulb moment. Other positive reasons comprise the descriptive design, general claims of BIRC being fun and a general interest in chemistry.
(Reasons for) enjoying BIRC, answers from positive only (n = 78) and ambivalent (n = 9) students | |||
---|---|---|---|
Positive | Ambivalent | All | |
Using simulations and diagrams | 27 | 3 | 30 |
Interaction, be active during learning | 26 | 3 | 29 |
Quiz-like (think first, check result, feedback), feeling of success | 23 | 4 | 27 |
Pleasure to learn, light bulb moment | 18 | 1 | 19 |
Descriptive, visual design | 10 | 1 | 11 |
BIRC units are generally fun | 9 | 9 | |
Interest in chemistry, new perspective on physical chemistry | 7 | 7 | |
Neutral reception (n = 7) | |||
• Neither boring nor enjoyable | |||
• Can’t associate learning with fun |
(Reasons for) BIRC being boring, answers from negative only (n = 14) and ambivalent (n = 9) students | |||
---|---|---|---|
Negative | Ambivalent | All | |
Physical chemistry is boring/too difficult | 6 | 3 | 9 |
Don’t like maths/equations | 1 | 4 | 5 |
Texts and explanation | 2 | 3 | 5 |
Other | 5 | 0 | 5 |
Reasons for a neutral reception comprise BIRC being helpful but neither boring nor enjoyable, and the statement that learning in itself is not associated with fun. On the negative side, nine students claim physical chemistry as a subject to be boring or not within their abilities. Five students disliked the mathematical content and five were partly bored by texts and/or explanations. Other reasons, such as an unsuitable level of difficulty and a general lack of interest, were mentioned by one to two participants each.
Participants of single-case studies are more positive in their statements (see also Fig. 7: post-test data in bar 3 compared to single-case data in bar 4), probably due to selection and social desirability effects. However, these findings are useful to explicate students’ reasons for enjoying BIRC. During think-aloud sessions, students not only observe the simulations passively but mentally interact with them and re-evaluate the situation. Second-year student TTW4¶ especially pinpoints his enjoyment while watching the simulation unfold.
“It has already started. Ok. […] Uii. […] Yes, right. Ok. Ok. […] Aah, the temperature roughly stays at 500 K. [That] means it would have been answer “(b)”. […] But…no. […] Now [the temperature] is rising, of the metal especially. […] So [that of] Argon stays a little bit behind […]. Why didn‘t we have something as cool as that? That would have been much cooler! Well, that is really descriptive.” (TTW4)
Positive feelings as a result of success are also mentioned by participants who are not fond of physical chemistry. IVA1 initially had quite negative expectations due to the abstract nature of physical chemistry. But the student enjoyed completing BIRC anyway, reasoning as follows:
“Because I experience success and I know, that I understand this now. […] These moments of success make it enjoyable. Especially, since I get the solution after drawing the conclusions on my own. I think it is really fun, although physical chemistry is definitely not my hobbyhorse.” (IVA1)
Many students rather agreed to BIRC being fun, but not fully (see Fig. 7). A recurring motif implies, that learning and work in general cannot be as much fun as other activities, even though BIRC is seen as comparatively enjoyable learning strategy. TVA7 and TRG11 state it like this:
“Well, I wouldn’t say I’d prefer BIRC to an amusement park.” (TVA7)
“If I compare that to the dreary learning one usually does, then it is fun for that matter. Compared to other learning contents, it is in any case better this way.” (TRG11)
Furthermore, interactive simulations and diagrams act as a motivator to deal with more abstract mathematical models in physical chemistry, as TIG4 puts it:
“It helps because it keeps the fun up. One has something interactive, where one […] can see, what happens and what's it all about. And that is why one has more motivation to look at the mathematical background in depth.” (TIG4)
![]() | ||
Fig. 8 Results on perceived support by post-test, incl. cross-checks to online inquiries (ol) and single-case studies (sc), translated from Schwedler, 2020, p. 362 and 364. |
Between 83 and 96% of post-test participants, who completed at least one BIRC unit, (rather) agreed to BIRC being helpful to imagine a concept on the submicro-level, to link submicroscopic conceptions with mathematical representations, to understand the concept and to generally cope with post-lecture work (Fig. 8, left). However, only 44% perceived BIRC as (rather) helpful to solve the weekly exercise (Fig. 8, middle).
Students were asked to explain in more detail, whether and why BIRC did or did not help them. 67% expressed feeling supported, 12% expressed ambivalent experiences, 3% of the students claimed BIRC was not helpful and 18% did not answer the question. On the whole, 110 open answers were given as to how and why BIRC supports students (see Table 7). 92 students found BIRC to induce understanding, 77 underscored its descriptive nature, 61 felt supported by preparing for and post-processing lessons, 33 students claimed BIRC helped them to deal with maths, 31 considered it helpful that it required them to think for themselves and be mentally active and 26 students expressed the support in general terms.
BIRC units help… (n = 110) |
---|
• To induce understanding (n = 92) |
• Because they are descriptive (n = 77) |
• With pre- and post-processing lessons (n = 61) |
• To deal with maths (n = 33) |
• Because one has to be active, think for oneself (n = 31) |
• In general (n = 26) |
These effects are elaborated during single-case studies. TVA6 describes, how BIRC changes her/his handling of equations from rote-learning to understanding their meaning:
“If I see such an equation, I memorize the equation. But I can’t really imagine, what is going on. And that is why these units help, […] because one can see what happens, if [some parameter] changes.” (TVA6)
One major goal is to foster mental simulation on the particle level. While TVA6's description sounds more like passive reception, TRG1 underscores how the concept encourages him to mentally simulate the situation.
“[These units] are helpful in any case. Instead of memorizing ‘if this happens, then that happens’, I can imagine in my own head ‘what is going to happen, if I squeeze the whole thing?” (TRG1)
That BIRC induces students to actively pursue a better conceptual understanding is regularly mentioned by students, for example by TVA7:
“Therefore, these units are really helpful, because otherwise you wouldn’t think about it as deeply. Else you only think ‘Damn, where is my mistake, where is my mistake?’ But you don’t think ‘How does this actually work?’ And that is promoted by this unit.” (TVA7)
Since these studies have been carried out during the term, little is known about their effect on students’ performance during exams. One second term student TRG2, who tested one unit during development, expressed positive experiences during the first term:
“[Physical chemistry] is not a subject that I like. In any case, [BIRC] helped me a lot to learn last term. I don’t know, whether I would have mastered the exam as well without it.” (TRG2)
This single statement can’t substitute systematic research. However, it indicates that at least a small number of students do profit profoundly from BIRC.
In summary, 95% of post-test participants that have worked with BIRC would (rather) recommend it to fellow students, and only 6% (post-test) agreed or rather agreed to BIRC being superfluous (see Fig. 9).
![]() | ||
Fig. 9 Recommendation of BIRC and cross-check, translated from Schwedler, 2020, p. 364. |
Participants, who tried at least one BIRC unit, are found to mostly experience it as a supporting resource, helping them to understand physical chemistry concepts on several levels and prepare or post-process lectures. Results of both single-case studies and anonymous online-inquiries are more favourable than those of the post-test. This discrepancy might stem from selection effects and the fact that post-test questioning is retrospective. Looking back, a support related to one specific topic is put into proportion to the whole term. Concerning closed questions, post-test findings are much more representative and relevant. However, single-case studies (especially think-aloud protocols) explore students’ interaction with BIRC in depth and can reveal valuable learning processes. Findings indicate that the quiz-like simulation-based concept induces students to mentally interact and simulate system behaviour in their mind, to re-evaluate prior conceptions and to change from rote learning equations to understanding their underlying meaning.
C1. Ability to imagine particle behaviour related to concepts and/or phenomena
C2. Ability to interpret, how equations and diagrams relate to submicroscopic conceptions
C3. Ability to further explain concept-related phenomena, terms and definitions
C4. Ability to explain/apply specific mathematical models and expressions
In inquiry, the expressions employed to describe competencies 1–4 are much more specific with regard to the corresponding scientific concepts. Findings of all BIRC units are summarized in Fig. 10.
![]() | ||
Fig. 10 Self-assessment of competencies 1-4 before and after working on BIRC, translated from Schwedler, 2020, p. 351. |
Before working on BIRC, 54% to 73% (rather) disagree to master these competencies; afterwards these rates range from 4 to 23%. Students clearly believe to have improved their skills. However, these are only self-assessing estimates, and the corresponding surveys took place directly after working on BIRC. Consequently, they do not prove long-term learning gains.
Multifactorial linear regression (n = 185)** was employed to analyse the relationship between the number of BIRC units completed and skilltest 2 scores. The five-factorial model includes final school exam scores, skilltest 1 scores, interest in physical chemistry, number of worked-on voluntary exercises and number of relevant BIRC activities to predict the outcome of skilltest 2. Other factors (such as the last school report mark in chemistry) have been explored but did not significantly contribute to the regression.
Regressions applying five factors yielded statistical significances of p < 0.001 for the whole model and explained between 24 and 31% of the variance observed. These findings are in line with Freyer et al. (2014). Table 8 lists the individual standardized coefficients as well as statistical significances pβ related to these coefficients. However, only three of the five factors contributed significantly to regression analysis: the number of relevant BIRC units completed, skilltest 1 scores and final school exam scores.
5 independent variables | 1a both cohorts | 1b WT 17/18 | 1c WT 18/19 | |||
---|---|---|---|---|---|---|
![]() |
p β |
![]() |
p β |
![]() |
p β | |
Final school exam scores | −0.143 | 0.033 | −0.051 | 0.610 | −0.245 | 0.008 |
Skilltest 1 scores | 0.229 | 0.001 | 0.193 | 0.043 | 0.247 | 0.010 |
Number of relevant BIRC units | 0.431 | <0.001 | 0.496 | <0.001 | 0.302 | 0.003 |
Number of voluntary exercises | −0.054 | 0.434 | −0.099 | 0.307 | −0.105 | 0.310 |
Interest in physical chemistry | 0.021 | 0.750 | 0.036 | 0.698 | 0.004 | 0.968 |
Corrected R2 | 0.27 | 0.25 | 0.31 |
3 independent variables | 2a both cohorts | 2b WT 17/18 | 2c WT 18/19 | |||
---|---|---|---|---|---|---|
![]() |
p β |
![]() |
p β |
![]() |
p β | |
Final school exam scores | −0.145 | 0.026 | −0.029 | 0.765 | −0.284 | 0.002 |
Skilltest 1 scores | 0.210 | 0.001 | 0.178 | 0.055 | 0.235 | 0.007 |
Number of relevant BIRC units | 0.423 | <0.001 | 0.488 | <0.001 | 0.357 | 0.001 |
Corrected R2 | 0.26 | 0.24 | 0.30 |
For all regressions, the number of relevant BIRC units has the strongest and statistically most significant influence on students’ performance in skilltest 2. The unstandardized coefficient in regression 1a reveals each BIRC unit to increase student performance on average by 7.4% of all points in skilltest 2. However, this effect is much more pronounced in cohort 1 than in cohort 2. Since two of the five factors do not seem to play a significant role, the model was further reduced to three factors using backward elimination and no notable loss in explained variance occurred. Furthermore, by reducing the amount of data needed, four additional respondents could be included into the regression.
Even though skilltest 2 scores varied over nearly the whole spectrum, they were quite low on average, underscoring students’ general struggle to understand physical chemistry concepts. Moreover, open explanations require students to having understood the concept in the first place and additionally remember it five to ten weeks later. However, this strategy suits a major strength of simulations in learning: reinforcing and reorganizing acquired content (Thomas and Hooper 1991, p. 510).
Although an increase of 7.4% of all skilltest 2 points per BIRC unit does not sound much at first glance, it is to be considered significant in view of the low average skilltest 2 scores (34%). These data also are in line with the doubling of average skilltest 2 scores for students completing all three units (43%) compared to students completing none (21%).
Most quality criteria of the multifactorial linear regressions (use of a linear model, metric variables, exogeneity as well as normal distribution of residua, non-collinearity of independent variables) were thoroughly fulfilled. Two constraints have been found: first, homoscedasticity is limited due to the fact that test scores cannot yield results below zero. Second, even though data of two consecutive cohorts have been used, sample sizes are, strictly speaking, too small to accurately quantify -coefficients of a five-factorial model (Maxwell, 2000). However, only three factors significantly contributed to the model, and for three-factorial models, sample-sizes were appropriate. Moreover, this study did not aim for precise quantification, but rather to detect a relationship between BIRC and conceptual understanding at all. The data are able to confirm this, since the number of BIRC units worked on has consistently proven to be the best predictor in all linear regressions performed.
Although the impact of BIRC activities cannot be completely separated from other influences such as motivation and work-ethics, completion of voluntarily weekly exercises as a factor to account for work-ethics and motivation does not significantly contribute to student performance in skilltest 2. Hence, the influence of BIRC is unlikely to be purely attributed to motivation.
It seems that students in cohort 1 learned more from BIRC than those in cohort 2. This effect matches students’ self-assessments on learning gains. During winter term 17/18, single-case studies (Schwedler, 2019; Schwedler and Lyczek, 2019) on BIRC units relevant for skilltest 2 have been carried out. This probably increased skilltest 2 scores, since some students not only worked on BIRC but additionally talked about it to researchers. Nevertheless, even for the second cohort, BIRC remained the most important factor to predict skilltest 2 scores. The other significant factors, skilltest 1 scores as a measure for prior knowledge and final school exam scores as a measure for cognitive abilities, have already been established as important predictors for first-year chemistry students’ academic success by Freyer et al. (2014). Furthermore, the significant relationship between skilltest 1 and 2 scores justifies our approach to employ students’ submicroscopic conceptions on familiar topics in skilltest 1 as a predictor for students’ submicroscopic conceptions and abilities to link them with abstract maths in skilltest 2.
Participants voluntarily choose to work on BIRC, hence students’ acceptance of BIRC and the impact BIRC has on their conceptual abilities is assessed under realistic circumstances. However, it is difficult to reliably separate the influence of BIRC from other factors (such as motivation) due to possible selection effects.
Questioning students during lectures reaches most of the active student body and hence allows for a reasonable sample size. However, the high course drop-out rate suggests an under-representation of students skipping lectures in order to better prepare for the upcoming exams and of students quitting the course (for various reasons) at an early stage. Hence, the study might inadvertently select (at least basically) motivated and successful students, which is very difficult to avoid for a term-long survey. From a holistic point of view, possibly not reaching early strugglers poses a severe limitation of this study. On the other hand, an early drop-out and hence professional reorientation does not have to be negative: It is neither the aim nor a useful benchmark for concepts such as BIRC to keep students, who quickly find their abilities, interests and motivations to severely mismatch their chosen courses (or even studying as a whole), into their major field of study.
What is more, testing during lectures also requires pre- and post-questionnaires to be rather short, allowing for not more than 15 minutes to complete skilltest 1 and skilltest 2 respectively. This limits reliability when measuring conceptual understanding. Although this approach might not be suitable to detect small impacts or quantify the effects with very high accuracy, it should reveal moderate or strong correlations between the use of BIRC and students’ performance, simultaneously accounting for other factors.
BIRC itself was created to match this high level of self-organisation and autonomy in learning, which might not be suitable for all students, let alone other learning cultures. Although BIRC does prompt students to perform certain tasks, students are not forced to do them. Holding students accountable might change students’ reception and performance. Furthermore, the concept was developed as a stand-alone, hence other learning activities (such as exercises, lectures and laboratory classes) do not specifically reference or rely on them. This approach makes it easier to separate the effect BIRC has on students’ conceptual abilities, but it restricts the power of these activities.
Although some aspects leave room for improvement, the majority of students regularly completed BIRC and experienced working on BIRC as mentally activating, comparably enjoyable and fostering their conceptual abilities. In retrospect, students viewed the units as valuable and descriptive support to increase understanding and help prepare and post-process regular lessons. However, only half of the students perceived BIRC as supportive concerning weekly exercise problems. Linear regressions reveal a moderate but significant impact of BIRC on long-term conceptual understanding.
Regarding these findings, we propose to further explore simulation learning as a tool to help students overcome learning obstacles in tertiary physical chemistry. One major task is to shift student behaviour from algorithmic rote learning to more meaningful learning processes. Although our results indicate a successful change in that direction, the impact of simulation learning in general, and BIRC specifically, on student learning processes in physical chemistry need to be explored in greater width and depth. It also remains to be investigated, whether and how simulation learning might facilitate students’ reasoning on the submicroscopic level in order to fill the diagnosed gap by Becker et al. (2013) and Hernández et al. (2014). Single teaching staff members reported on more students using the submicroscopic level in oral examinations of the following term, but no systematic study has been done. Moreover, according to student self-reports, BIRC does not seem to help students with solving weekly exercise problems as much as one could wish. This finding should be further investigated, e.g. by assessing BIRC's effect on student performances in weekly exercises. What is more, possible reasons for this phenomenon as well as alternative strategies to fill the gap remain to be explored. Since this study explored students’ emotional experience while working on BIRC in general, future studies should examine its impact on students’ attitude towards and interest in physical chemistry in greater depth.
Notwithstanding BIRCs flexibility as a teaching tool, the power of stand-alone best-practice supplementary material, which makes up only a small portion of the overall course, is restricted. Adjusting other classwork such as lectures and/or weekly exercises to further support simulation learning, for example giving students the opportunity to discuss their findings with colleagues, might possibly have an even greater impact on student understanding and attitude towards physical chemistry – as long as teaching strategies are made-to-measure physical chemistry characteristics as well as students’ needs. In addition, a suitable in-class concept might boost students’ interest in this oftentimes unpopular subject and perhaps even contribute to prevent dropout in the future.
Footnotes |
† This is the number of unique participant-BIRC-unit combinations, since each participant can give feedback to more than one unit. |
‡ These are thematically of lesser importance and for interested students only. |
§ Students were informed on this when BIRC was presented during the first lecture of the term. Still, some tried anyway. |
¶ TTW4 as an anonymous codes relates to the method of inquiry (T for think-aloud), the BIRC activity in question (TW temperature and heat) and a consecutive number (4 in this case). |
|| Students scoring 0% have all answered the questionnaire of the post-test, which was unrelated to chemical questions. Concerning the chemical part, some of them gave blank responses, some of them actually tried to answer the questions. |
** For some post-test questionnaires, assignment to a corresponding pre-test-questionnaire was not possible or pre-test data were incomplete. |
This journal is © The Royal Society of Chemistry 2020 |