View PDF VersionPrevious ArticleNext Article
 
DOI: 10.1039/C9RP00211A (Paper) Chem. Educ. Res. Pract., 2020, 21, 1132-1147

Linking the submicroscopic and symbolic level in physical chemistry: how voluntary simulation-based learning activities foster first-year university students’ conceptual understanding

Stefanie Schwedler * and Marvin Kaldewey
Universität Bielefeld, Bielefeld, Nordrhein-Westfalen, Germany. E-mail: stefanie.schwedler@uni-bielefeld.de

Received 19th September 2019 , Accepted 29th May 2020

First published on 15th June 2020

Abstract

Research in the past decades repeatedly revealed university students’ struggles to properly understand physical chemistry concepts. In contrast to school, tertiary teaching relies heavily on the symbolic level, mainly applying abstract representations such as equations and diagrams. To follow the lessons and generate conceptual understanding, students need to connect those representations with macroscopic and submicroscopic aspects of the scientific concept depicted. For German first-year chemistry students, this increase in abstraction in a major subject of study contributes to excessive demand and demotivation (especially during out-of-class learning) and increases the risk of early dropouts. We designed a simulation-based learning environment (BIRC: Bridging Imagination and Representation in Chemistry) to suit the needs of first-year students and support them when learning physical chemistry at home. Our approach, featuring molecular dynamics simulations, requires students to assess their own mental models on the submicroscopic level and connect them to equations and diagrams on the symbolic level. Prior studies did already highlight the potential of individual BIRC learning units to foster conceptual understanding on specific topics. In this paper, we investigate if and how students of a broader sample use these learning activities as voluntary supplement beside regular coursework. During the term, we used think-aloud protocols, interviews and eight online questionnaires to analyse students’ mental and emotional interaction while working on BIRC, assessing whether students perceived BIRC as a suitable, enjoyable and supportive resource to enhance learning. Via two paper & pencil achievement tests we examined students’ retention concerning submicroscopic mental models and their ability to connect these mental models with symbolic representations 5–10 weeks later. Our findings indicate a cognitively engaging, comparably enjoyable learning process, which strengthens conceptual understanding and – despite the necessary time and effort – induces a broad number of students to voluntarily work on several units at home.

1 Introduction

Transitioning from school to university is challenging for students of all fields of study. Although success during first year at university depends on numerous individual, social, institutional and other factors (Bornkessel and Asdonk, 2011), study requirements in first-year STEM courses pose an especially impeding barrier (Heublein et al., 2017). In 2008, roughly 49% of chemistry dropouts named struggles to fulfil study requirements as their main reason to leave – more than in any other major field of study (Heublein et al., 2010, p. 156). In this regard, students report on high levels of content difficulty compared to their prior knowledge, time pressure during the term and exams failure.

First year chemistry majors especially struggle to understand abstract concepts in mathematics and physical chemistry (Schwedler, 2017). Hence, feelings of excessive demand and stress occur frequently, especially during self-learning activities and homework. Many national projects aim to foster first-years’ abilities regarding mathematics (Dehling et al., 2014; Frettlöh and Hattermann, 2016), but similar strategies concerning physical chemistry have yet to be explored. Since physical chemistry as an important chemical sub-discipline seems to affect chemistry majors’ motivation stronger than minor subjects like mathematics and physics (Schwedler, 2017, p. 174), this task is of particular relevancy to improving chemistry students first-year experience.

In this paper, we report on the potential of a made-to-measure simulation-based teaching strategy (bridging imagination and representation in chemistry, BIRC, Schwedler, 2019) to support first-year chemistry students during self-learning activities in physical chemistry, aiming to foster their conceptual understanding, enable an enjoyable learning experience and possibly prevent feelings of excessive demand.

2 Theoretical background

Learning obstacles in physical chemistry

On the university level, elaborate mathematical models describe complex processes in physical chemistry (Tsaparlis and Finlayson, 2014). Consequently, conceptual and procedural mathematic skills are doubtless necessary to succeed in physical chemistry (Bain et al., 2014; Derrick and Derrick, 2002; Hahn and Polik, 2004). This might be one reason, why traditional teaching usually focuses on algorithmic strategies to solve physical chemistry problems (Stamovlasis et al., 2005). Science education researchers have criticized this approach as fostering rote learning of mathematic procedures instead of nurturing conceptual understanding (Hernández et al., 2014; Nakhleh and Mitchell, 1993).

Conceptual understanding is a complex phenomenon. According to Nieswandt (2007), it includes not only the understanding of simple as well as complex scientific concepts, but also refers to students ability to apply, relate and connect these concepts to scientific phenomena in a meaningful way. Beside a lack of mathematic skills, lack of conceptual understanding contributes to students’ struggles concerning “numbers, […] letters and Greek symbols” (Schwedler, 2017, p. 175), since applying these models and solving problems requires students to understand the chemical conceptual background (Goldhausen, 2015). Accordingly, Nicoll and Francisco (2001) found conceptual understanding to be one of the best predictors for academic success in physical chemistry. Unfortunately, even students of advanced courses struggle to understand basic concepts (Bain et al., 2014; Bennett and Sözbilir, 2007; Nilsson and Niedderer, 2014). Many students who are able to solve problems correctly (usually by algorithmic means) and achieve high scores in exams, nevertheless nurture misconceptions (Becker and Towns, 2012; Hadfield and Wieman, 2010) and fail to understand the phenomenon in depth. According to Sözbilir (2004), students explicitly wish university teaching to focus more strongly on conceptual understanding.

One major obstacle, which hinders students from creating accurate mental models, is the abstract nature of physical chemistry concepts (Carson and Watson, 2002; Sözbilir, 2004, Goedhart and Kaper, 2002). Learners struggle to differentiate between system and surrounding, between state and process, between thermodynamic and kinetic aspects (Sözbilir et al., 2010) and to acknowledge the limits of validity for the laws and concepts in question. Moreover, misconceptions from everyday life interfere with scientifically accepted conceptions (e.g. treating heat like a substance, Bain et al., 2014; Bain and Towns, 2016). Although many of these difficulties are pronounced in physical chemistry, they can be found in other areas as well.

Relating to all areas of chemistry, Johnstone (2000) proposed that teaching strategies should holistically integrate the macroscopic, submicroscopic and symbolic level of chemical concepts in order to facilitate understanding. Accordingly, it is not enough to observe a chemical process in every-day-life or a laboratory environment (macroscopic) and depict measured data in plots and diagrams (symbolic). One also has to explain observations and symbols on the particle level (submicroscopic) and connect all three levels appropriately. In Johnstone's eyes, meaningful learning in chemistry requires exploring the “interior of the triangle” (Johnstone, 1993, p. 703), which most learners are so very unfamiliar with, that Johnstone compared it to a black hole (see Fig. 1).


image file: c9rp00211a-f1.tif
Fig. 1 Johnstone's idea of a black hole within his triangle.

His idea of three essential levels to learning chemistry has echoed greatly through science education literature, inducing many researchers to expand on it. For example, Devetak created the related Interdependence of the Three Levels of Science Concepts model (ITLS, Devetak et al., 2009) to emphasize the importance of connecting the levels for building appropriate mental models und develop a deeper understanding. This triangle also applies to physical chemistry concepts, which are usually based on complex and often dynamic interactions between huge entities of submicroscopic particles. Maxwell's speed distribution, for example, models a specific aspect of the dynamic interaction between trillions of gas particles. Concerning kinetics, the relation to particle entities is obvious. For thermodynamics however, the connection is subtle. When Boltzmann established his statistical definition of entropy, he successfully linked this originally purely macroscopic discipline for the first time to particle behaviour. In line with these historical origins, the submicroscopic level is often underrepresented in physical chemistry textbooks (Nyachwaya and Wood, 2014) as well as teaching strategies (Becker et al., 2015), whereas both put much emphasis on the mathematically-symbolic level.

In view of the difficulty to imagine the complex and dynamic behaviour of great particle entities, we believe many students to lack appropriate mental models on the submicroscopic level for physical chemistry concepts, restricting their ability to understand related equations and diagrams. This is in line with Becker et al. (2015) and Hernández et al. (2014), who stress the importance of successfully connecting the submicroscopic and symbolic level to gain conceptual understanding. Furthermore, they call for the development and evaluation of suitable instructional strategies to explicitly foster this connection. Most research done in this area focuses on in-course teaching strategies for comparatively small groups. We report on the development and evaluation of simulation-based self-learning activities (BIRC, Bridging Imagination and Representation in Chemistry) to individually support independent, out-of-class learning for a large and very heterogeneous group of first-year students. Moreover, this is the first time such a concept to facilitate the connection between Johnstone's levels in physical chemistry has been explored for German tertiary education.

Research problem and research questions

Case studies already revealed the potential of single BIRC activities to enhance submicroscopic conceptions of individuals and link them to mathematic-symbolic representations (such as equations and diagrams), at least on the short term (Schwedler, 2019; Schwedler and Lyczek, 2019). But it still remains to be investigated, whether multiple BIRC units as a voluntary resource suit the needs of a broader sample and are hence widely used during first term, providing students with enjoyable learning experiences, and whether the use of BIRC facilitates conceptual understanding not only on the short term, but also leads to a measurable increase in conceptual abilities at the end of term. Therefore, the following research questions need to be addressed:

1. What portion of the student body does voluntarily use BIRC activities as supplementary learning material, how many activities do students complete and what time does it take on average? Do the activities suit students’ rather heterogeneous abilities?

2. To what extent and for what reasons are students mentally activated as well as emotionally involved when working on BIRC and perceive these activities as a supportive resource for out-of-class learning?

3. How does working on BIRC impact students’ conceptual understanding, especially regarding submicroscopic conceptions and their link to abstract representations, in a broader student sample? Will the number of BIRC units completed correlate with students’ conceptual abilities at the end of the term?

3 Methods

In order to answer the research questions, BIRC activities were used as an intervention during the term, assessing its impact via qualitative and quantitative methods of inquiry. Quantitative methods are able to investigate relations in a broader student sample. Hence, we assessed the use of BIRC as well as its impact on conceptual understanding via a pre-post-design (questionnaires and skilltests) for all students present during the lecture. However, these methods are neither open to unexpected findings and students’ individual views, nor can they capture the complexity of students learning experiences and conceptions in depth. Hence, qualitative single-case studies (think-aloud-protocol and interviews), focusing on students’ subjective interpretations, have been carried out with relatively few participants during and after the intervention to triangulate and explain quantitative findings.

Since this paper reports on German learning activities, which were evaluated in German, all materials, methods and data have been translated by the authors. Hence, nuances of the original may be lost in translation.

Participants

All participants are undergraduates studying chemistry or related subjects (see Table 1) and registered in a class on basics in physical chemistry. This course starts in week 7 of the term, since students visit a block course on introductory general chemistry during the first 6 weeks. Even though the physical chemistry course is scheduled for the first term for most participants, not all participants are first-years for two reasons: firstly, chemistry teaching students take this class later according to their schedule. Secondly, in line with the German higher education system, students may delay participation to later terms or revisit due to exams failure. Hence, of the participants, 66.1% are chemistry or biochemistry majors and 76.1% are in their first year (nearly all of them in their first term). 82% of participants are 18 to 20 years old, 2% are 17 years. Single-case studies exhibit similar sample proportions. Institutional Ethics Committee approval for this research was obtained and all students provided informed consent as research participants.
Table 1 Sample proportions concerning the field of study and year of study
Field of study Pre-test Single-case Year of study Pre-test Single-case
Chemistry, major 168 (44.1%) 38 (50.0%) 1 290 (76.1%) 68 (89.5%)
Biochemistry, major 84 (22.0%) 24 (31.5%) 2 61 (16.0%) 8 (10.5%)
Chemistry teaching 32 (8.4%) 4 (5.2%) 3 and more 30 (7.9%) 0
Chemistry, minor 97 (25.5%) 10 (13.2%) All 381 76
All 381 76

Since pre- and post-test were administered during class time, participants are limited to those choosing to attend lectures. Dropout rates from pre- to post-test (46%) are considerable and reflect course attendance. According to the corresponding lecturer, such rates are common due to high enrolment fluctuations as well as decreasing student presence during the term, since lecture attendance is not compulsory in Germany.

Design of BIRC activities

BIRC online activities (Schwedler, 2019) apply molecular dynamics simulations to facilitate students’ ideas on particle behaviour and connect them with mathematical representations such as equations and diagrams. The concept not only aims to cognitively engage students to improve their understanding on physical chemistry concepts and overcome potential learning obstacles, but also to enable a positive emotional experience concerning the oftentimes unpopular subject. In doing so, we hope to provide a valuable resource to possibly prevent feelings of overstrain, which result from a lack of understanding in physical chemistry and often occur during first-year out-of-class learning (Schwedler, 2017).

Molecular dynamics simulations are especially suitable to achieve the desired goals for several reasons. First, simulations are able to help students visualize the dynamic behaviour of large particle entities. Second, they can be designed as multiple representations, touching and connecting all three levels of Johnstone's triangle. Third, Landriscina (2009, p. 29) argues for computer simulations to be “the most suitable instructional method [to induce] a restructuring of the students’ individual mental models.” In order to achieve this, learners need to reflect on their own conceptions and assess them using simulations (Landriscina, 2013, p. 101). Especially mentally simulating the behaviour of a system in one's own head (mental simulation, Landriscina 2013, p. 135) has been shown to improve understanding and transfer knowledge to long-term-memory (imagination principle, Leahy and Sweller, 2004; Monaghan and Clement, 1999). And fourth, the emphasis on understanding and particle behaviour is in line with first-year chemistry students’ expressed interests (Sözbilir, 2004; Schwedler, 2017).

Since simulation learning is especially suited to restructure and reinforce pre-existing knowledge (Thomas and Hooper, 1991, p. 510), BIRC activities are designed to be worked upon at home after the corresponding lecture (see Fig. 2). Each unit takes roughly 15–30 minutes and is structured into three successive parts (Schwedler, 2019).


image file: c9rp00211a-f2.tif
Fig. 2 General structure of a BIRC unit and its intended schedule between lecture and regular homework.

Part 1 begins describing and depicting a chemical system (also on the submicroscopic level) which relates to a major concept of physical chemistry. Students are required to answer an open question on the (future) behaviour of the system. They need to activate their own molecular mental models – in many cases to mentally simulate how the system evolves in time – in order to answer it. For example, part 1 of BIRC activity 3 (real gas law) features a depiction of two containers with two different gases, varying in molecular size and intermolecular attractions (see screenshot Fig. 3).


image file: c9rp00211a-f3.tif
Fig. 3 Screenshot of part 1 (imagine), step 2.

Students are instructed to consider how these particle properties are going to impact gas pressure inside the two containers. Therefore, they have to mentally simulate the impact of a change in both attractive forces and particle size on dynamic particle interactions. If doing so properly, learners might come to the conclusion that particle size decreases the effective volume within the container, forcing particles to bump into each other more readily, and consequently increases pressure. By contrast, intermolecular attraction slows particle movement down and hence decreases pressure. Subsequently, four different answers are presented for learners to choose from, supporting their reflections and forcing them to take up a position, since they can’t navigate to the next step without choosing one.

With their choices in mind (and also displayed on the screen), students work on interactive molecular dynamics simulations in part 2 (try). These made-to-measure simulations are based on the interface and algorithms of molecular workbench next generation (Tinker and Xie, 2008). Guided exploration and observation allows students to discover formerly hidden relationships, such as the dependency of pressure on particle size and intermolecular attractions in activity 3, assessing and refining their own mental models. During this part, students are provided with individual feedback and explanation, but not before working on the simulations and possibly reconsidering their choice from part 1 by themselves. After focusing on submicroscopic mental models, the emphasis of instruction shifts towards the symbolic level.

Part 3 focuses on equations, diagrams and different mathematical models connected to the scientific concept. Instruction requests students to relate these representations to their observations in part 2 in order to link submicroscopic and symbolic aspects of their conceptions. For example, BIRC activity 3 focuses on van der Waals’ equation and related p–V diagrams. Van der Waals expanded on the ideal gas law, which describes the dependencies between pressure p, temperature T, volume V and amount of substance n, as long as gas particles can be approximated as point masses without attractive forces. He introduced two coefficients a and b into the equation to account for particle size and intermolecular attraction (eqn (1) and Atkins et al., 2017, p. 45).

 
image file: c9rp00211a-t1.tif(1)
p, pressure; n, amount of substance; T, temperature; V, volume; R, universal gas constant; a, coefficient of attraction; b, coefficient of particle volume.

Students – already familiar with the ideal gas law – are given the equation without specifying, which of the two coefficients a and b is related to particle size and which to intermolecular attractions. With help of an interactive p–V-diagram, learners explore the impact of a and b on the curve and compare this to the system behaviour observed during simulation. As a result, they can assign both coefficients to particle properties. Further tasks explore the behaviour and limits of van der Waals’ model at phase transition. Learning tasks and questions are, where appropriate, accompanied by optional support and explanatory feedback.

This instructional approach takes a middle ground between pure constructivist discovery learning and pure cognitivist reception learning, since simulation learning proves to be much more efficient in closely instructed settings than in settings with little guidance (Landriscina, 2013, p. 138; Mayer, 2004; Stieff and Wilensky, 2003). It encourages learners to actively explore and discover new relationships in order to change pre-existing mental models (diSessa and Sherin, 1998; Posner et al., 1982). But in contrast to more constructivist settings, students follow a well-orchestrated learning path comparable to the principles of programmed discovery. This path enables them to discover relationships and adjust the degree of difficulty (students can take their time, exploit optional support and further challenges, switch back and forth in the learning path at will), but stays focused on the core subject and refers to empirically proven principles of multimedia learning to enhance generative processing (such as imagination, worked examples, self-explanation, feedback, see Mayer, 2014). Furthermore, it is designed to take comparatively little time (20–30 minutes per unit on average). Considering first-year chemistry students’ notorious lack of time (Schwedler, 2017) this will probably enhance the acceptance and usability of BIRC as a voluntary learning tool considerably. As of yet, eight different BIRC activities have been developed according to this strategy (see Table 2).

Table 2 Physical chemistry concepts and mathematic tools addressed by BIRC units
No. Physical chemistry concept Mathematic tool
1 Ideal gas laws Partial differentials
2 Speed distribution in ideal gases Distribution functions
3 Real gas law (van der Waals) Functions and parameters
4 Change of inner energy Increments and integrals
5 Volume-related work Integrals
6 Energy distribution (entropy) Logarithms
7 Temperature and heat (addition to 4) Sums
8 Work and heat (addition to 5) Vectors

Single-case studies: think-aloud protocol and retrospective interview

Since subject-specific results exceed the scope of this paper and have been (and will be) published elsewhere, only general data of these instruments are included in this paper to triangulate quantitative findings. Hence, the instruments will be described in general terms. A more detailed description concerning specific chemical topics as well as corresponding results can be found by Schwedler and Lyczek (2019).

Concurrent think-aloud protocols (van Someren et al., 1994) consist of two parts: first, students verbalise their thoughts and emotions while working on the learning activity for 15–30 minutes. Participants are observed by a researcher, who encourages them to keep talking but is not allowed to provide help. Data collection include video-taping, recording desktop-streams and observation protocols. Second, in subsequent qualitative interviews (15–20 minutes, semi-structured) participants are questioned concerning their conceptions, learning outcomes, perceived support by the activity, emotional experience and individual attitude towards physical chemistry. The interview mainly consists of open questions to better access participants’ views and experiences. Additionally, closed questions (Likert, 4 and 5 steps, respectively) also assess students’ perceived level of competency, perceived support and emotional experience. Data collection focuses on audio-taping.

Retrospective interviews (30–45 minutes, semi-structured) take place a few days after students have been working on a BIRC unit at home. In the first part, students’ mental interaction with BIRC are reconstructed from memory. Students are prompted to relive their experience and narrate as much as they remember. The second part resembles the interview used in think-aloud protocols. Data collection focuses on audio-taping.

Both methods allow prompt, clarifying questions as well as deep insights into the conceptions and feelings of individual participants. The closed questions facilitate comparisons with findings from online-questionnaires and post-test.

Online questionnaires

Eight online questionnaires, one for each BIRC unit, are used to interrogate students directly after working on BIRC at home. They resemble a shortened interview to assess students’ perceived level of competency, perceived support and emotional experience, featuring similar open and closed questions. In contrast to the single-case-studies mentioned above, this strategy avoids social desirability, since students’ online answers are absolutely anonymous and are given to a computer and not face-to-face. It furthermore examines the effect of BIRC under more realistic learning conditions, since students can work on BIRC at home, in their own time and without being observed. However, written answers to open questions tend to be rather short and (since there is no chance for clarifying questions) less insightful than in personal interviews. An overall of N = 366† questionnaires have been collected. Table 3 indicates the number of questionnaires collected for each BIRC activity. Since activities 7 and 8 have been used as optional additions and in winter term 18/19 only, the smaller number of responses is not surprising.
Table 3 Number of completed online questionnaires per BIRC activity
Activity 1 2 3 4 5 6 7 8
N 72 51 57 43 46 54 24 19

Pre- and post-test (paper & pencil)

Both pre- and post-test are divided in two parts. The first part is a questionnaire, asking students for statistical data and feedback on their experiences during the term. The second part tests students’ conceptual skills in physical chemistry (skilltest).

The pre-test (N = 381) questionnaire collects demographic data (e.g. subject of study, year of study, final school exam scores etc.) and investigates students’ conceptual abilities (skilltest 1), focusing on submicroscopic conceptions relating to subjects known from school or preceding introductory chemistry courses, such as speed distribution, inner energy, particle model of ideal gases and entropy.

The post-test questionnaire examines students’ use of, support by and emotional response to BIRC learning units. The second part on conceptual abilities (skilltest 2), focusing on submicroscopic conceptions and links to equations and diagrams, is used to evaluate the influence of BIRC on conceptual understanding. Since these advanced topics (van der Waals equation for real gases, path-dependency of volume-related work, boundary conditions of the ideal gas law) are very unlikely to be taught before the physical chemistry course, the influence of prior knowledge on students test results should be comparatively low. Moreover, the post-test takes place five to ten weeks after most students have been working on the corresponding BIRC units 1, 3 and 5, underlining its follow-up-like nature.

In the following section, the structure and design of skilltests 1 and 2 are discussed in more detail.

Skilltest 1 and 2 on conceptual understanding

In order to investigate BIRC's impact on students’ conceptions, the second part of both pre- and post-test require students to work through conception-related questions. Diagnostic two-tiers, combining multiple-choice-questions and explanations, are acknowledged as especially suitable in order to test for conceptual understanding and students’ conceptions, (Treagust, 1988; Tan et al., 2002). Skilltest 1 and 2 mainly comprise two-tiers featuring open explanations as well as open explanations only. This approach avoids guessing, is more open to unforeseen answers and allows direct access to students’ thoughts. However, extensive qualitative content analysis is necessary to assess the data. Both tests were carried out during official lectures. Hence, assessment times had to be kept short (roughly 20 min), so that both skilltests are improvable as far as length and reliability are concerned.

Skilltest 1 contains four questions: an open question on entropy, a multiple-choice-question on Maxwells’ speed distribution function and two two-tiers (multiple choice combined with open explanation, see Fig. 4) on inner energy and ideal gas particles. Skilltest 2 contains three questions in equal proportions, hence results can be correlated with the number of BIRC units worked on. Question 1 and 3 are two-tiers (ideal gas and volume-related work), requiring students to apply their knowledge on experimental findings or settings. In questions 2, students have to explain as detailed as possible, how van der Waals’ equation on real gases is connected to particle properties.


image file: c9rp00211a-f4.tif
Fig. 4 Question 3 of skilltest 1, translated from Schwedler, 2020, p. 258.

All questions have been developed and qualitatively validated using prior single-case studies (between 7 and 20 single case studies per question) and a paper & pencil-questionnaire on second-year-students (N = 17). Skilltest 1 purposefully explores students’ (submicroscopic) conceptions on familiar subjects, while skilltest 2 touches advanced subjects, which are nearly unknown prior to university and related to, but not identical with those of skilltest 1. Using questions on unfamiliar topics in skilltest 1, students’ abilities concerning submicroscopic conceptions might be underestimated.

Research design

Six BIRC units related to six main concepts of thermodynamics (see Table 2) were accessible for students as voluntary tasks to accompany regular homework for a first-year introductory physical chemistry course during winter term 17/18 and 18/19. In winter term 18/19 two additional activities (7, 8) supplemented units 4 and 5.‡ The research design triangulates qualitative and quantitative methods of inquiry, as is schematically depicted in Fig. 5.
image file: c9rp00211a-f5.tif
Fig. 5 Research design.

To thoroughly investigate students’ mental interaction with BIRC, their learning gains, emotional experience and perceived support, 76 qualitative case studies (think-aloud, N = 44 and interviews, N = 32) on six BIRC activities have been conducted. Qualitative studies to activities 1, 3, 5 and 6 were carried out in winter term 17/18, those to activities 7 and 8 in winter term 18/19, respectively. Students’ use, emotional acceptance, perceived support and conceptual understanding of a broader sample have been evaluated via paper & pencil questionnaires as well as skilltests during the first and last lecture of the term (N = 381 and N = 205), and via online-questionnaires (N = 366), one for each activity, directly after working on BIRC. Considering the timeline, these online questionnaires more closely resemble individual post-tests, whereas the second paper & pencil test mirrors a follow-up examination.

While pre- and post-test are designed to include as many active students as possible and hence will include participants of single-case-studies and online-questionnaires, students participating in single-case-studies will not fill in the online questionnaire for the respective BIRC-unit.

Data analysis

Audio recordings were transcribed according to Kuckartz et al. (2008). Video and desktop recordings provided context to explicate otherwise ambiguous statements. All text passages were examined via qualitative data analysis (Mayring, 2002), combining summarising and structuring techniques. Additionally, consensual coding was applied to analyse open answers in skilltest 1 and 2. Points were assigned to scientifically appropriate (and internally consistent) content categories and added to students’ skilltest scores. Inter-coder-reliability was measured to be 92%.

Multifactorial linear regression via SPSS was carried out to examine, whether BIRC activities do influence students’ conceptual understanding. This technique assumes a linear relationship between multiple independent variables xi and the dependent variable Y (see eqn (2)). To better compare the impact of independent variables, standardized coefficients [small beta, Greek, circumflex]i have been calculated.

 
Y = β0 + β1·x1 + … + βi·xi + ε(2)
Y, dependent variable; xi, independent variable; βi, coefficient; ε, error term.

The regression model features five variables: final school exam scores, skilltest 1 scores, students’ interest in physical chemistry, the number of voluntary weekly exercises completed and the number of completed BIRC activities. Our approach is based on a general model for academic success (Schiefele et al., 1992), assuming cognitive abilities, motivation and interest to be major contributors. Freyer et al., (2014) did successfully specify the model to predict first-year chemistry students’ success in an introductory general chemistry exam and identify relevant contributors. In the same style, final school exam scores and skilltest 1 scores model cognitive abilities in this study. Freyer et al. asked students, whether studying chemistry was their first choice, in order to account for motivation. We used a different approach, since the motivation to study chemistry as a whole might be very different from studying physical chemistry. What is more, measuring participants’ motivation at one point in time does not necessarily account for student behaviour during the term. Hence, motivation to study physical chemistry is operationalised by the amount of voluntary coursework (number of voluntary weekly exercise and the number of BIRC activities) participants completed during the term. Students’ interest in physical chemistry is surveyed by adapting two items from Busker and Parchmann (2010). Four of these five factors yield metric data, only student interest is measured via Likert-scale. Regression itself was performed via SPSS using the five factorial model described above. Since two factors did not yield significant correlations, a three-factorial regression after backwards elimination was also carried out.

4 Findings

Participants’ use of BIRC and perceived degree of difficulty

To address our first research question (what portion of the student body does voluntarily use BIRC activities as supplementary learning material, how many activities do students complete and what time does it take on average? Do the activities suit students’ rather heterogeneous abilities?), participants of the post-test had to specify the BIRC activities they used during the term, and participants of all methods of inquiry reported on the perceived level of difficulty.

At the end of term, 33% of all post-test participants had voluntarily completed all six regular activities (see Table 4a and b). 60% had completed at least 4 of the 6 units. 16 participants expressed the plan to work on more activities during exam preparation. Each BIRC activity was designed to take between 15 and 30 minutes. 56% of the participants estimated the time needed accordingly (Table 5). 36% reported durations of less than 15 min, 9% more than 30 min. The average student completed 4.0 activities and needed 19.0 minutes per unit.

Table 4 (a) Student rates completing a certain amount of BIRC activities during the term. (b) Student rates for completing each BIRC activity
(a)
Number of BIRC units completed 0 1 2 3 4 5 6+
Student rate in % (N = 205) 18 5 9 8 12 15 33
(b)
BIRC unit 1 2 3 4 5 6 7 8
Student rate in % (N = 205) 77 66 63 51 52 53 54 41

Table 5 Estimated time needed to work through one unit, translated from Schwedler, 2020, p. 313
Duration for one unit <10 min 10–15 min 15–20 min 20–30 min >30 min
Student rate in % (N = 205) 4 32 35 21 9

Participants completing few or no units (usually the first ones) were asked to give their reasons. The question was answered by 79 of the 82 students completing 3 or less activities and nine students completing four activities.

Of the 88 answers given, 36 claimed lack of time, 23 stated to have other priorities coming first. In January, when excessive demand is particularly high (Schwedler, 2017), user rates dropped, since students focussed on preparations for other exams and laboratory work besides having to do regular classwork. 19 students expressed lack of discipline and/or ability to organize themselves, 16 students claimed they would work on more units during the upcoming exam preparation. 12 students professed a lack of motivation and/or interest concerning physical chemistry in general. Other factors, such as BIRC not being relevant to pass exams, BIRC being incompatible with smartphones§ and BIRC not matching students’ needs or preferred style to learn, were rare and mentioned by one to five students each.

164 of all post-test participants have completed at least one unit and can hence report on the level of difficulty. 74% of post-test students’ consider the degree of difficulty to be exactly appropriate (see Fig. 6); the remaining quarter distributes fairly evenly between (rather) easy (15%) and (rather) difficult (11%). Findings of other questioning methods (single-case studies, online questionnaires) are quite similar.


image file: c9rp00211a-f6.tif
Fig. 6 Perceived degree of difficulty, translated from Schwedler, 2020, p. 315.
Discussion of findings for question 1. On the whole, a significant proportion of participants completed BIRC on a regular basis and the reported time spans are in line with the targeted duration. Although the completion rates have been self-reported in order to ensure anonymity, high user rates (especially since students hesitate to reveal a lack of ability to the teaching staff) and avoid distortion due to attrition, completion rates do match data from student feedback and anonymous user statistics. Consequently, we believe them to be quite reliable. However, retrospective duration estimates are known to not being especially accurate.

Participants predominantly regard BIRC as matching their capabilities. Considering students’ heterogeneity (Schwedler, 2017) this highlights BIRCs ability to account for several levels of difficulty. Reasons for not (often) using BIRC are mainly related to a lack of time as well as different priorities, and much less frequent with the concept itself. Although students completing more than three activities were not specifically targeted, nine students completing four units felt the need to give their reasons for not completing all units anyway. Since their answers matched those of students completing two or three activities, they were kept in the dataset.

Participants’ mental and emotional involvement, perceived support

To address our second research question (to what extent and for what reasons are students mentally activated and emotionally involved when working on BIRC, and in what ways do they perceive these activities as a supportive resource for out-of-class learning?), participants of single-case studies reported on their thoughts and feelings while working on BIRC and assessed its helpfulness retrospectively. However, those cases might not be representative for the student body. Hence, this issue was addressed in the post-test via closed and open questions, using single-case studies to explicate these findings. In the following section, findings concerning participants’ mental and emotional involvement are presented first, followed by students’ assessment on BIRC as a supportive resource.
Students’ involvement. 164 of 205 post-test-participants have been working with BIRC and can hence answer questions on their emotional involvement. 70% of these 164 participants (rather) agreed in retrospect, that they enjoyed working on BIRC (see Fig. 7, middle). Similarly, only 15% of the student body (rather) agreed, that working on BIRC was boring (see Fig. 7, leftmost). On average, students perceived BIRC as more enjoyable than physical chemistry activities as a whole (Fig. 7, rightmost), agreement even exceeds interest in physical chemistry topics. Of 164 post-test-students questioned, 78 students (48%) named reasons for BIRC being enjoyable, but no negative aspects. 16 students (10%) professed either neutral (7) or ambivalent (9) emotions, the latter giving reasons for enjoyment as well as boredom:
image file: c9rp00211a-f7.tif
Fig. 7 Results on perceived enjoyment by post-test, including a cross-check to single-case-studies (sc), translated from Schwedler, 2020, p. 324.

“It was fun to use the simulations, and I liked the challenge to find the right answers […]. But working on equations is rather boring.”

14 students (8%) stated reasons for BIRC being boring, but named no positive aspects. The other 34% gave no answer.

Table 6 lists all student responses. In order to gain a better overview over the reasons for positive and negative emotions as well as their frequency, ambivalent feedback, containing separate reasons for enjoyment and boredom, was split and added to the categories reasons for enjoying BIRC and reasons for BIRC being boring respectively. Students liked to interact with simulations and diagrams, they express having fun with these tools (n = 30) as well as enjoying the interactivity (n = 29). 27 students enjoyed its quiz-like structure: being made to think and choose, check their answer themselves and get feedback, especially when doing so successfully. 19 students expressed a general pleasure to understand and learn, describing their enjoyment in having a light bulb moment. Other positive reasons comprise the descriptive design, general claims of BIRC being fun and a general interest in chemistry.

Table 6 Reasons for enjoyment/boredom, translated from Schwedler, 2020, p. 320
(Reasons for) enjoying BIRC, answers from positive only (n = 78) and ambivalent (n = 9) students
Positive Ambivalent All
Using simulations and diagrams 27 3 30
Interaction, be active during learning 26 3 29
Quiz-like (think first, check result, feedback), feeling of success 23 4 27
Pleasure to learn, light bulb moment 18 1 19
Descriptive, visual design 10 1 11
BIRC units are generally fun 9 9
Interest in chemistry, new perspective on physical chemistry 7 7
Neutral reception (n = 7)
• Neither boring nor enjoyable
• Can’t associate learning with fun
(Reasons for) BIRC being boring, answers from negative only (n = 14) and ambivalent (n = 9) students
Negative Ambivalent All
Physical chemistry is boring/too difficult 6 3 9
Don’t like maths/equations 1 4 5
Texts and explanation 2 3 5
Other 5 0 5

Reasons for a neutral reception comprise BIRC being helpful but neither boring nor enjoyable, and the statement that learning in itself is not associated with fun. On the negative side, nine students claim physical chemistry as a subject to be boring or not within their abilities. Five students disliked the mathematical content and five were partly bored by texts and/or explanations. Other reasons, such as an unsuitable level of difficulty and a general lack of interest, were mentioned by one to two participants each.

Participants of single-case studies are more positive in their statements (see also Fig. 7: post-test data in bar 3 compared to single-case data in bar 4), probably due to selection and social desirability effects. However, these findings are useful to explicate students’ reasons for enjoying BIRC. During think-aloud sessions, students not only observe the simulations passively but mentally interact with them and re-evaluate the situation. Second-year student TTW4 especially pinpoints his enjoyment while watching the simulation unfold.

“It has already started. Ok. […] Uii. […] Yes, right. Ok. Ok. […] Aah, the temperature roughly stays at 500 K. [That] means it would have been answer “(b)”. […] But…no. […] Now [the temperature] is rising, of the metal especially. […] So [that of] Argon stays a little bit behind […]. Why didn‘t we have something as cool as that? That would have been much cooler! Well, that is really descriptive.” (TTW4)

Positive feelings as a result of success are also mentioned by participants who are not fond of physical chemistry. IVA1 initially had quite negative expectations due to the abstract nature of physical chemistry. But the student enjoyed completing BIRC anyway, reasoning as follows:

“Because I experience success and I know, that I understand this now. […] These moments of success make it enjoyable. Especially, since I get the solution after drawing the conclusions on my own. I think it is really fun, although physical chemistry is definitely not my hobbyhorse.” (IVA1)

Many students rather agreed to BIRC being fun, but not fully (see Fig. 7). A recurring motif implies, that learning and work in general cannot be as much fun as other activities, even though BIRC is seen as comparatively enjoyable learning strategy. TVA7 and TRG11 state it like this:

“Well, I wouldn’t say I’d prefer BIRC to an amusement park.” (TVA7)

“If I compare that to the dreary learning one usually does, then it is fun for that matter. Compared to other learning contents, it is in any case better this way.” (TRG11)

Furthermore, interactive simulations and diagrams act as a motivator to deal with more abstract mathematical models in physical chemistry, as TIG4 puts it:

“It helps because it keeps the fun up. One has something interactive, where one […] can see, what happens and what's it all about. And that is why one has more motivation to look at the mathematical background in depth.” (TIG4)

BIRC as a supporting resource. All methods of inquiry examined the support students perceived by BIRC. Depending on the method of survey, 87% to 100% of the students agreed or rather agreed to the statement, that BIRC helped a lot during out-of-class learning (see Fig. 8, right).
image file: c9rp00211a-f8.tif
Fig. 8 Results on perceived support by post-test, incl. cross-checks to online inquiries (ol) and single-case studies (sc), translated from Schwedler, 2020, p. 362 and 364.

Between 83 and 96% of post-test participants, who completed at least one BIRC unit, (rather) agreed to BIRC being helpful to imagine a concept on the submicro-level, to link submicroscopic conceptions with mathematical representations, to understand the concept and to generally cope with post-lecture work (Fig. 8, left). However, only 44% perceived BIRC as (rather) helpful to solve the weekly exercise (Fig. 8, middle).

Students were asked to explain in more detail, whether and why BIRC did or did not help them. 67% expressed feeling supported, 12% expressed ambivalent experiences, 3% of the students claimed BIRC was not helpful and 18% did not answer the question. On the whole, 110 open answers were given as to how and why BIRC supports students (see Table 7). 92 students found BIRC to induce understanding, 77 underscored its descriptive nature, 61 felt supported by preparing for and post-processing lessons, 33 students claimed BIRC helped them to deal with maths, 31 considered it helpful that it required them to think for themselves and be mentally active and 26 students expressed the support in general terms.

Table 7 Perceived support, open answers, translated from Schwedler, 2020, p. 361
BIRC units help… (n = 110)
• To induce understanding (n = 92)
• Because they are descriptive (n = 77)
• With pre- and post-processing lessons (n = 61)
• To deal with maths (n = 33)
• Because one has to be active, think for oneself (n = 31)
• In general (n = 26)

These effects are elaborated during single-case studies. TVA6 describes, how BIRC changes her/his handling of equations from rote-learning to understanding their meaning:

“If I see such an equation, I memorize the equation. But I can’t really imagine, what is going on. And that is why these units help, […] because one can see what happens, if [some parameter] changes.” (TVA6)

One major goal is to foster mental simulation on the particle level. While TVA6's description sounds more like passive reception, TRG1 underscores how the concept encourages him to mentally simulate the situation.

“[These units] are helpful in any case. Instead of memorizing ‘if this happens, then that happens’, I can imagine in my own head ‘what is going to happen, if I squeeze the whole thing?” (TRG1)

That BIRC induces students to actively pursue a better conceptual understanding is regularly mentioned by students, for example by TVA7:

“Therefore, these units are really helpful, because otherwise you wouldn’t think about it as deeply. Else you only think ‘Damn, where is my mistake, where is my mistake?’ But you don’t think ‘How does this actually work?’ And that is promoted by this unit.” (TVA7)

Since these studies have been carried out during the term, little is known about their effect on students’ performance during exams. One second term student TRG2, who tested one unit during development, expressed positive experiences during the first term:

“[Physical chemistry] is not a subject that I like. In any case, [BIRC] helped me a lot to learn last term. I don’t know, whether I would have mastered the exam as well without it.” (TRG2)

This single statement can’t substitute systematic research. However, it indicates that at least a small number of students do profit profoundly from BIRC.

In summary, 95% of post-test participants that have worked with BIRC would (rather) recommend it to fellow students, and only 6% (post-test) agreed or rather agreed to BIRC being superfluous (see Fig. 9).


image file: c9rp00211a-f9.tif
Fig. 9 Recommendation of BIRC and cross-check, translated from Schwedler, 2020, p. 364.
Discussion of findings for question 2. On the emotional level, two thirds of the participants described working on BIRC as a (rather) enjoyable experience, especially compared to other modes of learning. They liked interacting with simulations and diagrams, were engaged due to the quiz-like nature of the first two parts and enjoyed feelings of success. Results of single-case studies were even more positive in this respect, probably because of selection effects and socially desirable answers. However, these answers emphasize the connection between active learning processes, conceptual understanding and students’ pleasure as well as wish to understand chemistry (Sözbilir, 2004), which seems to drive many first-year chemistry students.

Participants, who tried at least one BIRC unit, are found to mostly experience it as a supporting resource, helping them to understand physical chemistry concepts on several levels and prepare or post-process lectures. Results of both single-case studies and anonymous online-inquiries are more favourable than those of the post-test. This discrepancy might stem from selection effects and the fact that post-test questioning is retrospective. Looking back, a support related to one specific topic is put into proportion to the whole term. Concerning closed questions, post-test findings are much more representative and relevant. However, single-case studies (especially think-aloud protocols) explore students’ interaction with BIRC in depth and can reveal valuable learning processes. Findings indicate that the quiz-like simulation-based concept induces students to mentally interact and simulate system behaviour in their mind, to re-evaluate prior conceptions and to change from rote learning equations to understanding their underlying meaning.

Learning gains

To address our third research question (how does working on BIRC impact students’ conceptual understanding, especially regarding submicroscopic conceptions and their link to abstract representations, in a broader student sample? Will the number of BIRC units completed correlate with students’ conceptual abilities at the end of the term?), students were not only asked to assess their competencies before and after working on BIRC. What is more, achievement tests (skilltest 1 and 2) measured students’ conceptual skills at the beginning and end of term.
Closed self-assessment. In single-case-studies (N = 76) and online-questionnaires (n = 366), students reported on their learning gains. Content-specific results are presented elsewhere (Schwedler, 2019; Schwedler and Lyczek, 2019), since they exceed the scope of this paper. Students were also asked to retrospectively estimate their competency on a closed scale before and after working on BIRC relating to four areas of competency:

C1. Ability to imagine particle behaviour related to concepts and/or phenomena

C2. Ability to interpret, how equations and diagrams relate to submicroscopic conceptions

C3. Ability to further explain concept-related phenomena, terms and definitions

C4. Ability to explain/apply specific mathematical models and expressions

In inquiry, the expressions employed to describe competencies 1–4 are much more specific with regard to the corresponding scientific concepts. Findings of all BIRC units are summarized in Fig. 10.


image file: c9rp00211a-f10.tif
Fig. 10 Self-assessment of competencies 1-4 before and after working on BIRC, translated from Schwedler, 2020, p. 351.

Before working on BIRC, 54% to 73% (rather) disagree to master these competencies; afterwards these rates range from 4 to 23%. Students clearly believe to have improved their skills. However, these are only self-assessing estimates, and the corresponding surveys took place directly after working on BIRC. Consequently, they do not prove long-term learning gains.

Linear regression. Students (N = 205) achieved results between 0 and 87% in skilltest 2, scoring on average 34%.|| Since test questions addressed three units (1, 3 and 4) in equal measure, students can be divided into four groups: those having completed zero, one, two or three of these units, respectively. Students average scores increase with increasing number of BIRC activity from 21% (0 units) up to 43% (3 units).

Multifactorial linear regression (n = 185)** was employed to analyse the relationship between the number of BIRC units completed and skilltest 2 scores. The five-factorial model includes final school exam scores, skilltest 1 scores, interest in physical chemistry, number of worked-on voluntary exercises and number of relevant BIRC activities to predict the outcome of skilltest 2. Other factors (such as the last school report mark in chemistry) have been explored but did not significantly contribute to the regression.

Regressions applying five factors yielded statistical significances of p < 0.001 for the whole model and explained between 24 and 31% of the variance observed. These findings are in line with Freyer et al. (2014). Table 8 lists the individual standardized coefficients [small beta, Greek, circumflex] as well as statistical significances pβ related to these coefficients. However, only three of the five factors contributed significantly to regression analysis: the number of relevant BIRC units completed, skilltest 1 scores and final school exam scores.

Table 8 Findings of linear regression using five (top, 1a, b and c) or three (bottom, 2a, b and c) factors respectively. Black text indicates significant, italic text non-significant variables. Translated from Schwedler, 2020, p. 355
5 independent variables 1a both cohorts 1b WT 17/18 1c WT 18/19
[small beta, Greek, circumflex] p β [small beta, Greek, circumflex] p β [small beta, Greek, circumflex] p β
Final school exam scores −0.143 0.033 −0.051 0.610 −0.245 0.008
Skilltest 1 scores 0.229 0.001 0.193 0.043 0.247 0.010
Number of relevant BIRC units 0.431 <0.001 0.496 <0.001 0.302 0.003
Number of voluntary exercises −0.054 0.434 −0.099 0.307 −0.105 0.310
Interest in physical chemistry 0.021 0.750 0.036 0.698 0.004 0.968
Corrected R2 0.27 0.25 0.31
3 independent variables 2a both cohorts 2b WT 17/18 2c WT 18/19
[small beta, Greek, circumflex] p β [small beta, Greek, circumflex] p β [small beta, Greek, circumflex] p β
Final school exam scores −0.145 0.026 −0.029 0.765 −0.284 0.002
Skilltest 1 scores 0.210 0.001 0.178 0.055 0.235 0.007
Number of relevant BIRC units 0.423 <0.001 0.488 <0.001 0.357 0.001
Corrected R2 0.26 0.24 0.30

For all regressions, the number of relevant BIRC units has the strongest and statistically most significant influence on students’ performance in skilltest 2. The unstandardized coefficient in regression 1a reveals each BIRC unit to increase student performance on average by 7.4% of all points in skilltest 2. However, this effect is much more pronounced in cohort 1 than in cohort 2. Since two of the five factors do not seem to play a significant role, the model was further reduced to three factors using backward elimination and no notable loss in explained variance occurred. Furthermore, by reducing the amount of data needed, four additional respondents could be included into the regression.

Discussion of findings for question 3. Students perceived significant learning gains on submicroscopic conceptions, on the link between these conceptions and mathematical representations and on other issues concerning scientific concepts due to BIRC.

Even though skilltest 2 scores varied over nearly the whole spectrum, they were quite low on average, underscoring students’ general struggle to understand physical chemistry concepts. Moreover, open explanations require students to having understood the concept in the first place and additionally remember it five to ten weeks later. However, this strategy suits a major strength of simulations in learning: reinforcing and reorganizing acquired content (Thomas and Hooper 1991, p. 510).

Although an increase of 7.4% of all skilltest 2 points per BIRC unit does not sound much at first glance, it is to be considered significant in view of the low average skilltest 2 scores (34%). These data also are in line with the doubling of average skilltest 2 scores for students completing all three units (43%) compared to students completing none (21%).

Most quality criteria of the multifactorial linear regressions (use of a linear model, metric variables, exogeneity as well as normal distribution of residua, non-collinearity of independent variables) were thoroughly fulfilled. Two constraints have been found: first, homoscedasticity is limited due to the fact that test scores cannot yield results below zero. Second, even though data of two consecutive cohorts have been used, sample sizes are, strictly speaking, too small to accurately quantify [small beta, Greek, circumflex]-coefficients of a five-factorial model (Maxwell, 2000). However, only three factors significantly contributed to the model, and for three-factorial models, sample-sizes were appropriate. Moreover, this study did not aim for precise quantification, but rather to detect a relationship between BIRC and conceptual understanding at all. The data are able to confirm this, since the number of BIRC units worked on has consistently proven to be the best predictor in all linear regressions performed.

Although the impact of BIRC activities cannot be completely separated from other influences such as motivation and work-ethics, completion of voluntarily weekly exercises as a factor to account for work-ethics and motivation does not significantly contribute to student performance in skilltest 2. Hence, the influence of BIRC is unlikely to be purely attributed to motivation.

It seems that students in cohort 1 learned more from BIRC than those in cohort 2. This effect matches students’ self-assessments on learning gains. During winter term 17/18, single-case studies (Schwedler, 2019; Schwedler and Lyczek, 2019) on BIRC units relevant for skilltest 2 have been carried out. This probably increased skilltest 2 scores, since some students not only worked on BIRC but additionally talked about it to researchers. Nevertheless, even for the second cohort, BIRC remained the most important factor to predict skilltest 2 scores. The other significant factors, skilltest 1 scores as a measure for prior knowledge and final school exam scores as a measure for cognitive abilities, have already been established as important predictors for first-year chemistry students’ academic success by Freyer et al. (2014). Furthermore, the significant relationship between skilltest 1 and 2 scores justifies our approach to employ students’ submicroscopic conceptions on familiar topics in skilltest 1 as a predictor for students’ submicroscopic conceptions and abilities to link them with abstract maths in skilltest 2.

5 Limitations

Advantages and drawbacks of the research design

The triangulating research approach presented in this paper combines single-case studies (think-aloud protocols and interviews) and inquiries of a broader sample (online and paper & pencil questionnaires) at different points in time, yielding rich and complex data.

Participants voluntarily choose to work on BIRC, hence students’ acceptance of BIRC and the impact BIRC has on their conceptual abilities is assessed under realistic circumstances. However, it is difficult to reliably separate the influence of BIRC from other factors (such as motivation) due to possible selection effects.

Questioning students during lectures reaches most of the active student body and hence allows for a reasonable sample size. However, the high course drop-out rate suggests an under-representation of students skipping lectures in order to better prepare for the upcoming exams and of students quitting the course (for various reasons) at an early stage. Hence, the study might inadvertently select (at least basically) motivated and successful students, which is very difficult to avoid for a term-long survey. From a holistic point of view, possibly not reaching early strugglers poses a severe limitation of this study. On the other hand, an early drop-out and hence professional reorientation does not have to be negative: It is neither the aim nor a useful benchmark for concepts such as BIRC to keep students, who quickly find their abilities, interests and motivations to severely mismatch their chosen courses (or even studying as a whole), into their major field of study.

What is more, testing during lectures also requires pre- and post-questionnaires to be rather short, allowing for not more than 15 minutes to complete skilltest 1 and skilltest 2 respectively. This limits reliability when measuring conceptual understanding. Although this approach might not be suitable to detect small impacts or quantify the effects with very high accuracy, it should reveal moderate or strong correlations between the use of BIRC and students’ performance, simultaneously accounting for other factors.

Specifics of the BIRC concept and German tertiary education

BIRC as a concept was developed to fit the German tertiary education system, which puts little emphasis on mandatory learning activities but instead relies heavily on students’ self-reliance and autonomy in learning. Students often decide for themselves, whether and in what term they sign in for a specific course, and if they do, they are not forced to actually attend the lessons but can also prepare for the exams at home. This allows study flexibility and very individual schedules, but also creates very heterogeneous samples to assess.

BIRC itself was created to match this high level of self-organisation and autonomy in learning, which might not be suitable for all students, let alone other learning cultures. Although BIRC does prompt students to perform certain tasks, students are not forced to do them. Holding students accountable might change students’ reception and performance. Furthermore, the concept was developed as a stand-alone, hence other learning activities (such as exercises, lectures and laboratory classes) do not specifically reference or rely on them. This approach makes it easier to separate the effect BIRC has on students’ conceptual abilities, but it restricts the power of these activities.

6 Conclusions and implications

Simulation learning is known to be a versatile tool to facilitate understanding and engage students in active learning processes. But since completion of BIRC learning activities was voluntary and done individually at home, we were by no means sure students would spend the extra time and mental effort necessary to transform their mental models.

Although some aspects leave room for improvement, the majority of students regularly completed BIRC and experienced working on BIRC as mentally activating, comparably enjoyable and fostering their conceptual abilities. In retrospect, students viewed the units as valuable and descriptive support to increase understanding and help prepare and post-process regular lessons. However, only half of the students perceived BIRC as supportive concerning weekly exercise problems. Linear regressions reveal a moderate but significant impact of BIRC on long-term conceptual understanding.

Regarding these findings, we propose to further explore simulation learning as a tool to help students overcome learning obstacles in tertiary physical chemistry. One major task is to shift student behaviour from algorithmic rote learning to more meaningful learning processes. Although our results indicate a successful change in that direction, the impact of simulation learning in general, and BIRC specifically, on student learning processes in physical chemistry need to be explored in greater width and depth. It also remains to be investigated, whether and how simulation learning might facilitate students’ reasoning on the submicroscopic level in order to fill the diagnosed gap by Becker et al. (2013) and Hernández et al. (2014). Single teaching staff members reported on more students using the submicroscopic level in oral examinations of the following term, but no systematic study has been done. Moreover, according to student self-reports, BIRC does not seem to help students with solving weekly exercise problems as much as one could wish. This finding should be further investigated, e.g. by assessing BIRC's effect on student performances in weekly exercises. What is more, possible reasons for this phenomenon as well as alternative strategies to fill the gap remain to be explored. Since this study explored students’ emotional experience while working on BIRC in general, future studies should examine its impact on students’ attitude towards and interest in physical chemistry in greater depth.

Notwithstanding BIRCs flexibility as a teaching tool, the power of stand-alone best-practice supplementary material, which makes up only a small portion of the overall course, is restricted. Adjusting other classwork such as lectures and/or weekly exercises to further support simulation learning, for example giving students the opportunity to discuss their findings with colleagues, might possibly have an even greater impact on student understanding and attitude towards physical chemistry – as long as teaching strategies are made-to-measure physical chemistry characteristics as well as students’ needs. In addition, a suitable in-class concept might boost students’ interest in this oftentimes unpopular subject and perhaps even contribute to prevent dropout in the future.

Conflicts of interest

There are no conflicts of interest to declare.

Acknowledgements

This work is part of a German habilitation thesis (Schwedler, 2020) and was supported by the Fonds der chemischen Industrie (Chemical Industry Fund). Any opinions, findings and conclusions expressed in this material are those of the authors and do not necessarily reflect the view of the Chemical Industry Fund. We thank the lecturers, who supported us with advice from their physical chemistry perspective and allowed us to conduct our research during and around their lessons. We especially want to thank all students for participating in this study and taking the time, although having precious little of it to spare.

References

  1. Atkins P., de Paula J. and Keeler J., (2017), Physical Chemistry, University Press.
  2. Bain K., Moon A., Mack M. R. and Towns M. H., (2014), A review of research on the teaching and learning of thermodynamics at the university level, Chem. Educ. Res. Pract., 15(3), 320–335.
  3. Bain K. and Towns M. H., (2016), A review of research on the teaching and learning of chemical kinetics, Chem. Educ. Res. Pract., 17(2), 246–262.
  4. Becker N., Rasmussen C., Sweeney G., Wawro M., Towns M. and Cole R., (2013), Reasoning using particulate nature of matter: an example of a sociochemical norm in a university-level physical chemistry class, Chem. Educ. Res. Pract., 14(1), 81–94.
  5. Becker N., Stanford C., Towns M. and Cole R., (2015), Translating across macroscopic, submicroscopic and symbolic levels: the role of instructor facilitation in an inquiry-oriented physical chemistry class, Chem. Educ. Res. Pract., 16(4), 769–785.
  6. Becker N. and Towns M., (2012), Students’ understanding of mathematical expressions in physical chemistry contexts: an analysis using Sherin's symbolic forms, Chem. Educ. Res. Pract., 13(3), 209–220.
  7. Bennett J. M. and Sözbilir M., (2007), A study of turkish chemistry undergraduates’ understanding of entropy, J. Chem. Educ., 84(7), 1204.
  8. Bornkessel P. and Asdonk J., (2011), Der Übergang Schule-Hochschule [The transition school-university], VS Verlag.
  9. Busker M. and Parchmann I., (2010), Eingangsvoraussetzungen von Studienanfängern im Fach Chemie: Welches Vorwissen und welche Interessen zeigen Studierende? [Entry conditions of freshmen in chemistry studies - What prior knowledge and interests do students have?], Chem. Kon., 163–168.
  10. Carson E. M. and Watson J. R., (2002), Undergraduate students’ understandings of entropy and Gibbs free energy, Univ. Chem. Educ., 6, 4–12.
  11. Dehling H., Glasmachers E. and Griese B., (2014), MP 2 – Mathe/Plus/Praxis: Strategien zur Vorbeugung gegen Studienabbruch [MP 2 – Mathe/plus/praxis: strategies to prevent college dropout], Z. Hochschulentwicklung, 9, 39–55.
  12. Derrick M. E. and Derrick F. W., (2002), Predictors of success in physical chemistry. J. Chem. Educ., 79(8), 1013.
  13. Devetak I., Vogrinc J. and Glažar S. A., (2009), Assessing 16-year-old students’ understanding of Aqueous solution at submicroscopic level. Res. Sci. Educ., 39(2), 157–179.
  14. diSessa A. A. and Sherin B. L., (1998), What changes in conceptual change? Int. J. Sci. Educ., 20, 1155–1191.
  15. Frettlöh D. and Hattermann M., (2016), Konzeption eines Mathematik-Förderprogramms für Informatikstudierende der Universität Bielefeld [Conceptual design of a mathematics support programme for informatics students at Bielefeld university], in Lehren und Lernen von Mathematik in der Studieneingangsphase, Biehler R., Hochmuth R., Hoppenbrock A. and Rück H., (ed.), Springer, pp. 197–212.
  16. Freyer K., Epple M., Brand M., Schiebener J. and Sumfleth E., (2014), Studienerfolgsprognose bei Erstsemesterstudierenden in Chemie [Predicting Student Success of Freshmen in Chemistry]. Zeitschrift für Didakt. der Naturwissenschaften, 20, 129–142.
  17. Goedhart M. J. and Kaper W., (2002), From chemical energetics to chemical thermodynamics, in Chemical Education: Towards research-based practice, Gilbert J. K., De Jong O., Justi R., Treagust D. F. and Van Driel J. H., (ed.), Kluwer, pp. 339–362.
  18. Goldhausen I., (2015), Mathematische Modelle im Chemieunterricht [Mathematical models in chemistry instruction], Uni-Edition.
  19. Hadfield L. C. and Wieman C. E., (2010), Student interpretations of equations related to the first law of thermodynamics. J. Chem. Educ., 87(7), 750–755.
  20. Hahn K. E. and Polik W. F., (2004), Factors influencing success in physical chemistry. J. Chem. Educ., 81(4), 567.
  21. Hernández G. E., Criswell B. A., Kirk N. J., Sauder D. G. and Rushton G. T., (2014), Pushing for particulate level models of adiabatic and isothermal processes in upper-level chemistry courses: a qualitative study. Chem. Educ. Res. Pract., 15(3), 354–365.
  22. Heublein U., Ebert J., Hutzsch C., Isleib S., König R., Richter J. and Woisch A., (2017), Zwischen Studienerwartungen und Studienwirklichkeit. Ursachen des Studienabbruchs […] an deutschen Hochschulen [Between study expectations and reality: Reasons for dropout […] at German institutions for higher education], Forum Hochschule, 1(1), 1–318.
  23. Heublein U., Hutzsch C., Schreiber J., Sommer D. and Besuch G., (2010), Ursachen des Studienabbruchs in Bachelor- und in herkömmlichen Studiengängen [Reasons for student dropout in Bachelor programmes and conventional degree programmes]. Forum Hochschule, 2, 1–184.
  24. Johnstone A., (2000), Teaching of chemistry-logical or psychological? Chem. Educ. Res. Pract., 1(1), 9–15.
  25. Johnstone A. H., (1993), The development of chemistry teaching: a changing response to changing demand. J. Chem. Educ., 70(9), 701–705.
  26. Kuckartz U., Dresing T., Rädiker S. and Stefer C., (2008), Qualitative Evaluation. Der Einstieg in die Praxis. [Qualitative Evaluation. The entrance to practice], Verlag für Sozialwissenschaften.
  27. Landriscina F., (2013), Simulation and learning: A model-centered approach, Springer.
  28. Landriscina F., (2009), Simulation and learning: the role of mental models. J. e-Learning Knowl. Soc., 5(2), 23–32.
  29. Leahy W. and Sweller J., (2004), Cognitive load and the imagination effect. Appl. Cogn. Psychol., 18(7), 857–875.
  30. Maxwell S. E., (2000), Sample size and multiple regression analysis. Psychol. Methods, 5(4), 434–458.
  31. Mayer R. E., (2014), Cognitive theory of multimedia learning, in The Cambridge Handbook of Multimedia Learning, Mayer R. E., (ed.), Cambridge University Press, pp. 43–71.
  32. Mayer R. E., (2004), Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. Am. Psychol., 59(1), 14–19.
  33. Mayring P., (2002), Einführung in die qualitative Sozialforschung: Eine Anleitung zum qualitativen Denken [Introduction to qualitative social research: A guide to qualitative thinking], Beltz.
  34. Monaghan J. M. and Clement J., (1999), Use of a computer simulation to develop mental simulations for understanding relative motion concepts. Int. J. Sci. Educ., 21(9), 921–944.
  35. Nakhleh M. B. and Mitchell R. C., (1993), Concept learning versus problem solving: There is a difference. J. Chem. Educ., 70(3), 190.
  36. Nicoll G. and Francisco J. S., (2001), An investigation of the factors influencing student performance in physical chemistry. J. Chem. Educ., 78(1), 99.
  37. Nieswandt M., (2007), Student Affect and Conceptual Understanding in Learning Chemistry Martina. J. Res. Sci. Teach., 44(7), 908–937.
  38. Nilsson T. and Niedderer H., (2014), Undergraduate students’ conceptions of enthalpy, enthalpy change and related concepts. Chem. Educ. Res. Pract., 15(3), 336–353.
  39. Nyachwaya J. M. and Wood N. B., (2014), Evaluation of chemical representations in physical chemistry textbooks. Chem. Educ. Res. Pract., 15(4), 720–728.
  40. Posner G., Strike K. A., Hewson P. W. and Gertzog W. A., (1982), Accommodation of a scientific conception: toward a theory of conceptual change. Sci. Educ., 66(2), 211–227.
  41. Schiefele U., Krapp A. and Winteler A., (1992), Interest as a predictor of academic achievement: A meta-analysis of research, in The role of interest in learning and development, Renninger K. A., Hidi S. and Krapp A., (ed.), Lawrence Erlbaum Associates, pp. 183–212.
  42. Schwedler S., (2017), Was überfordert Chemiestudierende zu Studienbeginn? [What makes freshmen chemistry students feel overtaxed?], Zeitschrift für Didakt. der Naturwissenschaften, 23(1), 165–179.
  43. Schwedler S., (2019), Wie schnell sind die Teilchen denn jetzt? Studienanfänger des Fachs Chemie entwickeln dynamische Vorstellungen zur Maxwellverteilung mit BIRC [How fast are those particles? – BIRC helps first-year chemistry students to develop dynamic mental models on Maxwells distribution function]. Chem. Kon., 26(1), 12–22.
  44. Schwedler S. and Lyczek M., (2019), Punktmassen oder Mini-Billiardkugeln? Moleküldynamiksimulationen stärken Lernervorstellungen zu idealen Gasen im Chemiestudium [Point mass or little billiard ball? Molecular dynamics simulations enhance students’ mental models on ideal gas in tertiary chemistry studies]. Chem. Kon., 26(5), 179–187.
  45. Schwedler S., (2020), Analyse des Studienstarts im Fach Chemie - Lernen mit Simulationen als fachdidaktischer Weg aus der Überforderung [Analyzing the start in tertiary chemistry studies - simulation learning as a didactical approach to fight overstrain], Cuvillier.
  46. van Someren M. W., Barnard Y. F. and Sandberg J., (1994), The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes, Academic Press.
  47. Sözbilir M., (2004), What Makes Physical Chemistry Difficult? Perceptions of Turkish Chemistry Undergraduates and Lecturers. J. Chem. Educ., 81(4), 573.
  48. Sözbilir M., Pinarbasi T. and Canpolat N., (2010), Prospective Chemistry Teachers’ Conceptions of Chemical Thermodynamics and Kinetics. Eurasia J. Math. Sci. Technol. Educ., 6(2), 111–120.
  49. Stamovlasis D., Tsaparlis G., Kamilatos C., Papaoikonomou D. and Zarotiadou E., (2005), Conceptual understanding versus algorithmic problem solving: Further evidence from a national chemistry examination. Chem. Educ. Res. Pract., 6(2), 104–118.
  50. Stieff M. and Wilensky U. J., (2003), Connected Chemistry-Incorporating interacting simulations into the chemistry classroom. J. Sci. Educ. Technol., 12(3), 285–302.
  51. Tan K. C. D., Goh N. K., Chia L. S. and Treagust D. F., (2002), Development and Application of a Two-Tier Multiple Choice Diagnostic Instrument To Assess High School Students' Understanding of Inorganic Chemistry Qualitative Analysis, J. Res. Sci. Teach., 39(4), 283–301.
  52. Thomas R. and Hooper E., (1991), Simulations. J. Res. Comput. Educ., 23(4), 497–513.
  53. Tinker R. F. and Xie Q., (2008), Applying Computational Science to Education: The Molecular Workbench Paradigm, Comput. Sci. Eng., 10(5), 24–27.
  54. Treagust D. F., (1988), Development and use of diagnostic tests to evaluate students’ misconceptions in science. Int. J. Sci. Educ., 10(2), 159–169.
  55. Tsaparlis G. and Finlayson O. E., (2014), Physical chemistry education: Its multiple facets and aspects. Chem. Educ. Res. Pract., 15(3), 257–265.

Footnotes

† This is the number of unique participant-BIRC-unit combinations, since each participant can give feedback to more than one unit.
‡ These are thematically of lesser importance and for interested students only.
§ Students were informed on this when BIRC was presented during the first lecture of the term. Still, some tried anyway.
TTW4 as an anonymous codes relates to the method of inquiry (T for think-aloud), the BIRC activity in question (TW temperature and heat) and a consecutive number (4 in this case).
|| Students scoring 0% have all answered the questionnaire of the post-test, which was unrelated to chemical questions. Concerning the chemical part, some of them gave blank responses, some of them actually tried to answer the questions.
** For some post-test questionnaires, assignment to a corresponding pre-test-questionnaire was not possible or pre-test data were incomplete.
This journal is © The Royal Society of Chemistry 2020