Design and impact of a stoichiometry voluntary online course for entering first-year STEM college students†
Brayan
Díaz
*ab and
Arie
Aizman
b
aDepartment of STEM Education, North Carolina State University, Raleigh, USA. E-mail: badiaz@ncsu.edu
bDepartment of Learning and Teaching, Universidad Técnica Federico Santa María, Valparaíso, Chile
First published on 10th August 2023
Abstract
The paper presents the design and evaluation of a voluntary online introductory stoichiometry (VOIS) course aimed at facilitating the transition from secondary to higher education. The course utilized simple analogies and adaptive feedback through a formative scaffolding assessment. The study assessed the effectiveness of the VOIS course through pre- and post-knowledge tests, analysis of students' performance in general chemistry, and course evaluation surveys conducted at a Latin American University between 2019 and 2021. A total of 3995 first-year STEM students enrolled in the course voluntarily, and 358 students successfully completed it. The results showed a statistically significant improvement in stoichiometry-related knowledge, with the pre–post test scores increasing from 4.61 to 6.55 out of 10. The matched sample analysis, which only included students with 100% participation, demonstrated a statistically significant improvement in stoichiometry and related knowledge from 5.31 to 6.61. Furthermore, an analysis comparing the performance of students who completed the VOIS course with those who didn't reveal that the former group outperformed the latter by an average of 10.6 points in the general chemistry course. This statistically significant difference exhibited a large effect size (d = 0.8). In addition, a matching technique was employed to construct a synthetic control group in order to reduce bias in the quasi-experimental design. A successful propensity score analysis was conducted, controlling for variables such as gender, grade in high school, scores in the national test, and student ranking in their high school. The results of this analysis showed a statistically significant improvement of 8.6 points in the general chemistry performance for students who completed the VOIS course compared to those who did not enroll in the course. Furthermore, the feedback from 129 respondents indicated that 80% of the students either liked the VOIS course or liked it very much, with an overall satisfaction rating of 3.1 on a four-point scale. In conclusion, the VOIS course demonstrated positive outcomes in terms of enhanced stoichiometry knowledge, academic performance, and student satisfaction. These findings highlight the potential of online courses like VOIS in facilitating the transition to higher education.
Introduction
Transitioning from secondary to higher education presents a significant challenge for students (Coertjens et al., 2017), especially in STEM areas, where high dropout rates and low performance in the first year are common (Chen, 2013; Lytle and Shin, 2020). Previous studies have identified the importance of high school performance in physics, chemistry, and mathematics as strong predictors for bachelor completion and first-year GPA (De Winter and Dodou, 2011; Alkhasawneh and Hargraves, 2014; Li et al., 2022).General chemistry, a fundamental subject in STEM curricula (Harris et al., 2020; Bastyr et al., 2022), often poses difficulties for students (Cooper and Pearson, 2012). Stoichiometry is one of the essential topics covered in General Chemistry (Pienta, 2003Gulacar et al., 2013). Unfortunately, researchers have shown how complex stoichiometry can be for students and professors (Ralph and Lewis, 2019; Stott, 2021) and a poor performance in stoichiometry can lead to struggles in subsequent chemistry courses (Cook et al., 2013). Consequently, improving and supporting STEM students' preparation in chemistry before starting the first year is critical.
This article presents the design and evaluation of a voluntary online introductory stoichiometry (VOIS) course aimed at facilitating the transition from high school to university in chemistry. Implemented as a large-scale access strategy for first-year STEM students in a Latin American university, the VOIS course aimed to improve students’ knowledge of stoichiometry and related concepts. The impact of the course was assessed using a pre–post test, measuring changes in students' understanding, ESI.† Additionally, a survey was conducted to gather students' perceptions of the online course.
Framework
Cognitive development
Piaget (1972) proposed that students actively interpret new experiences based on their previous knowledge, highlighting the importance of prior knowledge in shaping learners' understanding of the world. Similarly, Vygotsky and Cole (1978) emphasized the role of scaffolding in learning, suggesting that students should be provided with cognitive models and resources that build upon their existing knowledge. Vygotsky's Zone of Proximal Development (ZPD) explains how learners can progress from what they already know to what they can learn independently or with assistance.Tools and technologies, such as online platforms and resources, mediate learners' cognitive development in the context of online education. Therefore, it is crucial to design learning experiences that facilitate student interaction with resources in a scaffolded sequence, gradually increasing their autonomy as they progress (Jamil et al., 2022).
Enhancing teaching and learning experiences in the first year
The first year of college is particularly crucial as it sees the highest number of student withdrawals (Wingate, 2007).
General chemistry is a common requirement for various STEM disciplines such as engineering, biology, chemistry, and medicine (Harris et al., 2020; Bastyr et al., 2022).
Performance in this subject is closely related to student retention in STEM fields (Harris et al., 2020; Bastyr et al., 2022). However, learning chemistry poses challenges, as students tend to focus on memorizing equations rather than understanding the underlying concepts (Hussein and Reid, 2009; Serna-Gallén et al., 2022). Additionally, the diverse preparation levels of students further complicate the teaching of chemistry, leaving little time for remediation within the semester.
Stoichiometry, a central topic in general chemistry (Pienta, 2003), is vital in establishing quantitative relationships in chemistry problems (Dahsah and Coll, 2008; Chandrasegaran et al., 2009; Sostarecz and Sostarecz, 2012; Kimberlin and Yezierski, 2016). The following section will explore the difficulties associated with this critical topic.
Instructors also face challenges due to the ambiguous and broad definitions associated with stoichiometry (Furió et al., 2002), leading to confusion among both teachers and students (Schmidt-Rohr, 2020).
Due to the difficulties of teaching–learning stoichiometry, many strategies have successfully been implemented. In-person experiences, as Forest and Rayne (2009), have been implemented to link curriculum learning with visits to companies where students could be in contact with practical situations where concepts such as stoichiometry, acid–base, organic, etc., are applied. Online initiatives also have been implemented (e.g., le Maire et al., 2018; Cai, 2022). Some have focused on supporting a specific concept; for example, le Maire et al. (2018) uses a virtual blog to explain the idea of a limiting reactant, while others have focused on correcting some of the misconceptions generated in general chemistry problems (Cai, 2022). However, these experiences have yet to be applied on a medium-large scale and do not cover the general stoichiometry concepts for first-year students. Therefore, this study proposes to design, implement, and evaluate a voluntary online introductory stoichiometry (VOIS) course to improve the success in chemistry in first-year STEM college students.
Research questions
This study examined the effectiveness of the VOIS course among STEM students by addressing the following research questions:1. RQ1. What are the participation, retention, and course completion rates for the VOIS course?
2. RQ2. How does the VOIS course change students' knowledge of stoichiometry and related concepts?
3. RQ3. How does students' participation in the VOIS course impact their academic performance in general chemistry?
4. RQ4. How do students perceive and evaluate the course structure, resources, activities, and feedback provided in the VOIS course?
By addressing these research questions, the study aimed to provide valuable insights into the effectiveness of the VOIS course and its potential benefits for STEM students, particularly in terms of enhancing their understanding of stoichiometry and improving their academic performance.
Method
Research context
This study was conducted at a leading engineering university in Latin America with a large student population across various engineering disciplines. The university admits over 1500 students annually, and the student body is predominantly male (70%) with a smaller female representation (30%). All first-year STEM students were extended an invitation to engage in the VOIS course, which was introduced to them as part of the enrollment process. Students were informed through various channels, including in-person at enrollment, email correspondence (both institutional and personal emails used for enrollment), as well as the university's introductory handbook and official social media platforms. The course was strategically designed to be undertaken during the summer break.In their first year, students are required to take courses in mathematics, physics, programming, sports, and general chemistry. General chemistry is mandatory for all engineering majors, typically taken in the first or second semester. Even though stoichiometry is a critical topic in this course, additional topics like acid–base reactions, electrochemistry, and environmental contamination are covered.
Despite students belonging to different majors, they follow the same coursework, assessments, and activities. A faculty member coordination team designs the activities and assessments for all students. The coordination team and instructors were unaware of students' participation in the VOIS course, ensuring an unbiased evaluation.
Participants
Between 2019 and 2021, a Latin American university invited all first-year STEM students to participate in the VOIS course. A total of 3995 students voluntarily enrolled in the course. The VOIS course is not mandatory for academic progression, and completing it is not used in calculating students' general chemistry grades.Measures
While some questions from the original survey were used directly, others were reformulated to align with our context. We retained the core concepts while modifying the wording or using explicit questions. For instance, question 4 from the original survey tested the understanding of the relationship between molecules and atoms to moles by asking the number of moles of sulfur in a sulfur molecule (S8). We used the same example and adapted it into question 7. Similarly, question 13 in the original survey assessed the law of conservation of mass (“in chemical reactions, the quantity of matter does not change” Dahsah and Coll, 2008, p. 583). We explicitly evaluated this concept through question 3 in our adapted questionnaire.
We also generated new questions to evaluate the concepts outlined by the SCQ authors. For instance, the initial test's question 14 aimed to assess the idea that “The number of atoms of each element on the reactant side and the product side must be equal; this is called a balanced chemical equation, and a chemical equation is a series of symbols that represent the formulae of reactants and products in (Reactants) Y (Products).” Consequently, we formulated question 4 where students are required to select the appropriate representation of a reaction, and the answer choices vary based on the balance of the equation.
SCQ questionnaire covers nine fundamental concepts related to stoichiometry: atomic mass, molar mass, mole, solution, empirical and molecular formulas, chemical equations, and quantity relationships in chemical reactions. To fit the learning outcomes of the VOIS course, the test was shortened into a ten-item multiple-choice questionnaire (see the appendix for the full instrument used).
The assessment employs a scoring system ranging from 0 to 10. While we can retrieve the aggregate score, a breakdown of individual question performance is not accessible. The instrument primarily aims to evaluate the understanding of stoichiometry and its associated concepts. Throughout this paper, we will collectively refer to these related concepts as “stoichiometry concepts,” acknowledging that a comprehensive grasp of these concepts is crucial for effectively solving stoichiometry problems.
In addition, the dataset for the cohort of 2020 includes additional variables that were not measured as part of the course but were provided directly by the university. The initial dataset consisted of 2075 observations; however, some observations were missing values for certain covariates. Therefore, a total of 1780 observations included complete values for all variables.
Data analysis
To evaluate the effectiveness of the VOIS course, a quasi-experimental design was implemented, consisting of two groups: ENROLL and CONTROL. The inclusion criteria and descriptions of each group are as follows:1. ENROLL: this group encompasses all students who registered for the VOIS course. Within this group, two sub-group can be delineated.
A. EXP I: students who enrolled in the course but did not complete at least one activity in each module. Although they might have undertaken initial or final tests, they did not meet the requirement of engaging in activities across all modules.
B. EXP II: students who enrolled in the course completed at least one activity in Module 1 and Module 2 and completed the pre-test (part of the welcome section) and post-tests (part of Module 3).
2. CONTROL: this group comprised students who did not enroll in the VOIS course but were freshmen enrolled in a STEM program.
It is important to note that the students in EXP II are distinct from those in EXP I. Students who enrolled in the course but did not complete any activities are classified under the “enroll” group and are not included in either EXP I or EXP II. All first-year STEM students can be categorized into one of these two groups.
The criteria for classifying students as part of EXP II were consciously established to embrace a range of engagement levels. Students might have chosen not to complete specific activities if they felt adequately prepared in those aspects. Furthermore, we positioned the post-test (a summative activity) within module 3 as a requirement rather than selecting another formative activity because it was used in the impact analysis. Hone and El Said (2016), demonstrated that students are highly unlikely to skip activities after engaging in and completing more than half of an online course. Consequently, if a student was actively participating in activities within modules 1 and 2, probably, they were equally engaged in module 3.
Furthermore, to be considered as completed, an activity in the course required the student to provide an answer, but no criteria of correctness or performance were applied to evaluate their responses. It is worth mentioning that no parameters such as time spent on the platform or metrics were available to differentiate between passive and active participation in the course. This limitation is discussed further in the limitations section of the study.
An ANOVA test was conducted using Stata 17.0 to assess the variance between the groups. The analysis encompassed the entire dataset, which comprised 2075 data points.
Conducting a randomized trial was not feasible due to university policies. All freshman students were invited to participate in the course, and there were no direct incentives or benefits for those who chose to participate. Consequently, it is highly probable that students who already possessed a better attitude, motivation, or prior preparation were the ones who opted to take the course.
To reduce bias, propensity score analysis, proposed by Rosenbaum and Rubin (1983), was employed to construct an artificial control group by matching similar characteristics between the experimental and control groups. This technique estimates the effect of receiving treatment by scoring the probability (propensity score) of receiving the treatment (or being in the experimental group) based on observed characteristics (Rubin, 1979). In this study, propensity score analysis computed the probability of students enrolling in the VOIS course based on observed characteristics such as high school grades, gender, national university admission test scores, and ranking.
The propensity score analysis was conducted using Stata 17.0, following the procedures outlined by Guo and Fraser (2014). The script used to run the analysis is described in Appendix B of the study. Here is a brief overview of the five key steps (Valojerdi and Janani, 2018) followed in the propensity analysis.
The dataset was initially constructed with 2075 observations. However, for the purpose of this analysis, 295 observations were excluded from the analysis due to missing values for at least one covariate. As a result, the propensity score was calculated based on the remaining 1780 observations, which had complete values for all covariates.
A binary treatment variable was established to distinguish between students who completed the course (EXP II) and those who did not enroll in the course (CONTROL):
1: students who completed the course (EXP II)
0: students who did not enroll in the course (CONTROL)
This treatment variable was used to differentiate between the two groups in the analysis. It is important to note that EXP I was not included in the PSA.
A logistic regression model was utilized to estimate the propensity score for this binary variable.
Random matching was performed by pairing observations in the treatment group with untreated subjects. Stratification was applied to ensure sufficient overlap in the propensity score predictions between the control and experimental groups. Three observations that did not meet this criterion were excluded.
The inverse probability of treatment was calculated as a weight using the propensity scores. This weight represents the inverse probability of receiving the treatment and can now be applied to the control group. By applying this inverse probability, the differences in covariates between the control and experimental groups were considerably reduced, rendering any remaining differences statistically insignificant.
Subsequently, a linear regression analysis was performed, utilizing the chemistry grades as the dependent variable and the completion of the VOIS course as the covariate. The inverse probability technique was applied to the control group to mitigate disparities.
Designing instruction for a seamless transition and effective leveling course
A variety of virtual resources, including videos, exercises, and readings, were developed to specifically address the challenges associated with stoichiometry. Drawing from over 30 years of experience working with freshman STEM students and the identified challenges discussed in the literature review, the course focused on critical aspects such as the definition and interpretation of the mole, composition of matter, representation of a chemical reaction, limiting reactant, and the mathematical skills required to solve quantitative problems. This section will explore different pedagogical approaches aimed at effectively teaching and learning stoichiometry in a virtual setting.Utilizing everyday analogies to enhance students' understanding of stoichiometry
The VOIS course incorporated simple analogies with everyday situations to enhance the understanding of abstract stoichiometry concepts, as supported by previous research (de Sanabia, 1993; Patel-Desai, 2020). The first analogy involved comparing the construction of a bicycle using frames and wheels to the construction of chemical compounds using elements. The second analogy involved creating relative scales using familiar objects, which helped introduce the concept of mole. This section will outline how each concept was taught using these analogies, providing a structured approach to the course content. | ||
| Fig. 1 Depicting the chemical notation for describing the bicycle assembly process. | ||
We highlight the differences between stoichiometry coefficients and a subscript. For example, the bicycle needs two wheels (2W) and one frame (1F). When we have assembled the bicycle, we can also write it in terms of its component (W and F). Nevertheless, because a bicycle has two wheels, we use the notation: FW2. It is essential to note that subscript 2 describes the number of bicycle wheels and makes the difference from a tricycle that needs three wheels (FW3).
Students were introduced to the concept of atoms and molecules using this analogy. Furthermore, an analogy was made between building a bicycle and forming water from hydrogen and oxygen.
 | ||
| Fig. 2 Video animations illustrating the analogy of a scale and a screw to represent atomic mass units. | ||
How many kilograms of Screws should be bought to have the same number of screws as nuts in one kg of nuts?
One possibility is to count the number of nuts and then buy the same number of units. Nevertheless, that would take a long time. Nevertheless, we can use relative mass scale concepts to answer the question. For example, if the mass ratio between a screw and a nut is 3![[thin space (1/6-em)]](https://www.rsc.org/images/entities/char_2009.gif)
:1, it means that for every 3 units of mass in nuts, there will be 1 unit of mass in screws. Using this information, students can estimate the quantity of objects they need without physically counting each one. Therefore, in the previous example, we must buy 3 kg of screws to have the same quantity. Then, students work in adaptive exercises (discussed below) to input various values of the nut unit of mass.
To illustrate this, consider the comparison between the mass of an oxygen atom and a hydrogen atom. The crucial point is that without knowing the number of oxygen atoms in 16 grams, there will be the same number of oxygen atoms as there are hydrogen atoms in 1 gram of hydrogen. This fixed number of units is known as Avogadro's number, which is approximately 6.023 × 1023.
This fixed number is given a special name, the mole, to simplify calculations. The mole represents a specific amount of matter that contains Avogadro's number of units, whether they are atoms, molecules, or any other entities. It is convenient to work with a fixed quantity of units across different substances.
After building all the bicycles, how many wheels are not used? This question and the calculations involved allow the introduction of excess reactants. Then, exercises with chemical reactions were added.
Assessment: adaptative problems and multiples choice
The importance of student interaction in online education has been highlighted by Jamil et al. (2022). Furthermore, informative feedback has been recognized as a crucial component of distance learning by Kanuka (2001), Eitemüller et al. (2023), and Wengrowicz et al. (2022). To address the challenge of student dropout in online courses, personalized and adaptive learning approaches have been proposed as alternatives (Ross et al., 2018). Three problems were used: adaptive problems, multiple choice with Hints and informative Feedback, and multiple choice without feedback. For instance, Fig. 3 illustrates an adaptive problem implemented in the course, where students enter a numerical value representing the mass they want to assign to a screw. Blue words are the English translation of the problem statement. | ||
| Fig. 3 Illustration of adaptive problems integrated into the VOIS course. | ||
The students enter a numerical value in grams of what mass they want to assign to a screw. Then all the subsequent questions will be based on this value made. In fact, all other follow questions in the activities were developed with parametric feedback (programmed in phyton), adapted to the first students' input. So, this type of exercise can be adapted to the students and be modeled as they want. Students can also re-do the activity with different initial values to check if they understand it.
 | ||
| Fig. 4 Example of multiple-choice problems with hints. | ||
Piloting the transition course
The primary objective of the course was to bridge the gap between secondary education and higher education by consolidating the knowledge acquired in secondary education and facilitating the transition to higher education. To evaluate the relevance, accuracy, and effectiveness of the course activities, resources, and topics, two pilot studies were conducted. The first pilot involved using the final VOIS course to support freshman engineering and sciences students in 2017. Approximately 30 students participated in this pilot, representing the same context as our target participants (same institution, course, achievements, etc.).The course materials were made available on a Moodle platform as a complementary resource to face-to-face classes. Moodle was chosen as the platform since students were already familiar with it. Students had the opportunity to provide feedback on the effectiveness of the resources, with the main comments focusing on the duration of the videos and the quality of the audio. To address these concerns, video scripts were developed to enhance the accuracy, duration, and content of each video.
The second pilot was conducted through face-to-face sessions with 26 secondary students. This pilot study specifically aimed to evaluate the activities and the effectiveness of the analogy used to introduce the mole concept. Although previous research has demonstrated the effectiveness of these analogies, they were conducted in different contexts. Therefore, we conducted this pilot study to ensure that the analogy is suitable and effective for our specific target students. Students were provided with work guides and were asked to comment on any areas where instructions or concepts could have been clearer. Students received written instructions that were consistent with the online course and worked individually or in pairs. The first author of the paper observed the students and made notes on any difficulties they encountered while completing the activity. Based on this feedback, adjustments were made to the online course.
A scaffolding course ensemble
Taking into account the feedback and insights gained from the piloting process, the course was structured using a cognitive sequencing learning approach, where each topic or concept related to stoichiometry followed a similar pattern of instruction. As is shown in Fig. 5, the final version of the VOIS course has three modules: chemical reaction, the mol, and stoichiometry in chemical reactions. | ||
| Fig. 5 Structure of the course and topics covered in each module. | ||
Each topic followed a similar pattern: introduction through videos and readings, application of analogies like assembling a bicycle, instruction on methodology and math operations, practice activities, and evaluations with feedback. This scaffolding approach, based on Vygotsky's theory, aimed to facilitate learning and understanding. The same structure was applied to other concepts in the course.
Results
Participation and engagement (RQ1)
The course had a total of 3995 voluntary student enrollments. The highest enrollment occurred in the year 2020, with 2354 students. Out of the total enrolled students, 1650 started the course by completing the initial test. The final tests received 397 valid answers.1331 students were categorized as part of EXP I, and 358 as part of EXP II. 2020 had the highest number of enrollments, participation, and completions. Table 1 summarizes the annual statistics for student participation in the course. The “Pre” column indicates the number of responses received for the pre-stoichiometry test.
| Year | Enrolled | EXP I | EXP II | Pre test | Post test |
|---|---|---|---|---|---|
| 2019 | 1240 | 382 | 135 | 512 | 140 |
| 2020 | 2354 | 860 | 209 | 1039 | 239 |
| 2021 | 401 | 89 | 14 | 99 | 18 |
| Total | 3995 | 1331 | 358 | 1650 | 397 |
To calculate the completion rate for the VOIS course, we divided the number of students in the EXP II group (358) by the total number of students enrolled in the course (3995), resulting in a completion rate of 9%. However, to account for the significant number of students who did not actively participate, we subtracted the number of students who completed the course (EXP II: 358) from those who started the voluntary online course (1650). This yielded a course finalization rate of 22% overall.
Furthermore, we analyzed the drop-off rates based on the initial knowledge test performance. The results revealed that the highest drop-off rate (84%) was observed among students who scored 0–2 points on the initial test. Similarly, students who scored 3–4 points and 7–8 points on the initial test also had a high drop-off rate of 83%. In contrast, students who performed well on the initial test (9–10 points) had a lower drop-off rate of 62%.
Impact on stoichiometry knowledge (RQ2)
A hypothesis t-test was applied to examine the difference between pre- and post-test scores. Table 2 shows the statistical parameters used to calculate the t-tests. The average difference between the pre–post evaluation was +1.94 points (on a scale of 0–10) and was statistically significant (p < 0.0001). Furthermore, the effect size was estimated by using Cohen's d, obtaining a d = 0.8 showing a large effect produced by the voluntary course (Cohen, 2013).An analysis was conducted to examine the difference in performance before and after completing the VOIS course for each year. The results showed gains of +2.66 (in 2019), +1.37 (in 2020), and +1.64 (in 2021). Multiple t-tests were performed on the same group, and to address the issue of multiple comparisons, Bonferroni's correction was applied, resulting in a critical p-value of 0.0125. The differences in performance before and after the course were statistically significant (p < 0.0001) for the years 2019 and 2020, indicating an improvement in students' knowledge and understanding of stoichiometry concepts. However, for the year 2021, the difference was not statistically significant.
Furthermore, considering the high drop-off on the course, an analysis focused on the performance of the students who and after the course. The results showed a gain of +1.3 (from 5.31 to 6.61) in the stoichiometry knowledge test score, which were categorized as EXP II (358 students). A paired t-test was conducted to examine the difference in performance before was statistically significant. This indicates a 24% improvement in students' understanding and application of stoichiometry concepts throughout the course. Table 3 summarizes the statistics results.
An additional analysis was conducted to examine the performance by year. The t-test values were 7.6 in 2019 (p-value of 0.0001), 10.1 in 2020 (p-value of 0.0001), and 2.6 in 2021 (p-value of 0.0210). Except for the difference in the year 2021, which was not statistically significant, all other differences were statistically significant. It should be noted that the 2021 cohort had a small sample size of only 14 students, which may have influenced the results.
Table 4 displays the gain results obtained by different groups of students based on their performance on the initial test. The group with the lowest performance (0–2 points) achieved the highest gain of 3.13 points, indicating a significant improvement in their understanding of stoichiometry. On the other hand, the group of students who initially performed excellently (9–10 points) showed only a slight gain of 0.05 points.
| Group | Number of students | Average pre-test | Average difference |
|---|---|---|---|
| Score pre-test (0–2) | 63 | 1.52 | 3.13 |
| Score pre-test (3–4) | 91 | 3.55 | 1.71 |
| Score pre-test (5–6) | 85 | 5.40 | 1.28 |
| Score pre-test (7–8) | 54 | 7.44 | 0.00 |
| Score pre-test (9–10) | 65 | 9.57 | 0.05 |
Effect of VOIS on students' performance (RQ3)
Based on the participation of students in the VOIS course in 2020, Table 5 presents a summary of the key statistics for each group. It is important to note that the descriptions of EXP I and EXP II remain the same as previously stated. However, there has been a change in the number of students within these groups due to the focus on analyzing only the cohort from 2020.| Group | Mean | Std. dev. | n |
|---|---|---|---|
| CONTROL | 61.3 | 14.8 | 1458 |
| EXP I | 64.8 | 12.2 | 467 |
| EXP II | 71.9 | 10.7 | 150 |
| Total | 62.8 | 14.3 | 2075 |
| Analysis of variance | |||||
|---|---|---|---|---|---|
| Source | SS | df | MS | F | Prob > F |
| Between groups | 177.4 | 2 | 887 | 45.1 | 0 |
| Within groups | 407.3 | 2072 | 196.6 | ||
| Bartlett's equal-variances test: chi2(2) = 44.1274 | |||||
| Prob > chi2 = 0.000 | |||||
An analysis of variance (ANOVA) was conducted to compare the means across the different groups. The analysis yielded a statistically significant difference (Prob < 0.0001) between the groups, indicating that there were significant variations in performance.
Furthermore, a Scheffe analysis (see Table 6) was performed to assess the statistical significance between the groups. The results demonstrated a statistically significant difference (p-value < 0.001), indicating that students who completed the VOIS course (EXP II) obtained 10.6 points more than their peers who did not participate (CONTROL) and 7.0 points higher than their peers who participated in the VOIS course but did not complete it (EXP I).
| Group | Control | Exp I |
|---|---|---|
| Exp I | 3.6 | — |
| 0 | — | |
| Exp II | 10.6 | 7.0 |
| 0 | 0 |
To determine the effect size, Cohen's d was calculated between the EXP II and CONTROL, yielded a value of 0.82. This suggests a relatively large effect size of the intervention.
Considering that the selection between the control and experimental groups was completely voluntary, it is possible that students who participated in the VOIS course were those who already had better preparation in stoichiometry or general chemistry.
Table 7 displays the mean and standard differences between the treated and untreated groups. It should be noted that the dataset used for this analysis included 1780 observations (not to be confused with the 2075 observations used in the ANOVA), as some covariate values were missing. The number of observations in each group were as follows: (a) control: 1231, (b) EXP I: 415, and (c) EXP II: 134.
| Covariable | Mean in treated (EXP II n = 134) | Mean in untreated (control n = 1231) | Standardised diff. |
|---|---|---|---|
| Gender | 0.29 | 0.23 | 0.147 |
| Highschool grades | 6.39 | 6.34 | 0.188 |
| Raking | 866.5 | 849.8 | 0.182 |
| NUST score-language | 753.4 | 728.4 | 0.301 |
| NUST score-science | 682.7 | 629.9 | 0.326 |
| NUST score-math | 757.8 | 726.9 | 0.247 |
The most substantial differences are observed in the scores of the national standardized tests, particularly in the areas of science, language, and mathematics. These disparities introduce bias into the results since previous student preparation in math and science strongly predicts performance in general chemistry.
To mitigate this bias, a propensity score analysis (PSA) was conducted using student gender, high school grades, school ranking, and scores on the national standardized tests as covariates. The PSA successfully recalibrated the weights of the control group, resulting in a reduction of the difference between the treatment and control groups to a statistically nonsignificant level. Table 8 provides a summary of the new differences with the weight correction applied to the control group.
| Covariable | Mean in treated | Mean in untreated | Standardised diff. |
|---|---|---|---|
| Gender | 0.29 | 0.29 | −0.002 |
| Highschool grades | 6.39 | 6.39 | −0.009 |
| Raking | 866.8 | 866.3 | −0.006 |
| NUST score-language | 752.6 | 753.3 | −0.009 |
| NUST score-science | 680.1 | 681.9 | −0.011 |
| NUST score-math | 756.2 | 757.3 | −0.008 |
| Lineal regression | |||
|---|---|---|---|
| Chemistry | Coefficient | t | P > t |
| Treatment | 8.66 | 8.83 | 0.000 |
| Gender | 1.52 | 1.28 | 0.202 |
| Highschool grades | 14.9 | 3.40 | 0.001 |
| Raking | −0.02 | −1.86 | 0.064 |
| NUST score-language | −0.00 | −0.42 | 0.673 |
| NUST score-science | 0.01 | 1.10 | 0.270 |
| NUST score-math | 0.02 | 2.78 | 0.051 |
Subsequently, a linear regression analysis with the weights was performed using the chemistry course grade as the dependent variable and including all covariates used in the PSA. This analysis revealed that students who completed the course (EXP II) had a statistically significant difference (p-value < 0.0001) of 8.6 points higher performance in general chemistry compared to students who did not participate in the course (CONTROL).
Satisfaction with the VOIS course (RQ4)
The satisfaction survey received one hundred fifty-six responses across the years—55 in 2019, 91 in 2020, and 10 in 2021. The number of responses received shows the same pattern of participation. The survey consisted of 8 items that evaluate the quality of the material, clarity, navigation, access, and sources of feedback within the course. All items had over 75% satisfaction.Table 9 provides the results of each item of the survey. The feedback mechanisms in the course received the lowest score, with 19.9% of students expressing that there were not enough feedback mechanisms. Despite including activities with feedback and adaptive exercises, there is still a need to enhance the value of feedback provided to students. On the other hand, the clarity and consistency between activities and learning objectives received the highest score of 94%, indicating that the instructional course design effectively facilitated scaffolding cognitive activities for students. Overall, students rated the course at 3.1 on a scale of 1–4.
| Question | Yes | No | N/A |
|---|---|---|---|
| The objective of the course is clearly specified | 94% (n = 145) | 2% (n = 3) | 4% (n = 6) |
| The required prior knowledge was reported | 72% (n = 113) | 21% (n = 34) | 7% (n = 9) |
| Navigation through the course was easy | 86% (n = 133) | 10% (n = 16) | 4% (n = 6) |
| Course materials are presented in an attractive structure and format | 83% (n = 129) | 13% (n = 21) | 4% (n = 6) |
| The course contents were presented in a logical sequence | 91% (n = 142) | 6% (n = 10) | 3% (n = 4) |
| Activities were consistent with the purpose of the course | 91% (n = 142) | 4% (n = 7) | 4% (n = 7) |
| Assessments into the course contributed to learning | 87% (n = 135) | 8% (n = 13) | 5% (n = 8) |
| Were there feedback mechanisms in the course? | 66% (n = 106) | 19% (n = 31) | 12% (n = 19) |
Discussion
Several studies have emphasized the difficulties linked with stoichiometry for both learners and educators (Bridges, 2015; Ralph and Lewis, 2019; Schmidt-Rohr, 2020; Stott, 2021). The VOIS course effectively addresses these challenges by demonstrating a statistically significant enhancement in students' comprehension of stoichiometry and related concepts.For future editions of this course, we would recommend that students who score more than 7 points on the pre-test should not take the course, while students with lower performance should be encouraged to complete all activities. Due to local policies, we cannot restrict the course to specific groups and must offer it to all students. Nonetheless, additional strategies could be implemented to provide incentives for students who complete the course.
To enhance success in STEM careers, previous remedial or transition courses have primarily focused on math and language skills (Zhao et al., 2021), yielding mixed results for STEM students (Chen and Wu, 2020; Zhao et al., 2021). For example, in chemistry, a longitudinal study by Bentley and Gellene (2005) found no significant effect on students' academic performance in the first year. In contrast, the VOIS course responds to the need for support in leveling STEM students in chemistry and demonstrates a statistically significant improvement in first-year general chemistry performance, with a large effect size.
The PSA technique effectively controlled for covariates such as gender, high school grades, school ranking, and scores on the national tests. Interestingly, among the covariates included in the PSA, only high school grades showed statistical significance (p-value 0.05) as a predictor of performance in general chemistry. Gender, school ranking, and scores on the national tests did not exhibit statistical significance in predicting general chemistry performance.
The completion rate of the course fell within the expected range for voluntary online initiatives (Ho et al., 2014). However, strategies to increase participation and completion rates could be explored. The COVID-19 pandemic may impact the completion and finalization rates, leading to lower participation and completion rates in 2021. Furthermore, our analysis revealed that the drop-off rate in the course was higher among students who had a poor performance on the pre-knowledge test. While these results align with previous findings in the literature, it emphasizes the need to develop additional strategies to engage students throughout the course. In the future, we aim to explore the creation of a more adaptive learning system that can tailor the cognitive difficulty of the course based on the student's performance.
In summary, the findings of this research provide valuable insights into the effectiveness of the VOIS course and its impact on first-year students' performance in chemistry. The course's ability to address challenges, improve stoichiometry knowledge, and positively impact students' academic journey underscores its value as a supportive tool in STEM education.
Limitation
While we have strived to conduct a rigorous analysis in this research, it is important to acknowledge certain limitations. Firstly, the stoichiometry knowledge test’s reliability in this study is a concern. Due to the lack of instruments available in the literature that have evidence of generating valid data on evaluating stoichiometry in Spanish tailored to comparable student groups, we adopted an existing instrument available that has evidence of generating reliable data as a reference to develop a new tool aligned with the course’s learning objectives. Unfortunately, we could not generate evidence about the reliability of this newly created instrument. Additionally, using a similar instrument for pre- and post-assessment may have led to potential recall bias. Our analysis, based on students’ grades in the first semester, indicates that the course significantly contributes to students’ success during this period, which helps to reduce this concern. In future work, we aim to conduct a psychometric analysis of the stoichiometry knowledge test.Secondly, while we employed matching techniques to reduce bias between the control and experimental groups, our measurements were limited to students' academic conditions. We did not incorporate measurements of factors such as attitude, motivation, and other influential factors that may impact student performance in their first year.
Thirdly, as discussed within the research context, our focus primarily revolved around stoichiometry, as it serves as the core and most significant topic covered in the general chemistry course at our university. However, this is not the exclusive topic covered in the general chemistry course. While other topics such as global contamination, acid–base chemistry, and more were touched upon in the course, we firmly believe that a solid foundation in stoichiometry is essential for students' success in their first semester. Future research could explore the impact of the course on other topics as well.
Fourthly, the course allowed free navigation, and there were no restrictions on student participation. Unfortunately, we lacked metrics such as time spent on the platform or clicks made by students, which could have provided further insight into their behavior during the course. However, we were able to categorize students' engagement based on their interactions and responses to formative and summative activities throughout the course, which also gave us a sense of how active and engaged students were in the course.
Future work
In our future work, we plan to enhance the stoichiometry knowledge test and validate it further. Additionally, we aim to improve the monitoring of student participation in the course.Furthermore, we intend to conduct a more comprehensive analysis of the impact of using analogies on students' understanding. This analysis will be followed by interviews with students at the conclusion of the VOIS course. These interviews will provide valuable qualitative data to complement the quantitative analysis and help us better understand students' experiences and perceptions.
By incorporating these improvements and conducting further research, we aim to refine the course and continue enhancing its effectiveness in facilitating the transition from secondary education to higher education in STEM fields.
Conclusion
This research presents an effective design for an online voluntary stoichiometry chemistry course that utilizes a scaffolding cognitive approach, employing analogies to introduce challenging stoichiometry concepts. Implementation of this course in a Latin American university resulted in statistically significant improvements in stoichiometry understanding, leading to enhanced performance in general chemistry during the first semester for STEM college students.This study addresses the need for innovative strategies that facilitate the transition from high school to university and can be easily scaled. By providing students with a structured and supportive learning experience, the course effectively bridges the gap between secondary education and higher education in the field of STEM. The findings of this research contribute to the broader body of knowledge on effective instructional approaches and can inform the development of similar transition courses in other educational contexts or topics.
Ethical statement
All procedures performed in studies involving human participants were in accordance with the ethical standards of the Human Research Ethics Committee (HREC) and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was approved and considered as not human subjects by the North Carolina State University Institutional Review Board (IRB) (Protocol ID: 25058).Author contributions
Conceptualization: Brayan Diaz, Arie Aizman; analysis: Brayan Diaz; writing – first draft: Brayan Diaz; writing – review and editing: Arie Aizman, Brayan Díaz; funding acquisition: Arie Aizman.Conflicts of interest
There are no conflicts to declare.Appendix
[X]: Show the correct answers.1. Indicate which of the following statements is correct:
(a) An atom consists of a nucleus containing protons and neutrons, and electrons outside the nucleus. [X]
(b) Most of an atom's space is occupied by the nucleus, which contains a significant portion of the atomic mass.
(c) In a neutral atom, the number of protons equals the number of electrons. [X]
(d) Atoms are arranged in the modern periodic table based on their mass.
(e) The atomic mass of an atom is equal to the number of protons plus neutrons in its nucleus. [X]
Note: Make sure to select all correct options – there may be more than one!
2. Indicate which of the following statements is correct:
(a) The mass of a hydrogen atom is 1.0 gram.
(b) Isotopes of an element differ in their mass because they contain a different number of neutrons in the nucleus. [X]
(c) The mass of the carbon isotope with mass number 12, the most abundant isotope of carbon, is 12.011 atomic mass units (as shown on the Periodic Table for Carbon).
(d) The mass of one mole of oxygen atoms is 16.0 atomic mass units.
(e) In the periodic table, the atomic mass of an element is a weighted average of the masses of all isotopes of that element. [X]
Note: Make sure to select all correct options – there may be more than one!
[explanation] If you didn't answer this question correctly, review chapters 1 and 2 carefully. Review the definitions of isotopes and the first part of the text “Isotopes, the definition of the standard element, and the mole” in this course, which discusses how atomic mass in the Periodic Table is related to them. [explanation]
3. When a chemical reaction occurs, what can be stated?
(a) Mass is conserved [X]
(b) The number of molecules is conserved
(c) The number of atoms is conserved [X]
(d) The number of moles is conserved
Note: Make sure to select all correct options – there may be more than one!
[explanation]
In a chemical reaction, the law of conservation of mass is fulfilled. From the microscopic point of view, the atoms are conserved (atoms are not created or destroyed in a chemical reaction; they are only rearranged to form products). Note that the conservation of atoms ensures the conservation of mass in a chemical reaction. [explanation]
4. The chemical equation representing the reaction of solid aluminum, Al(s), with gaseous chlorine, Cl2(g), to produce solid aluminum chloride, AlCl3(s), is:
(a) Al(s) + Cl2(g) → AlCl3(s)
(b) 2Al(s) + 3Cl2(g) → 2AlCl3(s). [X]
(c) 3Al(s) + 3Cl2(g) → 3AlCl3(s)
(d) 3Al(s) + Cl2(g) → 4AlCl3(s)
[explanation] Atoms must be conserved. [explanation]
5. In a reaction vessel, 200 molecules of H2 and 200 molecules of O2 are placed, which react according to: H2 + ½O2 → H2O. After a certain time t, it is observed that there are 80 molecules of H2O in the vessel. It can be stated that at that instant, in the vessel there are:
(a) 80 molecules of H2 and 40 molecules of O2
(b) 280 molecules of H2 and 240 molecules of O2
(c) 120 molecules of H2 and 160 molecules of O2 [X]
(d) It cannot be known until the reaction is complete.
[explanation] If 80 molecules of water were formed, then 80 molecules of H2 and 40 molecules of O2 must have reacted. Therefore, there should be (200 − 80) = 120 molecules of H2 and (200 − 40) = 160 molecules of O2 remaining. [explanation]
6. We can state that one mole of water molecules, H2O, and one mole of oxygen molecules, O2:
(a) Have the same mass
(b) Contain one molecule of each
(c) Have a mass of 1 gram each
(d) Contain the same number of molecules [X]
[explanation] Being a mole a fixed amount of units, then in one mole of different compounds, there is always the same number of units of that compound. [explanation]
7. A sulfur molecule contains 8 sulfur atoms, S8. Therefore, one mole of sulfur molecules will contain:
(a) 8 grams of sulfur
(b) 8 moles of sulfur atoms [X]
(c) 6.02 × 1023 sulfur atoms
(d) 8 sulfur atoms
[explanation] Being a mole has a fixed number of units, then in one mole of S8 there are 8 moles of S atoms. [explanation]
8. Ammonia, NH3, can be prepared by the reaction between nitrogen (N2) and hydrogen (H2), according to N2(g) + 3H2(g) → 2NH3(g). How many moles of hydrogen are required to prepare 1 mole of ammonia? (Assume that all the nitrogen needed is available).
(a) 1
(b) 3/2 [X]
(c) 2/3
(d) 3
(e) 2
9. When methane, CH4, reacts with oxygen (O2), water, H2O, and carbon dioxide (CO2) are formed. How many grams of methane are needed to produce 110 grams of water?
(a) 55 grams
(b) 110 grams
(c) 49 grams [X]
(d) 30 grams
(e) 3 grams
10. Aluminum reacts with hydrogen chloride to produce aluminum chloride and hydrogen gas according to: 2Al(s) + 6HCl(g) → 2AlCl3(s) + 3H2(g). Calculate the mass of aluminum chloride formed when 2.70 grams of aluminum react with 4.00 grams of hydrogen chloride.
(a) 4.93 grams [X]
(b) 13.3 grams
(c) 2.70 grams
(d) 1.70 grams
(e) 4.00 grams
Acknowledgements
We thank Dr K. C. Bush for her suggestions in the conceptualization. We would like to thank Dr Julie Edell for her insightful and outstanding comments while preparing the manuscript. The authors also thank the Online Learning department at Universidad Técnica Federico Santa María. We thank all anonymous reviewers for their outstanding and informative suggestions.Notes and references
- Alkhasawneh R. and Hargraves R. H., (2014), Developing a hybrid model to predict student first year retention in STEM disciplines using machine learning techniques, J. STEM Educ.: Innovations Res., 15(3), 35–42.
- Bastyr C., Johnson C., Lakhan R. and Wainman J. W., (2022), Reducing chemistry casualties: supporting women enrolled in general chemistry i not during their first term of college with active learning, J. Chem. Educ., 99(9), 3089–3095 DOI:10.1021/acs.jchemed.1c01279.
- Bentley D. and Gellene G., (2005), Longitudinal study on the implementation of a remedial chemistry course at Texas Tech University, J. Chem. Educ., 82(7), 1099–1104.
- BouJaoude S. and Barakat H., (2003), Students' problem solving strategies in stoichiometry and their relationships to conceptual understanding and learning approaches, Electron. J. Res. Sci. Math. Educ., 7(3), 42.
- Bridges C. D., (2015), Experiences Teaching Stoichiometry to Students in Grades 10 and 11 (Order No. 3684500). Available from ProQuest Dissertations & Theses Global, (1660768832), https://proxying.lib.ncsu.edu/index.php/login?url=https://www.proquest.com/dissertations-theses/experiences-teaching-stoichiometry-students/docview/1660768832/se-2.
- Cai S., (2022), Harry Potter themed digital escape room for addressing misconceptions in stoichiometry, J. Chem. Educ., 99(7), 2747–2753 DOI:10.1021/acs.jchemed.2c00178.
- Chandrasegaran A. L., Treagust D. F., Waldrip B. G. and Chandrasegaran A., (2009), Students’ dilemmas in reaction stoichiometry problem solving: deducing the limiting reagent in chemical reactions, Chem. Educ. Res. Pract., 10(1), 14–23 10.1039/B901456J.
- Chen X., (2013), STEM Attrition: College students’ paths into and out of STEM fields (NCES2014–001), Washington: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.
- Chen C. L. and Wu C. C., (2020), Students’ behavioral intention to use and achievements in ICT-Integrated mathematics remedial instruction: case study of a calculus course, Comput. Educ., 145, 103740.
- Coertjens L., Brahm T., Trautwein C. and Lindblom-Ylänne S., (2017), Students’ transition into higher education from an international perspective, Higher Educ., 73(3), 357–369 DOI:10.1007/s10734-016-0092-y.
- Cohen J., (2013), Statistical power analysis for the behavioral sciences, Academic Press.
- Cook E., Kennedy E. and McGuire S. Y., (2013), Effect of teaching metacognitive learning strategies on performance in general chemistry courses, J. Chem. Educ., 90(8), 961–967 DOI:10.1021/ed300686h.
- Cooper C. I. and Pearson P. T., (2012), A genetically optimized predictive system for success in general chemistry using a diagnostic algebra test, J. Sci. Educ. Technol., 21, 197–205.
- Dahsah C. and Coll R. K., (2008), Thai grade 10 and 11 students’ understanding of stoichiometry and related concepts, Int. J. Sci. Math. Educ., 6(3), 573–600 DOI:10.1007/s10763-007-9072-0.
- de Sanabia J. A., (1993), Relative atomic mass and the mole: a concrete analogy to help students understand these abstract concepts, J. Chem. Educ., 70(3), 233 DOI:10.1021/ed070p233.
- De Winter J. C. F. and Dodou D., (2011), Predicting academic performance in engineering using high school exam scores, Int. J. Eng. Educ., 27(6), 1343.
- Eitemüller C., Trauten F., Striewe M. and Walpuski M., (2023), Digitalization of Multistep Chemistry Exercises with Automated Formative Feedback, J. Sci. Educ. Technol., 1–15.
- Forest K. and Rayne S., (2009), Thinking outside the classroom: integrating field trips into a first-year undergraduate chemistry curriculum, J. Chem. Educ., 86(11), 1290 DOI:10.1021/ed086p1290.
- Furió C., Azcona R. and Guisasola J., (2002), The learning and teaching of the concepts amount of substance and mole: a review of the literature, Chem. Educ. Res. Pract., 3(3), 277–292 10.1039/B2RP90023H.
- Gulacar O., Overton T. L. and Bowman C. R., (2013), A closer look at the relationships between college students’ cognitive abilities and problem solving in stoichiometry, Int. J. Phys. Chem. Educ., 5(2), 144–163.
- Gulacar O., Eilks I. and Bowman C. R., (2014), Differences in general cognitive abilities and domain-specific skills of higher-and lower-achieving students in stoichiometry, J. Chem. Educ., 91(7), 961–968.
- Guo S. and Fraser M. W., (2014), Propensity score analysis: Statistical methods and applications, SAGE publications, vol. 11.
- Harris R. B., Mack M. R., Bryant J., Theobald E. J. and Freeman S., (2020), Reducing achievement gaps in undergraduate general chemistry could lift underrepresented students into a “hyperpersistent zone”, Sci. Adv., 6(24), eaaz5687 DOI:10.1126/sciadv.aaz5687.
- Ho A. D., Reich J., Nesterko S., Seaton D. T., Mullaney T., Waldo J. and Chuang I., (2014), HarvardX and MITx: the first year of open online courses, HarvardX and MITx Working Paper No. 1, Cambridge, MA, USA: Harvard University.
- Hone K. S. and El Said G. R., (2016), Exploring the factors affecting MOOC retention: a survey study, Comput. Educ., 98, 157–168.
- Hussein F. and Reid N., (2009), Working memory and difficulties in school chemistry, Res. Sci. Technol. Educ., 27(2), 161–185 DOI:10.1080/02635140902853632.
- Jamil N., Belkacem A. N. and Lakas A., (2022), On enhancing students’ cognitive abilities in online learning using brain activity and eye movements, Educ. Inf. Technol., 1–35.
- Kanuka H., (2001), University student perceptions of the use of the Web in distance-related programs, Canadian J. Higher Educ., 31(3), 49.
- Kimberlin S. and Yezierski E., (2016), Effectiveness of inquiry-based lessons using particulate level models to develop high school students’ understanding of conceptual stoichiometry, J. Chem. Educ., 93(6), 1002–1009 DOI:10.1021/acs.jchemed.5b01010.
- le Maire N. V., Verpoorten D. P., Fauconnier M.-L. S. and Colaux-Castillo C. G., (2018), Clash of chemists: a gamified blog to master the concept of limiting reagent stoichiometry, J. Chem. Educ., 95(3), 410–415 DOI:10.1021/acs.jchemed.7b00256.
- Li C., Herbert N., Yeom S. and Montgomery J., (2022), Retention Factors in STEM Education Identified Using Learning Analytics: A Systematic Review, Educ. Sci., 12(11), 781.
- Lytle A. and Shin J. E., (2020), Incremental beliefs, STEM efficacy and STEM interest among first-year undergraduate students, J. Sci. Educ. Technol., 29, 272–281.
- Patel-Desai K. J., Student Conceptualization of Stoichiometry, Doctoral dissertation, Clemson University, 2020.
- Piaget J., (1972), Development and learning, Read. Child Behav. Dev., 38–46.
- Pienta N. J., (2003), A placement examination and mathematics tutorial for general chemistry, J. Chem. Educ., 80(11), 1244 DOI:10.1021/ed080p1244.
- Ralph V. R. and Lewis S. E., (2019), An explanative basis for the differential performance of students with low math aptitude in general chemistry, Chem. Educ. Res. Pract., 20(3), 570–593 10.1039/C9RP00068B.
- Raviolo A., Farré A. S. and Schroh N. T., (2021), Students’ understanding of molar concentration, Chem. Educ. Res. Pract., 22(2), 486–497 10.1039/D0RP00344A.
- Rosenbaum P. R. and Rubin D. B., (1983), The central role of the propensity score in observational studies for causal effects, Biometrika, 70(1), 41–55.
- Ross B., Chase A. M., Robbie D., Oates G. and Absalom Y., (2018), Adaptive quizzes to increase motivation, engagement and learning outcomes in a first year accounting unit, Int. J. Educ. Technol. Higher Educ., 15(1), 1–14.
- Rubin D. B., (1979), Using multivariate matched sampling and regression adjustment to control bias in observational studies, J. Am. Stat. Assoc., 74(366a), 318–328.
- Schmidt-Rohr K., (2020), Analysis of two definitions of the mole that are in simultaneous use, and their surprising consequences, J. Chem. Educ., 97(3), 597–602 DOI:10.1021/acs.jchemed.9b00467.
- Sharif A. and Gisbert M., (2015), The Impact of Culture on Instructional Design and Quality, Int. J. Instruct., 8(1), 143–156.
- Serna-Gallén P., Fortuño-Morte M., Beltrán-Mir H. and Cordoncillo E., (2022), “MasterChemist”: a novel strategy for reviewing stoichiometry and introducing molecular gastronomy to chemistry students, J. Chem. Educ., 99(10), 3443–3451 DOI:10.1021/acs.jchemed.2c00250.
- Sostarecz M. C. and Sostarecz A. G., (2012), A conceptual approach to limiting-reagent problems, J. Chem. Educ., 89(9), 1148–1151 DOI:10.1021/ed200420h.
- Stott A. E., (2021), South African physical sciences teachers’ use of formulae and proportion when answering reaction-based stoichiometry calculation questions, Chem. Educ. Res. Pract., 22(2), 443–456 10.1039/D0RP00291G.
- Valojerdi A. E. and Janani L., (2018), A brief guide to propensity score analysis, Med. J. Islamic Republic Iran, 32, 122.
- Vygotsky L. S. and Cole M., (1978), Mind in society: Development of higher psychological processes, Harvard university press.
- Wengrowicz N., Lavi R., Kohen H. and Dori D., (2022), Modeling with Real-Time Informative Feedback: Implementing and Evaluating a New Massive Open Online Course Component, J. Sci. Educ. Technol., 1–14.
- Wingate U., (2007), A Framework for Transition: Supporting? Learning to Learn? in Higher Education, High. Educ. Quarterly, 61(3), 391–405 DOI:10.1111/j.1468-2273.2007.00361.x.
- Zhao Y., Bi H. and Zhang H., (2021), Effectiveness of remedial courses for underprepared college students: a systematic review and meta-analysis, J. College Read. Learn., 51(2), 197–218.
Footnote |
| † Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d3rp00179b |
| This journal is © The Royal Society of Chemistry 2024 |