Contribution of an instructional module incorporating PhET simulations to Rwandan students' knowledge of chemical reactions, acids, and bases through social interactions

Abstract

The current study ascertained the influence an instructional module had on enhancing students’ understanding of chemical reactions and acid–base topics. The sample size for this study consisted of 197 students, including 101 in an “experimental” group and 96 in a “control” group, selected from schools in two Districts (Rwamagana and Musanze) in Rwanda, Africa. The experimental and control groups received a pre-test and post-test to collect data. In addition, focus group discussions (FGDs) were conducted with students in the experimental group. Further, a test question analysis was used to evaluate the students’ content knowledge of chemical reactions and acids, bases, and pH. To analyze the research data, the Statistical Package for the Social Sciences (SPSS) software was used for quantitative analysis. The independent t-test results indicated no significant difference between the means of the control and experimental groups at the pre-test stage (df = 195, p = 0.380). At the post-test stage, a statistically significant increase was observed in the mean scores of the experimental group compared to the control group (df = 195, p < 0.001), showing that the intervention effectively improved student learning outcomes in chemistry education.

---

Introduction

Information and Communication Technology (ICT) tools play a crucial role in science education. They help learners visualize abstract concepts and clarify their understanding of the subject matter (Akramova and Rustamova, 2023). ICT tools are especially valuable when physical materials are limited or when dealing with hazardous chemical reactions (Nkemakolam et al., 2018). For students who lack laboratory experience, computer simulations and animations can provide a valuable substitute by representing chemical reactions at the particle level.

Computer-based simulations are a valuable teaching approach for demonstrating active, multifaceted learning principles that can be challenging to convey solely through words, equations, or traditional classroom lectures. They are particularly effective in explaining complex ideas in chemistry and enhancing students’ understanding of the subject (Khan, 2011; Mihindo et al., 2017). Studies have indicated that students taught with computer-based simulations achieve significantly higher levels of understanding of chemistry concepts compared to those taught through traditional methods (Tuysuz, 2010; Carpenter and Sullivan, 2017; Mihindo et al., 2017; Nkemakolam et al., 2018; Nsabayezu et al., 2023a). Simulations offer unique advantages in promoting active exploration, visualization of abstract concepts, and safe experimentation, complementing other teaching methods. Therefore, challenging chemistry concepts and processes can be better understood and addressed through students’ interactions with computer-based simulations. Computer-based simulations can significantly boost students’ content knowledge of chemistry (Khan and Chan, 2011). Studies have shown that students who use simulations in chemistry classes demonstrate increased performance and engagement compared to those who do not. For example, a study in Nigeria recommended using simulations for the topics involving hazardous experiments (Nkemakolam et al., 2018). Another study conducted in Tanzania indicated an increased performance for students who used simulations compared to students who did not (Beichumila et al., 2022). Carpenter and Sullivan (2017) also argued that simulations increase students’ engagement and enjoyment during the teaching and learning of chemistry. A study on computer pH simulation (Watson et al., 2020) indicates that students who completed the collaborative computer simulation modules demonstrated higher pH knowledge, confidence, and content knowledge compared to those in traditional groups. The study found that collaborative computer simulation group members experienced higher mean scores and increased pH-related confidence throughout the semester (Watson et al., 2020). In 2016, the Rwandan education system transitioned from a Knowledge-Based Curriculum (KBC) to a Competence-Based Curriculum (CBC) to foster 21st-century competencies among learners (Ndihokubwayo et al., 2019; Nsengimana et al., 2021). Within this context, the Education Sector Strategic Plan (ESSP) highlights the integration of ICT in teaching and learning to increase access to quality education, improve its quality, and strengthen its relevance to the labor market (Ministry of Education [MINEDUC], 2016). To achieve these goals, the Government of Rwanda (GoR) invested in equipping schools with smart classrooms and providing teachers with computers and continuous professional development (CPD).

Chemistry, is compulsory subject in Rwanda due to its importance for sustainable economic development (REB, 2015). However, some chemistry topics are challenging for learners due to their abstract nature and traditional teaching methods (Saricayir, 2010; Supasorn and Waengchin, 2014; Achor and Ukwuru, 2017). Teachers who have not received professional development on active learning often rely on didactic approaches involving lectures (talk and chalk) that can hinder student learning (Hassan et al., 2015; Iyamuremye et al., 2022; Iyamuremye et al., 2023a; 2023b). Additionally, those teachers who have not received professional development do not provide students opportunities to exploit the ICT tools though they are available in schools (Nkundabakura et al., 2023; Nyirahabimana et al., 2023). Integration of ICT tools, particularly computer based simulations, can enhance chemistry education and prepare students for future challenges. Studies in Rwanda have demonstrated the effectiveness of PhET simulations in teaching physics and chemistry concepts, as they offer interactive features, rapid feedback, and dynamic access to multiple representations (Ndihokubwayo et al., 2020; Nsabayezu et al., 2023a).

While PhET simulations offer a promising approach to teaching chemistry (Nsabayezu et al., 2023b), equitable access to technology devices remains challenging. In schools with limited resources, teachers can bridge the gap between physical experimentation and content knowledge by offering inquiry-based assignments and guided exploration activities that scaffold students’ interaction with the simulations (Nkundabakura et al., 2024).

Theoretical framework of the study

This study was guided by the social constructivism theory which emphasizes the role of social interaction in learning (Vygotsky and Cole, 1978). The theory asserts that students actively construct their knowledge by interacting with each other. Vygotsky believed learning occurs more in social and cultural contexts than within individuals (Schreiber and Valle, 2013). Effective teaching and learning depend greatly on interpersonal interaction and conversation, with the main goal being that the students grasp the topic (Schreiber and Valle, 2013).

Vygotsky's concept of the Zone of Proximal Development (ZPD), highlights the importance of the teacher guidance in helping students grasp new knowledge and skills. It outlines the tasks that students may complete independently and those that require instructor assistance. According to the ZPD, learners can comprehend and master knowledge and abilities that they could not achieve independently with a teacher's aid (Schreiber and Valle, 2013). The theory proposes that learners can advance their comprehension with the help of teachers or peers who possess more information. Physical proximity as a fundamental principle within social constructivism, facilitates interactions among learners. This study adopted a social constructivist framework with a specific focus on the role of physical proximity in learning. In this context, we examined how physical proximity fosters collaborative learning and knowledge construction. Our research clarifies how the physical environment affects learning processes within a social constructivist framework, ultimately enhancing the learning experience.

Objectives of the study

(a) To investigate how an instructional module incorporating PhET simulations contributed to the students’ content knowledge of chemical equations and reactions, acid–base and pH topics.

(b) To explore how students, perceive the use of PhET simulations in learning topics in chemistry.

Research questions

(a) How does an instructional module, incorporating PhET simulations, contributed to students’ content knowledge of chemical reaction, chemical equation; acids, bases and pH topics?

(b) What are students’ perceptions about using PhET simulations?

Methodology

Research approach

A mixed research approach was used to collect and analyze data. A chemistry achievement test was used to assess chemistry content knowledge of facts associated with chemical equations, chemical reactions, acids, bases and pH. Qualitative data was collected during focus group discussions to explore students’ perception about the use of PhET simulations.

Research design

The research used an explanatory sequential design (Creswell et al., 2003). According to this design, the qualitative data are collected and analyzed after the quantitative data to help explain or supplement the quantitative results (Ditsa, 2004; Ivankova et al., 2006). Before the intervention of the instructional module, quantitative data were collected through a pre-test for both control and experimental groups. Qualitative data were collected through focus group discussion to explore students’ perceptions about using PhET simulations. This mixed-methods approach allowed for a more complete analysis of the data (Toyon, 2021).

Description of the intervention

The intervention of the study involves using the instructional module incorporating PhET simulations into a chemistry course in lower secondary schools in Rwanda. These simulations presented representations of fundamental topics in senior one (Grade 7) for chemical reactions, balancing chemical equations, properties of acids and bases, and pH of solutions. Given their foundational nature and the logical sequence in which they build upon each other, these core chemistry concepts were specifically chosen for this study. Notably, despite the centrality to the field of these topics, many senior students struggle to grasp the content knowledge and concepts.

Before the intervention, two groups of students were used in this study: control and experimental groups. The experimental group had the advantage of using PhET simulations, which contained computer animations representing reactions at the particle level and simulations allowing for the manipulation of variables and interacting with their peers and their teacher. The control group received didactic instruction through lectures and discussions that primarily relied on chalkboards and other passive learning methods. Teachers in the control group prepared their lessons using textbooks provided by the Rwandan Basic Education Board. Students listened to the teacher's explanations of topics as there was no laboratory equipment to conduct hands-on activities.

Before the intervention, the teachers in the experimental group underwent training on using PhET simulations in the teaching and learning process. Teachers were exposed to different chemistry PhET simulations by accessing them through the link: https://phet.colorado.edu/. The teachers were further trained on how to download and use the simulations offline. Teachers conducted micro-teaching in small groups during training to become familiar with teaching and learning using simulations. Additionally, the teachers were trained on effectively integrating the simulations into their lessons. This training aimed to equip them with the skills necessary to create powerful and engaging learning experiences for their students. This involved learning how to (1) identify content knowledge, key concepts and skills students should gain from the instructional activity, (2) prepare a short introductory activity to prepare students and focus their exploration, (3) develop reflection questions to guide student interaction with the simulation in a way that addresses the learning objectives, and (4) facilitate a post-activity discussion that allows students to solidify their understanding and connect it to broader concepts.

The training including discussions about addressing known student misconceptions certain to arise during student exploration. Students had regular opportunities to interact with the PhET simulations throughout the intervention period. The intervention covered two Senior One units: chemical reactions and acids, bases, and pH. PhET simulations were utilized in the chemical reactions unit (reactants, products, leftovers, and balancing chemical equations), while two simulations were also employed in the bases and acids unit (acids–base solutions, pH-scale basics).

The lesson implementation spanned a period of six weeks. Since the lessons took place in smart classrooms, students were provided instructions and guided through the use of PhET simulations on their computers by the chemistry teacher. The smart classroom was rearranged to place tables and chairs in groups, with computers installed with PhET simulations, allowing students to sit next to each other and facilitate active learning and social interactions. This change in room design fostered a cooperative learning environment where students could share insights, troubleshoot challenges together, and collaboratively construct their understanding of chemistry concepts. Students were also encouraged to ask questions to their peers or the teacher, fostering student–student and student–teacher interactions. The researcher conducted observations during the teaching of these two Senior One units to gain a better understanding of the teaching practices in the two groups.

Population and sampling techniques

The population of this study consisted of Senior one students from two Provinces in Rwanda: the Northern and Eastern Provinces. Based on the study's design (quasi-experimental study), these two Provinces were selected randomly and included in this study. One District was randomly selected in each Province and in each District, two schools were purposively selected. Schools were chosen based on the condition that they were equipped with computer rooms and internet connectivity. One school from each district was randomly assigned as a control group and the other as an experimental group. Thus, the sample consisted of 197 Senior One students who were purposefully selected from four secondary schools in two Districts (Musanze and Rwamagana). In each school, an intact class was used as the study sample. The intact class was used because the research could not neglect any individual data and did not require changing the classroom setting. Senior One students were chosen because they are at the beginning of learning basic concepts in chemistry and are likely to struggle with writing chemical reactions (Yam et al., 2020; Mammino, 2023).

Research instruments

This study employed three research instruments. A Chemistry achievement test, focus group discussion, and classroom observation.

(a) Chemistry achievement test. The researchers designed a chemistry achievement test to evaluate students’ chemistry content knowledge taught during the lessons. The test was prepared based on the available literature and questions from the standard student textbook provided by the Rwanda Basic Education Board (REB), which are commonly used in their regular classes. The initial test comprised 40 questions. After a filtering process conducted by subject matter experts, the final test consisted of 30 questions. The experts removed certain questions based on their difficulty level and similarity index. Ultimately, the test was composed of 30 questions related to chemical reactions, acids, bases, and pH.

(b) Focus group discussion. A focus group discussion was used to collect data on students’ perceptions and experiences using PhET simulations while learning chemistry. The focus group discussion was conducted with six students from the experimental group to explore their experiences with using PhET simulations, their preferences for different simulation activities, and any challenges or barriers they may have encountered.

(c) Classroom observation. A classroom observation checklist was adapted from the Reformed Teaching Observation Protocol (RTOP) to explore the classroom environment and how students interact with ICT tools. The list comprised 13 items with “Yes” and “No” responses. These statements were classified into five categories: students’ engagement, teacher facilitation, technological accessibility, learning outcomes, and classroom environment (Appendix A). The researcher observed a teacher four times while teaching students in the experimental and control groups. Thus, two teachers from each group were observed, making a total of four observed teachers.

Validity and reliability

Prior to data collection, the research team sought to validate the data collection instruments. They consulted two subject matter experts (senior lecturers) from the University of Rwanda, College of Education, to review the focus group discussion (FGD) interview guide. The experts provided helpful feedback and recommendations to ensure the guide accurately measured what it was intended to measure. Based on the experts’ input, the research team made revisions, eliminating 10 questions to arrive at the final validated FGD guide. Similarly, the classroom observation checklist was reviewed, and grammatical errors were corrected.

In addition, the researchers developed a chemistry achievement test after carefully reviewing the curriculum and content taught in Senior One. To establish the test's reliability, they calculated the internal consistency using Cronbach's Alpha. The obtained value of 0.79 indicated a high level of internal consistency, suggesting the test would consistently measure the underlying chemistry knowledge. The final chemistry performance test consisted of 30 multiple-choice questions.

Ethical consideration

Before data collection, the authors sought approval from the University of Rwanda College of Education to conduct the study. At school, before data collection, participants were informed about the study's purpose. All participants were given a consent form to sign and participated voluntarily in the study. Confidentiality and privacy standards were upheld throughout the process of data collection.

Data analysis procedure

This section describes the methods used to analyze the collected data. Firstly, the quantitative data collected from the achievement test were filtered to include only the students who sat for both the pre-test and post-test. Therefore, the researchers analyzed the data for students who took both tests to avoid bias and control for other mediating variables. After filtering the data, MS Excel 2016 was used to mark each student's answers. The “IF EXACT” function was used to mark students’ responses, with the correct answer marked as “1” and the wrong answer as “0.” After marking, descriptive statistics (mean, percentage, and standard deviation) were analyzed, and inferential statistics (independent t-test) were used to evaluate whether there was a statistically significant difference in the mean scores between the control and experimental groups before and after the intervention.

On the other hand, the recorded focus group discussions were transcribed verbatim to ensure accuracy and facilitate analysis. Next, thematic and interpretive analysis was employed to identify recurring themes and patterns within the transcripts regarding students’ perceptions of using PhET simulations in chemistry lessons. This approach helped the researchers gain insights into the subjective experiences expressed by the participants. Data from classroom observations were analyzed by generating percentages showing the level at which each activity occurred. The following steps were followed to calculate the percentage of positive responses on the observation checklist: 1. Count all the items on the checklist. 2. Count the number of “Yes” responses. 3. Divide the number of “Yes” responses by the total number of items. 4. Multiply the result by 100% to express it as a percentage.

Results presentation

(i) The contribution of PhET simulations on students’ content knowledge of chemistry

For our data analysis, we used SPSS version 23. We began by entering the students’ scores for all 30 questions into the SPSS data view and assigning appropriate variable names in the variable view. The first column was labeled “students’ group,” with “1” representing the control group and “2” representing the experimental group. The subsequent columns contained the pre-test and post-test scores for both groups.

To determine the appropriate statistical tests, we first assessed the normality of the score distribution. This involved plotting histograms for the pre-test and post-test scores and overlaying normal curves to visualize the distribution (see Fig. 1).

Fig. 1 Score distribution at pre-test for both control and experimental group.

To determine the appropriate statistical tests, the normality of the score distribution was assessed by plotting histograms for both pre-test and post-test scores. These histograms were overlaid with normal curves to visualize the distributions (see Fig. 1).

Additionally, skewness and kurtosis were computed to further examine the normality of the pre-test scores. Skewness, which measures the symmetry of the distribution, showed values of 0.215 for the control group and 0.106 for the experimental group. Both values fell within the acceptable range (−2 to 2), indicating that the distributions were fairly symmetrical. Kurtosis, which assesses the distribution's peakedness or flatness, yielded values of −0.283 for the control group and −0.025 for the experimental group, also within acceptable bounds (−7 to 7, as per Hair et al., 2010). These findings suggested that the data followed a normal distribution, thus allowing the use of parametric statistical tests for further analysis.

An independent samples t-test was conducted to determine whether there was a statistically significant difference between the pre-test and post-test scores of the control and experimental groups. This test is appropriate for comparing the means of two independent groups to assess if there is a significant difference between them. By using this approach, the impact of the intervention could be evaluated. The mean scores for both groups were computed for the pre-test and post-test stages, and the results are summarized in Table 1.

Table 1 Mean and standard deviation at pre- and post-test for control and experimental

	Group	N	Mean	Std. deviation	Std. error mean	t	df	Sign.
Pre-test	Control	96	27.50	8.421	0.859	−0.879	195	0.380
	Experimental	101	28.52	7.840	0.780
Post-test	Control	96	33.58	9.890	1.009	−14.245	195	0.000
	Experimental	101	56.14	12.158	1.210

---

The independent samples t-test showed no significant difference between the pre-test means of the control group (M = 27.50, SD = 8.421) and the experimental group (M = 28.52, SD = 7.840), t(195) = −0.879, p = 0.380. However, at the post-test stage, a significant difference was observed between the groups. The control group had a mean score of 33.58 (SD = 9.890), while the experimental group scored significantly higher with a mean of 56.14 (SD = 12.158), t(195) = −14.245, p < 0.001. This indicates that the intervention had a significant impact on the performance of the experimental group. To quantify the magnitude of this difference, Cohen's d was calculated, yielding an effect size of 2.04. According to Magnusson (2021), effect sizes are generally categorized as small (0.2), medium (0.5), and large (0.8), with the current value indicating a very large effect in favor of the experimental group (see also Fig. 2).

Fig. 2 Estimated marginal means of control and experimental groups at pre- and post-test.

The results demonstrated a significant improvement in the post-test scores of the experimental group compared to the control group, as illustrated in Fig. 2.

While both groups started at similar levels in the pre-test, the experimental group made substantially greater gains following the intervention. This result underscores the effectiveness of the interventions’ instructional module.

(ii) Students’ perceptions on the use of PhET simulations in learning chemistry. The qualitative results of the study on the effect of PhET simulations to enhance chemistry education in lower secondary schools in Rwanda through physical proximity to learning revealed students’ insights and experiences with this approach in the classroom. Through focus group discussions, students expressed their views and opinions regarding using PhET simulations in their educational environment. The results were classified into two themes: (1) the perceived benefits of PhET simulations in facilitating students’ content knowledge and (2) the perceived challenges associated with using PhET simulations in teaching and learning chemistry.

Theme 1: perceived benefits of PhET simulations in facilitating deeper content knowledge development. Under this theme, most students reported that using PhET simulations in teaching and learning was engaging and motivating. Students mentioned that the PhET simulations provided them with a visual and interactive way of learning chemical reactions, such as acid-based and pH. They were enthusiastic about how the simulations made difficult and abstract ideas more concrete, facilitating their comprehension and increasing their interest in learning. Students also appreciated that they could perform virtual experiments in a safe and controlled environment without the risks and hazards that might come with an actual physical experiment. For instance, one student said, “PhET simulations improved our understanding as we were able to explore visualized balanced reaction”. Students also mentioned that using PhET simulations helped them develop their ICT skills by allowing them to explore simulations using computers. A student pointed out, “Exploring simulation through the use of the computer helped me to explore more other simulations and other content available on the internet” Another student complimented, “While initially logging into the PhET website and manipulating the simulations presented a bit of a learning curve due to my limited computer skills, with repeated use, I became comfortable navigating the platform and effectively using the simulations.”

Theme 2: perceived challenges of using PhET simulation in teaching and learning chemistry. Students noted some technical challenges when using the simulations in their learning. The issues mentioned were technical difficulties such as a lack of internet facilities and insufficient computer skills among teachers and students. Some students suggested that additional training on the use of the simulations could help them navigate the simulations more effectively.

Regarding physical proximity, the researcher observed student interactions at 78% with one another, 80% with teachers, and 89% with computers. During the lessons, teaching and learning process, teachers intervened to clarify chemistry content knowledge and concepts for students who struggled with manipulating the simulations, at a rate of 75%. While students sometimes found computers tricky, they were able to use them independently most of the time. Student's computer skills played a role in how well they acquired and understood the chemistry content and concepts taught through the simulations.

Discussion

This section discusses the quantitative and qualitative results obtained from this study. The analysis of students’ content knowledge indicated a significant improvement in performance of the treatment group. Specifically, the treatment instructional module seemed to foster students’ understanding of basic facts involving chemical equations, as well as acids, bases and pH. Students were able to learn from their mistakes and solidify their understanding of chemistry facts through a combination of student-to-student and teacher-to-student interaction (facilitated by the close physical proximity of the students), the rapid feedback and active learning provided by the simulations, and the effectiveness of using an active learning pedagogy.

Catlow et al. (2013) argued that simulations assist students in understanding abstract concepts and honing their critical thinking abilities by providing an interactive and visual learning environment. Previous studies and our findings add to our understanding of the benefits of integrating computer simulations and animations of reactions at the particle level into an instructional module. It takes a combination of strategies and techniques to reinforce fundamental knowledge and foster critical thinking, problem-solving, and creativity among students (Ukobizaba and Nizeyimana, 2021).

In terms of students' chemistry content understanding, the study indicated a positive impact of using an instructional module integrating PhET simulations on senior secondary school student’ performance in chemical reactions and acid, base, and pH. While both groups showed improvements in their mean scores from pre-test to post-test, the experimental group exhibited a higher increase, indicating the efficacy of the instructional module on students’ understanding of content associated with chemical reactions, chemical equations, acids, bases, and pH.

These findings could be attributed to the role that social interactions – student-to-student and student-to-teacher played as a result of proximity of learning. The experimental design of the study did not allow for the analysis that the PhET simulations added to the instructional module. We can speculate that PhET simulations provide a dynamic access to representations of chemical reactions, scaffold the inquiry process, and allow for multiple trials and rapid feedback cycle while at the same time being engaging and fun for both students and teachers (Saudelli et al., 2021). Similarly, the Mahtari et al. (2020)'s findings showed that interactive simulations serve as an effective implicit scaffolding method during experimentation, promoting guided-inquiry learning without overwhelming students.

The use of social interactions, facilitated by proximities of learning, an active learning pedagogy, and the incorporation of PhET simulations contributes to the explanation of how students learn complex scientific content knowledge. All of the components of the instructional module helped to facilitate social interactions and collaborative learning among students. Our findings are consistent with previous findings which indicate in the presence of suitable physical facilities, an effective instructional module can help teachers and students overcome a variety of obstacles (Jabeen and Afzal, 2020). This demonstrates that physical proximities enhanced social interaction while using computer simulations and this improved students’ learning outcomes in chemistry education (Oladejo, 2021).

Conclusion

The purpose of this study was to determine the effectiveness of an instructional module in helping to improve senior one students’ understanding of chemistry facts through social constructivism and physical proximities of learning. The study's findings provided some evidence of the usefulness of the instructional module in improving students' understanding of facts about chemical equations, chemical reactions; acids, bases, and pH. The PhET simulations seemed to enhance students’ enjoyment in learning chemistry. The study supports the incorporation of the instructional module into chemistry classes to improve the understanding of the targeted subject matter. This study recommends implementing professional development for teachers, ongoing monitoring, and adaptation of active learning pedagogy. Future research could delve deeper into the comparative effectiveness of PhET simulations and other active learning approaches. Such exploration would provide a more nuanced understanding of the factors contributing to enhanced student outcomes.

Ethical declaration

All participants signed consent forms to participate as well as consent forms to publish data.

Data availability

The datasets generated and/or analyzed during the current work are available on this link https://osf.io/ma8rj.

Conflicts of interest

The authors confirm that they have no potential conflicts of interest to disclose.

Appendix A. Classroom observation checklist

School name: ……………………………………………………………

Subject: …………………………………………………………………

Topic area: ……………………………………………………………

Lesson: ………………………………………………………………

Teacher's name: ………………………………………………………

Observer's name: …………………………………………………….

Date: …………………………………………………………………

Instructions: Circle on “Yes” or “No” option depending on your observation for each item during your classroom observation.

(A) Student engagement

(1) Are students actively interacting with the PhET simulations? (Yes/No)

(2) Do students demonstrate curiosity and enthusiasm while exploring the simulations? (Yes/No)

(3) Are students engaging in collaborative discussions with their peers to explore concepts and problem-solving strategies using the simulations? (Yes/No)

(B) Teacher facilitation

(4) Does the teacher provide clear instructions and guidance on how to use the PhET simulations effectively? (Yes/No)

(5) Does the teacher actively participate in discussions and exploration of concepts alongside students? (Yes/No)

(6) Does the teacher provide timely feedback and assistance to students when they encounter difficulties with the simulations? (Yes/No)

(7) Does the teacher encourage critical thinking and inquiry-based exploration during the PhET simulation activities? (Yes/No)

(C) Technological accessibility

(8) Are all students able to access the necessary technology devices (e.g., computers) to use the PhET simulations? (Yes/No)

(9) Are there any technical issues or barriers hindering students’ access to and use of the PhET simulations? (Yes/No)

(D) Learning outcomes

(10) Do students demonstrate improved understanding of chemistry concepts after engaging with the PhET simulations? (Yes/No)

(11) Are there observable changes in students’ attitudes or perceptions towards learning chemistry as a result of using PhET simulations? (Yes/No)

(E) Classroom environment

(12) Is the physical space conducive to PhET simulation activities, with appropriate seating arrangements and access to technology devices? (Yes/No)

(13) Is there a positive and collaborative atmosphere in the classroom during the PhET simulation activities? (Yes/No)

End of observation

Appendix B. Chemistry performance test

Dear Participant,

Thank you for your willingness to participate in this study. We are conducting research on the use of PhET simulations to enhance the teaching and learning of Chemical equations, acids, bases, and pH concepts at lower secondary schools, in Rwanda. This questionnaire consists of 30 multiple choice questions. It is expected to be accomplished within 60 minutes. You are kindly requested to answer all items in this questionnaire even if you are not sure of the answer.

Section A: Demographic information

1. Name of School: ……………

2. Student Name: …………

3. Sex: Male □ Female □

4. Age: ………

Section B: Multiple choice questions Circle the correct answer:

1. The chemicals that are created in a chemical reaction are:

a. Products

b. Reactants

c. Leftovers

d. None of these

2. The chemicals that are involved in a chemical reaction are:

a. Products

b. Leftovers

c. Reactants

d. None of these

3. The term coefficient in relation to a chemical equation is defined as:

a. The number written as subscripts to balance chemical equation

b. The number placed in front of chemical symbol or formulae to balance chemical equation

c. The number placed behind chemical symbol or formulae to balance chemical equation

d. None of the above

4. How many oxygens are in 2(NO₃)?

a. 3

b. 1

c. 6

d. 2

5. What are the coefficients needed to balance the chemical equation below?

Al + FeO → Fe + Al₂O₃

a. 2, 3, 3, 1

b. 3, 3, 3, 1

c. 2, 1, 1, 1

d. 1, 2, 1, 2

6. What coefficients would balance the equation below? NH₃ + NO → N₂ + H₂O

a. 3, 2, 2, 2

b. 4, 2, 5, 6

c. 4, 6, 5, 6

d. 4, 2, 2, 5

7. Which of the following reactions, represents the correct decomposition reaction of water

a. H₂O(l) → 2H₂(g) + O₂(g)

b. H₂O → H₂ + O₂

c. H₂O → H₂ + 2O₂

d. H₂O → 2H + O

8. The correct word equation of the following chemical equation:

Ca(s) + H₂O(l) → Ca(OH)₂(aq) is:

a. Solid sodium metal is added to water at room temperature and forms calcium hydroxide (dissolved in water) and hydrogen gas.

b. Solid calcium metal is added to water at room temperature and forms calcium hydroxide (solid) and hydrogen gas.

c. Solid calcium metal is added to water at room temperature and forms calcium hydroxide (dissolved in water) and hydrogen gas.

d. None of the above

9. The formula equation for the following word equation:

Iron + Chlorine → Iron(III) Chloride is:

a. 2Fe + 3Cl2 → 2FeCl3

b. 2Fe + 3Cl2 → FeCl3

c. 2Fe + Cl2 → 2FeCl3

d. 2Fe + 3Cl2 → 2FeI3

10. Which is correct among the following that can be used to balance this equation:

Na₂CO₃ + HCl → NaCl + CO₂ + H₂O

a. 1 2 1 1 1

b. 2 1 1 1 2

c. 1 2 2 1 1

d. 2 1 2 1 1

11. Which one of the following is acidic?

a. Lemon juice

b. Hand soap

c. Milk

d. All

12. Which one of the following will turn red litmus blue?

a. Vinegar

b. Baking soda solution

c. Lemon juice

d. Soft drinks

13. Which one of the following will turn blue litmus red?

a. Vinegar

b. Lime water

c. Baking soda solution

d. Washing soda solution

14. What is the pH range of our body?

(a) 7.0–7.8

(b) 7.2–8.0

(c) 7.0–8.4

(d) 7.2–8.4

15. Rain is called acid rain when its:

(a) pH falls below 7

(b) pH falls below 6

(c) pH falls below 5.6

(d) pH is above 7

16. An aqueous solution turns red litmus solution blue. Excess addition of which of the following solution would reverse the change?

(a) Baking powder

(b) Lime

(c) Ammonium hydroxide solution

(d) Hydrochloric acid

17. A drop of solution X is placed on a strip of a pH paper. A deep red colour is produced. The solution X is:

(a) NaOH

(b) HCl

(c) Water

(d) Sodium bicarbonate

18. A few drops of a liquid X were added to distilled water. It was observed that the pH of the water decreased. The liquid X is:

(a) Lemon juice

(b) Sugar solution

(c) Common salt solution

(d) Baking soda solution

19. A liquid sample turned red litmus paper blue. This indicates that the liquid sample is of

(a) An alcohol

(b) Distilled water

(c) Sodium hydroxide solution

(d) Hydrochloric acid

20. A solution with pH = 0 is

a. Acidic

b. Basic

c. Amphoteric

d. neutral

21. A student tested the pH of distilled water using paper and observed green colour. After adding a few drops of dilute NaOH solution, the pH was tested again. The colour change now observed would be:

a. Blue

b. Green

c. Red

d. orange

22. As the pH increases

a. Acidic strength increases

b. Basic strength increases

c. Basic strength decreases

d. None of these

23. If the pH of a solution is 7.5, its nature is

a. Neutral

b. Strongly acidic

c. Weakly acidic

d. Weakly basic

24. pH of a neutral solution

a. is more than 7

b. is less than 7

c. is 7

d. none of these

25. pH value of lime water is

a. 7

b. More than 7

c. Less than 7

d. zero

26. The two colours seen at the extreme end of the pH chart are

a. Red, blue

b. Red, green,

c. Green, blue

d. Orange, green

27. The correct method of finding the pH of a solution is to

a. heat the solution in a test tube and expose the pH paper to the vapours formed

b. pour a few drops of the solution from the test tube on the pH paper

c. drop the pH paper in the solution

d. put a drop of the solution on the pH paper using a dropper

28. The pH of a NaOH solution is 10. If water is added to it, its pH will

a. Remains same

b. Increases

c. Decreases

d. Becomes 7

29. The pH range of a universal indicator solution is

a. 3 to 14

b. 0 to 14

c. 11 to 14

d. 0 to 11

30. Which one of the following is not required to find the pH of a solution?

a. pH paper

b. litmus paper

c. universal indicator

d. Standard pH value chart

Appendix C. Focus group interview discussion protocoal

Dear Participant,

We appreciate your willingness to take part in this research. In lower secondary schools in Rwanda, we are conducting research on the use of simulations to improve electrolysis instruction and learning. This focus group interview is intended to gather data on how students feel about the use of simulations to teach electrolysis principles. Please take the time to respond to every question on the questionnaire. It shouldn’t take you longer than 20 minutes to complete this questionnaire. Since the results will only be used for academic purposes, the information you supply in this questionnaire will be treated with confidentiality and anonymity.

1. Can you share your initial thoughts and feelings about using PhET simulations in your chemistry classes? How did you feel before using them, and has your perception changed since then?

2. How do you feel about the level of engagement PhET simulations bring to your learning compared to traditional methods? Do you find yourselves more interested or motivated to learn chemistry through these simulations?

3. In what ways do you think PhET simulations help you better understand chemistry concepts compared to traditional teaching methods? Can you provide examples of concepts that were clearer to you after using the simulations?

4. How would you describe the experience of interacting with PhET simulations during your chemistry lessons? Do you find it more enjoyable or interactive compared to traditional lectures or textbook learning?

5. Have you noticed any connections between the chemistry concepts you learn through PhET simulations and real-life applications or experiences? How do these simulations help you bridge the gap between theoretical knowledge and practical applications?

6. Are there any aspects of PhET simulations that you think could be improved to enhance your learning experience further? What additional features or functionalities would you like to see in these simulations to better support your chemistry education?

Acknowledgements

This research was financially supported by Masters Card Foundation through Leaders in Teaching (LIT) project in collaboration of university of Rwanda College of Education (UR-CE).

References

1. Achor E. E. and Ukwuru J. O., (2017), An Examination of the Facilitative Effect of the Computer Assisted Instruction (CAI) in Students’ Achievement in Chemical Reaction and Equilibrium, Education, 4(1), 7–11 DOI:10.2139/ssrn.2401088.
2. Akramova L. and Rustamova N., (2023), Computer-human interaction: Visualisation of the educational process as a means of increasing the efficiency of the education, AIP Conf. Proc., 2789(1), 050013.
3. Beichumila F., Bahati B. and Kafanabo E., (2022), Students’ Acquisition of Science Process Skills in Chemistry through Computer Simulations and Animations in Secondary Schools in Tanzania, Int. J. Learn. Teach. Educ. Res., 21(3), 166–195.
4. Carpenter Y. and Sullivan E. R., (2017), The Role of Interactive Digital Simulations in Student Conversations About Visualising Molecules, Proc. Univ. Calgary Conf. Learn. Teach., 2, 16–23.
5. Catlow C. R. A., Cox P. A., Jackson R. A., Parker S. C., Price G. D., Tomlinson S. M. and Vetrivel R., (2013), Industrial Applications of Simulation Studies in Solid State Chemistry, Mol. Simul., 3(1–3), 49–69 DOI:10.1080/08927028908034619.
6. Creswell J. W., Clark V. L. P., Gutmann M. L. and Hanson W. E., (2003), Advanced mixed, Handbook of Mixed Methods in Social & Behavioral Research, vol. 209.
7. Ditsa G., (2004), A Research Design and a Methodological Approach to an Explanatory User Behavior Testing: Lessons Learnt, pp. 762–768.
8. Hair J. F., Black W. C., Babin B. J. and Anderson R. E., (2010), Multivariate Data Analysis, 7th edn, New Jersey: Prentice Hall, Inc.
9. Hassan A. A., Ali H. I., Salum A. A., Kassim A. M., Elmoge Y. N., (2015), Factors affecting students’ performance in chemistry in Rwandan secondary schools, Int. J. Soc. Behav. Educ. Econ. Bus. Ind. Eng., 9(1), 166–171.
10. Ivankova N. V., Creswell J. W. and Stick S. L., (2006), Using Mixed-Methods Sequential Explanatory Design: From Theory to Practice, Field Methods, 18(1), 3–20 DOI:10.1177/1525822X05282260.
11. Iyamuremye A., Mbonyubwabo J. P., Mbonyiryivuze A., Hagenimana F., Butera M., Niyonderera P. and Ukobizaba F., (2023a), Enhancing Understanding of Challenging Chemistry and Physics Concepts in Secondary Schools of Kayonza District through Computer Simulation-Based Learning, J. Classroom Pract., 2(2), 1–28 DOI:10.58197/prbl/thrf5883.
12. Iyamuremye A., Mukiza J., Nsabayezu E., de Dieu Kwitonda J. and Habimana C., (2023b), Exploration of Students’ Social Presence in Web-Based Discussion for Conceptual Learning of Organic Chemistry, J. Sci. Educ. Technol., 32(1), 111–126 DOI:10.1007/s10956-022-09997-6.
13. Iyamuremye A., Mukiza J., Nsabayezu E., Ukobizaba F. and Ndihokubwayo K., (2022), Web-based discussions in teaching and learning: Secondary school teachers’ and students’ perception and potentiality to enhance students’ performance in organic chemistry, Educ. Inf. Technol., 27(2), 2695–2715 DOI:10.1007/s10639-021-10725-7.
14. Jabeen F. and Afzal M. T., (2020), Effect of Simulated Chemistry Practicals on Students’ Performance At Secondary School Level, J. Educ. Educ. Dev., 7(1), 119 DOI:10.22555/joeed.v7i1.2600.
15. Khan S., (2011), New pedagogies on teaching science with computer simulations, J. Sci. Educ. Technol., 20, 215–232.
16. Khan S. and Chan V., (2011), An exploration of digital representations in chemistry education, J. Res. Center Educ. Technol., 7(2), 2–37.
17. Magnusson K., (2021), Understanding statistical power and significance testing, https://rpsychologist.com/cohend/.
18. Mahtari S., Wati M., Hartini S., Misbah M. and Dewantara D., (2020), The effectiveness of the student worksheet with PhET simulation used scaffolding question prompt, J. Phys.: Conf. Ser., 1422(1), 012010 DOI:10.1088/1742-6596/1422/1/012010.
19. Mammino L., (2023), The spontaneity of chemical reactions: challenges with handling the concept and its implications, Phys. Sci. Rev., 8(11), 4099–4123.
20. Mihindo W. J., Wachanga S. W. and Anditi Z. O., (2017), Effects of Computer-Based Simulations Teaching Approach on Students’ Achievement in the Learning of Chemistry among Secondary School Students in Nakuru Sub County, Kenya. J. Educ. Pract., 8(5), 65–75.
21. MINEDUC, (2016), ICT in Education Policy, April, 1–17.
22. Ndihokubwayo K., Habiyaremye H. T. and Rukundo J. C., (2019), Rwandan New Competence Base: Curriculum Implementation and Issues; Sector-Based Trainers, J. Contemp. Res., 16(1), 24–41.
23. Ndihokubwayo K., Uwamahoro J. and Ndayambaje I., (2020), Effectiveness of PhET simulations and YouTube videos to improve the learning of optics in Rwandan secondary schools, Afr. J. Res. Math. Sci. Technol. Educ., 24(2), 253–265 DOI:10.1080/18117295.2020.1818042.
24. Nkemakolam O. E., Chinelo O. F. and Jane M. C., (2018), Effect of Computer Simulations on Secondary School Students’ Academic Achievement in Chemistry in Anambra State, Asian J. Educ. Train., 4(4), 284–289 DOI:10.20448/journal.522.2018.44.284.289.
25. Nkundabakura P., Nsengimana T., Nyirahabimana P., Nkurunziza J. B., Mukamwambali C., Dushimimana J. C., Uwamariya E., Batamuliza J., Byukusenge C., Nsabayezu E., Twahirwa J. N., Iyamuremye A., Mbonyiryivuze A., Ukobizaba F. and Ndihokubwayo K., (2023), Usage of modernised tools and innovative methods in teaching and learning mathematics and sciences: A case of 10 districts in Rwanda, Educ. Inf. Technol., 28(9), 11379–11400 DOI:10.1007/s10639-023-11666-z.
26. Nkundabakura P., Nsengimana T., Uwamariya E., Nyirahabimana P., Nkurunziza J. B., Mukamwambali C., Dushimimana J. C., Nsabayezu E., Twahirwa J. N. and Ndihokubwayo K., (2024), Contribution of Continuous Professional Development (CPD) Training Programme on Rwandan Secondary School Mathematics and Science Teachers’ Pedagogical, Technological, and Content knowledge, Educ. Inf. Technol., 29(4), 4969–4999.
27. Nsabayezu E., Iyamuremye A., Mukiza J., Mbonyiryivuze A., Gakuba E., Niyonzima F. N. and Nsengimana T., (2023a), Impact of computer-based simulations on students’ learning of organic chemistry in the selected secondary schools of Gicumbi District in Rwanda, Educ. Inf. Technol., 28(3), 3537–3555 DOI:10.1007/s10639-022-11344-6.
28. Nsabayezu E., Iyamuremye A. and Mbonyiryivuze A., (2023b), Digital – based formative assessment to support students’ learning of organic chemistry in selected secondary schools of Nyarugenge District in Rwanda, Education and Information Technologies (Issue 0123456789), Springer US DOI:10.1007/s10639-023-11599-7.
29. Nsengimana T., Mugabo L. R., Ozawa H. and Nkundabakura P., (2021), Science Competence-based Curriculum Implementation in Rwanda: A Multiple Case Study of the Relationship between a School's Profile of Implementation and its Capacity to Innovate, Afr. J. Res. Math. Sci. Technol. Educ., 25(1), 38–51 DOI:10.1080/18117295.2021.1888020.
30. Nyirahabimana P., Minani E., Nduwingoma M. and Kemeza I., (2023), Multimedia-aided technologies for effective learning of quantum physics at the university level, J. Sci. Educ. Technol., 32(5), 686–696.
31. Oladejo A. I., (2021), Teaching chemistry with computer simulation: Would senior crawford journal of multidisciplinary research (cjmr) 2021, Crawford J. Multidiscip. Res., 2(2), 16–32.
32. REB, (2015), Ordinary Level Chemistry Syllabus.
33. Saricayir H., (2010), Teaching electrolysis of water through drama, J. Balt. Sci. Educ., 9(3), 179–186.
34. Saudelli M. G., Kleiv R., Davies J., Jungmark M. and Mueller R., (2021), PhET simulations in undergraduate physics: Constructivist learning theory in practice, Brock Educ. J., 31(1).
35. Schreiber L. M. and Valle B. E., (2013), Social constructivist teaching strategies in the small group classroom, Small Group Res., 44(4), 395–411.
36. Supasorn S. and Waengchin S., (2014), Development of Grade 8 Students’ Learning Achievement on Chemical Reaction by Using Scientific Investigation Learning Activities, Procedia Soc. Behav. Sci., 116, 744–749 DOI:10.1016/j.sbspro.2014.01.291.
37. Toyon M. A. S., (2021), Explanatory sequential design of mixed methods research: Phases and challenges, Int. J. Res. Bus. Soc. Sci., 10(5), 253–260 DOI:10.20525/ijrbs.v10i5.1262.
38. Tuysuz C., (2010), The effect of the Virtual Laboratory on Students’ Achievement and Attitude in Chemistry, Int. Online J. Educ. Sci., 2(1), 37–53.
39. Ukobizaba F. and Nizeyimana G., (2021), The Use of the Revised Bloom’ s Taxonomy Levels in Enhancing Students’ Higher Order Thinking Skills in Mathematics, Am. J. Humanit. Soc. Sci. Res., 4(5), 22–28.
40. Vygotsky L. S. and Cole M., (1978), Mind in society: Development of higher psychological processes, Harvard University Press.
41. Watson S. W., Dubrovskiy A. V. and Peters M. L., (2020), Increasing chemistry students’ knowledge, confidence, and conceptual understanding of pH using a collaborative computer pH simulation, Chem. Educ. Res. Pract., 21(2), 373–381 10.1039/C9RP00235A.
42. Yam C. H. K., Griffiths S. M. and Yeoh E.-K., (2020), What helps and hinders doctors in engaging in continuous professional development? An explanatory sequential design, PLoS One, 15(8), e0237632.

---