Li
Ye
*a,
Jack F.
Eichler
*b,
Alex
Gilewski
ac,
Lance E.
Talbert
b,
Emily
Mallory
a,
Mikhail
Litvak
d,
Emily
M. Rigsby
b,
Grace
Henbest
e,
Kiana
Mortezaei
b and
Cybill
Guregyan
b
aDepartment of Chemistry and Biochemistry, California State University, Northridge, 18111 Nordhoff Street, Northridge, California 91330, USA. E-mail: li.ye@csun.edu
bDepartment of Chemistry, University of California, Riverside, 501 Big Springs Road, Riverside, California 92521, USA. E-mail: jack.eichler@ucr.edu
cDepartment of Chemistry, Glendale Community College, 1500 N Verdugo Rd, Glendale, California 91208, USA
dDepartment of Biology, California State University, Northridge, 18111 Nordhoff Street, Northridge, California 91330, USA
eGraduate School of Education, University of California, Riverside, 501 Big Springs Road, Riverside, California 92521, USA
First published on 14th May 2020
Science educators have developed a variety of assessment techniques to help students connect their scientific knowledge and bridge conceptual gaps. In chemistry, concept maps and creative exercises are the two notable assessments that have been implemented into multiple chemistry courses and indicated promising effects on students’ conceptual learning and connection-making between chemistry concepts. These two assessment techniques were usually implemented individually in research studies. Herein, we employed a quasi-experimental, mixed-methods approach to explore whether combining concept maps and creative exercises would reveal any synergistic effects for student learning of chemical equilibrium and acid–base chemistry in a college general chemistry course. In this study, student perceptions of the use of the two assessments were examined by open-ended surveys. Interestingly, students perceived creative exercises as an assessment technique while concept maps were viewed as a learning tool for studying or reviewing exams. Additionally, Students believed that concept maps assisted them in answering creative exercises, but not vice versa. The four study groups (control group, concept maps only, creative exercises only, and both concept maps and creative exercises) were compared through concept inventory pre and post-test questions. The results of an ANCOVA indicated that participation in the experimental groups did not significantly impact conceptual learning gains, as measured by the concept inventory post-test scores. However, focus group interviews indicated students from the experimental group that used both concept maps and creative exercises were able to provide more sophisticated scientific explanations for conceptual questions related to the topics of chemical equilibrium and acid–base chemistry. Implications of these research results, best practices for implementation of the two assessments, and future research are discussed.
In chemistry, CMs have been implemented as learning and assessment tools in several college chemistry courses, including general chemistry, organic chemistry, and analytical chemistry. A subset of studies has incorporated CMs in the general chemistry curriculum. For instance, Francisco and colleagues implemented concept maps in general chemistry as homework, pre-, and post-laboratory exercises, quizzes, and exams (Francisco et al., 2002). The students were given a list of terms and asked to construct individual maps during the above activities. The concept maps gave insight as to why many students were failing to correctly answer the algorithmic questions. That is, a lack of conceptual understanding led to poor algorithmic question performance. In another study, Burrows and Mooring conducted interviews including a concept map exercise of organic chemistry students who had recently completed one year of general chemistry (Burrows and Mooring, 2015). They graded the concept maps and found that students who scored in the highest tercile had a more thorough conceptual understanding of electronegativity in the context of chemical bonding than those who scored in the lowest tercile. A recent quasi-experimental study by Talbert and colleagues (Talbert et al., 2020) implemented concept maps in a large-enrollment general chemistry course. Students receiving the concept maps assessment built upon their maps every week throughout the course. This group of students performed significantly better on a chemical concept post-test than the corresponding control group (who completed weekly journal entries instead of concept maps) as demonstrated through t-tests, though the results were insignificant when covariates related to incoming academic preparation were included in the statistical analysis.
Moreover, concept maps have been used as a diagnostic assessment for students who completed general chemistry. Lopez and colleagues (Lopez et al., 2011) interviewed organic chemistry students four times and assigned them a concept map to complete in each interview in addition to textbook problems. Concept map scores were significant predictors of problem-solving and exam scores, indicating that concept mapping and conceptual organization improved the students’ problem-solving skills and academic performance in organic chemistry. Students in analytical chemistry have shown learning gains after using concept maps. Turan-Oluk and Ekmecki (Turan-Oluk and Ekmekci, 2018) implemented a concept inventory and four concept maps as pre- and post-tests for gravimetric analysis and demonstrated quantitative gains with large effect sizes. Qualitatively, they determined that students viewed the concept maps in a highly positive manner, as evidenced through Likert-type attitude scales and content analysis of open-ended survey responses. While no specific themes were presented, the authors note that students claimed that concept maps greatly helped their conceptual understanding of the material.
Another emerging technique for promoting connection-making and conceptual understanding of chemistry concepts in chemistry education literature is the Creative Exercise (CE). In CEs, a student is given a chemistry prompt that is designed based on newly learned concepts in the course and asked to write as many accurate, distinct, and relevant statements as they can, relating their previous chemistry knowledge to the newly learned concepts (Ye and Lewis, 2014). Creative exercises are open-ended activities that are designed to incorporate the gains a student would achieve through an activity, and are easier to objectively assess than traditional concept maps. Rubrics, comprised of a list of possible responses from instructors, can be generated beforehand. Additionally, students are explicitly informed as to the three criteria of accuracy, distinction from previously used concepts, and relevance to the prompt and concepts learned in the course. Student responses that meet the aforementioned criteria are added to the instructor-generated rubrics for grading. Recently, Gilewski and colleagues (Gilewski et al., 2019) have shown that creative exercises implemented in two similar introductory chemistry courses at two institutions increased student academic performance with small and medium effect sizes. CEs were given to the treatment groups one week before exams as a group activity and then individually on exams. Control groups completed close-ended (multiple-choice or short-answer) questions on exams written on the same topics as the CE prompt. Students appeared to make more connections between topics over time and had an overall positive appraisal of the assessment. Four themes emerged from open coding survey responses: conceptual understanding, effective study habits, knowledge integration, and flexibility. These positive themes synergize with the quantitative gains endemic to the treatment groups. Warfa and Odowa (Warfa and Odowa, 2015) implemented CEs three times in a process-oriented guided inquiry learning (POGIL) biochemistry course and demonstrated their utility in supporting foundational concepts in a biochemistry course. Students were able to use CEs to link concepts such as acid/base chemistry and chemical equilibrium to structural biochemistry. The authors noted the enzyme kinetics CE, in particular, was able to illuminate learning gaps regarding free energy diagrams and the connections of chemical equilibrium, kinetics, and energy.
Scaffolding is a process through which an educator provides support for building students’ experiences and knowledge while they are learning new skills, and this support can be removed once the students master the skills. The idea of scaffolding is closely related to the Zone of Proximal Development theory (ZPD). The ZPD originally pioneered by Vygotsky, describes three cognitive areas: one where students can perform tasks entirely on their own, another in which students cannot perform cognitive tasks even with assistance, and one area in between the two, where students can complete tasks with assistance (Fernández et al., 2001). It is this middle area that is of interest to educators and researchers, where the potential development of students’ capabilities can be determined through learning activities under expert guidance or in collaboration with more capable peers. CMs and CEs fill this role well. These tasks are structured with an appropriate level of difficulty, that is, neither trivial nor frustratingly difficult. With proper guidance, students can make their own connections between concepts, and in doing so, recognize that new concepts can be integrated alongside their prior knowledge and advance their learning to more meaningful learning. While the ZPD is traditionally framed in an asymmetric learning scenario (teacher–student interaction), the paradigm can be shifted to symmetric learning when students collaborate together (Fernández et al., 2001). In fact, the collective ZPD is expanded when a group works together to complete a task and offers a fertile ground for learning to occur. Therefore, collaboration between students with diverse abilities can also fall within the ZPD. When students work as a group to help each other to complete CM and CE tasks, they have opportunities to discuss and learn from each other's answers. This support from peers is likely to enhance connection making and deepen meaningful learning of individuals. In addition to helping students to explicate conceptual connections, CMs have been shown to reduce cognitive load of learners, especially in those with limited verbal capacity (Schroeder et al., 2017). This is achieved through the simplicity of the grammatical structures required to connect concepts together, compared to textual structures, which can be much more complex.
2. (a) Do students believe the assessments improve their conceptual understanding and connection making in chemical equilibrium and acid–base chemistry? (b) How do coupling concept maps and creative exercises affect the efficacy of learning as compared to using each assessment alone?
The S19 course consisted of a large-enrollment section, which included two 80 minute lectures and one 50 minute recitation section each week (there were 25–30 students in each recitation section). Approximately one-third of the lecture meetings used flipped classroom modules (Eichler and Peeples, 2016), while the remaining class periods included a mixture of lecture, peer-to-peer discussion, and interactive clicker questions. The recitation sections were taught by Teaching Assistants (TAs) who had prepared in advance to implement the CM and/or CE assessments. One TA (TA-1) taught the CM-only (two recitation sections; total n = 55) and CM + CE (two recitation sections; total n = 59) groups. The second TA (TA-2) taught the CE-only (two recitation sections; total n = 60) and control group (two recitation sections; total n = 59). Students enrolled in the discussion group sections based on their schedule availability, and were not aware of the study during the enrollment period. The TAs were also assigned to the discussion group sections based on schedule availability, and TA-1 was assigned to teach the CM-only and CM + CE sections because this TA had implemented the concept map activities in a previous course. The demographic and academic backgrounds of the students were not known when the TAs were assigned to various study groups/discussion group sections. Students were awarded extra credit for completing the CMs and/or CEs, and the students in the control group sections were awarded equivalent extra credit for attendance. In lieu of completing CMs or CEs in groups, students in the control group sections completed collaborative group problem-solving exercises that included free-response calculation-based questions. Detailed protocols describing how the TAs structured the weekly recitation groups for the different study groups are provided in Appendices 1 and 2 (ESI†). It is noted here that the CMs and CEs were collected on a weekly basis to ensure students were making progress in regard to building up their conceptual frameworks.
Same lecture by author J. F. E. | Week 1 recitation concept inventory pre-test | Week 2 recitation | Week 3 recitation | Week 4 recitation | Week 5 midterm 1 | Week 6 concept inventory post-test |
---|---|---|---|---|---|---|
Control by TA1 | Calculation-based questions | Calculation-based questions | Calculation-based questions | Calculation-based questions | Equally valued multiple-choice questions | Focus group interview by author L. Y. + open-ended survey questions |
CM-only by TA2 | CM | CM | CM | CM | Equally valued multiple-choice questions | |
CE-only by TA1 | CE1 | CE1 | CE2 | CE2 | CE3 | |
CM + CE by TA2 | CM + CE1 | CM + CE1 | CM + CE2 | CM + CE2 | CE3 |
Second, for students’ verbal explanations of the conceptual questions, an analytical framework for coding student reasoning about acid–base reactions developed by Cooper and colleagues (Cooper et al., 2016) was modified to include both acid–base chemistry and chemical equilibrium topics. According to the nature of participant responses to the conceptual questions during the focus group interview, the responses were coded into four major categories: no response; non-normative; principle/theory descriptive; principle/theory mechanistic. One-fourth of participant responses to conceptual questions in the focus groups were then coded into those four categories independently by authors A. G. and M. L. An inter-coder reliability of 81% was reached, and the discrepancies were resolved after deliberation. One author (A. G.) then coded the remainder of the conceptual question data.
The concordance of the two assessments is shown at the top of the map and represent common themes present in all focus groups. The first theme, metacognition, describes students thinking about their own learning process. For example, a student from one of the groups that used CMs mentioned, “[Concept maps] make you think about what you learned.” Another example quotation for metacognition from the groups used CE, “I believe that it made me think in another way. It made me calculate and write down everything that I know about the question and statement and allowed me to see what are things that I would be able to solve for…”. Metacognition and self-reflection have been shown to be beneficial to learning in many science disciplines, including chemistry, by allowing students to refine ideas about concepts (Rickey and Stacy, 2000; Cook et al., 2013).
Another common theme from the interview data was the reinforcement of prior concepts. This means that students understand that these assessments strengthen their understanding of topics covered in current and previous chemistry courses. A student from the CE group said, “…it kind of forces you to kind of recall basic, basic gen chem concepts to help you come up with statements.” From the CM group perspective, another response was: “…how the [ideas] connected to the acid–base chemistry and also the chemical equilibrium and I also connected some ideas from things I have learned in Chem 1A and Chem 1B.” The setting for the study was the third quarter of the general chemistry series, the first two quarters of which are Chem 1A and Chem 1B. Ideally, these assessment tools would help students not only see the connections between concepts learned in the course, but also make connections to chemistry concepts learned in previous courses.
Two additional common themes between the two assessments were noted: the assessment helped students to connect chemistry concepts and deepen conceptual understanding of the topics of chemical equilibrium and acid–base chemistry. “So the concept map was just – was just essential to me, just make those connections and remember it better than I did in previous classes,” claimed a student. Another mentioned, “[creative exercises] establish more connections to understand Gen Chem better.” With regards to conceptual understanding, a student from one of the groups that used CEs, when asked if he believed he needed to understand concepts rather than just knowing them, said, “These aren't statements that you can just memorize and copy exactly over to something. Because the prompts all differ, so you have to know pretty much everything you're learning. There can't be gaps in your knowledge.” In one of the groups that used CMs, a student said, “I believe that concept maps do help me understand chemistry conceptually because pretty much, concept maps, you can't really lay out – I mean you can lay out equations on there, but you can't really like doing the math so you are forced to like lay out the concepts and actually think about what you are talking about.”
The above themes align with Ausubel's Meaningful Learning framework, as students believe that building upon and making connections with previous concepts are helping them to reinforce their understanding of chemistry and take a more active role in making connections and thinking about their learning process. The themes of deepened conceptual understanding and connection-making resonate with student comments found from previous studies using concept maps (Turan-Oluk and Ekmekci, 2018) or creative exercises in chemistry (Gilewski et al., 2019) and some other science-related disciplines (Veronese et al., 2013; Chan, 2017). Chan found the use of concept maps motivated students in a problem-based nursing class to learn more actively and nurture their creativity. Veronese and colleagues found concept maps were useful to first-year college medical and dental students for systems physiology integration, identification of knowledge deficiencies, and re-examination of learned material.
In addition to similarities between the two assessments, as shown in Fig. 1, many different themes endemic to one assessment arose from the analysis. Representative quotations of the codes unique to CMs and CEs alone can be found in Appendix 9 Tables 1 and 2 (ESI†). The quotations in these two tables are from all three experimental groups. These themes are closely related to the nature of the assessment itself and the implementation of the assessment. Some highlights include students believing that creative exercises were well-suited to group work and collaboration, while concept maps were more personalized. As a result, the CMs might be difficult for others to interpret. Indeed, along those same lines, students identified CEs as an assessment tool while CMs were identified more as a learning tool for self-study for exams. Participants offered suggestions for how to improve the implementation of each assessment. In particular, students wanted thorough and timely feedback for their responses to CEs. To prevent a steep learning curve and to be able to incorporate math equations and mathematical symbols into their conceptual frameworks, students stressed that they want a more comprehensive guidance to new technologies employed and word banks for generating concept maps.
The above survey results suggest that students believed coupling CMs and CEs improves their conceptual understanding in chemistry and the ability to make connections between chemistry concepts. When students were given either activity individually, no more than half of the students believed the activity helped them understand chemistry conceptually. However, coupling both activities increased the agreement of conceptual understanding by 23% compared to the CM-only group and 18% relative to the CE-only group. Chi-square tests were conducted to examine whether the differences in the percentages of agreement between groups were statistically significant. The results indicated that the proportion of students in CM + CE group believed the assessments helped their conceptual understanding was significantly higher than CM-only group (χ2 = 3.643, p = 0.049) with medium effect size, Cramer's V = 0.233 (Gravetter and Wallnau, 2014). However, the difference between the CM + CE group and CE-only group was not statistically significant (χ2 = 2.119, p = 0.138). In terms of making connections between chemistry concepts, the majority of the students believed each activity helped them make connections. There was a small improvement between pairing both activities compared to the individual activity (2% and 13% respectively), especially when comparing the CM-only group with the CM + CE group. This is likely due to the high agreement already perceived for the CM-only group for making connections. Chi-square tests did not show significant differences between groups for connection making (between CM + CE group and CM-only group, χ2 = 0.044, p = 0.834; between CM + CE group and CE-only group, χ2 = 1.509, p = 0.207). Overall, the evidence from survey data and analysis showed that students believed coupling CMs and CEs was more effective for helping students with conceptual understanding, compared to when the CMs and CEs were implemented individually. Considering these results within the context of the theoretical frameworks described in the introduction (Novak et al., 1984), the coupling of both CMs and CEs offers more opportunities for students to link prior and newly learned knowledge with a deepened understanding of chemistry concepts, therefore more meaningful learning might occur during students' learning process.
CE + CM (n = 59) | CE-only (n = 60) | CM-only (n = 55) | Control (n = 59) | |
---|---|---|---|---|
Concept inventory pre-test | 6.03 ± 2.48 | 6.21 ± 2.03 | 5.65 ± 1.98 | 6.50 ± 1.73 |
Concept inventory post-test | 9.25 ± 1.81 | 8.82 ± 2.06 | 8.71 ± 2.05 | 9.02 ± 2.01 |
CE3 score | 13.2 ± 4.4 | 12.5 ± 3.3 | n/a | n/a |
Female | 78% | 43% | 44% | 54% |
Male | 27% | 51% | 54% | 46% |
Asian | 51% | 54% | 59% | 56% |
African American | 3% | 3% | 5% | 0% |
Hispanic/LatinX | 22% | 30% | 23% | 27% |
White | 14% | 5% | 5% | 8% |
Multi-racial | 16% | 0% | 3% | 8% |
An ANCOVA was carried out to determine if the concept inventory post-test scores differed significantly between any of the study groups. In particular, it was of interest to determine if the CE + CM study group appeared to positively impact the conceptual learning gains relative to the other study groups. Though the CE + CM study group was observed to have the highest concept inventory post-test scores (see Table 2), the omnibus ANCOVA indicates there was no statistically significant difference in concept inventory post-test scores between any of the groups (i.e., the null hypothesis stating the mean concept inventory post-test scores are equivalent for the study groups could not be rejected; F = 0.911; p = 0.436; see Table 3). It is noted this analysis accounted for the impact incoming content knowledge had on the variance of the dependent variable by including the concept inventory pre-test scores as a covariate in the model. Bonferroni-corrected between-groups pairwise comparisons were also carried out, and indicate there were no statistically significant differences in concept inventory post-test scores between any of the study groups (see Appendix 7, ESI†). It was also desired to determine if the combination of CEs and CMs might lead to greater gains on the final CE assessment scores, therefore a second ANCOVA was carried out in which the final CE scores were compared between the CE + CM and CE-only groups. Because there was a significant correlation between the concept inventory pre-test scores and final CE scores (see Appendix 10, ESI†), the concept inventory pre-test scores were included as a covariate in the model. Though the CE + CM group was observed to have higher CE scores (see Table 2), the ANCOVA indicates this difference in CE scores was not statistically significant (i.e., the null hypothesis stating the mean final CE scores are equivalent between CE and CE + CE groups could not be rejected; F = 1.136; p = 0.289; see Table 3).
Analysis | Degrees of freedom | Sum of squares | Mean square | F | p | η 2 |
---|---|---|---|---|---|---|
Comparison of means on concept inventory post-test between all groups | 3 | 9.711 | 3.237 | 0.911 | 0.436 | 0.012 |
Comparison of means on final CE scores between CE-only and CE + CM groups | 1 | 14.670 | 14.670 | 1.136 | 0.289 | 0.100 |
The fact none of the experimental groups appeared to significantly improve student performance on the concept inventory post-test assessment is likely explained by two inter-related factors. First, the CM and CE assessments were employed over a four-week period, and this relatively short exposure may have limited the impact of the CM and CE treatments on concept inventory performance. Second, the experimental conditions were embedded within a larger course framework that student groups were given the same lectures on the two topics. This confounding factor may have diminished the impact of the assessments relative to the control group, and relative to one another. If the CM and CE treatments had been administered over a longer timeframe, and/or had been compared to a “teaching as usual” control that was embedded in a different course that had less focus on conceptual learning in the lecture portion of the course, it is possible these treatments would have significantly impacted the concept inventory post-test scores.
CM-only (n = 3) | CE-only (n = 4) | CM + CE (n = 2) | |||||||
---|---|---|---|---|---|---|---|---|---|
a All the names are pseudonyms. b n/a means the data is unavailable or the student did not submit certain assessments. | |||||||||
Namea | Linda | Daisy | James | David | Katie | Rose | Felix | Chris | Jenny |
Gender | Female | Female | Male | Male | Female | Female | Male | Male | Female |
Ethnicity | Asian | Black | Asian | Asian | Asian | Multi-racial | Asian | Multi-racial | Hispanic |
High school GPA | 3.6 | 4.4 | 3.8 | 4.2 | 3.5 | n/ab | 3.6 | 4 | 4.3 |
Math SAT | 640 | 670 | 670 | 770 | 560 | n/a | 570 | 650 | 610 |
Concept inventory post-test scores | 6 | 11 | 10 | 7 | 9 | 6 | n/a | 12 | 9 |
CE scores | n/a | n/a | n/a | 16 | 15 | 11 | 12 | 16 | 20 |
CM scores | 6 | 8 | 5 | n/a | n/a | n/a | n/a | n/a | 6 |
Categories and characterizations | Examples of student quotations |
---|---|
1. No response | “I didn’t really get that question.” |
Students did not provide an answer | “I do not really have anything more to add.” |
2. Non-normative | “Basically, one's just like the reactants and the products or something like that, I'm not sure.” |
Students provide non-normative or unrelated explanations | “I said that the conjugate base, CH3COOH, was able to neutralize the acid by reacting with it.” |
3. Principle/theory descriptive (what) | |
Students provide simplistic description including | “The reaction shifts to the products and that came from Le Chatelier's Principle.” |
• description of chemical equilibrium as reaction rates are constant for forward and reverse reactions | “I also just put down like, HCl acts as the Brønsted–Lowry acid because it donates the proton and NH3 acts as the base in Brønsted–Lowry because it accepts it.” |
• identification of the shift of the reaction or mention of Le Chatelier's principle without stating how it changes the chemical equilibrium | |
• Identification of a strong acid is completely dissociated into its ions while a weak acid is partially dissociated into its ions in water | |
• identification of acids and bases based on Brønsted–Lowry or Lewis theory | |
• identification of CH3COOH as weak acid and its conjugate base CH3COONa | |
4. Principle/theory mechanistic (what and how) | |
Students provide more scientific explanation including | “If you have one of Le Chatelier's that the reaction can react some kind of stress placed on the reaction, so if you increase the concentration of the reactants, then the reaction will thus favor the-like it'll favor and proceed towards the products side to counteract the addition of reactants and form more products.” |
• explanation on chemical equilibrium constant or differences between solid/gaseous/aqueous reactions | “You have HCl and NH3 so according to Brønsted–Lowry acids are proton-proton donors and bases are proton acceptors. So, in this case, HCl is your acid which is the proton donor. So when you draw the reaction, you have your two separate reactions: HCl plus NH3 and that forms – that would form NH4 plus and Cl minus. So what happened was the HCl, the acid, donated its acidic hydrogen to the ammonium to form NH4 plus, and the Cl minus became its – the, the conjugate base.” |
• application of Le Chatelier's principle and explanation on how it changes the chemical equilibrium | |
• connections between strong and weak acids with chemical equilibrium or concentration of H+ or pH | |
• description of the products of the acid–base reaction or the role of proton in Brønsted–Lowry theory or the role of lone pair in Lewis theory | |
• description of chemical reactions between HCl and CH3COONa or NaOH and CH3COOH |
CM-only | CE-only | CM + CE | |
---|---|---|---|
No response | 3 (8%) | 3 (9%) | 0 (0%) |
Non-normative | 9 (25%) | 11 (32%) | 5 (16%) |
Principle/theory descriptive | 9 (25%) | 12 (35%) | 15 (48%) |
Principle/theory mechanistic | 15 (42%) | 8 (24%) | 11 (36%) |
Total | 36 (100%) | 34 (100%) | 31 (100%) |
Fig. 4 The comparison between the three experimental groups in proportions for the four categories of conceptual understanding. |
As indicated in Fig. 4, students in the CM + CE group were able to provide more scientific explanations in the last two categories: Principle/theory Descriptive and Principle/theory Mechanistic, 48% and 36%, respectively. There were no statements coded as No response and relatively low percentage (16%) of statements were coded as Non-normative (i.e., incorrect or irrelevant). Students who had CM-only as an assessment had 8% coded as No response and 25% coded as Non-normative. The rest of the student responses were coded as 25% of Principle/theory Descriptive and 42% Principle/theory Mechanistic category. The CE-only group had similar percentages for No response (9%), and higher percentages for (32%) Non-normative as compared to the CM-only groups. Non-normative and Principle/theory Descriptive were 35% and 24% for this group. When combing the two higher-level categories (Principle/theory Descriptive and Principle/theory Mechanistic) for the three experimental groups, which represented the more scientific and sophisticated explanations of chemical equilibrium and acid–base chemistry concepts, response rates were 59%, 67%, and 84% for CE-only group, CM-only group, and CM + CE group, respectively. Chi-square tests were conducted to examine whether the differences in the proportions of higher-level and lower-level categories (No response and Non-normative) between groups were statistically significant. The results indicated that there was a significant difference between CE-only and CM + CE groups (χ2 = 4.918, p = 0.027) with medium effect size, Cramer's V = 0.275 (Gravetter and Wallnau, 2014). However, the difference between the CM-only and CM + CE groups was not statistically significant (χ2 = 2.604, p = 0.107). These results suggest the students in CM + CE group were able to more frequently describe scientific concepts pertaining to chemical equilibrium or acid–base chemistry in a scientific manner, relative to the CE-only group. In addition, students in the CM-only group were also more likely to provide more scientific descriptions about similar concepts, or what was taking place during the chemical equilibrium or acid–base chemistry, compared to the CE-only group.
Though the CE + CM group appeared to have higher concept inventory post-test scores and higher CE scores relative to the CM-only and CE-only study groups, the ANCOVA results indicate the combined assessment did not significantly impact conceptual learning gains. However, the analysis of survey responses and focus group interviews suggest the promising impact of pairing concept maps and creative exercises on student learning in chemistry. In particular, the focus group interviews suggest the combination of concept maps and creative exercises led to greater gains in conceptual understanding compared to either individual assessment. During the focus group interviews, students who used both concept maps and creative exercises were able to provide more sophisticated scientific explanations for the concepts related to the topics of chemical equilibrium and acid–base chemistry.
Based on the evidence obtained in this study we encourage instructors and educators to adopt concept maps and creative exercises into their curricula and to implement these two assessments together. The benefits of pairing the CMs with CEs include: (1) the combinations of CEs and CMs provides students with multiple tools for understanding and connecting chemistry concepts in addition to traditional assessments; and (2) one assessment is complementary to the other due to the nature of each assessment; CMs are ideal for formative assessment that assists students to transfer textual concepts to visual representations while CEs are suitable for summative assessment because they can be graded in a less subjective manner. To facilitate the implementation of the two assessments for instructors, a set of creatives exercise and example student responses, and detailed teaching notes are provided in the appendices (see Appendices 1 and 2, ESI†). Instructors can use these ready-to-use materials directly or use them as templates to develop their own course materials. Best practices for implementation can be derived from student responses in this study. In particular, instructors should keep in mind students perceive creative exercises as an assessment technique, whereas they view concept maps as a study tool that helps them prepare for the CEs. Students also indicate they would like to generate personalized concept maps with the assistance of word banks and technology tools that incorporate math equations and mathematical symbols, that they prefer to work collaboratively with others for creative exercises, and that prompt and constructive feedback from instructors is essential. Thus, instructors should consider these recommendations when implementing the two assessments. Additionally, since students believe concept mapping helps with answering creative exercises, giving students opportunities to generate concept maps before creative exercises are assigned will likely improve performance, especially if the CEs are incorporated into high-stakes exams.
Lastly, future directions of research with regards to the two assessments could put more thought on how to measure the differences between study groups using alternative modes for collecting student. For instance, one potential measure could ask students to explain relevant concepts and connections between chemistry concepts and submit the explanations in the form of video or audio files. A rubric similar to our analytic framework in Table 5 for student conceptual understanding in chemical equilibrium and acid–base chemistry can be developed for grading and comparing student explanations between different groups of students. Such a study might increase the sample sizes of participants from each study group because it will not be restricted by location and time, and could allow for a repeated measures study design. Another interesting area of future work might be an investigation of the impact of the TAs on students' concept mapping and connection-making. In our study, participants expressed anecdotally in the interview about how TAs influence their proficiency in completing the assessments. Do students benefit more from working with their peers in groups or TAs? How does each contribute to the potential development of students’ capabilities? A further investigation of these questions might be worthwhile and gain insight into the theory of the Zone of Proximal Development.
Footnote |
† Electronic supplementary information (ESI) available. See DOI: 10.1039/d0rp00038h |
This journal is © The Royal Society of Chemistry 2020 |