Analysis of the role of a writing-to-learn assignment in student understanding of organic acid–base concepts

Abstract

Acid–base chemistry is a foundational topic that is taught in courses across the chemistry curriculum. Students often have difficulty distinguishing between the different theories of acid–base chemistry—Brønsted–Lowry and Lewis acid–base chemistry—and applying these two definitions correctly in unfamiliar scenarios. To help students learn these definitions and be able to apply them, an acid–base Writing-to-Learn assignment was developed and evaluated. The Writing-to-Learn assignment involved a three-step process where students constructed an initial draft in response to a writing prompt, participated in peer review, and made revisions based on peer review feedback, before submitting a final draft. This process is informed by sociocultural theory applied to writing, which states that student learning of concepts increases through engagement with their peers’ work and receiving peer feedback on their own writing. To test the efficacy of the acid–base writing assignment, an external assessment, comprised of conceptual questions related to acid–base chemistry and students’ confidence when responding to them, was administered in two groups; a treatment group who completed the Writing-to-Learn assignment, and a comparison group who completed a separate assignment. Additionally, students who completed the Writing-to-Learn assignment were interviewed about their experiences. Regression analysis revealed that students in the treatment group had a greater increase in their conceptual understanding and confidence as compared to the students in the comparison group. The results demonstrate the students could successfully write about the Brønsted–Lowry and Lewis acid–base models separately, but were less successful with connecting these two concepts together in their writing. These results demonstrate the efficacy of Writing-to-Learn as an approach for promoting conceptual learning of acid–base chemistry.

---

Introduction

Organic chemistry is often described as a “gatekeeping” course for many students (Grove et al., 2008; Widanksi and McCarthy, 2009). One of the reasons why organic chemistry can be challenging for students is because the content is less mathematical and more conceptual in nature (Anderson and Bodner, 2008; Grove and Bretz, 2012; Grove et al., 2012). To be successful, students have to have a high degree of conceptual knowledge in organic chemistry. The need for a deeper conceptual understanding of organic chemistry provided the rationale for this study, which focused on improving students’ conceptual understanding of acid–base chemistry, a notoriously challenging topic, through the use of a Writing-to-Learn (WTL) activity.

Acid–base chemistry is a foundational topic in introductory chemistry that is relevant in later chemistry courses, as well as in other science courses: it is first introduced in general chemistry and again is revisited in organic chemistry, analytical chemistry, and physical chemistry. Acid–base chemistry is viewed by faculty as the most fundamental concept that students need to understand for successful continuation in the study of chemistry (Duis, 2011). Acid–base reactions are often the first reaction type covered in an organic chemistry course (Stoyanovich et al., 2014). In an analysis of 28 commonly taught organic and biochemistry reactions, 86% were found to involve acid–base reaction chemistry (Stoyanovich et al., 2014). A recent survey of nurse educators identified the core concepts of the definitions of the Brønsted–Lowry acid–bases and pH to be extremely important topics to teach, highlighting the importance of these concepts for STEM majors and non-majors alike (Brown et al., 2018). In addition, in a survey study, organic chemistry faculty reported that organic acid–base chemistry is the second hardest topic for students to learn (Duis, 2011).

Though acid–base chemistry is foundational for chemistry understanding and considered important by faculty, students sometimes struggle to understand the definitions and apply them appropriately. This struggle may be attributed to the everyday notions of acids and bases that students hold before learning about them formally in chemistry class, such as knowledge that bases are often common household cleaners. Information about acid–base chemistry is often presented in textbooks in a way that leads students to learn about these topics through rote memorization rather than through developing a deep conceptual understanding (Drechsler and Schmidt, 2005; Kousathana et al., 2005; Lin and Chiu, 2007; Stoyanovich et al., 2014). Students struggle with the specific definitions of acids and bases from the Brønsted–Lowry and Lewis acid–base theories. In particular, Lewis acid–base chemistry has been reported by multiple studies as being particularly difficult for students to define and apply to novel contexts (Bhattacharyya, 2006; Cartrette and Mayo, 2011; McClary and Talanquer, 2011; Cooper et al., 2012). If students struggle with distinguishing among the different theories of acid–base chemistry, then they might struggle with applying the principles of each theory to novel organic acid–base reaction contexts.

Several studies have reported on the commonly held misconceptions about acid–base chemistry. These misconceptions can persist from freshman level (Cros et al., 1986) into sophomore level university chemistry courses (Cros et al., 1988). Further, student difficulty with identifying and classifying acids and bases persists to the graduate level—with Bhattacharyya (2006) reporting how advanced students were able to provide definitions for acids and bases but were less able to apply their conceptual models in multiple situations. Cartrette and Mayo (2011) reported that even when students had a good grasp of the concepts of acid–base theories from general chemistry, their knowledge did not necessarily translate successfully into organic chemistry. McClary and Talanquer (2011) reported that students generally hold multiple distinct mental models of acid–base chemistry, but their models depend on the surface features present in a molecule. A study by Dood et al. (2018), which classified what mental models of acid–base chemistry students used on a formative assessment, reported that students who used the Lewis model over the Brønsted–Lowry model performed better on a summative assessment of their acid–base knowledge. Cooper et al. (2016) reported that, while students are often able to identify what is happening in an acid–base reaction, they are often not able to explain why or how an acid–base reaction is proceeding. These studies revealed particular misconceptions about acid–base chemistry held by students: that many students do not have a cohesive mental model of acid–base chemistry. Furthermore, even if they do have cohesive models of each theory, students often struggle to apply them appropriately.

Another area of acid–base chemistry that students struggle with relates to understanding the relationship between pK_a values and pH, and how this relationship can apply to various organic contexts (Orgill and Sutherland, 2008; Stoyanovich et al., 2014). Nakhleh and Krajcik (1994) proposed that one reason students may struggle with applying pH and pK_a is that in order for students to fully understand the concepts of acid–base chemistry, they must first understand the properties of atoms and molecules. Students struggle at finding and estimating pK_a values (Flynn and Amellal, 2015), which can be an important skill for solving complex organic chemistry problems where specific pK_a values may not be available. Students may (incorrectly) conflate a stronger acid with having a higher pH level (Ross and Munby, 1991). These studies suggest that students often cannot apply the properties of pH and pK_a to an unfamiliar situation, because they lack a strong foundational understanding of these concepts.

Instructional interventions have been proposed as a way to combat students’ difficulty with understanding acid–base concepts. Yaman and Ayas (2015) found that computer-assisted activities, which prompted students to evaluate acid–base reactions through a predict–observe–explain procedure, increased the complexity of students’ acid–base explanations on a concept inventory after the activity was completed. Explicitly targeting and providing instruction on topics, such as pK_a (Flynn and Amellal, 2015), can help mitigate and improve students’ understanding. McClary and Talanquer (2011) suggest that assignments focused on acid–base chemistry could help increase students’ meaningful learning about those topics. Cooper et al. (2016) also proposed that the nature of a task can influence students’ ability to create correct explanations of acid–base chemistry and suggested that students can develop a greater conceptual understanding of acid–base concepts through constructing explanations. A writing intervention that was done with secondary school students found that students’ performance on conceptual assessments could be improved through writing (Visser et al., 2018). Cox et al. (2018) studied students with less chemistry experience who completed short writing activities about acid–base chemistry in general chemistry and participated in a peer review process, and compared them with students with more chemistry experience who did not complete the writing activities or participate in peer review. The study found that after the intervention there was no difference in the two groups’ conceptual understanding or confidence. They concluded that writing assignments can help close performance gaps between students. Collectively, research indicates that writing assignments and peer review activities have the potential to increase student understanding and confidence about organic acid–base concepts.

Writing-to-Learn

The goal of this research was to design an instructional intervention, specifically implementing a WTL activity to help students learn about organic acid–base concepts. WTL is a pedagogy that encourages students to increase their understanding about designated topics through the process of writing about them (Rivard, 1994; Fry and Villagomez, 2012; Reynolds et al., 2012; Prain and Hand, 2016). Previous research has shown that WTL assignments can help increase students’ understanding of particular concepts in chemistry and biology (Shultz and Gere, 2015; Finkenstaedt-Quinn et al., 2017; Halim et al., 2018; Moon et al., 2018). These WTL assignments can provide in-depth feedback to instructors on how students are describing and integrating important core concepts in their writing (Shultz and Gere, 2015; Finkenstaedt-Quinn et al., 2017; Halim et al., 2018; Moon et al., 2018). Importantly, WTL assignments are much more focused on how students explain content and phenomena, rather than on features of writing itself. The WTL activity presented herein is distinct from the instructional interventions that have previously targeted students’ acid–base understanding in an organic chemistry context. In this activity, students defined two acid–base models separately and then connected and applied both models together. This prompt is designed to encourage students to apply foundational concepts in acid–base chemistry by presenting them with a specific and novel scenario. We hypothesized that having students explain and apply these concepts in a novel setting would increase their conceptual understanding of these topics.

Along with responding to the WTL prompt, students participate in peer review (Zhang et al., 2017; Finkenstaedt-Quinn et al., 2019). Students receive feedback on their writing and also read how other students describe the same concepts in their papers; then they make changes in response to peer review and submit a revised version of their writing. The peer review component is especially significant since feedback from peers has been shown to be just as effective as an experts’ feedback in improving students’ writing (Cho et al., 2006; Patchan et al., 2009). Participation in a peer review and revision process improves the quality and content of students’ writing (Cho and Schunn, 2007; Finkenstaedt-Quinn et al., 2019). Participation in writing, reviewing, and revising serves to not only reinforce but also to help students connect ideas related to organic chemistry concepts through the process of writing explanations. Through the writing process, students can uncover how they are thinking about certain concepts and they are able to see what content areas they need to improve their explanations of chemistry concepts (Halim et al., 2018). Therefore, the focus of this research was to evaluate how students shifted their thinking about organic acid–base chemistry concepts after participating in Writing-to-Learn.

The objective of this project was to examine the influence of a single Writing-to-Learn activity on students’ understanding of acid–base concepts. To investigate how the writing activity may have influenced students’ understanding, the WTL assignment was administered to a group of students during one semester, who served as the treatment group. The treatment group was compared to a different group of students (the comparison group) who completed a different assignment. The study was guided by the following research questions:

(1) Do students who complete a WTL assignment improve their understanding of acid–base concepts compared to students who do not?

(2) How does this WTL assignment support students’ understanding of acid–base concepts?

(3) Do students who complete the WTL assignment improve their confidence when answering acid–base problems?

Theoretical framework

The design, data collection, and findings of this study were informed by Vygotsky's sociocultural theory (Vygotsky, 1962; Vygotsky, 1980; Prior, 2006). According to sociocultural theory, learning occurs and is facilitated by social interactions between people (Vygotsky, 1962; Vygotsky, 1980). Writing can be viewed as a social activity, where the author is writing to convey their understanding and communicate a particular phenomenon. Writing can also be viewed as a social learning activity when students can read each other's work in order to experience a different perspective and uncover new insights about particular phenomenon (Zhang et al., 2017; Finkenstaedt-Quinn et al., 2019). Through participating in writing as a social activity, students are constructing knowledge together—both during the process of reviewing each other's work and receiving feedback on their own writing. When revising their assignments, students are provided an opportunity to incorporate the new knowledge they gained from the peer review process and revision of their writing. When writing is viewed as a social activity, a zone of proximal development is created (Vygotsky, 1980). The zone of proximal development is in the middle of three concentric circles, as shown in Fig. 1. This middle ring is the optimal level where instruction should be aimed. The innermost ring represents what a learner can do on their own, without help from others. The outermost ring represents what the learner cannot do, even with assistance. This model assumes that student would not be able to reach the outermost ring, even if they had extensive, quality feedback from their peers. With regards to writing, the zone of proximal development represents how students could alter their writing through feedback from their peers.

Fig. 1 The zone of proximal development depicting what a learner can do with assistance.

Through the WTL activity in this research study, students were provided an opportunity to read their peers’ writing and see what their peers wrote. This is another form of the social engagement inherent in the peer-review process (Patchan et al., 2009; Cho and MacArthur, 2010; Cho and MacArthur, 2011; Zhang et al., 2017). This exchange of feedback creates a zone of proximal development where all of the students who participate in the WTL process are receiving feedback and have the potential to improve their writing. Participating in the writing as a social activity can increase students’ confidence because it offers students corroboration of their conceptual understanding by others (Mahn and John-Steiner, 2008). The findings of this study were interpreted through Vygotsky's sociocultural theory of writing (1962) to investigate to what extent the WTL activity, as a social endeavour, contributed to students’ increase in their conceptual understanding of acid–base concepts (Prior, 2006).

Methods

This study was conducted across two semesters. In both semesters, the students were enrolled in an organic chemistry laboratory II course. Students were assigned to the treatment or comparison group based on the semester of the course they were enrolled in. In the treatment group, students completed the organic acid–base WTL activity. The comparison group students completed a different writing activity, which included shorter writing in response to questions on prepared worksheets on acid–base chemistry and related laboratory topics. Students in both groups spent a similar amount of time on each of their respective writing tasks (Fig. 2).

Fig. 2 Timeline of treatment and comparison group assignments over a two week period.

Description of the Writing-to-Learn assignment

The WTL assignment was based upon a topic that was chosen by two experienced organic chemistry instructors: the contraindication of levothyroxine. The reason this topic was chosen is that it provided an opportunity for students to define the two theories of acid–base chemistry, Brønsted–Lowry and Lewis acid–base, and apply them in a real-world scenario. Another important feature of WTL assignments is that the students write to an intended audience specified in the prompt. When directing their writing to a specific audience, students respond to the WTL prompt taking on a specific scientific role. These contextual features help create an authentic context for students to construct their responses. For the acid–base prompt, students responded to the prompt as if they were a medicinal chemist, and they were directed to write an email to their collaborator, who was a physician. The goal of using a designated rhetorical context and audience was to encourage students to write as an expert and to define and explain any organic chemistry terms they used in their responses, since they were writing to a collaborator who was not an expert in the field of organic chemistry. The WTL prompt can be seen in Fig. 3.

Fig. 3 Contraindication of levothyroxine Writing-to-Learn assignment description.

The assignment described here was one of a series implemented in the course. Each of the WTL assignments were designed to help students develop a conceptual understanding of organic chemistry topics through writing. Students were directed towards the target concepts during each of the three core components of the WTL activity: the first draft, peer review, and revised draft. Students were encouraged to seek help on their WTL assignments from peer writing tutors who were undergraduate students who had successfully completed the course and demonstrated good communication skills. The peer writing tutors participated in formal training that included attending a one-hour, once per week class, where they learned about evidenced-based tutoring practices. The peer writing tutors also met regularly with the course instructor to discuss content concerns. The role of the peer writing tutors was distinct from graduate teaching assistants because they were trained to specifically answer students’ questions about the WTL activity. They responded to students’ questions via email and held in-person office hours.

Description of the comparison group assignment

Students in the comparison group completed two worksheets that included short answer essay questions on acid–base chemistry. Students provided short written answers (3–4 sentences on average) in response to questions like “Why is sodium acetate used to deprotonate the compound rather than NaOH?”. Although many questions in the two worksheets were focused similarly on acid–base chemistry, they were not strictly aligned with the Writing-to-Learn assignment. The students worked in small groups to discuss the concepts that were on the worksheet to mirror the peer interactions experienced by the students who did the WTL assignments. These discussions occurred both in and out of class time and were not mediated by the instructor. However, students could seek assistance from their graduate student teaching assistant, who also graded the worksheets. Students completed the first worksheet as a group. Students completed the second worksheet individually two weeks after the initial worksheet. In this way, the duration of the overall assignment was similar to that of the treatment group.

The primary differences between the WTL activity and the comparison group activity were in the nature of the writing students completed and how they completed it. For the WTL activity, students wrote a longer response (1–2 pages) to a complex prompt. The comparison group wrote shorter responses to a set of simpler prompts. Both activities included elements of collaborative writing. The WTL activity involved peer review whereas students read each other's writing, provided and received feedback. In the case of the comparison group, students drafted responses to the first worksheet together as a group and the second worksheet individually.

Implementation

The writing activity was completed during a two-week period by students in the WTL (treatment) group. Students had one week to submit initial drafts, after which they were assigned three anonymous peer reviews through an online course management system. They then had three to four days to read and evaluate their peer's work. A peer review rubric with content-directed criteria guided the students’ evaluation of their peers, where each of the criteria were aligned with learning objectives from the prompt. In addition, students were provided general guidelines on how to complete peer review. Students then had about three to five days to read and incorporate the peer feedback they received before submitting their revised draft.

For the grading of the WTL activity, students were assigned points on their initial draft if they turned in a draft where reasonable effort was shown. Students were awarded points for completion of peer reviews if they provided quality and detailed feedback. For the revised draft, students were awarded points if they made significant changes to their initial draft. A “significant” revision was defined as having changes in the revised draft of at least three sentences. Requiring revisions to receive full credit on the assignment provided students with an incentive to revisit their thinking after conducting peer review and in response to the review comments they received.

Data collection

Data collection involved a three-pronged approach consisting of three pre-post conceptual questions, students’ writing, and interviews with students after they had completed the WTL assignment. All of the data was approved by the Institutional Review Board (IRB) at the participating institution, and all of the participating students provided consent. The concept questions were administered to students in both groups, WTL (treatment) and non-WTL (comparison).

The acid–base conceptual questions were adapted and modified from two previously published organic chemistry questionnaires (McClary and Talanquer, 2011; McClary and Bretz, 2012). The structure for each question on the concept test was modelled after that reported by McClary and Bretz in that it consisted of a multiple-choice question, a self-efficacy question where students ranked how confident they were with their answer, and an explanation question. However, the explanation question prompted open-ended explanation where students had to provide their reasoning behind the multiple-choice answer they selected instead of selecting one of a set of possible explanations. The open ended format was included in order to elicit all of the possible explanations that students might generate after a participating in writing to learn. The questionnaire included questions on both Brønsted–Lowry and Lewis acid–base theories that were ranking tasks similar to those reported by McClary and Bretz, and McClary and Talanquer, but in which some of the specific compounds students were asked to rank differed. The conceptual questions were selected by two experienced organic chemistry instructors as being aligned with the learning goals of the WTL activity and comparison group worksheet. A copy of the three conceptual test questions that were used in this study are located in Appendix 1. All three questions were multiple choice, and students’ answers were scored as being either correct or incorrect. The confidence components of the questions were scored on a 1–5 Likert scale, with a 1 representing a “Not confident” rating and a 5 representing a “Very confident” rating. The questionnaire was piloted with students who had previously completed the organic chemistry II laboratory course. The purpose of piloting the questionnaire was to ensure that the language of the questions would be understandable to students and that the question content would be able to be answered by students by the end of organic chemistry II.

Registrar data was also collected from both WTL and non-WTL students who consented to participate in the study. This data included the students’ overall cumulative GPA at the end of the semester that they participated in the study, the grade they received at the end of the organic chemistry II laboratory course, and their math placement percentile score. The math placement exam is a school administered exam that all students take before they begin classes at the university.

The second source of data was the students’ WTL assignments, which included both the initial and the revised drafts. The data presented in this paper is from students that completed both drafts. 302 (out of 328) students turned in both an initial and revised version of the WTL assignment.

The third data source was reflective semi-structured interviews, which were conducted with students in the treatment group after students had completed all components of the WTL activity. The goal of the reflective interviews was to ascertain students’ experiences with the activity. Three students volunteered and were interviewed. They were consistent in their responses to posed questions and no additional students were interviewed. Prior to the interview, the interviewer reviewed the students’ documents and annotated them to ask the students targeted questions specific to their assignments and peer reviews. The interviewer followed a consistent process, where they first provided each student with a copy of the prompt and a sample of their writing and then asked the students a series of questions based upon their writing. The interviewer took field notes during the interview and the interviews were audio-recorded and transcribed verbatim by an external transcription service.

Data analysis

The data were analysed using both quantitative and qualitative methods. The questionnaire data were cleaned prior to analysis by removing responses from the analysis if students omitted any one component of the three-tiered question. Students’ responses were also omitted if they did not complete both the pre- and post-questionnaire. After data cleaning, for the WTL group there were 186 complete responses out of 328 enrolled students (57% percent response rate), and for the non-WTL group there were 335 complete responses out of 757 enrolled students (44% response rate).

Quantitative data analysis. All statistical analyses were performed using SPSS statistical software. Statistical comparisons were made on the students’ performance between the initial and revised draft for the acid–base WTL assignment. To test for normality, the students’ scores were plotted in histograms fitted with a normal Gaussian curve. Additionally, the skewness and kurtosis metrics of the student scores were analysed to evaluate their normality. Results found that the scores were not normally distributed, so non-parametric statistical tests were chosen for all analyses. Bonferroni-corrected McNemar Chi-Square tests were performed on each category in the 12-point rubric used to analyse the students’ WTL responses, located in Appendix 2. Additionally, Bonferroni-corrected Wilcoxon signed-rank tests were used to compare the students’ initial and revised draft performance on the three major rubric categories, as well as between their total initial and revised score on their drafts. To quantify the changes in performance on the questionnaire from pre to post, Cohen's effect sizes were calculated for each statistical comparison. According to the Cohen's effect size guidelines, an effect size from 0.20–0.49 indicates a small effect, an effect size of 0.50–0.79 indicates a medium effect, and an effect size of 0.80 or greater indicates a large effect (Cohen, 1992).

Multiple linear regression with bootstrapping (1000 bootstrap samples) and binary logistic regression models were the primary methods used to investigate the relationship between participation in the WTL activity and outcomes measures. Neither of these methods assume that the data is normally distributed. The outcome variable that the regression models predicted was the post-questionnaire score that students received including whether students had the multiple choice question correct, the score that was awarded for their open-ended explanation, and their confidence self-ranking. Course GPA, cumulative GPA, and math placement perentile score were correlated with each outcome variable to investigate which of these three variables could serve as proxies of overall academic performance (Theobald and Freeman, 2014). Cumulative GPA was found to be significantly correlated with the most outcome variables and was used in all of the regression models. The predictor variables that were used in the models included group membership (WTL or non-WTL), cumulative GPA, and the pre-score on the questionnaire component of interest. For every regression model, the non-WTL group was used as the reference group, so the group variable was equal to zero for the non-WTL group and one for the WTL group. An additional constant term was calculated for each regression model. It was hypothesized that the treatment or comparison group would have higher pre-scores on the questionnaire. To test this hypothesis, an interaction term was added into each model, which was the product of what group (WTL or non-WTL) students were in multiplied by their pre-score. The interaction variable was tested to see whether it was statistically significant. The interaction term mathematically accounted for whether the relationship of the pre and post-questionnaire scores was different for either the treatment or comparison group. For each component of the questionnaire, a regression model was conducted in addition to individual regression models for each of the three questionnaire questions.

Qualitative data analysis. Rubrics were developed to analyse the students’ written explanations to the conceptual questions and students’ responses to the WTL responses. The interviews were analysed by open-coding to examine how students interpreted the WTL prompts, how they felt the WTL prompts might have contributed to improving their understanding of acid–base chemistry topics, and how students incorporated the feedback they received from the peer review.

A rubric was developed to analyse the explanation portion of all three conceptual questions. The explanations were scored on a four point scale based on the accuracy of their multiple choice selections and explanations: 0 = incorrect multiple choice selection with an incorrect explanation; 1 = correct multiple selection with an incorrect explanation or incorrect multiple choice selection with a correct explanation; 2 = correct multiple choice selection with a partially correct explanation; and 3 = correct multiple choice selection with a correct explanation.

Two researchers initially scored a subset of the questionnaire responses to ensure they were both applying the rubric consistently. After the initial calibration of applying the rubric to this subset of explanations, the two researchers scored the remaining multiple-choice selections along with their corresponding open-ended explanations. Two researchers scored 25% of the responses independently, and then met and discussed the coding. If there were disagreements about the coding, the two researchers came to a consensus on a score. A Kohen's Kappa value of 0.719 was calculated on the responses that the researchers scored together, which is considered substantial agreement (Cohen, 1960; McHugh, 2012).

For the analysis of student writing, a rubric was created to assess the learning objectives from the WTL prompt. Three of the researchers constructed the preliminary rubric, which consisted of three broader rubric categories that contained four sub-categories each, for a total of twelve categories. The initial rubric was developed based on the learning objectives from the WTL prompt and how students were writing about these concepts in drafts. After the initial rubric was created, four researchers met and calibrated themselves with the rubric to score a subset of the students’ responses. They discussed how to apply the rubric and, based on the discussion, refined the rubric category definitions. The final version of the rubric used to assess the students’ WTL assignments is presented in Appendix 2.

Four researchers independently coded 18% of the student responses, including both the initial and revised drafts, and then met to discuss the coding. Any coding discrepancies were discussed and resolved and a consensus score was reached. The four researchers scored the remainder of the student assignments independently. A two-way mixed random intra-class correlation (ICC) analysis was used to calculate inter-rater reliability on the coding based on the average scorer's agreement of twelve the rubric categories. An overall ICC value of 0.892 was calculated, which is considered good agreement (Koo and Li, 2016).

A single author analysed the interview transcripts using the qualitative software NVivo. The interviews were analysed through open-coding to examine how students interpreted the language of the prompt, how students used the peer review feedback, and what acid–base concepts students reported having a deeper understanding of and if they felt differently about their self-efficacy in organic acid–base chemistry after completing the WTL activity. Initial themes were discussed with the research team after which the same author completed the analysis.

Results and discussion

Do students who complete a WTL assignment improve their understanding of acid–base concepts compared to students who do not?

Comparison of the increase in understanding of acid–base concepts between the treatment and comparison group was made using regression analysis. Cumulative GPA was included in the model to account for differences in prior academic performance among the students (Theobald and Freeman, 2014). A multiple-linear regression model was used to predict what factors contributed to the change in students’ post-questionnaire performance on the post-test explanations, and the results are presented in Table 1.

Table 1 Results of linear regression analysis with bootstrapping to evaluate factors that predict students' average performance on post-test score explanations

Variable	Coefficient estimate β (standard error)	Bias	95% confidence interval (lower–upper)	p-Value
Regression parameters: adjusted R-squared = 0.077 and F-value = 15.467.
Constant	0.255 (0.292)	0.010	−0.282–0.845	0.397
Group (WTL or non-WTL)	0.149 (0.058)	0.002	0.042–0.261	0.009
Cumulative GPA	0.056 (0.081)	−0.003	−0.103–0.207	0.513
Average pre-questionnaire explanation	0.305 (0.060)	0.001	0.186–0.420	0.001

---

Participation in the WTL activity, the group variable, was found to be statistically significant in predicting students’ explanation performance on the post-questionnaire. Even when accounting for the students’ pre-questionnaire scores and cumulative GPA, group membership was a meaningful and significant predictor of students’ post-questionnaire performance. The coefficient for the group variable was 0.149, which indicates that the WTL students had, on average, an increase of 0.149 rubric levels per explanation, compared to the non-WTL students.

Additionally, binomial logistic regression was used to predict student responses on each of the three concept questions. The results from these binomial logistic regression models are presented in Appendix 3. On one of the questionnaire questions, Rank Lewis Acidity, the students in the WTL group were found to have a higher statistical likelihood of answering this question correctly as compared to the non-WTL students. There was no statistically significant difference between groups on the other two questions. These results suggest that participation in the WTL activity improved students’ conceptual understanding of difficult Lewis acid–base chemistry topics particularly.

How does the WTL assignment support students’ understanding of acid–base concepts?

The role of the WTL assignment in students developing an understanding of acid–base concepts was examined through analysis of the student writing and interviews. Student writing was quantitatively transformed using a scoring rubric (Appendix 2) for each learning goal in the assignment. The initial and revised drafts were compared across the three broad categories from the rubric: Theory, Brønsted–Lowry, and Lewis (Table 2).

Table 2 Rubric scores for both drafts of the writing assignment. Each score represents the average for each broad category on a scale of 0–4 points and were derived from twelve separate rubric categories. Statistical comparison was made using the Wilcoxon signed-rank test. The Bonferroni corrected value was α = 0.0125

Category	Initial draft average	Revised draft average	p-Value	Effect size
Theory	1.00	1.50	<0.001	0.48
Brønsted–Lowry	1.96	2.60	<0.001	0.63
Lewis	1.41	1.76	<0.001	0.28
Total score	4.38	5.86	<0.001	0.64

---

From Table 2, all three broad rubric categories and the total rubric score showed that students statistically improved their scores from the initial to the revised draft. One of the categories, Brønsted–Lowry, along with the total rubric score, had meaningful medium effect sizes of 0.63 or higher. The Theory category had a meaningful small to medium effect size of 0.48. The Lewis acid–base category had an effect size of 0.28. Students, on average, scored the lowest in the Theory category, both on initial draft (with an average of 1.00) and revised draft (with an average of 1.50). The category that students scored highest on was the Brønsted–Lowry category, with an average of 1.96 on the initial draft and 2.60 on the revised draft. Students performed better both on their initial and final drafts in the Brønsted–Lowry categories compared to the Lewis acid–base categories. Students were able to increase their total average essay score by greater than a factor of one, which corresponds to incorporating more explanations and details into their revised drafts.

The subcategories from the rubric (Appendix 2) were compared by looking at the Cohen's effect sizes to examine whether the changes from the initial draft were meaningful; these results are presented in Table 3. Of the twelve, there were nine subcategories that had at least small to medium effect sizes. The two categories where students made the most improvement between drafts where the ‘Identified all acidic sites on levothyroxine’ and ‘Correctly defined pH/pK_a relationship’. Both of these categories are related to concepts from the Brønsted–Lowry model, and provides evidence to suggest that students were able to more accurately describe these concepts by the completion of the WTL activity. Even though the category of ‘Explained why Ca²⁺ is a Lewis acid’ had a small effect size difference between the initial and final draft, only 38% of the students actually included this explanation in the revised draft. Students were able to identify that the calcium ion was the Lewis acid, but were less likely to be able to describe how the calcium ion acts as a Lewis acid. Table 4 provides an excerpt from a student's writing from this category who made changes on their draft. On initial draft, the student did not explain why the calcium ion would be the interactor and scored a ‘0’ on the rubric (Appendix 2) in the ‘Explained why Ca²⁺ is a Lewis acid’ category but on the revised draft they included an explanation and scored a ‘1’. On their initial draft, the student does not define distinctly what part of the calcium carbonate molecule is acting as a Lewis acid and provides no definition of how the calcium ion acts as a Lewis acid. In their revised draft, the student clearly identifies the calcium ion from the calcium carbonate molecule as the Lewis acid that would interact with the deprotonated levothyroxine molecule and also provides a clear definition of a Lewis acid.

Table 3 Rubric scores for the initial and final draft of the writing assignment. Each score represents the average score from each of the 12 rubric categories. p-Values and effect sizes for the comparison twelve rubric categories between initial and revised drafts. McNemar's test was used to determine the statistical difference between the initial and final drafts. The maximum score in each category was 1. The Bonferroni corrected value was α = 0.004

Rubric category	Initial draft average	Revised draft average	p-Value	Effect size
Defined rønsted–Lowry acid–base theory	0.18	0.30	<0.001	0.28
Defined Lewis acid–base theory	0.34	0.46	<0.001	0.25
Contrasted acid–base theories	0.07	0.09	0.092	0.07
Correctly defined pH/pKa relationship	0.42	0.65	<0.001	0.47
Identified most acidic proton	0.80	0.92	<0.001	0.35
Identified all acidic sites on levothyroxine	0.48	0.71	<0.001	0.48
Explained effect of stomach pH on levothyroxine	0.45	0.62	<0.001	0.35
Explained the theory behind pKa	0.24	0.36	<0.001	0.26
Correctly identified Ca^2+ as interactor	0.71	0.76	0.025	0.11
Identified where/how Ca^2+ interacts	0.29	0.36	0.010	0.15
Explained why Ca^2+ is a Lewis acid	0.27	0.38	<0.001	0.24
Compared carbonate/citrate and interactions with Ca^2+	0.14	0.27	<0.001	0.33

---

Table 4 Example revisions from student writing that resulted in an improved score on the revised draft from rubric criteria (Appendix 2) related to comparison of acid–base theories

Criteria	Initial draft (score 0)	Revised draft (score 1)
Contrasted the acid–base theories	The student did not include any comparison of the two theories in their Initial Draft.	Firstly, acids and bases are defined in two ways: Brønsted–Lowry acid/bases and Lewis acid/bases. You may be more familiar with the Brønsted–Lowry definition, which names any molecule that donates a proton an acid and accepts a proton a base. Lewis acids, however, are molecules that accept an electron pair, and Lewis bases donate these electrons. Both of these types of acids and bases are relevant to your trial of Levothyroxine.
Explained why Ca^2+ is a Lewis acid	In this reaction, calcium carbonate will act as a Lewis acid and accept electrons from the alcohol group attached to the aromatic ring in levothyroxine. Specifically, the carbon double bonded to the oxygen will act as the electrophile and accept electrons from the oxygen atom in the alcohol functional group. The two molecules will undergo a complexation reaction to form a new product containing both levothyroxine and calcium carbonate as one molecule. The calcium ion will still be present and interact with the negatively charged oxygen in Levothyroxine to prevent the drug's absorption.	Levothyroxine will interact with calcium carbonate, which will prevent its absorption. In this reaction, calcium carbonate will first deprotonate the carboxylic acid in Levothyroxine, which will leave a negatively charged carboxylate in Levothyroxine. The calcium ion (Ca ^2+ ) will act as a Lewis acid and accept electrons from the negatively charged carboxylate. This interaction between the calcium ion and Levothyroxine will prevent the drug's absorption
Correctly identified Ca^2+ as the interactor	Next let's take a look at how Ca ^2+ acts as a Lewis acid and how the Ca ^2+ in calcium carbonate can interact with levothyroxine. Because the majority of groups in levothyroxine are acting as Lewis Bases, the Lewis Acid in calcium carbonate deprotonates interacting possibly with the effects of the already deprotonated sodium salt form as the neutral form does not need to be deprotonated any farther.	Next let's take a look at how Ca ^2+ and Levothyroxine interacts. When taken together, there is a Lewis acid–base interaction, where the base will donate an electron pair and the acid will accept an electron pair. In this instance, Ca ^2+ is the Lewis Acid.

---

Two subcategories that were part of the larger Lewis category had non-significant results; these categories were ‘Correctly identified Ca²⁺ as the interactor’ and ‘Identified where/how Ca²⁺ interacts’. These two rubric aspects of the WTL assignment required students to correctly apply both the of Brønsted–Lowry and Lewis acid–base models together in their writing. Most students (71%) correctly identified the calcium ion as being the Lewis acid in their initial draft on the ‘Correct identified Ca²⁺ as the interactor’, which may help explain the lack of a meaningful effect size. However, on the ‘Identified where/how Ca²⁺ interacts’ rubric category, only 36% of the students included this component in their writing. Table 4 presents excerpts from a students’ writing demonstrating how students were unable to connect the two acid–base models on the initial draft (rubric score of 0), but successfully integrated these two models when describing the levothyroxine and calcium ion interactions on the revised draft (rubric score of 1). For example, on the initial draft, the student identified the carbonyl group in the carbonate molecule as acting as the electrophile, and not the deprotonated carboxylic acid group from the levothyroxine molecule. On the revised draft, the student more clearly and succinctly identified how the calcium ion would interact with the deprotonated oxygen group. Together these results indicate that students were less successful in writing about concepts related to Lewis acid–base chemistry. These results are aligned with previous literature that states that students may struggle to apply the principles of Lewis acid–base chemistry to a novel situation (Cartrette and Mayo, 2011; McClary and Bretz, 2012).

The writing prompt was designed to help students compare and contrast these two models of acid–base chemistry; however, students were less able to connect and apply the two models than other aspects of the assignment. This was evident in the comparison between the two drafts for the ‘Contrasted the Acid–Base Theories’ rubric category (p > 0.004, effect size = 0.07), where there was no meaningful difference. Performance on other criteria indicate that, on average, students were able to define and apply the two models of acid–base chemistry separately, but less able to discuss the theories together. Table 4 provides an example of student writing where they directly compared the two models on the revised draft and were scored a ‘1’. Most students did not attempt to compare the two models in their initial draft, which resulted in a ‘0’ score, as was the case in this example. As the example of the students’ writing shows, they were able to convey and connect both acid–base models in the revised draft. However, most students did not succeed in improving this component for the revision.

Overall, the results from the analysis of the WTL assignments indicate that students improved between the initial and revised draft. This improvement could be attributed to the social interactions provided by the WTL peer review and revision process (Zhang et al., 2017). To gain more insight into how students responded and thought of the WTL assignments, the interview transcripts were analysed to see how the students might have incorporated peer review feedback.

Reflective interviews were conducted with three students (referred to as Abbey, Aaron, and Margaret) after they completed all three components (initial draft, peer review, and revision) of the WTL activity. The results from the interviews were used to contextualize the findings from the analysis of the students’ initial and revised drafts to see why students may have been prompted to change their drafts.

Interviewer: Could you tell me, then, how were you thinking about the content when you made the statement that, “Additionally, calcium carbonate is a Lewis acid?”

Margaret: And not just the calcium. Yeah. I think that is something that I may have glazed over, because it says specifically the calcium is the Lewis acid. I think I might have just been thinking of it as the molecule itself. That's probably where I was going wrong with that, but even still thinking about it, maybe I'd have to ask more questions, but I just don't know.

Even after receiving peer reviews and feedback on her initial draft, Margaret still had some confusion about whether the calcium ion or the entire calcium carbonate molecule would act as a Lewis acid. On the other hand, another student, Aaron, was initially confused about how the calcium ion acted as a Lewis acid, but by the completion of the activity he was able to figure out calcium's role.

Interviewer: Did you get stuck at any point while writing?

Aaron: Not that I can think of. Maybe initially when figuring out how calcium actually acts as a Lewis acid, but other than that, no.

Aaron admitted to being initially unsure of how the calcium ion performs as a Lewis acid, but through looking and thinking about it further, had no trouble explaining it on his essay. Additionally, Aaron mentioned how the assignment prompted him to think about how the different acid–base models would work together to help him accurately respond to the prompt.

Aaron: So, it was, like, interesting having to figure out what exactly Levothyroxine does, and then also, why it's important that it's, like … Why its structure is important to what it does, and how that is also affected with it. So, I kind of started with thinking about that, and then also about how, like, the different interactions … Like that acid-based interactions, how those kind of worked together.

Even though the evidence from the analysis of the students’ WTL assignments showed that students had a hard time connecting the two models in their writing, evidence from the interviews show how the assignment at least prompted the students to start to consider how the two models might be connected together. Margaret also expressed how the activity prompted her to think about how the calcium ion, as a Lewis acid, interacted with deprotonated levothyroxine molecule. However, Margaret expressed some hesitance with her thinking about how these two models would interact together.

Interviewer: Could you tell me, then, how were you thinking about the content when you made the statement that, “Additionally, calcium carbonate is a Lewis acid?” It appears that you're looking at the entire molecule as a Lewis acid.

Margaret: I feel like the levothyroxine in its salt form, it's negative. Calcium is positive. The calcium would probably surround that negative charge, cancel it out, or just balance it. Then I don't know what the calcium carbonate's doing, or the carbonate part of it, the actual molecule is doing, and I don't know why that would be any different from calcium citrate, either. Why would the calcium leave this molecule (calcium carbonate) to hang out over here (on the levothyroxine) versus this (calcium carbonate)? I just don't know.

All three students discussed their participation in the peer review process; both as a reviewer and receiver of feedback. In this process, if the students seemed confused about any of the concepts from the prompts, they were able to use the peer reviews to identify areas that they needed to expand upon in their essays. The interviewed students also leveraged their peer review feedback to improve from their initial draft to their final daft. The following quote from Margaret illustrates that the peer review comments she received caused her to double check the pK_a values she included in her initial draft.

Interviewer: Did you notice anything in your peers' writing that you didn't include in yours or that you approached differently?

Margaret: Yeah. Like I said, some of the pK_avalues are a little bit different, so I wanted to go back and check mine over again. Some people said different protons would be removed first. I got mine from a scholarly article, and I believe that myself, too, because I looked up the pK_a. That challenged me a little bit to think about it.

Ultimately the peer review comments did not prompt Margaret to change her essay, however, it did encourage her to reflect on the data she included in her revised draft.

Aaron used the peer review comments to try and polish his essay, especially when he was explaining concepts such as the pK_a of the levothyroxine sites.

Interviewer: Okay. So, how did you go about approaching the revision process just in general?

Aaron: So, I just looked at what the reviewer said and what I could do that could make the letter better. So, I looked it and I was like, okay, I can try making it a bit more professional and then also trying to, a few changes regarding pK_aand things like that, but that was pretty much it.

Though Aaron only made small changes based upon the peer review feedback he received, evidence from the entire analysis of the students’ initial and final drafts of the WTL assignment indicate that many students made substantial enough edits to their initial draft to have an improved revised draft.

Abbey elaborates on how the peer reviews helped improve her essay through making substantial changes to her essay.

Interviewer: Could you tell us how you approached revising?

Abbey: Begrudgingly. I relied a lot on the knowledge I gained through the peer reviews….I took notes based on what I thought I was missing from my draft from the peer reviews and then when I looked at my review comments, more or less, it aligned for the information that was missing. Then I used the peer reviews to try and find ways to break up my paragraphs.

Abbey reported using the peer reviews in a systematic way to make meaningful content and structural changes that improved her revised draft. Since the evidence suggests that the WTL students improved their conceptual understanding of acid–base concepts between drafts from participating in the WTL activity, we investigated whether participation in the WTL activity was related to students’ confidence with acid–base topics.

Do students who complete the WTL assignment improve their confidence when answering acid–base problems?

Finally, we aimed to examine how participation in the WTL assignment contributed to students’ self-reported confidence. First, we investigated to see if there was a difference between students’ average pre and post-questionnaire confidence rankings. The lowest confidence ranking on each question was a 1; the highest was a 5. A multiple linear regression model was used to test to examine what factors contributed to students’ average confidence ranking on the post-questionnaire.

The results of this multiple linear regression model are shown in Table 5. The group variable was significant in predicting the average post-questionnaire confidence ranking. According to the multiple-linear regression model, membership in the WTL group resulted in an average post-questionnaire ranking of 0.202 units greater than the non-WTL group. These results indicate that by participating in the WTL assignment, students increased their confidence when answering questions about organic acid–base chemistry. The interaction term, which was group multiplied by the pre-questionnaire confidence ranking, was not statistically significant; therefore the interaction term was omitted from the final regression model. Additionally, students’ cumulative GPA were not significant factors in predicting students’ average post-confidence, which provides evidence to suggest that participation in a WTL activity can help increase students’ confidence even when accounting for differences in overall academic ability. Because these results indicate that students benefitted from completing WTL assignments, we investigated whether there was a difference in confidence on the each of the individual three questions from the questionnaire.

Table 5 Results of linear regression analysis with bootstrapping to evaluate factors that predict student confidence in post-questionnaire

Variable	Coefficient estimate β (standard error)	Bias	95% confidence interval (lower–upper)	p-Value
Regression parameters: adjusted R-squared = 0.206 and F-value = 46.091.
Constant	1.335 (0.358)	0.009	0.660–2.085	0.001
Group (WTL or non-WTL)	0.202 (0.065)	−0.004	0.066–0.326	0.004
Cumulative GPA	0.171 (0.090)	−0.001	−0.002–0.349	0.058
Average pre-questionnaire confidence ranking	0.442 (0.045)	−0.001	0.354–0.529	0.001

---

Because the students interviewed indicated that they were less confident about their understanding of the Lewis acid–base model, the three confidence components of the three questions were analysed separately also using individual multiple-linear regression models for each question. From these results (Appendix 4), the group variable was found to be a significant predictor for predicting the students’ post-questionnaire confidence ranking for the Rank Acidity Single Compound question; students in the WTL group had greater post-score confidence rankings on this question. The material on Rank Acidity Single Compound question was similar to the types of activities that students responded to on the WTL activity when writing about which site on the levothyroxine molecule was the most acidic and discussing the pK_a values of each site. In their writing, the WTL students performed the highest when writing about the Brønsted–Lowry model, so their increased confidence may have resulted from their familiarity with this type of question. Similarly, in the interviews the students expressed more ease and confidence when talking about pK_a values and their interaction with levothyroxine. Similar to the lack of confidence that the students expressed with the Lewis acid–base concepts in the interviews, the WTL students did not have greater post-score confidence rankings than the non-WTL students on the Rank Lewis Acidity question. These results tell us that while the WTL students improved their ability to write about the Lewis model of acid–base chemistry on their WTL responses, there was no difference in the WTL and non-WTL students’ post-score confidence rankings on this topic. When viewing all of the results about students’ confidence rankings together, there is evidence to support that the WTL activity assisted students in becoming more confident when responding to acid–base conceptual questions.

Limitations

This study used a quasi-experimental design in which students were assigned to treatment or comparison groups according to their section and the term they were enrolled in the course. As this was a quasi-experimental study, the internal validity of the study is not as robust as a study with a true experiment and alternative explanations that caused the observed differences between the treatment and comparison group cannot be completely ruled out. Although the three students who were interviewed after completing the WTL activity didn’t describe substantially different experiences, the claims that can be drawn from the interview data are limited in scope. No students were interviewed from the comparison group about their experiences with the acid–base activity; therefore, less is understood about how the activities differently influenced student learning and is unclear to what degree the different social interactions that were present in the two groups impacted the outcomes of the study. The acid–base questionnaire questions were only one component of a larger questionnaire that students in both the WTL and non-WTL groups completed. The length of the questionnaire could have contributed to students’ fatigue and not responding optimally, especially with regards to their open-ended explanations. Student fatigue in responding to questions may potentially contribute to an underestimate of what both the WTL and non-WTL students’ true understanding of these topics was. Additionally, the response rate on the questionnaire was different between the WTL and non-WTL groups, which could mean that the missing data could have had different effects across the two groups. Another limitation of this study is that there were different instructors for the WTL and non-WTL groups; however, the course curriculum, including the number of assignments and content, were consistent between terms. The WTL intervention was implemented in only one course, so in the future the activity should be replicated in additional organic chemistry courses to provide more evidence to support the conclusions presented in this paper.

Conclusions

Results from this study provide evidence to support the claim that a Writing-to-Learn activity helped students to articulate acid–base chemistry concepts that are relevant to the interaction of levothyroxine with the acidic environment of the stomach. Students in the WTL group had more accurate explanations on the post-questionnaire compared to students in the non-WTL group.

Students in both groups were less successful at providing open-ended explanations to the conceptual question related to Lewis acid–base chemistry. However, the WTL students had higher post-scores than the non-WTL students on the multiple-choice component of the Lewis acid–base question. Students also had more difficulty describing Lewis acid–base concepts on their responses to the WTL assignment prompt, compared to Brønsted–Lowry concepts. It is known from prior literature that students have a harder time understanding Lewis acid–base concepts and our results align with those that have been previously reported (Bhattacharyya, 2006; Cartrette and Mayo, 2011; McClary and Talanquer, 2011; Cooper et al., 2012). A study by Dood et al. (2018) showed that students who use the Lewis acid–base model of reasoning over the Brønsted–Lowry are more successful on exams, so encouraging students to use the Lewis acid–base model in their essays would potentially increase their understanding of those concepts. In the future, modifying this WTL assignment to place additional emphasis on the Lewis acid–base interactions may help further increase students’ understanding of these concepts.

Students were able to successfully define and apply each of the separate models of acid–base chemistry in the context of the prompt. However, the students were less successful at directly comparing and applying the two acid–base concepts, which was one of the learning goals of the WTL assignment. Modifying the WTL activity to help elicit more connections among students when applying the Lewis and Brønsted–Lowry acid–base model in the same assignment, could further enhance the students’ understanding or organic acid–base chemistry.

Findings from this study also provide evidence to suggest that students benefitted from participating in the peer review process; students improved the accuracy of their writing about acid–base concepts, as they scored statistically higher on the revised draft compared to the initial draft on eleven of the twelve categories that were analysed. These findings suggest that the peer review component of the WTL activity helped increase students’ ability to convey their conceptual understanding through their writing, which can be explained through the sociocultural theory of writing (Prior, 2006; Zhang et al., 2017). As students reported in the interviews, the peer review feedback that they received helped them improve their writing between drafts.

Our findings suggest that the WTL assignment increased students’ confidence when responding to acid–base questions. Results from the multiple linear regression model indicated that students who completed the WTL assignments had a higher average post-test confidence ranking than students who did not complete the WTL assignments, even when taking into account the students’ pre-questionnaire confidence rankings along with their cumulative GPA. Additionally, the WTL students had a higher post-confidence ranking on the question related to Brønsted–Lowry acid–base chemistry. However, in the interviews talking about the WTL assignments, students still expressed a lack of confidence when thinking about or discussing concepts related to Lewis acid–base chemistry. These interview results support the lack of an increase in the WTL students’ confidence ranking observed for the Rank Lewis Acidity question on the post-questionnaire. The WTL assignments helped students develop more self-efficacy when it comes to explaining organic acid–base concepts, particularly with Brønsted–Lowry topics.

The findings from the conceptual test questionnaire, WTL student responses, and interviews suggest that WTL activity supported students’ ability to articulate their thinking about acid–base concepts on an external assessment. In addition, the findings demonstrate that students improved their writing about two different models of acid–base chemistry during peer review and revision, which is a new finding that has not been previously explored in a writing intervention in the context of an organic chemistry course. Our findings suggest that students were less successful at making connections and applying both models to the same scenario. The WTL activity was related to students’ confidence when answering questions about acid–base chemistry. This work also adds to the growing body of work on how WTL assignments can contribute to an increase in student conceptual understanding, particularly with topics that are considered traditionally challenging or difficult (Shultz and Gere, 2015; Finkenstaedt-Quinn et al., 2017; Cox et al., 2018; Halim et al., 2018; Moon et al., 2018).

Implications

Using WTL activities offers students an opportunity to review one another's work; students can benefit from reviewing one another's work, and from the individualized feedback they receive. Prior literature has shown the feedback from peers is just as effective as feedback from an expert, such as a course instructor or a teaching assistant (Patchan et al., 2009; Cho and MacArthur, 2010; Cho and MacArthur, 2011). Peer review eases the burden of having one instructor read through each student's essay and provide individual feedback as this responsibility is shifted to the students. The process of the WTL activity can also be used as a way to help increase students’ confidence when answering questions about organic acid–base chemistry (Cox et al., 2018). Future work could involve incorporating other measures of students’ conceptual understanding of acid–base chemistry, such as incorporating already published and complete acid–base concept inventories (McClary and Bretz, 2012), into the evaluation of students’ performance on the WTL assignments. Future iterations of the WTL activity could look at ways to modify the language of prompt to help increase students’ ability to simultaneously and correctly apply the theories of Brønsted–Lowry chemistry and Lewis acid–base methods to the same scenario. Additionally, instructors could use examples of student writing in class for discussion about the more challenging aspects of the assignment.

Conflicts of interest

There are no conflicts to declare.

Appendices

Appendix 1: survey content questions

(1) Select the order that correctly ranks the following compounds from most acidic to least acidic.

How confident are you in your answer? Not at all; somewhat unconfident; Neutral; Confident; Very Confident.

Briefly explain your reasoning for selecting your answer.

(2) Select the order that correctly ranks the indicated protons from most acidic to least acidic.

How confident are you in your answer? Not at all; somewhat unconfident; Neutral; Confident; Very Confident. Briefly explain your reasoning for selecting your answer.

(3) Select the order that correctly ranks the following from strongest Lewis acid to weakest Lewis acid.

How confident are you in your answer? Not at all; somewhat unconfident; Neutral; Confident; Very Confident.

Briefly explain your reasoning for selecting your answer.

Appendix 2: expanded rubric to score student responses to WTL assignment across 12 sub-categories

Broad category	Sub-category	Binary score
		0 – not present or scientifically inaccurate	1 – present and scientifically accurate
Theoretical principles	Defines Brønsted–Lowry acid–base theory	Does not define Brønsted–Lowry acid–base theory or defines Brønsted–Lowry as electron transfer	Defines Brønsted–Lowry acid–base theory as proton donor/acceptor, gain/loss of protons
	Defines Lewis acid–base theory	Defines Lewis acid–base as protons, pKa, deprotonation, etc. Talks about how Lewis acid–base do not form actual bonds	Defines Lewis acid–base theory as electron pair donor/acceptor
	Contrasts Brønsted–Lowry and Lewis acid–base theories	Does not contrast the two theories; says the two acid–base theories are the same and that all Brønsted–Lowry acid–base as Lewis acid–base or vice-versa	Discusses the differences in each of the acid–base definitions- Brønsted acid–base theory focuses on proton donation/acceptance, while Lewis acid–base theory focuses on electron donation/acceptance; usually the definitions of the theories are present in the students’ assignment in close proximity to one another
	Defines pH/pKa relationship	Does not relate pH to pKa; misinterprets pKa and pH relationship; does not mention how either pH or pKa are measured	Correctly defines pH/pKa relationship; answer should note effect of pH > pKa (deprotonated) and/or pH < pKa (protonated), may define pH as [H+], mentions how pKa is a measure of acidity
Brønsted–Lowry application	Identifies the most acidic proton	Identifies any site on Levothyroxine, other than the carboxylic site, as being the most acidic	Identifies the carboxylic acid site on the levothyroxine as being the most acidic
	Identifies all of the acidic sites on levothyroxine	Identifies the number of acidic sites as being more or less than three	Identifies three and only three acidic sites; if pKa values are mentioned in the response it is still acceptable if the values are not completely accurate as long as the lowest pKa value is attributed to the carboxylic acid group and the pKa of the carboxylic acid group is higher (>3.5) than the pH of the stomach acid (1.5–3.5)
	Explains effect of stomach pH on levothyroxine	Discusses the effects of the acidic stomach pH on carbonate or citrate; discusses how one or more sites of the Levothyroxine will be deprotonated under the acidic conditions of the stomach pH	Discusses how due to the pH value <pKa levothyroxine will be protonated at all sites in the acidic conditions of the stomach
	Explains underlying principles of pKa	Does not discuss the underlying principles of pKa or how structure determines pKa	Discusses how electronegativity and/or resonance determine the relative stability (or instability) of the conjugate base that influence the pKa values
Lewis application	Correctly identifies Ca^2+ as the interactor	Discusses how carbonate or citrate interacts with the levothyroxine	Discusses how Ca^2+ interacts with levothyroxine
	Discusses where and how Ca^2+ interacts with the Levothyroxine	Discusses how Ca^2+, as a Lewis acid, deprotonates any molecules and/or atoms and/or suggests that Ca^2+ is acting as a Brønsted–Lowry acid–base	Discusses how Ca^2+ acts as Lewis acid by interacting with the deprotonated oxygen of the carboxylic acid group
	Explains why Ca^2+ is a Lewis acid	Discusses how Ca^2+ acts an electron acceptor	Discusses how Ca^2+ is an electron acceptor due to the 2+ charge
	Compares how carbonate and citrate interact differently with calcium	Student mentions how the steric hindrance of either the carbonate or citrate impacts the interaction of Ca^2+ with levothyroxine, the key is that carbonate or citrate do not really interact with levothyroxine only Ca^2+ does	Discusses the differences in chelating abilities between carbonate and citrate; citrate has more oxygen atoms, thus making citrate more electronegative, thus Ca^2+ has a higher affinity/chelating ability with the citrate over the carbonate; if student mentions sterics that is OK if the sterics are mentioned in addition to how Ca^2+ interacts with Levothyroxine; student discusses the ability of calcium citrate to participate in intra-molecular interactions

Appendix 3: results from the regression models for predicting students’ post-scores on multiple-choice correctness

Table 6.

Table 6 Binomial logistic regression analysis to evaluate factors that predict student correctness on individual post-test questions

Multiple-choice question	Variable	Coefficient β estimate (standard error)	Odds ratio	Chi-squared	p-Value
a p < 0.05. b p < 0.01. c p < 0.001. The regression parameters for the Rank Brønsted acidity model were as follows: Cox & Snell R square = 0.041, Chi-square = 21.620, p-value = 0.000.^a The regression parameters for the Rank Single Compound Brønsted acidity model were as follows: Cox & Snell R square = 0.046, Chi-square = 21.620, p-value = 0.000.^a The regression parameters for the Rank Lewis acidity model were as follows: Cox & Snell R Square = 0.052, Chi-square = 27.804, p-value = 0.000.^a
(1) Rank Brønsted acidity	Constant	0.474 (0.911)	1.606	0.270	0.603
	Group (WTL or non-WTL)	0.154 (0.188)	1.167	0.675	0.411
	Cumulative GPA	−0.287 (0.253)	0.751	1.291	0.256
	Pre-questionnaire multiple-choice correctness	0.812 (0.185)	2.253	19.193	0.000^c
(2) Rank single compound Brønsted acidity	Constant	−3.252 (1.037)	0.039	9.831	0.002^b
	Group (WTL or non-WTL)	0.273 (0.198)	1.313	1.891	0.169
	Cumulative GPA	0.627 (0.287)	1.873	4.779	0.029^a
	Pre-questionnaire multiple-choice correctness	0.938 (0.234)	2.556	16.074	0.000^c
(3) Rank Lewis acidity	Constant	−1.375 (0.991)	0.253	1.923	0.166
	Group (WTL or non-WTL)	0.401 (0.201)	1.494	3.991	0.046^a
	Cumulative GPA	0.030 (0.277)	1.031	0.012	0.913
	Pre-questionnaire multiple-choice correctness	1.006 (0.203)	2.735	24.578	0.000^c

---

Appendix 4: results from the linear regression with a 1000 bootstrap samples analysis to predict students’ post-score confidence on individual post-test questions

Table 7.

Table 7 Results of linear regression analysis with bootstrapping to evaluate factors that predict student confidence ranking on individual post-test questions

Question confidence	Variable	Coefficient β estimate (standard error)	Bias	95% confidence interval lower	95% confidence interval upper	p-Value
a p < 0.01. The regression parameters for the Rank Brønsted acidity model were as follows: adjusted R square = 0.122, F = 24.988, p-value = 0.000. The regression parameters for the Rank Single Compound Brønsted Acidity model were as follows: adjusted R square = 0.089, F = 17.979, p-value = 0.000. The regression parameters for the Rank Lewis acidity model were as follows: adjusted R square = 0.094, F = 19.067, p-value = 0.000.
Rank Brønsted acidity	Constant	1.222 (0.435)	0.006	0.314	2.102	0.004^a
	Group (WTL or non-WTL)	0.087 (0.081)	−0.004	−0.073	0.240	0.286
	Cumulative GPA	0.385 (0.121)	−0.001	0.147	0.624	0.002^a
	Pre-questionnaire confidence ranking	0.295 (0.042)	−0.001	0.213	0.376	0.001^a
Rank single compound Brønsted acidity	Constant	1.789 (0.468)	−0.16	0.917	2.694	0.001^a
	Group (WTL or non-WTL)	0.291 (0.088)	−0.001	0.128	0.470	0.001^a
	Cumulative GPA	0.242 (0.127)	0.004	−0.006	0.495	0.060
	Pre-questionnaire confidence ranking	0.262 (0.045)	0.000	0.177	0.353	0.001^a
Rank Lewis acidity	Constant	1.893 (0.516)	0.034	0.925	2.944	0.001^a
	Group (WTL or non-WTL)	0.188 (0.101)	−0.005	−0.010	0.380	0.056
	Cumulative GPA	0.031 (0.141)	−0.007	−0.251	0.310	0.836
	Pre-questionnaire confidence ranking	0.325 (0.043)	−0.002	0.235	0.409	0.001^a

---

Acknowledgements

We would like to thank the Keck Foundation for generously providing funding for this research. We would like to thank all of the organic chemistry students who participated in this research.

References

1. Anderson T. L. and Bodner G. M., (2008), What can we do about ‘Parker’? A case study of a good student who didn’t ‘get’ organic chemistry, Chem. Educ. Res. Pract., 9, 93–101.
2. Bhattacharyya G., (2006), Practitioner development in organic chemistry: how graduate students conceptualize organic acids, Chem. Educ. Res. Pract., 7, 240–247.
3. Brown C. E., Henry M. L. M. and Hyslop R. M., (2018), Identifying Relevant Acid–Base Topics in the Context of a Prenursing Chemistry Course To Better Align Health-Related Instruction and Assessment, J. Chem. Educ., 95, 6, 920–927.
4. Cartrette D. P. and Mayo P. M., (2011), Students' understanding of acids/bases in organic chemistry contexts, Chem. Educ. Res. Pract., 12, 29–39.
5. Cho K. and MacArthur C., (2010), Student revision with peer and expert reviewing, Learn. Instruct., 20, 328–338.
6. Cho K. and MacArthur C., (2011), Learning by reviewing, J. Educ. Psychol., 103, 73–84.
7. Cho K. and Schunn C. D., (2007), Scaffolded writing and rewriting in the discipline: a web-based reciprocal peer review system, Comput. Educ., 48, 409–426.
8. Cho K., Schunn C. D. and Charney D., (2006), Commenting on writing: typology and perceived helpfulness of comments from novice peer reviewers and subject matter experts, Written Commun., 23, 260–294.
9. Cohen J., (1960), A coefficient of agreement for nominal scales, Educ. Psychol. Meas., 20, 37–46.
10. Cohen J., (1992), A power primer, Psychol. Bull., 112, 155–159.
11. Cooper M. M., Underwood S. M. and Hilley C. Z., (2012), Development and validation of the implicit information from Lewis structures instrument (IILSI): do students connect structures with properties? Chem. Educ. Res. Pract., 13, 195–200.
12. Cooper M. M., Kouyoumdjian H. and Underwood S. M., (2016), Investigating Students’ Reasoning about Acid–Base Reactions, J. Chem. Educ., 93, 1703–1712.
13. Cox C. T., Poehlmann J. S., Ortega C. and Lopez J. C., (2018), Using Writing Assignments as an Intervention to Strengthen Acid–Base Skills, J. Chem. Educ., 95, 8, 1276–1283.
14. Cros D., Maurin M., Amouroux R., Chastrette M., Leber J. and Fayol M., (1986), Conceptions of first-year university students of the constituents of matter and the notions of acids and bases, Eur. J. Sc. Educ., 8, 305–313.
15. Cros D., Chastrette M. and Fayol M., (1988), Conceptions of second year university students of some fundamental notions in chemistry, Int. J. Sci. Educ., 10, 331–336.
16. Dood A. J., Fields K. B. and Raker J. R., (2018), Using Lexical Analysis To Predict Lewis Acid–Base Model Use in Responses to an Acid–Base Proton-Transfer Reaction, J. Chem. Educ., 95, 8, 1267–1275.
17. Drechsler M. and Schmidt H.-J., (2005), Textbooks' and teachers' understanding of acid-base models used in chemistry teaching, Chem. Educ. Res. Pract., 6, 19–35.
18. Duis J. M., (2011), Organic chemistry educators’ perspectives on fundamental concepts and misconceptions: an exploratory study, J. Chem. Educ., 88, 346–350.
19. Finkenstaedt-Quinn S. A., Halim A. S., Chambers T. G., Moon A., Goldman R., Gere A. R. and Shultz G. V., (2017), Investigation of the Influence of a Writing-to-Learn Assignment on Student Understanding of Polymer Properties, J. Chem. Educ., 94, 1610–1617.
20. Finkenstaedt-Quinn S. A., Snyder-White E. P., Connor M. C., Gere A. R. and Shultz G. V., (2019), Characterizing Peer Review Comments and Revision from a Writing-to-Learn Assignment Focused on Lewis Structures, J. Chem. Educ., 96, 227–237.
21. Flynn A. B. and Amellal D. G., (2015), Chemical Information Literacy: pK_a Values. Where Do Students Go Wrong? J. Chem. Educ., 93, 39–45.
22. Fry S. W. and Villagomez A., (2012), Writing to learn: benefits and limitations, Coll. Teach., 60, 170–175.
23. Grove N. P. and Bretz S. L., (2012), A continuum of learning: from rote memorization to meaningful learning in organic chemistry, Chem. Educ. Res. Pract., 13, 201–208.
24. Grove N. P., Hershberger J. W. and Bretz S. L., (2008), Impact of spiral organic curriculum on student attrition and learning, Chem. Educ. Res. Pract., 9, 157–162.
25. Grove N. P., Cooper M. M. and Cox E. L., (2012), Does mechanistic thinking improve student success in organic chemistry?, J. Chem. Educ., 89, 850–853.
26. Halim A. S., Finkenstaedt-Quinn S. A., Olsen L. J., Gere A. R., Shultz G. V. and Pelaez N., (2018), Identifying and Remediating Student Misconceptions in Introductory Biology via Writing-to-Learn Assignments and Peer Review, CBE-Life Sci. Educ., 17, ar28.
27. Koo T. K. and Li M. Y., (2016), A guideline of selecting and reporting intraclass correlation coefficients for reliability research, J. Chiropr. Med., 15, 155–163.
28. Kousathana M., Demerouti M. and Tsaparlis G., (2005), Instructional misconceptions in acid–base equilibria: an analysis from a history and philosophy of science perspective, Sci. Educ., 14, 173–193.
29. Lin J. W. and Chiu M. H., (2007), A case study about teacher's pedagogical content knowledge influencing in students’ mental models in acids and bases, Chem. Educ. J., 9(Serial No. 17), 1–5.
30. Mahn H. and John-Steiner V., (2008), The Gift of Confidence: A Vygotskian View of Emotions, in Learning for Life in the 21st Century, Blackwell Publishing, pp. 46–58.
31. McClary L. M. and Bretz S. L., (2012), Development and assessment of a diagnostic tool to identify organic chemistry students’ alternative conceptions related to acid strength, Int. J. Sci. Educ., 34, 2317–2341.
32. McClary L. and Talanquer V., (2011), College chemistry students' mental models of acids and acid strength, J. Res. Sci. Teach., 48, 396–413.
33. McHugh M. L., (2012), Interrater reliability: the kappa statistic, Biochem. Medica, 22, 276–282.
34. Moon A., Zotos E., Finkenstaedt-Quinn S., Gere A. R. and Shultz G., (2018), Investigation of the role of writing-to-learn in promoting student understanding of light–matter interactions, Chem. Educ. Res. Pract., 19, 807–818.
35. Nakhleh M. B. and Krajcik J. S., (1994), Influence of levels of information as presented by different technologies on students' understanding of acid, base, and pH concepts, J. Res. Sci. Teach., 31, 1077–1096.
36. Orgill M. and Sutherland A., (2008), Undergraduate chemistry students’ perceptions of and misconceptions about buffers and buffer problems, Chem. Educ. Res. Pract., 9, 131–143.
37. Patchan M. M., Charney D. and Schunn C. D., (2009), A validation study of students' end comments: Comparing comments by students, a writing instructor, and a content instructor, J. Writ. Res., 1, 124–152.
38. Prain V. and Hand B., (2016), Coming to know more through and from writing, Educ. Res., 45, 430–434.
39. Prior P., (2006), A sociocultural theory of writing, in Handbook of writing research, pp. 54–66.
40. Reynolds J. A., Thaiss C., Katkin W. and Thompson R. J., (2012), Writing-to-learn in undergraduate science education: a community-based, conceptually driven approach, CBE-Life Sci. Educ., 11, 17–25.
41. Rivard L. O. P., (1994), A review of writing to learn in science: implications for practice and research, J. Res. Sci. Teach., 31, 969–983.
42. Ross B. and Munby H., (1991), Concept mapping and misconceptions: a study of high-school students’ understandings of acids and bases, Int. J. Sci. Educ., 13, 11–23.
43. Shultz G. V. and Gere A. R., (2015), Writing-to-Learn the Nature of Science in the Context of the Lewis Dot Structure Model, J. Chem. Educ., 92, 1325–1329.
44. Stoyanovich C., Gandhi A. and Flynn A. B., (2014), Acid–base learning outcomes for students in an introductory organic chemistry Course, J. Chem. Educ., 92, 220–229.
45. Theobald R. and Freeman S., (2014), Is it the intervention or the students? Using linear regression to control for student characteristics in undergraduate STEM education research, CBE-Life Sci. Educ., 13, 41–48.
46. Visser T., Maaswinkel T., Coenders F. and McKenney S., (2018), Writing Prompts Help Improve Expression of Conceptual Understanding in Chemistry, J. Chem. Educ., 95, 1331–1335.
47. Vygotsky L., (1962), Thought and language, Cambridge, MA, US: MIT Press.
48. Vygotsky L. S., (1980), Mind in society: the development of higher psychological processes, Harvard University Press.
49. Widanksi B. B. and McCarthy W. C., (2009), Assessment of chemistry anxiety in a two-year college, J. Chem. Educ., 86, 1447.
50. Yaman F. and Ayas A., (2015), Assessing changes in high school students' conceptual understanding through concept maps before and after the computer-based predict–observe–explain (CB-POE) tasks on acid–base chemistry at the secondary level, Chem. Educ. Res. Pract., 16, 843–855.
51. Zhang F., Schunn C. D. and Baikadi A., (2017), Charting the routes to revision: an interplay of writing goals, peer comments, and self-reflections from peer reviews, Instruct. Sci., 45, 679–707.

---