How do we know when students are learning? Shining a light on chemistry education practitioner research articles

The nuance in our journal title, Chemistry Education Research and Practice (CERP), continues to present a challenge to authors when deciding whether or not to submit their research article for publication. Previous Editors of this journal have published detailed explanations defining the scope of our journal, also acknowledging that the word ‘Practice’ in the journal title can cause confusion (Taber, 2016; Seery, 2019). While their advice and insights were clearly communicated and remain current, it appears that uncertainty about what represents ‘research’ of chemistry education practice persists amongst authors. I am taking this opportunity to both amplify my predecessors’ guidance and confirm that CERP accepts classroom practice-based studies for publication. However, all articles submitted to the journal should aim to meet the journal's criteria for publication. We value the contributions of teacher researchers (also known as practitioner researchers) as part of our CERP community. Their perspectives and contextual insights complement the perspectives of those who study educational practice but are external to the classroom context. Advances in our understanding of chemistry education rely on the combination of multiple research perspectives.

Many of the submitted manuscripts that we receive describe good practice but are rejected at the pre-screening stage – this is no doubt disappointing for the authors when the idea or activity reported shows strong potential to be effective. The reason for rejection is usually due to minimal evaluation of whether the proposed effective teaching practice has impacted student learning through the implementation. The scope of our journal is focused on examining pedagogies, practices and student learning through the collection of research and evaluation data.

The intent of this editorial is therefore an attempt to make the criteria for publication of acceptable practice-based research papers more explicit and support authors in their writing. In the same way that a teacher might consider using question prompts to scaffold their students’ navigation and understanding of a challenging concept, I felt it might be useful to do the same to guide authors in their evaluation of their manuscript for submission to CERP. As part of this process, it is critical to recognise the complexity of classroom-based practice research which inherently presents associated hurdles in communicating the transferability and generalisability of research findings. Real teaching contexts tend to be organic, dynamic and messy, involving diverse learners which can influence the quality of data collected.

Duality as teacher and researcher

Teachers, as education practitioner researchers, are deeply connected to their own context and their students – they seek to disseminate findings that emerge from changes in practice or interventions. These innovative practices are often introduced based on an assumption that there is likely to be some form of positive impact on student learning (it would be unethical to introduce change that is anticipated to have a negative impact on learning). Our CERP audience engages with published research studies to connect any findings to its own teaching contexts, practices and/or research. There needs to be sufficient rigor in the communication of practitioner research to facilitate adoption, adaptation and transfer into other contexts.

Practitioner researchers possess a deep knowledge of their own students and are likely to make observations influenced by their epistemic values, beliefs and experiences in student learning. Their pedagogical content knowledge is dependent on their professional development and their stage of career – their teaching practice is likely to be inherently reflective. It is probable that these attributes drew them into conducting research, but they can also introduce subjectivity and potential bias into their approaches to classroom research. For this reason, a research article submitted to CERP should include a rich description of the relationship between the teacher, the students, the study context and processes of data collection to make this nexus between practice and research explicit.

Teachers often also partner with other teachers in their research or collaborate with academics who are external to the teaching and learning setting. Collaborative research into practice offers many advantages in terms of the clarity of processes and methodological approaches. To highlight the considerations that may arise, and in some cases potentially hinder publication of this genre of research, I have summarised three possible practice-oriented educational research scenarios.

Scenario 1: a classroom teacher adopting good teaching practice in their own classroom and collecting evaluation data

A teacher decides to investigate how a new approach to teaching or pedagogical intervention impacts on their students’ learning. This simplest scenario comprises the teacher, their students, the learning environment, and some form of assessment of learning. The teacher may consult chemistry education literature to inform their instructional innovation and implementation of pedagogies or tools as part of good teaching practice. The evaluation data that they collect might include tests of students’ knowledge, in situ observations of student approaches to learning and examples of student work. This scenario can be described as scholarly teaching or practical inquiry (Richardson, 1994) and often involves the adaptation or transfer of an idea or intervention sourced from another teacher's practice. Any positive outcomes for student learning in this type of study are usually celebrated in terms of the teacher receiving teaching excellence awards, being recognised as a leader of change and encouraged to share their practice more widely. Indeed, diffusion of good practice is highly valued and encouraged through CERP (Seery, 2019).

However, when a teacher decides to try and publish the findings of their evaluation, they may encounter many barriers that arise from ethical considerations and the perceived transparency of the processes of data collection and analysis. The transactional distance between the researcher (as the teacher) and their student participants is very close, hence there is a risk of perceived bias (in terms of a desire to see the initiative succeed) or potential for power dynamics to exist (students might respond in a way that they believe is expected by the teacher). The practitioner researcher should be mindful of these risks and clearly describe the safeguards that they have applied to ensure neutrality, particularly for studies in which they taught both the control and the experimental group. Further, the methods of data collection in this scenario also need to be formally endorsed by a local ethics committee or other designated ‘gatekeepers’ such as the State Department of Education or School Principal. The quality of this type of study can be increased by collaborating with other teachers who can act as independent observers or collect data on behalf of the practitioner researcher.

Scenario 2: a group of teachers collaborating in research that involves multiple classes in the study context

The collaboration between multiple teachers who assume roles as either practitioners or as practitioner researchers offers greater transparency in terms of the process of data collection through development of shared understanding and protocols. While the same ethical considerations apply as in Scenario 1, there is now the potential for observations to be made indirectly by practitioner researchers who are not teaching the class. Collaboration enables greater neutrality and control of variables by increasing the transactional distance between the teacher as the researcher and their students.

Scenario 3: teachers as formal researchers collaborating with academic researchers who are outsiders in the study context

This scenario has become increasingly common and can, in part, be attributed to growth in the professional development of teachers in practitioner-led inquiry (Cochran-Smith and Lytle, 2015). The voice of teachers as researchers has been long-established (Richardson, 1994) including the affordance of formal partnerships with academic education researchers who are operating in a tertiary institution or college of education. They will contribute to guidance in the ethical procedures, sampling, data analysis and reporting of findings. A recent study, applying the theoretical framework of the concept of quality in research, considered the different lenses that teacher–researchers and academic–researchers bring to the quality of practice-oriented education research (Groothuijsen et al, 2020). These authors recognised that teacher–researchers and academic–researchers derive from different worlds of practice and hence bring different perspectives, beliefs, and values to their partnerships.

Practitioner research is an important aspect of chemistry education research

The nature of practitioner research has attracted significant attention because different stakeholders seek to define what represents ‘quality’ in this genre, including how it intersects with traditional academic research. Concerns about the quality of practice-oriented research that arise include: applicability, consistency, neutrality, cumulativity, recognisability, effectivity and truth value (Groothuijsen et al., 2020). The development of an orientation framework that seeks to increase the rigor and quality of practitioner research involves ethical protocols, transparent processes, collaboration, a receptive audience and transformation based on findings (Mockler, 2014).

As part of establishing the motivation and rationale for a study, formal education research draws on a range of research paradigms, theoretical stances and conceptual frameworks. Practice-oriented education research tends to be framed towards either interpretivist/constructivist or pragmatic research paradigms (Creswell, 2003; Morgan, 2007). Further, connections are frequently drawn between practitioner research and either action research where the teacher is also the researcher, or design-based research where there is a collaboration between teachers and researchers (Anderson and Shattuck, 2012). Each of these methodological frameworks requires cycles of evaluation, reflection and iteration to generate high quality data that can inform future practice or advance theoretical positions. The practitioner-researcher should establish the paradigm that they are working within, their methodology and then consider whether the data that they have collected provides evidence or new insights into student learning.

Why a rich description of the research context is recommended

Authors who are practitioner researchers often fail to provide sufficient information in the description of their research context in their manuscripts. In the absence of appropriate detail, their findings are more likely to be viewed as irrelevant by readers who cannot connect these into their own practitioner contexts. Teachers are only likely to engage with education research and subsequently embed outcomes and initiatives in their own practice when that research identifies problems that they have experienced in their own practice and context.

This aspect of research is known as ecological validity and is an important consideration that applies to research conducted in authentic teaching environments (as a contrast to experimental or quasi-experimental conditions) where findings become part of normal practice (Black and Wiliam, 1998). Vanderlinde and van Braak (2010) have observed that teachers need specific information about the elements of an intervention, evidence of student engagement and measurements of successful learning. Recommendations for practice and solutions to problems in research articles need to align with their epistemological beliefs and experiences as teachers.

Several question prompts provided in Fig. 1 are formulated to encourage authors to reflect on the quality of their research studies and to elicit a richer description of the context and methods that were applied by the researchers in a manuscript. The provision of greater depth in information can increase the potential for findings to become sustained as part of authentic teaching practice.

Fig. 1 A scaffold of question prompts that guide practitioner researcher authors to reflect on the detail that should be included in their manuscript.

Checking the fit for practice-based research for submission to CERP

The consideration of the different facets of practitioner-led research in this Editorial is intended to encourage reflection by authors prior to submission of their manuscript. Our editorial team are disappointed that we must return manuscripts to authors when there is insufficient transparency in the communication of the research methods, or the data does not respond to the research question posed. One of the most common issues with the quality of data is that researchers have only relied on asking students whether they have enjoyed a new approach to teaching and learning. Emotional engagement is just one dimension that can influence student learning – effective learning is sometimes difficult and not always enjoyable. Combining emotional engagement with additional measures of student learning outcomes will assist readers to consider the effectiveness of their practice.

Only for a limited audience

One final comment regarding intended audience, we receive many submissions where the findings of classroom-practice research are discussed to recommend proposed actions targeting local education authorities within a specified context, such as a particular country. If the aim of the research has been an intention to influence local curricula and policies, then the manuscript is not likely to be a good fit for informing the readership of CERP. These authors are advised to either submit to their national education journal or consider reframing their findings in terms of critical appraisal and discussion of (mis)alignment with international practices, informed by chemistry education research literature.

Returning to the original position of this editorial, the genre of chemistry education practitioner research is highly valued by our CERP audience who represent chemistry education researchers, practitioner researchers and teachers in all sectors. We look forward to, and encourage, more submissions that promote constructive discourse and debate in our field through high quality evaluation of good teaching practice leading to new insights into student learning and improvements in practice.

Acknowledgements

I would like to thank David Treagust, current Chair of our Editorial Board, for his comments and careful reading of this Editorial article.

References

1. Anderson T. and Shattuck J., (2012), Design-based research: A decade of progress in education research? Educ. Res., 41(1), 16–25.
2. Black P. and Wiliam D. (1998), Assessment and classroom learning, Assess. Educ.: Princip., Pol. Pract., 5(1), 7–74.
3. Cochran-Smith M. and Lytle S. L., (2015), Inquiry as stance: Practitioner research for the next generation, Teachers College Press.
4. Creswell J. W., (2003), Research design: Qualitative, quantitative, and mixed methods approaches, 2nd edn, Thousand Oaks: Sage.
5. Groothuijsen S. E. A., Bronkhorst L. H., Prins G. T. and Kuiper W., (2020), Teacher-researchers’ quality concerns for practice-oriented educational research, Res. Pap. Educ., 35(6), 766–787.
6. Mockler N., (2014), When ‘research ethics’ become ‘everyday ethics’: The intersection of inquiry and practice in practitioner research, Educ. Action Res., 22(2), 146–158 DOI:10.1080/09650792.2013.856771.
7. Morgan D. L., (2007), Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods, J. Mixed Methods Res., 1(1), 48–76.
8. Richardson V., (1994), Conducting research on practice, Educ. Res., 23(5), 5–10.
9. Seery M. K., (2019), Influencing the practice of chemistry education, Chem. Educ. Res. Pract., 20(3), 448–451.
10. Taber K. S., (2016), What is wrong with ‘practice’ papers, Chem. Educ. Res. Pract., 17(4), 639–645.
11. Vanderlinde R. and van Braak J., (2010), The gap between educational research and practice: Views of teachers, school leaders, intermediaries and researchers, Br. Educ. Res. J., 36(2), 299–316.

---