Writing a review article: what to do with my literature review

Introduction

Science education and chemistry education articles have proliferated in the last two decades. For researchers new to the field, it can be hard to get an overview about a research area, for example what has been studied and general trends over time. This increases both the importance of, and the demand for, review articles. This type of article is typically based on already published work and is meant to summarize and collate available studies about a research topic. Reviews play an integrative role in synthesizing the body of literature under a thematic umbrella. A reviewing lens on published work can integrate and outline state-of-the-art research in a field, provide a discussion of controversies and inconsistencies in prior research, evaluate existing methodological approaches or possibly propose future research endeavors. Some review articles adopt a more quantitative effects estimation approach, whereas others are more narrative, seeking to synthesize qualitative findings. Not every individual review article can cover all of these potential objectives.

Review articles allow the readers to get a landscape view of a topic, but readers can also use the collection of references cited in a review article to dig deeper into a topic. Thus, they are valuable resources to consult. Well written review articles are often highly cited and could increase the visibility and reputation of the authors.

Decisions to make before starting to write a review article

Before starting to write a review article, it is helpful to clarify whether, especially for researchers early in their careers, review articles count towards their promotion or tenure benchmarks. Depending on the country or requirements, articles reporting original research work can have higher value than review articles for these processes. If tenure requirements are related to citation indices, a review article is probably worth the investment of time. Making this decision about writing a review article or not should be guided by this economic lens, as a well-written and well-researched review article can be very time-consuming. Time might be considered to be precious when just having started an academic position.

It might be tempting to consider adapting a literature review, that is part of an article, proposal or dissertation, into a published review article. Such a literature review can be used as a starting point to build a review article upon. However, a literature review often does not follow the quality criteria of a formal review article or specific types of reviews and therefore should be reworked based on the steps illustrated in this editorial.

Types of review articles suitable for chemistry education research and practice

The denomination of review types can vary depending on the field and on the resources used. This editorial does not encompass all types of review articles that are possible, but we endeavor to list and further explain the main types of review articles. Chemistry Education Research and Practice publishes three manuscript types: (1) original research articles, (2) perspectives and (3) review articles. The latter category includes narrative, integrative or systematic reviews and meta-analyses. Perspectives serve a different purpose than review articles, although like review articles perspectives should also be based on, or discuss, published research. Review articles need to align with the goals and scope of the journal. Thought experiments outlining a theoretical position or personal opinion without including a literature basis, pedagogical recommendations or evidence of implementation are not considered in the journal.

Perspectives

This manuscript type is typically shorter than a normal article and provides a focused analysis of a controversial or emergent topic in our field. In contrast to reviews, perspectives often discuss stances or perspectives which are illustrated by providing a critical view on a research topic or an outline for future avenues in chemistry education research. A good example is Taber's (2014) perspective article on the notion of implicit knowledge in chemistry learning, calling for its recognition in chemistry education research. There is no cookie-cutter approach for an optimal structure of a perspective. It could best be compared to the format of a commentary, common in other journals, but with a strong focus on chemistry education research and practice.

Narrative and integrative reviews

A narrative review follows a more topical approach, with a more general discussion of a subject or with the aim of reinterpreting or highlighting a connection. Narrative reviews often consider qualitative studies and do not per se follow a strict selection process. Choosing publications for a narrative review can be more subjective and unsystematic. The notion of being an “integrative review” highlights a stronger focus on synthesizing and integrating aspects in the review, such as comparing and contrasting research findings or deriving structured implications, see for example Castro-Alonso et al. (2021).

Systematic reviews

Systematic reviews aim to provide a descriptive overview of a body of literature with a clearly-defined research topic/question and a structured search methodology in place to take into account all published studies on a particular topic. All steps should ideally be systematic, including the identification of keywords for the search, the criteria for inclusion and exclusion of publications (e.g. methods, study design), and the information extraction process for the review. Illustrative examples for this type of review include the review by Kahveci (2013), which reports an overview of diagnostic assessment research and instruments exploring particulate nature of matter conceptions and the review by Flaherty (2020) on chemistry education research studies with a focus on the affective domain.

Meta-analyses

Meta-analyses synthesize multiple studies that share a common quantitative evaluation of an intervention or description of a relationship between characteristics. In so doing, meta-analyses describe the average effect and range of effects observed across studies and offer a means to explore how characteristics among the studies can influence the average effects. Some of the steps in presenting a meta-analysis are common to other review articles, including making explicit the rationale for conducting the analysis, the process for the systemic identification of articles and the criteria for screening the articles. In addition, meta-analyses also need to describe how missing data and nested data are handled, the effect observed in each study and the confidence interval associated with each synthesized effect, as well as any investigations into the sensitivity of the results to the methodological choices and potential for bias presented within the sample. The organization Preferred Reporting Items for Systematic Reviews and Meta-Analyses offers a more thorough checklist that may be helpful for presenting a meta-analysis (http://prisma-statement.org/documents/PRISMA_2020_checklist.pdf). Consulting meta-analyses published in the field provides a good starting point (e.g.Rahman and Lewis, 2020; Alfieri et al., 2013; Freeman et al., 2014; Bisra et al., 2018; Theobald et al., 2020).

Appropriate ways of approaching a (systematic) review – writing a review step-by-step

The following steps should be considered to be a short guide about what to have in mind when starting to write a review article. As narrative/integrative reviews and perspectives are more fluid in their approach, these steps are mainly focusing on systematic approaches (i.e. systematic reviews and meta-analyses). Nevertheless, they can guide the writing process for other types of reviews, as well. There are many resources online available to further guide the authors in this process. There are many digital resources available to further guide the authors in this process that are provided in university and publisher websites, for example Taylor and Francis (2021).

Step 1. Topic and research question

When deciding on the topic/aim of a review, a consideration could be to either choose a very mature topic, that is suitable for a holistic conceptualization, or an emergent one. The former perspective can be overwhelming when it is not chosen properly, e.g. writing a review on learning in general chemistry could be an endless endeavor. An emergent topic could be one that has received a lot of interest and is increasingly important in the literature. For both topic choices, it is important to evaluate whether the selected topic offers new insights and research directions, which means the review has a scope that ensures sufficient breadth and depth and, foremost, is of interest to a larger audience.

After the topic is chosen, it may be helpful to narrow the review down to a clear aim or question that the review seeks to answer. This helps to facilitate the selection of the publications to be reviewed. In addition to the topic, the author should seek to clarify the aim of the review by identifying the likely audience for such a review and how these individuals would benefit from this particular review. Review articles should explicitly mention the nature and scope of the intended review, as well as making a case for who would benefit from the review and how they would benefit.

Step 2. Determine the search and selection criteria

Searching for articles that fit the topic or research question of the review is the key for review articles. Prior to consulting a search engine, keywords and keyword combinations should be defined and included in the review article's methods section. Commonly used search engines include Google Scholar, PubMed, Web of Science, ProQuest and Scopus. Authors will need to set boundaries for the time period covered and the scope of journals included.

Step 3. Inclusion or exclusion of publications

As reviews are highly read and cited, the process of filtering articles to include in the review requires a systematic approach to avoid a subjective bias or individual opinions in the presentation process. Review articles should therefore openly discuss the process of search, selection and the inclusion/exclusion criteria employed that led to the final selection of articles in their methods. Reproducibility can be a guiding principle for this section.

Step 4. Synthesis of results

Readers of a review article appreciate it if the authors provide a clear outline, which is related to the search and selection criteria defined in the previous step. Additionally, when writing an integrative review, presenting critical aspects, discussing a controversy, identifying patterns and gaps or providing ideas for future investigations are often valued by readers. While discussing the chosen articles, readers should not get the impression that the authors were ‘cherry-picking’ certain articles over others or implicitly promoting their own opinion.

Step 5. Check for clarity and bias

The authors should be aware of their impact, as the way that publications are presented and discussed can have an impact on how these cited articles are perceived and interpreted, e.g. meta-analyses can influence study considerations by laying out possible future research directions. They may also paint a picture of how effective/ineffective certain approaches are. Seeking feedback from critical friends or asking colleagues in the field to evaluate review drafts can ensure the quality and wider impact.

How should review articles be cited as a reference in CERP manuscripts?

For those relying on review articles when developing future work, the question might arise whether to cite a review article or the original research in a manuscript. Foremost, review articles provide an overview of a research topic, but they are not per se a substitute for reading original papers, especially when only certain aspects of a review article are interesting. When citing a review article and original work in a manuscript, we encourage authors to make a clear difference between citing original work and review articles. If citing a conclusion made in a review article or illustrating the importance of the topic is desired, it is sufficient to cite the review, and not the original research, e.g. indicating the review as (review: in-text reference). However, when referencing results or observations made in one of the articles cited in a review, then the original work should be cited as the primary source and ideally read beforehand to confirm that it has been accurately considered in the review. If the purpose is highlighting conclusions that have been drawn in a review article on original work, this idea can be introduced by using sentences such as, “As reviewed by (authors of the review article), work by (authors of original work) showed promising avenues.”

Reflections on the impact of a review article

Sometimes, the sheer number of publications can be overwhelming. Review articles are, in this respect, a powerful tool to provide clarity and an overview that allows easy access to a research field, especially for young researchers and graduate students or experienced researchers starting a new line of inquiry. Therefore, authoring reviews bears a certain responsibility in terms of how published work is reviewed and discussed. A lack of critical appraisal or systematic approach can propagate erroneous conclusions. Well-synthesized findings, for example of meta-analyses, often guide future studies and might thus influence the research in a field.

One can raise the question of whether a review article is actually supportive or harmful for the original articles included (Ketcham and Crawford, 2007). Authors tend to cite a review article more often compared to original work, thus lowering the number of citations for the respective articles. However, on the other hand, if studies are included and discussed in a review, readers who would like to learn more or access the original perspectives tend to download, read and possibly cite them as well. The benefits of publishing review articles clearly outweigh any potential shortcomings, and their scarcity in the field of chemistry education opens up a venue for publication calls.

References

1. Alfieri L., Nokes-Malach T. J. and Schunn C. D., (2013), Learning Through Case Comparisons: A Meta-Analytic Review, Educ. Psychol., 48, 87–113.
2. Bisra K., Liu Q., Nesbit J. C., Salimi F. and Winne P. H., (2018), Inducing Self-Explanation: a Meta-Analysis, Educ. Psychol. Rev., 30, 703–725.
3. Castro-Alonso J. C., de Koning B. B., Fiorella L. and Paas F., (2021), Five Strategies for Optimizing Instructional Materials: Instructor- and Learner-Managed Cognitive Load, Educ. Psychol. Rev.,  DOI:10.1007/s10648-021-09606-9.
4. Flaherty A. A., (2020), A review of affective chemistry education research and its implications for future research, Chem. Educ. Res. Pract., 21, 698–713.
5. Freeman S., Eddy S. L., McDonough M., Smith M. K., Okoroafor N., Jordt H. and Wenderoth M. P., (2014), Active learning increases student performance in science, engineering, and mathematics, Proc. Natl. Acad. Sci. U. S. A., 111, 8410–8415.
6. Kahveci A., (2013), in Tsaparlis G. and Sevian H. (ed.), Concepts of Matter in Science Education, Dordrecht: Springer, pp. 249–278.
7. Ketcham C. M. and Crawford J. M., (2007), The impact of review articles, Lab. Invest., 87, 1174–1185.
8. Rahman M. T. and Lewis S. E., (2020), Evaluating the evidence base for evidence-based instructional practices in chemistry through meta-analysis, J. Res. Sci. Teach., 57, 765–693.
9. Taber K. S., (2014), The significance of implicit knowledge for learning and teaching chemistry, Chem. Educ. Res. Pract., 15, 447–461.
10. Taylor and Francis, (2021), What is a review article?, https://authorservices.taylorandfrancis.com/publishing-your-research/writing-your-paper/how-to-write-review-article/.
11. Theobald E. J., Hill M. J., Tran E., Agrawal S., Arroyo E. N., Behling S., Chambwe N., Cintrón D. L., Cooper J. D., Dunster G., Grummer J. A., Hennessey K., Hsiao J., Iranon N., Jones L., Jordt H., Keller M., Lacey M. E., Littlefield C. E., Lowe A., Newman S., Okolo V., Olroyd S., Peecook B. R., Pickett S. B., Slager D. L., Caviedes-Solis I. W., Stanchak K. E., Sundaravardan V., Valdebenito C., Williams C. R., Zinsli K. and Freeman S., (2020), Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math, Proc. Natl. Acad. Sci. U. S. A., 117, 6476–6483.

---