Jon-Marc G.
Rodriguez†
*a,
Jocelyn Elizabeth
Nardo†
b,
Solaire A.
Finkenstaedt-Quinn†
c and
Field M.
Watts†
a
aDepartment of Chemistry & Biochemistry, University of Wisconsin – Milwaukee, Milwaukee, WI 53211, USA. E-mail: rodrigjg@uwm.edu; wattsf@uwm.edu
bDepartment of Chemistry & Biochemistry, The Ohio State University, Columbus, OH 43210, USA. E-mail: nardo.11@osu.edu
cDepartment of Chemistry, University of Michigan, Ann Arbor, Michigan 48109-1055, USA. E-mail: quinnsa@umich.edu
First published on 1st September 2023
Extant literature has emphasized the importance of education research being theory-based. To this end, many research articles have a distinct “theoretical framework” section describing the theoretical underpinnings that inform the research. Nevertheless, there is large variation in how explicit articles are regarding their use of frameworks in the research process. This work describes a literature review focusing on the use of frameworks (broadly defined) in chemistry education research. Our sample draws on research articles published in Chemistry Education Research and Practice and the Journal of Chemical Education from 2018 to 2021 (n = 457). The longitudinal analysis revealed general trends about the presence of frameworks in research articles over four years as well as the types of frameworks commonly used. In addition, we analyzed how frameworks were used within individual research articles published in 2021, focusing on chemistry education research articles and research articles published across biology, engineering, mathematics, and physics education research journals (n = 595). Our goal is to describe how frameworks were used to open a dialogue and inform future chemistry education research.
“…a system of ideas, aims, goals, theories and assumptions about knowledge; about how research should be carried out; and about how research should be reported that influences what kind of experiments can be carried out and the type of data that result from these experiments” (p. v, “Prologue”).
One of the key points we would like to draw attention to is that theoretical frameworks should influence interpretive and methodological decisions throughout the research process. This is highlighted in Bodner and Orgill's (2007) book, where each framework is described along with implications for analysis and interpretation processes and techniques. That said, theoretical frameworks are important because different theoretical frameworks may lead to different approaches toward data interpretation, with implications for methodological choices, conclusions reached, and the resulting suggestions made to researchers and practitioners. This has been illustrated across DBER, with various studies applying different theoretical frameworks to the same dataset, illustrating how different frameworks lead to different predictions, explanations, and inferences related to students’ reasoning (e.g., Elby, 2000; Southerland et al., 2001; Harrer et al., 2013; Gouvea and Simon, 2018; Lira and Gardner, 2020; Rodriguez and Towns, 2021). In this sense, we can view a theoretical framework as a model that we use to explain and predict phenomena, recognizing that often there are multiple models to describe the same phenomenon, such as different perspectives related to the nature and structure of knowledge (Elby, 2000; Rodriguez and Towns, 2021). Thus, criteria related to robust scientific models [e.g., consistency with data, explanatory and predictive power, etc.; (Passmore et al., 2016)] can and should be applied to how we think about theoretical frameworks.
With the various ways theoretical frameworks can inform a study, it is important to consider the research goals when selecting a framework to ensure alignment. To provide researchers with guidance in selecting an appropriate theoretical framework, Bodner and Orgill (2007) broadly grouped frameworks thematically into three categories: constructivist frameworks that emphasize the continuous and active development of concepts based on individual and collective observations (e.g., constructivism and social constructivism, symbolic interactionism, models and modelling, pedagogical content knowledge); hermeneutic frameworks that emphasize iterative and cyclic analysis of the relationship between part of a text (broadly defined) and the whole text to generate meaning, often related to understanding human experiences (e.g., hermeneutics, phenomenology, phenomenography, action research, ethnology and ethnomethodology, situated cognition, communities of practice, narrative analysis); and critical frameworks that emphasize social structures and concerns such as the uneven distribution of power (e.g., critical theory, feminism, Afrocentricity). This organization of theoretical frameworks is not exhaustive, and there may be some overlap of frameworks across categories, as well as potentially productive alternative approaches toward classiyfing frameworks; nevertheless, the grouping based on Bodner and Orgill (2007) provides a useful starting point for characterizing theoretical frameworks.
The three categories of frameworks presented in Bodner and Orgill (2007) also generally align with three paradigms which often guide different education research studies: positivist (and post-positivist), interpretivist, and critical paradigms (Treagust et al., 2014). For instance, studies using constructivist frameworks tend to focus on objectively characterising the nature of students’ content knowledge or conceptual learning, aligning with the (post-)positivist paradigm which is characterised by seeking scientific objectivity and using experimental designs to develop causal explanations. In contrast, studies using hermeneutic frameworks are more concerned with how students and teachers experience the learning environment, which reflects the values of the interpretivist paradigm to focus on the situational and contextual nature of human experience. Lastly, studies using critical theory frameworks are aligned with the critical paradigm in that they share a focus on equity and challenging power dynamics in learning contexts. However, the alignment between frameworks and paradigms is complex, as exemplified when considering constructivism which can range from personal to critical constructivism (Bodner and Orgill, 2007). Further, as paradigmatic perspectives are often unstated in research articles, it is important to recognize that any alignment suggested is based on our understanding of how different frameworks and paradigms are described in the literature.
Within DBER, researchers also use frameworks that are not necessarily theoretical in nature [e.g., analytical frameworks (Luft et al., 2022; Magana, 2022)]. While there is a distinction between theoretical frameworks and other types of frameworks that may not be theoretical, other types of frameworks can still be beneficial for guiding studies and vary in purpose and intended use. From an engineering education perspective, Magana (2022) identifies the purpose of theoretical and conceptual frameworks as “defining, grounding, and explaining the focus of a study”; methodological and analytical frameworks “for planning and executing the methods of a study”; and, lastly, instructional design, pedagogical, and evaluation frameworks “for planning, delivering, and evaluating instruction.” These different types of frameworks are seen in chemistry as well. For example, the Anchoring Concepts Content Map (ACCM) released by the American Chemical Society – Exams Institute describes the content covered in an undergraduate chemistry curriculum (Murphy et al., 2012). This framework, developed by content experts, reflects the goals and organization of content to guide curriculum and assessment and can be viewed as belonging to the category of instructional design, pedagogical, and evaluation frameworks (Magana, 2022). Importantly, the ACCM is not theoretical in nature; that is, it would provide little interpretive, predictive, or explanatory power in a study. Nevertheless, the ACCM can still be used as part of data analysis; for example, deductive coding using the ACCM can be a productive approach to develop themes related to content coverage within studies in a systematic review (Bain and Towns, 2016; Hunter et al., 2022). Similarly, the chemistry triplet has been used extensively within CER to guide data analysis (Johnstone, 1982), and although it can provide insight regarding what makes chemistry challenging, it not inherently theoretical. To add to the complexity of framework use in CER, multiple frameworks may be compatible and useful for a single study and different types of frameworks may be used in tandem to shape an investigation. Furthermore, different types of frameworks (e.g., theoretical, conceptual, analytical, methodological) are often conceptualized in different ways with overlapping meanings, leading to challenges with differentiating between types of frameworks. Therefore, for this review, it is necessary to focus on frameworks broadly, and frameworks (without any modifier) will be used throughout this work to encompass all types of frameworks encountered in studies.
Transformative change also requires interactions across disciplinary communities. Exploring framework use in CER and across DBER fields can facilitate change in STEM higher education, with frameworks specifically having the ability to connect multiple domains of knowledge that enable needed connection amongst the currently siloed STEM higher education landscape (Reinholz and Andrews, 2019). CER often draws frameworks from other communities, such as looking at mathematics education research for ways to investigate how students use mathematics in the context of chemistry (Bain et al., 2019). Therefore, it is more than shared disciplinary skills, language, and concepts that connect DBER communities. We are connected by theories and frameworks related to concerns such as how students learn and how to promote conceptual change. To this end, we are motivated by the broader goal of connecting communities of practice by using frameworks to facilitate the flow of knowledge across STEM disciplinary domains.
(1) For CER articles published from 2018 to 2021 (n = 457), what trends emerge related to the presence of frameworks (a) by year and (b) methods used?
(2) For DBER articles published in 2021 (n = 595), what trends emerge related to the (a) presence of frameworks and (b) use of frameworks across each article?
(3) What types of frameworks were used in CER articles published from 2018 to 2021 (n = 457)?
In addition, we would like to acknowledge that we are authors on 28 research articles in our sample. We believe our experiences successfully publishing within and outside the field of CER helped us consider what would be meaningful to share with researchers regarding the use of theory and frameworks. To account for potential bias, our thematic analysis consisted of independent and group coding until consensus was reached and all discrepancies were resolved through discussion. Additionally, we sought to draw from supporting texts to determine framework types (e.g., Bodner and Orgill, 2007). We do not believe we are authorities to dictate how researchers should or should not use frameworks. That said, the goal of this work is not to evaluate the quality of individual research articles. We echo the sentiment by Ashwin (2012) that a published study can be viewed as the shared product of a community, produced through a social process involving peer-review and context-specific group norms. Thus, we believe it would not be productive to critique features of individual studies.
The final sample involved articles collected from two journals for each community (except for physics). Across each field, the journals selected were: Biochemistry and Molecular Biology Education and CBE-Life Sciences Education (biology); the International Journal of Engineering Education and the Journal of Engineering Education (engineering); Educational Studies in Mathematics and the Journal for Research in Mathematics Education (mathematics); and Physical Review Physics Education Research (physics). Including the CER articles published in 2021 in CERP and JCE (i.e., a subset of the CER sample discussed previously), our final sample of DBER articles was n = 595. Provided in Table 1, we have additional information regarding the DBER sample, including the journal author guidelines regarding framework use. For the remainder of the article, we use the journal abbreviations provided in Table 1.
DBER field | Journal | Journal description | Author guidelines |
---|---|---|---|
Biology | Biochemistry and Molecular Biology Education | “Biochemistry and Molecular Biology Education is an international journal aimed to enhance teacher preparation and student learning in Biochemistry, Molecular Biology, and related sciences such as Biophysics and Cell Biology, by promoting the world-wide dissemination of educational materials,” (BAMBED, 2023) | No discussion of frameworks or theory in journal author guidelines (BAMBED Author Guidelines, 2023) |
N = 128 | Abbreviation: BAMBED | ||
Impact factor: 1.160 | |||
N = 60 | |||
CBE-Life Sciences Education | “LSE is written by and to serve professionals engaged in biology teaching in all environments, including faculty at large research universities who teach but do not view teaching as their primary mission, as well as those whose teaching is the major focus of their careers, in primarily undergraduate institutions, museums and outreach programs, junior and community colleges, and K–12 schools,” (LSE, 2023) | “Authors of this type of article are encouraged to draw from the diverse social science theories, methods, and findings to inform their work, and to clearly define terms and approaches that may be unfamiliar to a biologist audience,” (LSE Author Guidelines, 2023) | |
Abbreviation: LSE | |||
Impact factor: 3.365 | |||
N = 68 | |||
Chemistry | Chemistry Education Research and Practice | “Chemistry Education Research and Practice (CERP) is the journal for teachers, researchers and other practitioners at all levels of chemistry education. … Coverage includes the following: research, and reviews of research, in chemistry education; evaluations of effective innovative practice in the teaching of chemistry; in-depth analyses of issues of direct relevance to chemistry education,” (CERP, 2023) | “the studies reported should have all features of scholarship in chemistry education, that is they must be … theory-based … It is highly desirable that such contributions should be demonstrably based, wherever possible, on established educational theory and results,” (CERP Author Guidelines, 2023) |
N = 116 | Abbreviation: CERP | ||
Impact factor: 1.902 | |||
N = 65 | |||
Journal of Chemical Education | “The Journal of Chemical Education publishes peer-reviewed articles and related information as a resource to those in the field of chemical education and to those institutions that serve them. The journal typically addresses chemical content, laboratory experiments, instructional methods, and pedagogies. JCE serves as a means of communication among people across the world who are interested in the teaching and learning of chemistry,” (JCE, 2023) | “Researchers use particular theoretical and methodological frameworks to inform their work. Discussion of these frameworks and why they are informative is very helpful to readers. Further, these frameworks should be used to guide the analysis of data and interpretation of results or findings,” (JCE Author Guidelines, 2023) | |
Abbreviation: JCE | |||
Impact factor: 1.385 | |||
N = 51 | |||
Engineering | International Journal of Engineering Education | “The International Journal of Engineering Education (IJEE) is an independent, peer-reviewed journal. It has been serving as an international archival forum of scholarly research related to engineering education for over thirty years,” (IJEE, 2023) | No discussion of frameworks or theory in journal author guidelines (IJEE Author Guidelines, 2023) |
N = 173 | Abbreviation: IJEE | ||
Impact factor: 1.280 | |||
N = 131 | |||
Journal of Engineering Education | “The Journal of Engineering Education is more than a place to publish papers—it is a vital partner in the global community of stakeholders dedicated to advancing research in engineering education from pre-college to post-graduate professional education. The Journal of Engineering Education seeks to help define and shape a body of knowledge derived from scholarly research that leads to timely and significant improvements in engineering education worldwide. The Journal of Engineering Education serves to cultivate, disseminate, and archive scholarly research in engineering education, “ (JEE, 2023) | “Aspects of validity, reliability, and trustworthiness are taken into consideration, including but not limited to instrument and protocol development, data collection, handling of the data and interpretation of theory,” (JEE Author Guidelines, 2023) | |
Abbreviation: JEE | |||
Impact factor: 1.569 | |||
N = 42 | |||
Mathematics | Educational Studies in Mathematics | “Educational Studies in Mathematics presents new ideas and developments of major importance to those working in the field of mathematics education. It seeks to reflect both the variety of research concerns within this field and the range of methods used to study them. It deals with methodological, pedagogical/didactical, political and socio-cultural aspects of teaching and learning of mathematics, rather than with specific programmes for teaching mathematics. Within this range, Educational Studies in Mathematics is open to all research approaches,” (ESM, 2023) | No discussion of frameworks or theory in journal author guidelines (ESM Author Guidelines, 2023) |
N = 76 | Abbreviation: ESM | ||
Impact factor: 2.402 | |||
N = 59 | |||
Journal for Research in Mathematics Education | “JRME is a forum for disciplined inquiry into the teaching and learning of mathematics. The editors encourage submissions including: research reports, addressing important research questions and issues in mathematics education; brief reports of research; research commentaries on issues pertaining to mathematics education research,” (JRME, 2023) | “The study is guided by a theoretical framework that influences the study's design; its instrumentation, data collection, and data analysis; and the interpretation of its findings. The literature review connects to and supports the theoretical framework. Make it clear to the reader how the theoretical framework influenced decisions about the design and conduct of the study.” (JRME Author Guidelines, 2023) | |
Abbreviation: JRME | |||
Impact factor: 3.676 | |||
N = 17 | |||
Physics | Physical Review Physics Education Research | “PRPER publishes detailed research articles, review articles, and replication studies. Descriptions of the development and use of new assessment tools, presentation of research techniques, and methodology comparisons or critiques are welcomed,” (PRPER, 2023) | No discussion of frameworks or theory in journal author guidelines (PRPER Author Guidelines, 2023) |
N = 102 | Abbreviation: PRPER | ||
Impact factor: 2.412 | |||
N = 102 |
Reflected in our research questions, the initial stage of our analysis was relevant to both the longitudinal CER sample and the 2021 DBER sample. Here, we were interested in identifying the presence of a framework and contextualizing emerging trends using the journal author guidelines (see Table 1). The coding scheme for this analysis is presented in Table 2. Coding for distinct framework sections identified whether authors included clear headings distinguished as describing theory or frameworks, including phrases such “theoretical framework”, “conceptual framework”, “analytical framework”, and other similarly-worded heading variations. When coding for discussed framework(s) used in the research, we identified if the authors described a framework or theory as guiding the study, irrespective of placement within a distinct framework section, and we wrote memos identifying the specific frameworks the authors described. To receive this code, the authors needed to define or describe key constructs for the framework or theory such that they could be used to guide and inform the study. Articles that only provided a literature review of studies that used the framework, as opposed to reviewing the framework itself, did not receive this code. Lastly, we coded for the research methodology (qualitative, quantitative, mixed methods, or reviews).
Item coded | Definition | Examples | Percent agreement (%) | Cohen's kappa | ||
---|---|---|---|---|---|---|
A | B | A | B | |||
a Reliability measures represent the agreement between two pairs of coders. b Reliability measures represent the agreement between two independent coders. c The specific framework(s) discussed were noted within memos for articles receiving this code. | ||||||
Distinct framework section | The article includes an explicit section that describes the framework(s) and clearly indicates that the framework(s) were used to guide the study | “Theoretical Framework(s),” “Guiding Frameworks,” “Meaningful Learning” | 93a | 100b | 0.86a | 1.00b |
Discussed framework(s) used in the researchc | The article specifically identifies the framework(s) guiding the study, providing definitions and/or descriptions of key constructs such that the framework/theory can be used to guide/inform the study | Pedagogical content knowledge, representational competence, social cognitive, 3D learning, meaningful learning | 90b | 94b | 0.78b | 0.88b |
Research methodology | The research methodology used in the study | Qualitative, quantitative, mixed methods (includes studies where qualitative data is quantitatively transformed and studies that utilise both qualitative and quantitative analysis), reviews (both literature and methodological) | 89b | 94b | 0.83b | 0.88b |
This analysis was then extended beyond the CER context to confirm its application to the DBER sample. Here, reliability was established by two researchers (J-MGR & JEN) analysing a randomized subset consisting of 5% of the (non-CER) DBER articles (n = 30). This involved: (1) applying the codes provided in Table 2 and (2) assigning the previously discussed inductive categories used to characterize framework use across individual articles (Tiers 1–4). See Table 2 for the percent agreement and kappa values for the previously discussed codes. For the inductive categories related to tracking framework use across the DBER sample, the percent agreement was 88% and Cohen's kappa was 0.76, indicating moderate agreement (Watts and Finkenstaedt-Quinn, 2021). Disagreements were then discussed to reach consensus. Lastly, we sought feedback from members of the biology, engineering, mathematics, and physics education research communities to lend validity to our claims related to each field, analogous to member-checking (Carlson, 2010). We sent an excerpt of the manuscript to disciplinary experts, asking for comments on the claims and their impression of the results to provide insight regarding consistency with their experience in their community and their familiarity with the relevant journals, especially in terms of expectations regarding framework use. This process provided confirmation regarding the claims made to ensure we were adequately capturing community norms regarding framework use. Experts commented on preferences related to publication destination: for mathematics, a tendency to publish equally within both high-impact journals of JRME and ESM; for biology, a preference in publishing in LSE over BAMBED; for engineering, a preference in publishing in JEE over IJEE; and for physics, PRPER was confirmed as the primary discipline-specific journal.
2018 | 2019 | 2020 | 2021 | Total (N = 457) | χ2 | p-Value | |
---|---|---|---|---|---|---|---|
Distinct framework section | 66 | 65 | 71 | 80 | 283 | 5.4288 | 0.143 |
Discussed framework(s) used in the research | 80 | 77 | 84 | 91 | 333 | 5.0248 | 0.170 |
N Year | 120 | 110 | 111 | 116 |
Furthermore, we examined trends regarding the inclusion of frameworks between different research methodologies used in the articles (Table 4). Here, there was a statistically significant difference from the expected distribution across methodologies for both the presence of a distinct framework section (χ2 = 30.6148, p < 0.001) and whether articles contained some discussion of the framework(s) guiding the study (χ2 = 38.2645, p < 0.001). For each set of analyses, the standardised residuals were calculated to determine which methodologies differed significantly from the expected frequencies. For the presence of a distinct framework section, the standardised residuals were significant for articles using only qualitative (z = ±4.663) and only quantitative methodologies (z = ±3.954), compared to z = ±0.302 and 2.049 for mixed methods and reviews, respectively. Similarly, the standardised residuals for the chi-square examining whether framework(s) were discussed were also significant for articles that used only qualitative (z = ±5.042) and only quantitative (z = ±4.887) methodologies, compared to z = ±0.070 and 1.784 for mixed methods and reviews, respectively. Examining the frequencies in conjunction with the significant residuals, it appears that more qualitative studies and fewer quantitative studies contain a distinct framework section or description of the framework(s) than would be expected.
Qualitative | Quantitative | Mixed methods | Reviews | Total (N = 457) | χ2 | p-Value | |
---|---|---|---|---|---|---|---|
Distinct framework section | 114 | 52 | 112 | 4 | 282 | 30.6148 | <0.001 |
Discussed framework(s) used in the research | 130 | 62 | 134 | 6 | 332 | 38.2645 | <0.001 |
N Methodology | 148 | 113 | 184 | 12 |
As shown in Fig. 2, there was large variation in whether studies had a framework across the disciplinary journals. Focusing on specific disciplines, in chemistry, 82% of the articles in CERP and 75% of the articles in JCE discussed a framework. This aligns with the authorship guidelines for both journals that emphasize authors should discuss framework(s) in relation to their work (CERP Author Guidelines, 2023; JCE Author Guidelines, 2023). Framework use is also high in the mathematics education research articles, where 90% of the articles published in ESM and 88% of the articles published in JRME discussed frameworks. The high percentage of framework use across mathematics education research articles is notable considering the journal guidelines. Specifically, the ESM journal author guidelines do not explicitly mention the use of frameworks in submitted manuscripts (ESM Author Guidelines, 2023), while the JRME author guidelines have a subsection on theoretical frameworks and how they should influence the study design (JRME Author Guidelines, 2023). The high presence of framework use across mathematics education articles despite the difference in authorship guidelines suggests the use of frameworks may be an implicit expectation and norm within the mathematics education research community of practice. As for engineering and biology education research, the discussion of frameworks is mixed across their respective disciplinary journals. In engineering education research, 69% of studies discussed a framework in JEE, but only 14% of studies discussed a framework in IJEE. Biology education exhibits a similar difference across journals, where LSE has 40% of the research articles involving a discussion of frameworks, but only 5% of articles within BAMBED. In both cases, this reflects the authorship guidelines related to framework use for the respective journals (BAMBED Author Guideline, 2023; IJEE Author Guidelines, 2023; JEE Author Guidelines, 2023; LSE Author Guidelines, 2023). Within PRPER, the primary discipline-specific journal for physics education research, 41% of articles involved a discussion of a framework. This is a similar percentage to that in LSE, even though the incorporation of a framework or theory is not discussed in the journal's author guidelines (PRPER Author Guidelines, 2023). While there is variation across discipline and journal for whether a framework or frameworks are described, a general trend across disciplines was that most studies discussing a framework also included a distinct framework section.
Tier 1 | Tier 2 | Tier 3 | Tier 4 | Framework was used to … |
---|---|---|---|---|
✓ | ✓ | ✓ | ✓ | Provide supporting literature to contextualize the study |
✓ | ✓ | ✓ | Make connections to data in the findings and conclusions | |
✓ | ✓ | Guide the data analysis | ||
✓ | ✓ | Organize the findings | ||
✓ | Guide the data collection |
In the case of Tier 1, articles primarily used a framework as supporting literature to situate and provide context for the work described. In addition to using the framework to contextualize the study, articles in Tier 2 drew connections between the framework and the data in the Findings and Conclusions. This was often observed as making a claim using the data presented and then commenting on how that relates to the framework. In the case of Tier 3, articles also used the framework and related constructs to analyze the data. Moreover, for articles in Tier 3, the organization and framing of the Findings were influenced by the framework, with the framework integrated throughout the Findings (as opposed to only being referenced). In the case of qualitative studies, this often stemmed from the framework's more explicit role in data analysis and thus generation of themes; quantitative studies typically emphasized the assumptions from the frameworks used to determine the constructs being measured (e.g., performance, identity, and belonging, etc.). Lastly, studies in Tier 4 included the components of the previous tiers but also discussed how the framework(s) informed the data collection process; thus, articles in this tier discussed the framework and how it was relevant across the primary sections of the article (Data Collection, Data Analysis, Findings, Conclusions). Based on the patterns in how each group used frameworks, the primary feature that separates Tiers 1 and 2 from Tiers 3 and 4 is that studies in Tiers 3 and 4 incorporated the framework into the data analysis process (which then subsequently informed the organization of the Findings).
As suggested by Fig. 3, the previously discussed findings pertaining to community norms regarding the presence of frameworks generally corresponds with how the articles use frameworks. For the biology and engineering journals, LSE and JEE author guidelines include the expectation for a framework, whereas BAMBED and IJEE author guidelines do not, and this appears to be reflected in the prevalence of articles in Tiers 3 and 4 for LSE and JEE, whereas there are proportionally more articles in the Tier 1 and 2 categories for BAMBED and IJEE. In contrast, the mathematics journals (JRME and ESM) both include a higher proportion of Tier 3 or Tier 4 articles versus Tier 1 or Tier 2 articles. This suggests that not only is the use of frameworks an established community norm (Fig. 2), but also that frameworks may be expected to be used in a particular way (i.e., as part of data analysis), despite just one of the journals (JRME) emphasizing frameworks in the author guidelines (it is also interesting to note that JRME does not have any Tier 1 articles). Similarly, PRPER author guidelines do not discuss framework use, though a larger proportion of articles use frameworks in alignment with Tiers 3 and 4. Notably, for the CER journals, both sets of author guidelines mention framework use and both journals include a high proportion of articles using frameworks that aligned with Tiers 3 and 4. Thus, in the case of CER, the norms of the community appear to be well-aligned with the guidelines for authoring articles in both journals.
Constructivist frameworks | Hermeneutic frameworks | Critical theory frameworks | Organization of chemistry knowledge |
---|---|---|---|
Frameworks aligned with constructivism | |||
Conceptions-based constructivism (e.g., conceptual change; Park et al., 2020) | Action research (Sansom et al., 2021) | Belonging and identity (Fink et al., 2020) | Chemical thinking framework (Caushi et al., 2021) |
Constructivism (broadly stated; e.g., Nakiboğlu and Nakiboğlu, 2019) | Attitudes toward chemistry (Kousa et al., 2018) | Culturally responsive teaching (Reimer et al., 2021) | Johnstone's triangle/the chemistry triplet (Irby et al., 2018) |
Fine-grained constructivism (e.g., the resource-based model of cognition; Bain et al., 2018) | Design-based research (Wu et al., 2021) | Cultural competence and social justice (Clark et al., 2020) | Mechanistic reasoning (Macrie-Shuck and Talanquer, 2020) |
Meaningful learning (Enneking et al., 2019) | Phenomenology (Burrows et al., 2021) | Gender performativity (Miller-Friedmann et al., 2018) | Systems thinking (Talanquer, 2019) |
Mental models (Bongers et al., 2019) | Science and chemistry identity (Hosbein and Barbera, 2020) | Science capital (Rüschenpöhler and Markic, 2020) | Three-dimensional learning/A Framework for K-12 Science Education (Underwood et al., 2021) |
Pedagogical content knowledge (Akinyemi and Mavhunga, 2021) | Motivation/self-determination theory (Partanen, 2020) | ||
Representations and representational competence (Ferreira and Lawrie, 2019) | Self-efficacy (Willson-Conrad and Grunert Kowalske, 2018) | ||
Teacher-centred systemic reform model (Rupnow et al., 2020) | |||
Teacher noticing (Schafer and Yezierski, 2021) | |||
Frameworks aligned with social constructivism | |||
Argumentation (Shah et al., 2018) | |||
Communities of practice (Santos-Díaz and Towns, 2021) | |||
Situated learning (Reynders et al., 2019) | |||
Social constructivism (broadly stated; e.g., Bancroft et al., 2020) | |||
Sociocultural learning theory (Schmidt-McCormack et al., 2019) | |||
Frameworks aligned with learning and cognition | |||
Bloom's taxonomy (Lu et al., 2020) | |||
Cognitive load theory (Karch et al., 2019) | |||
Information processing (Galloway et al., 2019) | |||
Metacognition (Heidbrink and Weinrich, 2021) |
Researchers generally operationalized the specific frameworks used in alignment with the goals of the broader categories to which the frameworks belong. For example, studies using conceptions-based or fine-grained constructivist theories sought to understand the nature of students’ content knowledge and conceptual learning (e.g., Bain et al., 2018; Park et al., 2020), and studies using the pedagogical content knowledge framework focused on how instructors enact or develop their knowledge for teaching (e.g., Akinyemi and Mavhunga, 2021). In both cases, the use of frameworks was consistent with the constructivist framework category and its focus on the nature, structure, and development of knowledge. In contrast, studies using phenomenology or other hermeneutic theories such as self-efficacy generally sought to characterise how students and teachers experience features of the learning process or environment (e.g., Willson-Conrad and Grunert Kowalske, 2018; Burrows et al., 2021), reflecting the hermeneutic framework category and its attention to deriving meaning from individual experiences. Lastly, studies leveraging critical theories such as belonging and identity sought to investigate power structures that relate to inequities within teaching and learning in chemistry (e.g., Fink et al., 2020), mirroring the critical theory framework category and its emphasis on social systems and the distribution of power. As discussed by Bodner and Orgill (2007), specific frameworks lend themselves to particular study designs and are relevant for specific research topics, and researchers should select frameworks that align with the scope and research aims.
In addition to characterizing the presence of frameworks, our analysis also sought to identify how frameworks are used within individual articles published across DBER fields. For this analysis, we found that across all DBER disciplines, many studies presented a framework that was involved in the Data Analysis, informed the Findings, and was revisited in the Conclusions (Tiers 3 and 4), whereas a smaller subset of articles presented a framework that contextualized the study but was not discussed in relation to methodological decisions (Tiers 1 and 2). Most DBER journals had a higher fraction of articles that were categorized as Tier 4 in comparison to articles categorized as Tier 1 but there was variation (both across disciplines and within disciplines’ respective journals). Lastly, building on the correspondence between author guidelines and whether a framework is present for each disciplinary community (Fig. 2), we observed a correspondence between author guidelines and the varied ways in which frameworks are used (Fig. 3).
Lastly, we examined the common frameworks used within CER studies, which we grouped into four broad categories: constructivist frameworks, hermeneutic frameworks, critical theory frameworks, and frameworks related to the organization of chemistry knowledge. Articles mostly employed frameworks under the umbrella of constructivist frameworks, which included the subcategories of frameworks related to both social constructivism as well as learning and cognition. Our findings suggest that fewer studies use critical theory frameworks and (to a lesser extent) hermeneutic frameworks.
We argue it is necessary as a community to clarify and revisit expectations regarding the need to incorporate pertinent frameworks into research articles. Although authorship guidelines and recommendations suggested by editorials provide a great starting point for researchers to ensure their research and writing meets the standards of the respective journals, in communities such as CER, researchers continue to document and shape the norms of the field. Thus, it may be necessary to periodically reflect on author guidelines and consider whether they accurately reflect the current state of the field. We advocate for the importance of rethinking framework expectations given 27% of CER articles published in the last four years did not include a discussion of a guiding framework, indicating that it was not deemed necessary as it went through the journal review process. Stated differently, as a shared product of our community (Ashwin, 2012), it was decided that in some cases the project design did not warrant the inclusion of a framework. The variation in alignment between the community norms of disciplines (as reflected by the peer-reviewed articles analyzed) and the expectations described in the author guidelines can be seen across our DBER sample. In fact, as the co-authors discussed our literature review, we reflected on whether we needed to incorporate a framework. As we brainstormed, one potential framework we considered was communities of practice (Wenger, 1998). Literature related to communities of practice certainly informed our project by providing language to discuss this project and support our rationale; however, communities of practice did not explicitly inform the research process (e.g., data analysis). Rather than presenting a framework that did not track throughout different sections of the manuscript or forcing a framework (e.g., coding for constructs outlined in communities of practice like boundary objects or brokers, which would not add to our analysis), we chose to discuss the communities of practice literature without including it as an explicit “framework.” Importantly, whether a study needs a framework depends on the scope and goals of the study. For example, some investigations may not involve readily “theorizable” ideas, such as a study focusing on collecting data to inform future iterations of a course (e.g., through surveying student preferences related to in-person, hybrid, or online instruction). In contrast, other study designs, such as investigating students’ motivation to pursue a graduate program in chemistry, would be challenging without a framework. Motivation is a broad concept and there are multiple theories available with assumptions and definitions that could guide an investigation. As part of this, it is necessary to clarify whether the community holds the same expectations with respect to framework use for qualitative versus quantitative studies.
Furthermore, it is important to consider the ways frameworks can be used to support a study. Our analysis demonstrates that frameworks in CER articles often serve either as a picture “frame” (i.e., Tiers 1 and 2; the framework surrounds and contextualizes the study, and while it may influence the “big picture” narrative it could be removed without changing the specific findings of the research) or a house “frame” (i.e., Tiers 3 and 4; the framework provides a necessary structure and organizing scheme that is critical for the outcome of the research). Clarifying how frameworks inform and shape research is valuable both for increasing transparency about the research process while improving the ways emerging researchers and practitioners can identify connections between research findings and their own contexts.
We identified that constructivist frameworks are the most prevalent in our field, followed by a similar level of utilization for hermeneutic and organization of chemistry knowledge frameworks, and lastly an underutilization of critical theory frameworks. The distribution of framework usage suggests the current focus and values of our discipline and provides broader implications regarding the direction of future research within CER. In particular, recent calls for an increased focus on investigating diversity, equity, inclusion, and justice in CER (Winfield et al., 2020; Ryu et al., 2021) suggest a need to increase research using hermeneutic and critical frameworks, which are essential frameworks for social justice research (Metcalf et al., 2018). In general, the common usage of constructivist (e.g., personal and social) and organization of chemistry knowledge frameworks may indicate an unspoken post-positivist (or positivist) paradigm underlying many education research studies within the CER community, and it is necessary for researchers to engage in reflexivity regarding the traditional associations of positivist paradigms with privileging majority perspectives. We posit that research within the positivist paradigm can provide valuable contributions to our understanding of teaching and learning in chemistry, but that it is necessary to reflect on how research under these paradigms may lend itself to perpetuating structures that reinforce inequities. Hence, it is necessary for researchers to understand frameworks drawing from other paradigms and to recognize the value of approaching shared problems through the lenses of multiple paradigms and frameworks (Treagust et al., 2014).
In addition to contextualizing frameworks within their broader paradigms, it is important for researchers to reflect on the specific assumptions of frameworks and evaluate the validity of their application to chemistry. We noted that CER tends to borrow frameworks from other communities, as opposed to developing our own models that have explanatory and predictive power. Borrowing frameworks from other communities can be extremely productive, especially when investigating topics that align well with the goals of other communities (Bain et al., 2019); however, to continue to advance our community of practice, more work is needed in CER that uses data to develop chemistry-situated frameworks and theories, as well as work that critically evaluates the limitations of the frameworks we adapt from other research communities. Projects designed using a grounded theory approach are particularly well-suited to accomplish this goal (Strauss and Corbin, 1990; Charmaz, 2006).
In the context of the larger discussion related to bridging the research/practice divide, we echo the concerns related to accessibility and practicality (Rodriguez and Towns, 2019; Johnson, 2022; Sweeder et al., 2023). In terms of accessibility, we provided a general overview of how frameworks are used within CER and other DBER fields, focusing on use rather than categorizing frameworks into the categories of theoretical, conceptual, analytical, or methodological frameworks which are somewhat interchangeably used across the CER community. Additionally, as a community, we must consider the extent to which a “framework requirement” could make our work less accessible by potentially creating barriers toward entry into our community of practice. We aim to make our review more practical by providing a resource for instructors regarding frameworks and how they can serve as helpful tools to structure classroom interventions. Specifically, Table 6 summarizes types of frameworks that are organized thematically. As part of this, we draw the readers’ attention toward the variety of potentially useful instructional design, pedagogical, and evaluation frameworks. These frameworks can be used to shape assessment and instructional choices in the way they clearly articulate the organization of chemistry knowledge and outline target competencies related to developing expert reasoning.
Lastly, as discussed previously, our DBER communities are connected not just by shared content and skills, but by the frameworks we use as part of the research process. Although it was beyond the scope of the current review to compare the specific frameworks used across the DBER articles, we note this as a potential area for future inquiry and discussion. For example, examining how disciplines may use different frameworks to describe the same construct (e.g., identity) or how disciplines may be using the same framework in different ways (e.g., knowledge-in-pieces) could serve as a useful area to begin a dialogue across communities. Comparatively, multiple communities using the same frameworks may indicate shared assumptions that can serve as bridges between communities and demonstrate the broad utility of a framework. Considering how different communities may be operationalizing theoretical constructs differently, with open conversations across communities, has the potential to advance our collective knowledge about STEM education more broadly.
Footnote |
† Authors contributed equally to this work. |
This journal is © The Royal Society of Chemistry 2023 |