View PDF VersionPrevious ArticleNext Article

Open Access Article
This Open Access Article is licensed under a Creative Commons Attribution-Non Commercial 3.0 Unported Licence
DOI: 10.1039/D0RP00066C (Paper) Chem. Educ. Res. Pract., 2020, 21, 940-952

Effects of Scrum methodology on students’ critical scientific literacy: the case of Green Chemistry

Johannes Vogelzang *ab, Wilfried F. Admiraal b and Jan H. van Driel c
aGreijdanus College, Zwolle, The Netherlands. E-mail: j.vogelzang@greijdanus.nl
bICLON, Leiden University Graduate School of Teaching, Leiden University, The Netherlands
cMelbourne Graduate School of Education, The University of Melbourne, Australia

Received 27th February 2020 , Accepted 25th April 2020

First published on 2nd May 2020

Abstract

Secondary science education plays a key role in students’ process to become scientifically literate citizens. However, teaching students to acquire the necessary skills and knowledge to deal with complex societal issues is challenging. This paper reports about a study in which Scrum – a methodology to manage complex projects – was implemented in secondary chemistry classrooms to increase students’ conceptual understanding as well as their critical scientific literacy. A quasi-experimental design was used with 198 Grade 11 students from eight different classes. The experimental condition (99 students, 4 classes, 25 groups with 3 or 4 students, 2 teachers) used Scrum methodology during a context-based course on Green Chemistry. The comparison condition (99 students, 4 classes, 29 groups of 3 or 4 students, 3 teachers) completed the same module about Green Chemistry, without using Scrum methodology. At the end of the course students formulated a written advice on the greenest synthesis of adipic acid. A pre-test on prior knowledge of Green Chemistry principles and a post-test on conceptual understanding of the chemistry concepts involved were administered. In addition, the Standard Observed Learning Outcomes taxonomy (SOLO) was used to analyse the quality of the written advices as a measure for students’ critical scientific literacy. Students from the experimental condition outperformed their peers from the comparison condition in their conceptual understanding. Moreover, the quality of the advices of students from the experimental condition were rated higher than the quality of advices of students in the comparison condition. These findings are discussed and connected to Scrum methodology as teaching approach to scaffold both students’ conceptual understanding and its potential to promote the development of their critical scientific literacy.

Introduction

An important goal of secondary science education is to promote students’ competences to become scientifically literate citizens. Secondary education enables students to engage with science-related dilemmas that play a role in their personal lives (Eilks and Rauch, 2012). This is of the highest importance as society faces a broad variety of challenges (UN, 2015), including ecological issues such as climate change and chemicalisation (Ekberg, 2007; Sjöström et al., 2016). Inevitably, students, need to learn how to discuss dilemmas related to science, industry and the environment. Therefore, there is a clear need for students to acquire substantial science knowledge to understand underlying concepts. Moreover, students should develop appropriate skills to deal with complex issues, which enables them to participate in societal processes of democratic decision making (Sjöström et al., 2015). In short: the reality that society evolves towards greater complexity requires students to develop their scientific literacy. However, a challenging question is how to educate students to become scientifically literate citizens. Therefore, there is a need for teaching strategies that scaffold students on their way to become scientifically literate citizens.

This research explored to what extent the implementation of Scrum, a methodology intended to monitor and manage complex projects, can contribute to enhance students’ scientific literacy.

Scientific literacy and science education

The term ‘scientific literacy’ has been used extensively in the educational literature (Holbrook and Rannikmae, 2009). Although there are many, often overlapping, definitions, it seems that there is consensus that this term includes students’ ability to understand, use and apply scientific knowledge (Norris and Phillips, 2003). In addition, scientific literacy entails the ability to recognize topical real-world questions and, furthermore, the skill to draw evidence-based conclusions in order to generate well-informed decisions required for participation in democratic societies (Yacoubian, 2018). Based on this concept of scientific literacy, Roberts (2011) elaborated two viewpoints of science education. Vision I focuses on scientific theories, its underlying concepts and the scientific method. Typical examples are found in teacher-directed, traditional teaching approaches in which students develop understanding of concepts in a rather isolated way, without regard to how these concepts might be transferred to other contexts (Aikenhead, 2007). Within Vision I, contexts are used to illustrate concepts. In contrast, teaching based on Vision II starts with a meaningful, real-life context, and devotes particular attention to its societal aspects. Within these authentic practices, concepts are introduced on a ‘need-to-know’ basis (Pilot and Bulte, 2006). Moreover, students work collaboratively with classmates and are encouraged to explain course materials to other students. In addition, students receive timely and frequent feedback intended to enhance their learning process. Furthermore, they are invited to transfer the concepts learned and competencies acquired to new contexts. They are challenged to recognise and solve real-world problems by using relevant information sources. In sum, students work in small groups, deploy versatile communication skills to become independent and lifelong learners (Overton and Randles, 2015) as well as scientifically-literate citizens (Bennett, 2017, 23). Learning within Vision II can be characterised as student-centred. Its characteristics can be found in several teaching strategies, including problem-based learning (Barrows and Tamblyn, 1980), project-based learning (Krajcik and Shin, 2014) and context-based approaches (Pilot et al., 2016).

Worldwide, the major trend in secondary chemistry education reform is a shift from Vision I to Vision II (Pilot et al., 2016; Sevian et al., 2018). However, recently, a more elaborated form of Vision II is proposed, called Vision III (Sjöström and Eilks, 2018). It emphasizes that students need to develop a critical-reflexive attitude towards the context and concepts presented to them. Vision III is value-based and intended to take into account the complexity of life, of society and their mutual interactions (Sjöström, 2015) and is similar to critical scientific literacy (Sjöström and Eilks, 2018). Vision III aims at strengthening students’ learning beyond content (Vision I) and contexts (Vision II). Within Vision II students learn about a specific context and its underlying concepts, whereas within Vision III students develop a critical-reflexive attitude that goes beyond the context and concepts and that helps them to make well-informed, data-based, and value-based decisions. Vision III is intended to stimulate students to take responsibility for their personal lives and to participate in society. Although there is some overlap between Vision II and Vision III with regard to learning objectives and learning strategies (Table 1), it seems reasonable to assume that measurement of students’ learning progress within these learning environments requires additional, innovative assessment strategies. In contrast to a Vision I learning environment, in which teachers often use standard, straightforward, summative assessments, with multiple choice questions or questions with well-defined answers, within Vision II and Vision III special attention is paid to feedback and reflection. Therefore, formative assessments might play an important role as an appropriate assessment tool, although summative assessments are not without relevance and often used (Orpwood, 2007).

Table 1 Characteristics of Vision I, II and III. Based on and adapted from Pilot et al. (2016, 228–229) and Sjöström et al. (2018)
Vision I: traditional chemistry education with contexts as illustrations Vision II: context-based chemistry education, with authentic practices as context Vision III: critical-reflexive chemistry education
Learning objectives
Rationale Emphasis on concepts Emphasis on authentic practices as context Emphasis on socio-scientific issues
Cognitive Decontextualized concepts, rules, theories and processes Contextualized concepts, rules, theories, processes and transfer skills Contextualized concepts, rules, theories, processes, transfer skills and value-based decisions
Affective Preparing for the test Valuating the relevance of chemistry Emphasizing critical-reflexive thinking
Metacognitive Learn to reproduce and vary on standard procedures Learn to develop knowledge (need-to-know principle) as coherent and useful patterns of understanding Learn to develop a critical-reflexive attitude grounded in substantive understanding of relevant concepts
Learning process
Rationale Behavioural learning Learning by doing Learning by doing
Situation Chemistry concepts and the textbook are central A real-life question is central Critical-reflection on socio-scientific issues
Social setting Most individual learning in the implicit role resembling that of a ‘copy monk’. Participating in teams, taking up roles that are typical in the field of chemistry to search for and create answers Participating in teams, taking up relevant roles to search for appropriate and value-based answers
Control Teacher control, students follow teachers’ instructions Shared control, the real-life question structures students’ learning Shared control, reflecting on the role of (chemistry) concepts in socio-scientific issues
Cognitive Ideas can be mistakes and may be pointed out as wrong Ideas are shared and welcomed by both students and teacher Ideas are shared, welcomed and weighed by students and teacher
Creating and exercising with concepts on examples simplified to fit the theory Using of concepts on realistic contexts and tasks Testing and reflecting of concepts on realistic contexts and tasks
Leading to abstractions with universal meaning Leading to knowledge with proven value in various contexts Leading to knowledge with proven value in various contexts
No specific attention to transfer skills Learning to de-contextualise and re-contextualize knowledge and skills Learning to use knowledge in personal life
Affective Valuing the correct reproduction and use of standard situations Valuing relevance for reality and joint efforts to both understand and improve understanding and products Valuing relevance for and critical reflection on knowledge contributing to responsible citizenship
Metacognitive Little room for students to practice or learn reflecting, planning, steering their learning process Continuous challenge to improve on defining problems, planning, steering both individual and collaborative learning Continuous challenge to improve on reflecting, defining problems, planning, steering both individual and collaborative learning
Closing Stimulating to check for lacks in learning and knowing Challenge to reflect on relevance and opportunities for transfer Challenge to reflect on relevance, opportunities for transfer and connection with personal life and society
Assessment Focus on summative assessment Both formative and summative assessment Summative, formative and additional, alternative assessments

Clearly, due to the multifacetedness of socio-scientific issues, assessing students’ critical scientific literacy is rather complex, as students might propose different solutions for the real-world issue. Assessing and subsequently quantifying the multiplicity of students’ solutions, is challenging (Romine et al., 2017). Romine et al. (2017) showed that a broad variety of assessment tools, including tools to measure informal reasoning, argumentation, and reflective decision making, have been used to measure students’ scientific literacy. However, it seems reasonable to expect that a Vision III learning environment might benefit from additional, innovative teaching and assessment strategies to measure students’ critical scientific literacy.

The key-characteristics of Vision I, II and III with regard to learning goals, learning process and assessment have been brought together in Table 1. The characteristics are adapted from Pilot et al. (2016, 228–229) and Sjöström et al. (2018) and complemented.

Enhancing students’ critical scientific literacy (Vision III). The selection of an appropriate context or task is a crucial first step to enhance students’ critical scientific literacy. A suitable context meets with the following criteria: (1) it consists of a real-life question, (2) students experience the context as relevant, (3) it promotes discussion, and, preferably, (4) it has a controversial character in society (Marks and Eilks, 2009; Stolz et al., 2013). Such contexts force students to think about potential (dis)advantages, and invite them to propose solutions based on values and scientific data (Eilks and Hofstein, 2014, 10). There are numerous examples of suitable contexts available, including socio-scientific issues on climate change (Flener-Lovitt, 2014), and genetic modification (Lederman et al., 2014). Furthermore, the field of green chemistry provides a broad variety of socio-scientific issues, that can be used in an educational context: e.g. on biofuels (Mamlok-Naaman et al., 2015), and biopolymers (Sjöström et al., 2015). Green chemistry takes into account the effects of chemicals on people and planet and focuses on sustainability. It intends to stimulate people to reduce or eliminate the use and generation of substances which are harmful to human health and the environment by investigating alternative processes to synthesize chemicals (Anastas and Warner, 1998; Zuin and Mammino, 2015). Green chemistry takes into account the entire life cycle of a substance, including its design, use and disposal. Green chemistry education aims to (1) provide information how chemical reactions could be designed to be more eco-friendly; (2) deepen students’ knowledge of underlying chemistry concepts; and (3) teach students to develop their scientific literacy and corresponding skills (Zuin and Mammino, 2015, vii). Clearly, green chemistry education goes beyond the objectives of Vision I and meets with the criteria of Vision II. In addition, it offers the ingredients that might contribute to Vision III education in which students acquire skills to act as responsible future citizens.

Furthermore, if the socio-scientific issue is embedded in a classroom environment that scaffolds students’ learning, the development of their scientific literacy will be strengthened (Presley et al., 2013). A socio-scientific issue, for example derived from the field of green chemistry, is by its nature multifaceted: conceptual, contextual and societal aspects are strongly intertwined. Students deploy both cognitive and metacognitive skills to address its complexity. In such student-centred learning environments students might perceive difficulties in connecting concepts, context and social aspects, which, in turn, might slow down their learning. Constable et al. (2019) showed that systems thinking approaches might assist students in ameliorating the challenges associated with students connecting all these aspects. However, given the fact that, in general, such learning environments provide less guidance to students, they need scaffolds to apply concepts and to connect the context to their personal lives (Broman et al., 2018). Thus, teachers might use teaching strategies that support students to recognise and understand key concepts present in a socio-scientific issue. Scaffolds help students to manage and monitor their learning process, support mutual collaboration and are intended to provide tools to stimulate students to reflect critically on their learning process as well as on their own role as future citizens.

There are many examples of teaching strategies intended to scaffold students’ learning in such a way. Marks and Eilks (2009) describe a lesson series with authentic media, using newspaper articles to introduce the context and to prompt questions. In addition, they used role-play activities, in which students adopted the role of journalist to produce a news item, and panel discussions, with students in the role of chemist, engineer or environmental protection activist. Students learned chemistry with a combination of practical lab work, cooperative learning techniques and conceptual learning. Marks and Eilks (2009) found that both teachers and students appreciated the approach. Teachers and students characterised the teaching strategy as motivating, intense and relevant for their personal lives. Furthermore, Marks and Eilks (2009) suggest that the use of socio-scientific issues might induce changes in students’ attitude towards chemistry in general and improves their communication skills.

In addition, Barraza and Ruiz-Mallén (2017) explored and investigated the 4D-approach, a teaching strategy based on dialogue, divergent thinking, discussion and debate, intended to enforce higher cognitive skills such as critical thinking. Barraza and Ruiz-Mallén (2017) showed that the implementation of the 4D-approach in a classroom enhances students’ ability to deal with controversial socio-scientific issues. Moreover, they report that the 4D-approach scaffolds students to make well-informed and balanced decisions as future citizens in a democratic society.

However, despite these examples, enhancing students’ scientific literacy remains a complex endeavour. Although the criteria for selecting appropriate teaching contexts to improve students’ critical scientific literacy are clear (Stolz et al., 2013), it remains challenging to implement these approaches in the classroom. In addition, Sevian et al. (2018) emphasized the need for additional studies which focus on how students’ progress can be monitored and what teaching strategies might scaffold students’ learning process. We argue that these rather complex learning environments might benefit from the implementation of Scrum methodology.

Scrum methodology

Scrum methodology was introduced in the mid-1990s to manage complex projects in the field of software development (Schwaber and Sutherland, 2017). Basic tenets of Scrum methodology are transparency, inspection and adaptability. The framework consists of ceremonies, artefacts and roles that contribute to visualization of progress of the project, provide information on the quality of intermediate products and help employees to adjust their work to customers’ desires. The characteristics of Scrum methodology have been transferred to educational contexts, including writing courses (Pope-Ruark, 2015), courses on software engineering (Mahnic, 2010) and context-based approaches in secondary chemistry education (Vogelzang et al., 2019; 2020).

Scrum methodology is an iterative process and evokes feedback moments systematically. The teacher, in their role as product owner, introduces a social-scientific issue, e.g. on a green chemistry topic. Students are divided into groups of approximately four persons. All students commit themselves to deploy their skills (e.g. writing skills, planning skills) to the team explicitly. The product owner provides each team with a product backlog, which consists of assignments, exercises and practical work, necessary to formulate an answer to the real-world issue. Every group has the autonomy to plan their own work. Their planning is visualised on the Scrum board, which basically consists of three columns, ‘to do’, ‘doing’ and ‘done’. Students write tasks on Post-Its and stick them on the Scrum board. Every lesson starts with a stand-up ceremony, in which students discuss what they will do during the lesson. When a task is completed, the accompanying Post-It is positioned in the column ‘done’. The lessons of approximately two weeks form a ‘sprint’. A sprint concludes with a review ceremony in which the students receive feedback on their learning progress. A review ceremony might have different forms, including a check on the quality of an intermediate product, a panel discussion or a formative assessment on concepts covered in the sprint. The review ceremony sheds light on misconceptions in an early stage and offers opportunities for students to reconsider specific concepts and for the teacher to adjust their teaching to the specific needs of their students.

A review ceremony is followed by a retrospective ceremony, in which students discuss the quality of their learning process. They discuss their mutual collaboration and formulate one point of improvement for the next sprint. After two or three sprints, students release their final product, for example, an answer to the real-world question associated with the socio-scientific issue, a written advice or a final product. Depending on school policy, a summative assessment might be used to finish the entire project. Especially the review ceremony and the retrospective ceremony might scaffold students by evoking critical reflection on conceptual, contextual as well as societal aspects of the socio-scientific issue in a natural and systematic way. Together, these ceremonies intend to stimulate both a reflective attitude and an ethical awareness concerning the consequences of the solution proposed. In addition, social interactions of students, in the form of discussions and mutual feedback might shape and influence their critical scientific literacy.

This study

This study was conducted in order to conceive, implement, and assess the influence of the use of Scrum methodology in a context-based approach on Green Chemistry.

The following research questions (RQ) guided this study:

RQ1. What are the effects of Scrum methodology on students’ understanding of chemistry concepts involved in the Green Chemistry module?

RQ2. What are the effects of the implementation of Scrum methodology on the development of students’ critical-reflexive scientific literacy?

Method

Context of the study

A module on Green Chemistry was implemented in secondary context-based chemistry classrooms (grade 11) in The Netherlands (Jansen-Ligthelm et al., 2010). The goal of the module was that students formulate an advice in which they argue what the greenest synthesis is to produce adipic acid, a precursor for nylon polymers and a preservative (E355) for food. The module started with exercises and assignments to deepen students’ knowledge of the twelve principles of Green Chemistry (Table 2) and concepts concerning reaction enthalpy and block diagrams.
Table 2 Green Chemistry principles (Bodner, 2015)
1. Prevent waste
2. Maximize atom economy
3. Design less hazardous chemical synthesis
4. Design safer chemicals and products
5. Use safer solvents and reaction conditions
6. Increase energy efficiency
7. Use renewable feedstocks
8. Avoid chemical derivatives
9. Use catalysts
10. Design chemicals and products to degrade after use
11. Analyse in real time to prevent pollution
12. Minimize the potential for accidents

The assignments and exercises were embedded in illustrative contexts, suggesting that this part of the module typically fits within Vision I, where focus is mainly on concepts. In this stage of the course students became aware of how chemical reactions can be designed eco-friendlier.

The second part of the module was designed in line with the characteristics of context-based approaches (Vision II). In this part, full focus was on the real-world question of the greenest synthesis of adipic acid. Answering the real-world question required that students used concepts connected to this context on a ‘need-to-know’ basis. Students were challenged to apply and transfer the twelve principles of Green Chemistry, to a new situation; i.e. to the case of adipic acid. They received two different routes to synthesize adipic acid; (1) a multistep oxidation starting from cyclohexanol and nitric acid; (2) a single step oxidation of cyclohexene with hydrogen peroxide (Fig. 1). For both routes they applied the twelve principles of Green Chemistry, that is, they calculated atom efficiencies, Environmental-factors, theoretical yields and reaction enthalpies and searched for information on toxicity for the chemicals involved (Bodner, 2015).


image file: d0rp00066c-f1.tif
Fig. 1 Two synthesis routes for adipic acid.

Subsequently students were invited to interpret their outcomes and reflect critically on the societal consequences of the synthesis routes (Vision III). They were asked to formulate a written advice in which they balance both routes and made an informed decision on the route they preferred. The assignment was: “Within a radius of 10 kilometres of your school a new chemical plant will be built to produce adipic acid. There are two routes to produce this chemical and you will receive information on both. As junior professionals, you will provide the block council near your school with a substantiated, scientific advice which route is preferable. Feel free to add any information or arguments to underpin your final decision.”

Implementing Scrum methodology. A pre-test/post-test control condition design was used to explore the effects of the use of Scrum methodology on students’ understanding of chemistry concepts and the development of students’ critical-reflexive scientific literacy. In both the experimental condition and the comparison condition there was no difference with regard to learning objectives (rationale, affective and (meta)cognitive, Table 1). The objective of the module was to evoke critical reflection on the impact Green Chemistry can have on the environment students live in. In both the experimental and comparison condition, students prepared a written advice.

Both conditions did differ with regard to how students’ learning process was organised and with regard to intermediate assessments during the lessons (Table 3). Students participating in the experimental condition used the Scrum methodology framework to plan their work, and to monitor their progress on a scrum board. Their teachers reported that they executed the Scrum ceremonies as intended. During the stand-up and retrospective, according to their teachers, the students reflected on the quality of their learning process and discussed how they could improve their learning approaches. Students of the comparison condition did not receive specific scaffolds to manage their planning and monitor their progress. Just as the students of the experimental condition, they worked in groups. However, they were free to use their own strategies to plan and monitor their progress.

Table 3 Learning process and assessment of critical-reflexive chemistry education connected to teaching strategies used in the experimental and comparison condition
Vision III: critical-reflexive chemistry education (see also right column ofTable 1) Scrum methodology (experimental condition) Comparison condition
Learning process
Rationale Learning by doing. Students plan and monitor their progress systematically, using Scrum board and stand-up ceremonies. Students are free to plan their work as they prefer.
Situation Critical-reflection on socio-scientific issues. Ultimate objective is to evoke critical reflection Ultimate objective is to evoke critical reflection
Social setting Participating in teams, taking up relevant roles to search for appropriate and value-based answers. Teams are based on qualities. Students promise to deploy their personal qualities to their team. Teams are formed by students. No external regulation. Teams are often based on personal friendships.
Control Shared control, reflecting on the role of (chemistry) concepts in socio-scientific issues. Shared control, students have the lead, the teacher in their role as product-owner is near. Ceremonies are used to guide and monitor students’ learning. Shared control, students have the lead. The teacher is available on request and has a facilitating and stimulating role. No specific procedures were used.
Cognitive Ideas are shared, welcomed and weighed by students and teacher. Ideas are shared etc. However, not systematically evoked by Scrum methodology. Ideas are shared etc., although not systematically evoked by the teaching strategy.
Testing and reflecting of concepts on realistic contexts and tasks. Review ceremony evokes testing and reflecting on intermediate products and concepts, explicitly. No systematic reviews. Reflecting on learning process and progress is stimulated by the teacher and takes place on-the-fly.
Leading to knowledge with proven value in various contexts. Not specifically induced by a ceremony of Scrum methodology. Not specifically induced by the teaching strategy.
Learning to use knowledge in personal life and socio-scientific issues. Not specifically induced by a ceremony of Scrum methodology. Not specifically induced by the teaching strategy.
Affective Valuing relevance for and critical reflection on knowledge contributing to responsible citizenship. Not systematically. However, ceremonies such as review and retrospective might support the socio-scientific issue to evoke critical reflection. Not systematically. The socio-scientific issue might evoke critical reflection.
Metacognitive Continuous challenge to improve on reflecting, defining problems, planning, steering both individual and collaborative learning. Stand-up, review and retrospective challenge students to plan, reflect on and monitor their individual and collaborative learning. Although the socio-scientific issue is intended to enforce reflection on problems and challenges students to plan and monitor their progress, systematic and planned reflection does not take place.
Closing Challenge to reflect on relevance, opportunities for transfer and connection with personal life and society. Scrum ceremonies might scaffold the socio-scientific issue and induce reflection on relevance, and promote transfer and connection with personal life and society. No systematic scaffolding of the socio-scientific issue.
Assessment Focus on alternative assessments (although formative and summative assessment can play a role). Formative assessments at the end of each sprint to get insight in conceptual development as well as quality of intermediate products. In the final stage of the module students produce a written advice. No use of formative assessments, or other assessments to check conceptual development or quality of intermediate products. In the final stage of the module students produce a written advice.

Another salient difference between the experimental and comparison condition is the intermediate assessment during the review ceremony. Students of the experimental condition worked in iterative sprints with a review at the end of them, in which they checked their understanding of Green Chemistry principles. The reviews had the form of formative assessments. Students made them individually and discussed their answers with team mates as well as the teacher. On request, the teacher provided additional explanations.

An overview of the differences between teaching approaches of the experimental and comparison condition is provided in Table 3.

Participants

The module described above was implemented in eight classes, taught by five teachers with at least ten years of teaching experience. Of these classes two teachers (both male, four classes) used Scrum methodology as teaching strategy, forming the experimental condition. The comparison condition was formed by three teachers (two females, four classes), who implemented the module with their regular teaching style (see Table 3). The research was carried out following the guidelines for research ethics and integrity of Leiden University. All students and their teachers were informed about the aim of the study, which was to gather information on how they learn the principles of Green Chemistry and to improve classroom teaching. They were told that their participation was voluntary and that they had the opportunity to opt out at any stage of the study. Students received information that their answers were anonymised and therefore could not influence their grades. Students and teachers were informed that with participation they provided their consent to use their responses for research purposes. In total 198 students, distributed over 54 groups of three or four persons, participated. In the experimental condition 25 groups (99 students, 44 females) participated and the comparison condition consisted of 29 groups (99 students, 56 females).

The participating teachers worked at different school from all over The Netherlands. They were familiar with both teacher-centred learning environments and context-based, student-centred learning environments. They responded to an email invitation, written by the first author, and distributed by teacher trainers of several teacher education institutions. They voluntarily choose whether they participated in the experimental or in the comparison condition. Teachers participating in the experimental condition participated in a professional development program (five sessions of four hours) over a period of nine months in which they studied the ceremonies, roles and artefacts of Scrum methodology, shared and discussed their experiences during the implementation of the framework in their chemistry lessons.

Data and instruments

During the study three types of data were gathered: (1) a test on students’ previous knowledge of Green Chemistry principles; (2) a group task on the application of these principles to answer RQ1, and (3) a joint group advice on the greenest synthesis of adipic acid. These advices were used to answer RQ2.
Students’ previous knowledge. At the start of the module, all students individually completed twelve open questions about the twelve principles of Green Chemistry. For instance, students were asked to define the Green Chemistry term ‘E-factor’ and to calculate the atom-efficiency as well as the energy-enthalpy of different reactions. Maximum scores on the 12 open questions varied from 1 to 3 (Appendix A). Overall maximum score was 23. Scores of individual group members were combined to an average group score and converted in a percentage (23 points = 100%). The average scores of the experimental and the comparison condition did not differ significantly (F(1, 52) = 0.093, p = 0.762), suggesting that students in these conditions are comparable with regard to their prior knowledge. The answers of 15 students (12.6%, 300 items) were checked by a second rater, and resulted in an inter reliability score of Cohen's kappa k = 0.925 (p < 0.001), 95% CI (0.980, 0.987), suggesting a good reliability between the raters. Descriptive statistics are presented in Table 6.
Students’ understanding of chemistry concepts involved in the Green Chemistry module. All student groups were asked to apply the twelve principles of Green Chemistry to the two different synthesis routes of adipic acid. All student groups planned and monitored their work themselves and released a joint report in which they answered items about the two synthesis routes. The 40 items included a variety of calculations (atom efficiency, E-factor, reaction enthalpy), development of block diagrams, toxicity of chemicals as well as on reusability of the chemicals involved (Appendix A). The maximum scores per item varied from 1 to 3 points. The overall maximum score was 46 points. The average scores were converted in a percentage (46 points = 100%). The answers of 6 groups (11%, 240 items) were checked by a second rater, resulting in an inter reliability score, of Cohen's kappa k = 0.854 (p < 0.001), 95% CI (0.944, 0.966), and showed a good reliability between the raters. Descriptive statistics are presented in Table 6.
Students’ critical science literacy. The last source of data was formed by the written advices, which were used to measure students’ critical-reflexive scientific literacy. All 54 groups were required to release their final product, i.e. their written advice, in which they provided a balanced response to the greenest synthesis of adipic acid. Two characteristic excerpts of the written advices, translated from Dutch, are presented in Appendix B. The complexity of the written advices was analysed with the SOLO-taxonomy, originally developed by Biggs and Collis (1982). Their Structure of Observed Learning Outcomes (SOLO) has been used extensively in a variety of disciplines (Stewart, 2012), including mathematics and chemistry to analyse students’ responses (Tomperi and Aksela, 2014). It comprises five levels of understanding: (1) pre-structural, (2) unistructural, (3) multi-structural, (4) relational and (5) extended abstract. In Table 4 an overview of different SOLO-levels is presented, including sublevels, corresponding scores and examples of verbs.
Table 4 SOLO-levels. Based on and adapted from Biggs et al. (1982) and Stewart (2012)
SOLO level Sub-level Descriptions of student responses Score Examples of verbs
Pre-structural Question not understood; no relevant information. 0
Unistructural Mentions one relevant piece of information or variable. 1 Identify, name, recall, state
Multi-structural Low Contains 2 of 3 independent aspects related to the topic but without further elaboration. 2 Combine, describe, classify
Medium Contains a number of related pieces of information but presented serially or in isolation with no connections between underlying concepts. 3
High Contains many related aspects and elaborates each, but with no connection between concepts. 4
Relational Low Connections drawn between variables and concepts in one or two parts of the assignment. 5 Analyse, apply, argue, compare, relate, contrast
Medium Connections drawn between variables and concepts in many parts of the assignment. 6
High Overall generalisation of concepts showing high levels of integration throughout the assignment. 7
Extended abstract Consistent generalisation and synthesis of concepts throughout the assignment and high-level critical analysis. 8 Create, formulate, reflect, generalise, predict, evaluate

In the pre-structural level, there is little evidence of learning. The student did not approach the socio-scientific issue on Green Chemistry appropriately. At the unistructural level, the student focused on one relevant aspect without making connections to other aspects. At the multi-structural level, students’ writings comprised several relevant aspects without making connections between them. This level can be characterised as quantitative in nature, which means that the amount of details and the number of aspects mentioned, increased. When students used two or three Green Chemistry principles their advice was characterised as multi-structural (low) and thus rewarded with only 2 points. Groups that used all twelve principles received 4 points. These advices were characterised as multi-structural (high), containing many related aspects, however without making connections between the principles. At the relational level, students treated the different aspects of the socio-scientific issue as an integrated whole. This level can be characterised as qualitative (Tomperi and Aksela, 2014). In their advices they showed that conceptual, contextual as well as societal aspects of the Green Chemistry principles are closely connected and mutually related. Advices at the relational level had a minimum score of 5 points. Scores of 6 or 7 points were assigned to advices in which students made respectively several or many connections between the Green Chemistry principles. At the extended abstract level, students are supposed to conceptualise the previous integrated whole on a higher level of abstraction. Their writings go beyond the requirements of the assessment and comprise a critical reflection on the socio-scientific issue from multiple perspectives, including a personal perspective. At this level, students generalize, create and transfer ideas to new contexts. Obviously, the extended abstract level is strongly connected to students’ critical scientific literacy.

All advices were scored independently by two raters. Further details with regard to the SOLO-scores are provided in Table 5. Data in the matrix reveal the instances where rater 2 (dis)agreed with the score assigned to an advice by rater 1. The scores are ordinal data; therefore, Cohen's kappa was calculated. The inter-rater reliability measure was found to be k = 0.773 (p < 0.001), 95% CI (0.971, 0.990), suggesting that the scores assigned by the raters have substantial reliability. Therefore, the scores of rater 1 and rater 2 were used to calculate a mean score for all separate 54 advices. Descriptive statistics are presented in Table 7.

Table 5 Matrix showing agreement between rater 1 and rater 2 of SOLO-scores
SOLO-scores of rater 1
1 2 3 4 5 6 7 8 Total
Notes: rater 1 = first author; rater 2 = independent verifier.
SOLO-scores of rater 2 1
2 3 3
3 8 8
4 12 12
5 3 4 1 8
6 8 8
7 6 9 15
8
Total 3 8 15 4 15 9 54/54

Table 6 Converted mean scores (0–100) on 12 pre-test items as well as on 40 items concerning chemistry concepts present in the Green Chemistry module (0–100 points)
n Pre-test Post-test
M SD M SD
Scrum 25 groups 26.88 10.68 60.20 17.83
Non-scrum 29 groups 26.07 8.87 42.93 18.99

Table 7 Distribution of SOLO-scores by two independent raters over the different levels (mean score, (sd) and percentages per level). Overall average SOLO-scores of rater 1 and 2
SOLO-level Score Experimental condition (n = 25 groups) Comparison condition (n = 29 groups)
Rater 1 Rater 2 Rater 1 Rater 2
Pre-structural 0 — — — —
Unistructural 1 — — — —
Multi-structural 2–4 32% 28% 62% 55%
Relational 5–7 68% 72% 38% 45%
Extended abstract 8 — — — —
Mean score rater 1 and 2 (SD) 5.42 (1.38) 4.53 (1.61)

Data analyses

To answer the first research question analysis of covariance were carried out with the experimental and comparison condition as factor, the pre-test scores as covariate and the knowledge test scores at the group level as dependent variable. To answer the second research question analysis of variance were performed at the group level with experimental and comparison condition as factor and the quality of group advises indicated by their SOLO levels as dependent variable.

Results

Students’ understanding of chemistry concepts involved in the Green Chemistry module

Students participating in the experimental condition outperformed students of the comparison condition. Analysis of data revealed a large effect-size (see Cohen, 1988), suggesting that the implementation of Scrum methodology enhances students’ understanding of Green Chemistry concepts: (F(1, 52) = 11.912, p < 0.002, η2 = 0.189). Pre-test scores were used as covariate. Descriptive statistics are presented in Table 6.

Students’ critical science literacy

Data analysis of the SOLO-scores of both the experimental and the comparison condition revealed that the advices of all 54 groups had a score between 2 and 7 and therefore fit within the SOLO-levels, multi-structural and relational (Table 7). None of the advices was characterised as pre-structural, unistructural or extended abstract. However, 70% of the groups using Scrum-methodology delivered an advice on the relational level, whereas only 41.5% of the comparison condition produced an advice on this level (Table 7). Obviously, the groups participating in the experimental condition outperformed groups of the comparison condition. ANOVA-analysis revealed a medium effect-size: (F(1, 52) = 4.427, p < 0.05, η2 = 0.080) (Table 7).

Discussion

In this study, effects of the use of Scrum methodology were examined on understanding of concepts connected to Green Chemistry education and the quality of written advices concerning the greenest synthesis of adipic acid.

Impact on students’ understanding of concepts

The results reveal benefits of Scrum methodology for students’ conceptual understanding. It provided scaffolds for students participating in teams, using a realistic real-life context, in which they were challenged to discuss, monitor and improve their learning (see Vision II, Table 1). Inherently, the ceremonies in general, and the review in the form of a formative assessment at the end of a sprint in particular, promoted mutual feedback and enforced students to reflect on their understanding of the concepts involved. Consequently, both teacher and students might be confronted with misconceptions in an early stage, increasing the probability that teacher and students discuss these issues together. Ideas and questions were shared and welcomed. Obviously, the features of Scrum methodology contribute to a classroom climate in which students reflect on chemistry concepts, which, in turn, might explain the increased student learning achievements. The findings align with educational research on the implementation of formative assessments in the classroom, which, in general, suggest an increase in conceptual understanding and learning outcomes (Wiliam et al., 2004).

Impact on students’ critical scientific literacy

In general, advices developed in Scrum classes were more elaborated than advices composed by students of the comparison condition. It seemed that the Scrum ceremonies, roles and artefacts guided students through their learning process and enforced them to discuss and reflect on the Green Chemistry principles. Moreover, approximately 70% of the student groups using Scrum methodology reached a relational level, whereas the majority of the comparison condition (>55%) remained at the multi-structural level. This can be understood as a clear indication that the implementation of Scrum methodology scaffolds students in their process to become scientific literate. However, none of the 54 groups reached the extended abstract level. None of them connected the issue to other actual, societal issues or to their personal lives. This might be explained in various ways. The students were unfamiliar with this kind of written assessments and were not used to go beyond its requirements to formulate their own personal opinion. The student assignment focused on principles of Green Chemistry (E-factor, atom-efficiency etc.) and their mutual relations. It might be that an extra phrase, e.g. Describe how the principles of Green Chemistry might affect your personal life or Describe whether the Green Chemistry module influences your personal choices with regard to sustainability issues, would have increased the chance that students would have reflected critically on the principles of Green Chemistry and connect them to other societal issues or to their personal lives (Vision III); (Sjöström et al., 2018). Nevertheless, it seems that alternative assessments, such as these written advices, contribute to encourage students to think about and reflect on the impact of chemistry on society as well as their personal circumstances. Students are stimulated to explore the concepts and context, and, moreover, they are invited to add and reflect on their own ideas. In addition, the SOLO-taxonomy provides an appropriate working tool to obtain a representation of students’ critical thinking (Vision III). Furthermore, the results suggest that the features of Scrum methodology scaffolds students to organise their learning process and enforce them to converge their thinking to answer the real-world question.

However, the implementation of Scrum methodology is not the only factor that impacts the development of students’ critical scientific literacy. Other important factors include the role of the teacher and the classroom climate (Boss and Larmer, 2018). A teacher who is able to create a classroom climate in which students work collaboratively on a shared objective, increases the opportunities to enhance students’ critical scientific literacy. Although all teachers in the experimental condition and the comparison condition were experienced teachers, it is impossible to exclude a teacher effect. Admittedly, even when there is a positive classroom climate, a context-based learning environment remains rather complex, and the results seem to suggest that Scrum methodology benefits students when they work together on a rather complex real-world issue.

Limitations and directions for future research

There are some limitations in the study that should be taken into account. First, students participating in the experimental condition were unfamiliar with Scrum methodology. As a consequence, they needed some time to become accustomed to the ceremonies, roles and artefacts. Therefore, the differences between the experimental and comparison condition might be underestimated.

Secondly, the number of participating teachers, in both the experimental and comparison condition, is confined. This limits the generalizability of the results. A replication of this study with more teachers, and, in addition, with a larger sample size of students who are familiar with Scrum methodology might shed light on the generalizability of the results.

Conclusions

Findings suggest that the implementation of Scrum methodology might scaffold students’ learning in at least three ways. First, it enhances students’ conceptual understanding (Vision I). Secondly, its ceremonies, roles and artefacts scaffold students when they collaboratively work on rather complex real-world questions. Scrum invites to think critically, to provide feedback to each other (Vision II) and scaffolds students to apply the concepts in new contexts. Scrum methodology as framework might strengthen the shift to student-centred learning environments, because it might decrease feelings of overwhelmingness among students. Thirdly, in combination with a challenging socio-scientific issue that evokes interest and discussion, it reinforces their communication and collaboration skills and helps them to take responsibility in their socio-cultural environment, promotes participation in a democratic society, and appreciates skills and talents of other citizens (Vision III). It is worthwhile to perform additional research to explore whether the potential benefits of frameworks such as Scrum methodology and probably other agile project management frameworks work in other cultures and with other subjects.

Conflicts of interest

There are no conflicts of interest to declare.

Appendix A

Pre-test items (23 points)

1. Explain what is meant with the word ‘sustainability’ (1 point).

2. Write down as many of the characteristics of Green Chemistry you are aware of (3 points).

3. Describe what is meant with reaction yield (1 point).

4. Provide a description of E-factor (1 point).

5. A manufacturer wants to produce a specific chemical. It turns out that there are two different synthesis routes available. Method 1 has an atom efficiency of 50%, whereas method 2 has an atom efficiency of 75%. Explain which method is preferable (2 points).

Methyl-tert-butylether (MBTE, C5H12O) is added to petrol to increase its anti-knock rating. MTBE is synthesized from methylpropene (C4H8) and methanol (CH3OH).

6. Explain whether this reaction is an addition reaction (2 points).

7. Calculate the E-factor. Assume that the yield of the reaction is 100% (2 points).

8. In an experiment a researcher started with 20 g methylpropene and an excess of methanol. Finally, she isolated 30 g pure MTBE. Calculate the yield of the reaction (3 points).

9. Calculate the reaction-enthalpy of the MTBE synthesis. Given: the heat of formation of MTBE = −3.2 × 105 J mol−1 (3 points).

The industrial production of MTBE is represented in this simplified block diagram (Fig. 2).


image file: d0rp00066c-f2.tif
Fig. 2 Simplified block diagram.

In reactor R methylpropene and an excess of methanol are mixed. In this situation all substances are liquid. The mixture that leaves reactor R consists of methanol and traces of methylpropene. In three successive steps (S1, S2 and S3) the mixture is separated in MTBE, methylpropene and methanol. For the separation step S2 water is added.

10. Explain on micro level what happens in S2 (1 point).

11. In S3 water and methanol are separated. Methanol is recycled. Explain whether the process in S3 is endothermic or exothermic (2 points).

12. Why is it necessary to add extra methanol during the reaction process (2 points)?

Post-test items

The main-items listed below were used for both synthesis routes. Each item was divided in sub-items with separate scores. In total there were 40 items (46 points).

1. Design a detailed block-diagrams, for multi-step synthesis route 1 and for synthesis route 2. Include recirculation of substances if possible (10 points)

2. Provide equations for all reaction steps of synthesis route 1 as well as for route 2 (7 points).

3. Calculate the atom economy for synthesis route 1 and synthesis route 2 (7 points).

4. Calculate the E-factor for synthesis route 1 and route 2. Take into account the reaction yields as provided in the description of the module (6 points).

5. Distinguish potential waste-products and discuss their impact on the E-factor (for both route 1 & route 2) (5 points).

6. Discuss the hazardousness of all substances involved in route 1 and route 2 (4 points).

7. Calculate the reaction enthalpy for route 1 and route 2. Use heats of formation as provided in the module (7 points).

Appendix B: two characteristic excerpts extracted from the written advices

SOLO level (sub-level) Score Examples from the written advices of students Explanation
Multi-structural (high) 4 Finally, they wrote: We advise route 2 for several reasons. It does not use solvents, there are fewer steps, and a catalyst is used. Chemicals are less hazardous. Although that the atom efficiency is lower, when we combine our findings, we think that route 2 is eco-friendlier. For both synthesis routes group 17 used the 12 principles. They described their opinion for all the 12 aspects. Therefore, their advice could be rated with at least 4 points. However, they made some calculations errors (e.g. atom efficiency was miscalculated). For all principles, they combined data for both routes. However, they made no connections between different principles. They did not reach the relational level. In addition, sometimes their argumentation was incomplete or wrong. Both raters awarded this advice with 4 points.
Relational (high) 7 The final score of route 1 is −7, whereas route 2 scores +9. Obviously, synthesis route 2 is preferable. This route has fewer reaction steps, uses less harmful chemicals, waste products are biodegradable and its chemistry is overall less hazardous. In addition, route 1 is more expensive. It comprises corrosive chemicals and therefore there is a need for stainless reactors. On the other hand, route 2 is still in its infancy. A lot of research is necessary whether there are alternatives in the form of cheaper and/or reusable chemicals. Route 2 can only gain, and therefore we choose route 2: synthesis of adipic acid from cyclohexene with hydrogen peroxide. Group 3 compared the two synthesis routes by awarding points (++/+/0/−/−−) to all twelve principles. They explained their argumentations carefully, including correct calculations and to-the-point descriptions. This group compared both synthesis routes and applied their data to new situations (e.g. corrosive chemicals require expensive stainless reactors). They critically reflected on the preferred route and suggested some alternatives. They did not connect their advice to their personal lives or other societal issues. Both raters awarded this advice with 7 points.

References

  1. Aikenhead G., (2007), Expanding the research agenda for scientific literacy, Promoting scientific literacy: Science education research in transaction, p. 64.
  2. Anastas P. and Warner J., (1998), Green chemistry: Theory and practice, New York: Oxford University Press.
  3. Barraza L. and Ruiz-Mallén I., (2017), The 4D's: A pedagogical model to enhance reasoning and action for environmental and socio-scientific issues, in Corcoran P. B., Weakland J. P. and Wals A. E. J. (ed.), Envisioning futures for fnvironmental and sustainability education, Wageningen: Wageningen Academic Publishers, pp. 257–269.
  4. Barrows H. S. and Tamblyn R. M., (1980), Problem-based learning: An approach to medical education, New York: Springer Publishing Company.
  5. Bennett J., (2017), Bringing science to life, in Taconis R., den Brok P. and Pilot A. (ed.), Teachers creating context-based learning environments in science, Rotterdam: Sense Publishers, pp. 21–39.
  6. Biggs J. B. and Collis K., (1982), Evaluating the quality of learning: The SOLO taxonomy, New York: Academic Press.
  7. Bodner G. M., (2015), Understanding the change toward a greener chemistry by those who do chemistry and those who teach chemistry, in Eilks I. and Hofstein A. (ed.), Relevant chemistry education: From theory to practice, Rotterdam: Sense Publisher, pp. 263–284.
  8. Boss S. and Larmer J., (2018), Project based teaching: How to create rigorous and engaging learning experiences, Alexandria (VA): ASCD.
  9. Broman K., Bernholt S. and Parchmann I., (2018), Using model-based scaffolds to support students solving context-based chemistry problems, Int. J. Sci. Educ., 1–22.
  10. Cohen J., (1988), Statistical power analysis for the behavioral sciences, 2nd edn, Hillsdale, New Jersey: Lawrence Erlbaum Associates.
  11. Constable D. J. C., Jiménez-González C. and Matlin S. A., (2019), Navigating complexity using systems thinking in chemistry, with implications for chemistry education, J. Chem. Educ., 96(12), 2689–2699.
  12. Eilks I. and Hofstein A., (2014), Combining the question of the relevance of science education with the idea of education for sustainable development, in Eilks I., Markic S. and Ralle B. (ed.), Science education research and education for sustainable development, Aachen: Shaker, pp. 3–14.
  13. Eilks I. and Rauch F., (2012), Sustainable development and green chemistry in chemistry education, Chem. Educ. Res. Pract., 13(2), 57–58.
  14. Ekberg M., (2007), The parameters of the risk society: A review and exploration, Curr. Sociol., 55(3), 343–366.
  15. Flener-Lovitt C., (2014), Using the socioscientific context of climate change to teach chemical content and the nature of science, J. Chem. Educ., 91(10), 1587–1593.
  16. Holbrook J. and Rannikmae M., (2009), The meaning of scientific literacy, Int. J. Environ. Sci. Educ., 4(3), 275–288.
  17. Jansen-Ligthelm K., Scheffers-Sap M., Verhofstad A. and Van der Reijt V., (2010), Groene Chemie (Green Chemistry), viewed 2020-04-21, http://www.scheikundeinbedrijf.nl/Module/index.rails?id=6.
  18. Krajcik J. S. and Shin N., (2014), Project-based learning, in Sawyer R. K. (ed.), The Cambridge handbook of the learning sciences, 2nd edn, New York: Cambridge University Press, pp. 447–484.
  19. Lederman N. G., Antink A. and Bartos S., (2014), Nature of science, scientific inquiry, and socio-scientific issues arising from genetics: A pathway to developing a scientifically literate citizenry, Sci. Educ., 23(2), 285–302.
  20. Mahnic V., (2010), Teaching scrum through team-project work: Students' perceptions and teacher's observations, Int. J. Enig. Educ., 26(1), 96.
  21. Mamlok-Naaman R., Katchevich D., Yayon M., Burmeister M., Feierabend T. and Eilks I., (2015), Learning about sustainable development in socio-scientific issues-based chemistry lessons on fuels and bioplastics, in Zuin V. and Mammino L. (ed.), Worldwide trends in green chemistry education, Cambridge: The Royal Society of Chemistry.
  22. Marks R. and Eilks I., (2009), Promoting scientific literacy using a sociocritical and problem-oriented approach to chemistry teaching: Concept, examples, experiences, Int. J. Environ. Sci. Educ., 4(3), 231–245.
  23. Norris S. P. and Phillips L. M., (2003), How literacy in its fundamental sense is central to scientific literacy, Sci. Educ., 87(2), 224–240.
  24. Orpwood G., (2007), Assessing scientific literacy: Threats and opportunities, Promoting Scientific Literacy: Science Education Research in Transaction, p. 120.
  25. Overton T. L. and Randles C. A., (2015), Beyond problem-based learning: Using dynamic PBL in chemistry, Chem. Educ. Res. Pract., 16(2), 251–259.
  26. Pilot A. and Bulte A. M. W., (2006), Why do you “need to know”? Context-based education, Int. J. Sci. Educ., 28(9), 953–956.
  27. Pilot A., Taconis R. and den Brok P., (2016), Concluding reflections on context-based learning environments in science, in Taconis R., den Brok P. and Pilot A. (ed.), Teachers creating context-based learning environments in science, Rotterdam: Sense Publishers, pp. 225–242.
  28. Pope-Ruark R., (2015), Introducing agile project management strategies in technical and professional communication courses, J. Bus. Tech. Commun., 29(1), 112–133.
  29. Presley M. L., Sickel A. J., Muslu N., Merle-Johnson D., Witzig S. B., Izci K. and Sadler T. D., (2013), A framework for socio-scientific issues based education, Sci. Educ., 22(1), 26–32.
  30. Roberts D. A., (2011), Competing visions of scientific literacy, in Linder C., Östman L., Roberts D. A., Wickman P.-O., Ericksen G. and MacKinnon A. (ed.), Exploring the landscape of scientific literacy, New York: Routledge, pp. 11–27.
  31. Romine W. L., Sadler T. D. and Kinslow A. T., (2017), Assessment of scientific literacy: Development and validation of the Quantitative Assessment of Socio-Scientific Reasoning (QuASSR), J. Res. Sci. Teach., 54(2), 274–295.
  32. Schwaber K. and Sutherland J., (2017), The Scrum guide. The definitive guide to Scrum: The rules of the game, viewed 2018/10/23, https://www.scrumguides.org/scrum-guide.html.
  33. Sevian H., Dori Y. J. and Parchmann I., (2018), How does STEM context-based learning work: What we know and what we still do not know, Int. J. Sci. Educ., 40(10), 1095–1107.
  34. Sjöström J., (2015), Vision III of scientific literacy: Science education for sustainability, Malmö, 2019, http://muep.mau.se/handle/2043/19283.
  35. Sjöström J. and Eilks I., (2018), Reconsidering different visions of scientific literacy and science education based on the concept of Bildung, in Dori Y. J., Mevarech Z. R. and Baker D. R. (ed.), Cognition, metacognition, and culture in STEM education: Learning, teaching and assessment, Cham: Springer International Publishing, pp. 65–88.
  36. Sjöström J., Rauch F. and Eilks I., (2015), Chemistry education for sustainability, in Eilks I. and Hofstein A. (ed.), Relevant chemistry education: From theory to practice, Rotterdam: Sense Publishers, pp. 163–184.
  37. Sjöström J., Eilks I. and Zuin V. G., (2016), Towards eco-reflexive science education, Sci. Educ., 25(3), 321–341.
  38. Stewart M., (2012), Joined up thinking? Evaluating the use of concept-mapping to develop complex system learning, Assess. Eval. Higher Educ., 37(3), 349–368.
  39. Stolz M., Witteck T., Marks R. and Eilks I., (2013), Reflecting socio-scientific issues for science education coming from the case of curriculum development on doping in chemistry education, Eurasia J. Math., Sci. Technol. Educ., 9(4), 361–370.
  40. Tomperi P. and Aksela M., (2014), In-service teacher training project on inquiry-based practical chemistry, Lumat, 2(2), 12.
  41. UN, (2015), Transforming our world: The 2030 agenda for sustainable development, New York: United Nations, viewed March 27, 2020, https://sustainabledevelopment.un.org/content/documents/21252030%20Agenda%20for%20Sustainable%20Development%20web.pdf.
  42. Vogelzang J., Admiraal W. F. and Van Driel J. H., (2019), Scrum methodology as an effective scaffold to promote students’ learning and motivation in context-based secondary chemistry education, Eurasia J. Math., Sci. Technol. Educ., 15(12), em1783.
  43. Vogelzang J., Admiraal W. F. and Van Driel J. H., (2020), A teacher perspective on Scrum methodology in secondary chemistry education, Chem. Educ. Res. Pract., 21(1), 237–249.
  44. Wiliam D., Lee C., Harrison C. and Black P., (2004), Teachers developing assessment for learning: Impact on student achievement, Assess. Educ.: Princ., Policy Pract., 11(1), 49–65.
  45. Yacoubian H. A., (2018), Scientific literacy for democratic decision-making, Int. J. Sci. Educ., 40(3), 308–327.
  46. Zuin V. and Mammino L., (2015), Worldwide trends in green chemistry education, Cambridge: The Royal Society of Chemistry.
This journal is © The Royal Society of Chemistry 2020