A model for evaluating programs to promote pro-environmental attitudes and behavior

Abstract

Many educational programs seek to promote students' pro-environmental attitudes and behaviors, yet few are explicitly designed or examined in light of evidence-based design features known to support such change. This study presents and examines the validity of an evaluation model, the Environmental Attitudes and Behavior Model, through its application to an independently developed intervention program targeting ninth-grade students' attitudes and behaviors related to SDG 13 (Climate Action). The model is grounded in eight design features identified in the literature as effective in fostering pro-environmental attitudes and behaviors. Using a mixed-methods approach, we examined pre-post changes in the attitudes and behaviors of 23 teachers and 127 students through questionnaires, alongside interviews with six teachers and eight student groups. The findings indicate improvements in students' environmental attitudes and behaviors, as well as in their perceived importance of learning about these issues. Analysis of interview data further examined the presence of the model's design features in the intervention, leading to the suggestion of an additional design feature and the refinement of the model into a nine-feature framework. We recommend that evaluations of environmental education programs adopt a dual approach that combines assessment of learning outcomes with analysis of the presence of evidence-based design features that promote pro-environmental attitudes and behavior.

---

Sustainability spotlight

This paper introduces an innovative educational program that engages students with the United Nations Sustainable Development Goals (SDGs), focusing on SDG 13: Climate Action. Through data-driven exploration and student-led dissemination of climate initiatives, the program fosters awareness, responsibility, and practical sustainability skills. Beyond the program's implementation, the paper contributes a replicable model for curriculum developers seeking to influence students' attitudes and behaviors toward sustainability, thereby advancing SDG 4: Quality Education and SDG 13. By aligning pedagogical design with global goals, this work empowers educators to cultivate informed, proactive learners capable of driving climate action and promoting sustainable values within their communities.

1 Introduction

The field of environmental education (EE), places a great deal of emphasis on promoting positive attitudes and behavior regarding the environment.¹ In fact, many educational programs that aim to change attitudes and behavior regarding the environment have been designed and implemented.² While frameworks to design and evaluate such programs exist,² few evaluation models are based on design features known in the research literature to influence learners' attitudes and behavior. The purpose of the present study is to present such an evaluation model and to study its validity, based on the examination of an intervention program (IP) that was developed independently from the model.

This paper presents a theoretical framework for developing positive environmental attitudes and behavior through key components of environmental literacy, including knowledge, affect, skills, and behavior. The paper then introduces a design and evaluation model for promoting positive environmental attitudes and behavior, along with the characteristics of the IP. The two-part evaluation methodology that follows combines a pre-post quantitative questionnaire and a qualitative analysis that implements the evaluation model. Finally, the findings are presented on the value of this approach.

2 Theoretical framework

2.1 Environmental literacy

Environmental literacy, a term coined by Roth in 1968, is one of the key terms in EE. It is defined as the ability to diagnose the general state of environmental systems and to take appropriate measures in order to preserve and improve them.³ In addition, environmental literacy embraces a person's environmental feelings, knowledge, attitudes, skills, values, and sense of responsibility toward active social engagement.³

Environmental literacy comprises four interrelated components:

(a) Knowledge: an understanding of ecological concepts, human-nature interrelationships and environmental issues, including their causes, effects, and potential solutions.^3,4

(b) Affect: positive environmental attitudes, values, sensitivity, a sense of responsibility, and a belief in an individual's capacity to contribute to environmental solutions.³

(c) Skills: critical thinking, problem-solving, communication, and high-order civic skills necessary for informed decision-making and effective environmental action.^4,5

(d) Behavior: every human activity has an environmental impact. Hsu and Roth ⁶ identified five environmental behavior types: practical actions, consumer decisions, political activity, persuasion, and law enforcement.

Environmental literacy is essential for creating a more sustainable future. A person with a high level of environmental literacy can discuss environmental problems based on scientific findings and social and economic considerations, and can actively and openly apply this knowledge through acquired environmental skills.²

Environmental education, emerging in the 1970s, fosters environmental literacy,³ critical thinking, and values while promoting active citizenship. It cultivates environmental citizens who critique unsustainable behaviors, solve problems, and implement solutions. In this way, educators can influence public opinion by promoting realistic, achievable goals with students to establish environmentally friendly behaviors.²

Attitudes reflect a person's evaluations, preferences, and views.⁷ Environmental attitudes shape behaviors toward preservation or exploitation⁸ and influence environmental actions.⁹

Environmental behavior aims to reduce human impact on the environment.¹⁰ While knowledge is seen as key to pro-environmental behavior,¹¹ it has little direct effect.¹² Instead, social learning, emotional connection,¹² early nature experiences,^13,14 and hands-on resource management¹⁵ are crucial to the development of pro-environmental attitudes and behavior.

2.2 A challenge for environmental education design and evaluation

Evaluations of EE programs often focus exclusively on outcomes, such as changes in attitudes and behaviors, while overlooking key design features, such as social learning, emotional connection, early childhood experiences, and resource management, that have been shown to foster pro-environmental attitudes and behaviors. This exclusive focus on outcomes is incomplete if the goal is to provide knowledge for program design and improvement.¹⁶ While measuring attitude and behavior changes offers insights, it does not reveal the design features that drive these changes. Therefore, identifying and evaluating both outcomes and the design features that contribute to them are both essential for providing useful knowledge to program designers.

2.3 An evaluation model based on evidence-based design features to develop environmental attitudes and behavior

The study by Karpudewan, Ismail¹⁷ shows that integrating environmental content and values into teacher education through experiential learning such as green chemistry labs enhances both knowledge and environmental attitudes among pre-service teachers. These findings support the current study's focus on the teacher's role in promoting pro-environmental attitudes and behavior. In order to develop a model for evaluating programs that effectively cultivate positive environmental attitudes and behaviors, a narrative review of the research literature was conducted. A narrative review is an approach to synthesizing and interpreting existing literature that constructs a coherent account of a field's development. Unlike systematic reviews, narrative reviews allow for a more qualitative, flexible, and wide-ranging integration of scholarship, drawing on the expertise and interpretive judgment of subject-matter experts.¹⁸ This review focused on references that provided solid evidence to identify key design features shown to influence learners' environmental attitudes and actions; emphasis was placed on references that include research reviews and/or large sample sizes. The resulting design features are described in the following section and summarized in Fig. 1.

Fig. 1 The Environmental Attitudes and Behavior Model. The graphic illustrates 8 design features that promote pro-environmental attitudes and behavior.

(1) Social interactions: Eagly and Chaiken¹⁹ found that the formation and evolution of particular attitudes are intricately intertwined with social interactions. Human beings, as social creatures, engage in a continuous exchange of ideas, opinions, and experiences within their social environments. These interactions significantly contribute to the molding of their attitudes, since individuals absorb and integrate various perspectives from interpersonal dialogues.^20,21

(2) Early childhood experiences in nature: the formative years of childhood are profoundly impacted by experiences in nature, offering children not only cognitive and emotional benefits, but also imprinting a lasting influence on their development of lifelong environmental attitudes and behaviors. Exposure to natural environments during early childhood fosters cognitive development by stimulating curiosity, problem-solving skills, and creativity.^14,22 Programs that connect student activity to their memories of positive childhood experiences in nature might also have a positive effect.

(3) Diverse teaching resources: the utilization of a diverse range of teaching resources is essential to foster the development of attitudes in education. These resources encompass a wide spectrum of materials, including textbooks, multimedia presentations, interactive tools, real-world examples, integrating digital technologies and research-based projects.²³ By incorporating such diverse teaching resources, educators can cater to varied learning styles and preferences, thus creating a more inclusive and engaging learning environment.^19,24,25

(4) Environmental knowledge: broadening one's environmental understanding contributes significantly to the transformation of one's environmental attitudes and behaviors. Deepening environmental knowledge fosters a corresponding shift towards a more conscientious and sustainable approach to the natural world.^20,25–28

(5) Exposure to and recognition of environmental problems: an increased awareness and acknowledgment of environmental issues play a pivotal role in shaping environmental behavior. As individuals become more familiar with various environmental problems, their understanding and recognition of these issues become integral drivers for cultivating environmentally conscious attitudes and actions.^28–30

(6) Emotional value and agency: the emotional significance attached to a subject, coupled with the sense of agency or control that individuals perceive in their learning, collectively shape their attitudes and actions. As learners develop a heightened awareness of their emotional and self-driven learning, meaningful shifts in their overall attitudes and behavioral responses tend to follow.^20,25,29

(7) The intention to preserve the environment for learners: as individuals express a genuine intent to safeguard the environment, this fosters a nuanced shift in the students' conduct and actions. The conscientious decision to prioritize environmental preservation becomes a catalyst for transformative changes in students' behaviors, signifying a deeper integration of ecological responsibility into their daily lives.^31,32

(8) Learner's ability to effectively manage environmental resources: nurturing the students' skills for resource stewardship is a complex process, since ecological values are intricately linked to individual perspectives and priorities. Strengthening these skills can exert a profound influence on students' orientation to the environment. When students develop a heightened ability to care for resources, they not only contribute to sustainable practices, but also undergo a transformative shift in their attitudes and behavior, aligning more closely with principles of ecological responsibility and thoughtful resource utilization.^31,33

The design features, shown in the literature to positively affect environmental attitudes and behavior, are presented in Fig. 1. We used these design features to create the Environmental Attitudes and Behavior evaluation model, which we used to evaluate an independently-designed IP, as described below.

Many environmental educational programs have been developed to change teachers' and students' attitudes and behavior (e.g., ref. 1 and 34). One context for developing these programs is the Sustainable Development Goals (SDGs); these goals consist of 17 global issues that were established as a global agenda for sustainable development by the UN Council; this resolution was adopted by all 194 countries in the UN General Assembly in September 2015, with the intent of reaching the targets of the SDGs by 2030.³⁵

2.4 The intervention program (IP)

The IP is based on five SDGs, with this paper focusing on SDG-13 (Climate Action). The IP emphasizes scientific and mathematical skills through a “data-driven pedagogy” that fosters arguments, reasoning, and activism. Students analyze data to draw evidence-based conclusions and engage in environmental activism.³⁶ Furthermore, the program links different fields of knowledge – science and mathematics – and can serve as a basis for designing interdisciplinary courses that link mathematics and sustainability. Such courses not only deepen students' values and abilities in the field of sustainability, but also support the professional development of future teachers. Çibik and Boz-Yaman³⁷ showed that an interdisciplinary science-mathematics course in the field of sustainability education contributed to improving pre-service teachers' attitudes towards sustainable development and strengthened their self-efficacy in mathematical modeling.

Each unit in the IP concludes with a summative activist task, where students apply their learning to propose local solutions with a potentially global environmental impact. Using program resources and external tools, students address local sustainability issues, fostering societal influence. This task promotes emotional engagement and agency, enhancing environmental attitudes and behavior.²⁹ For example, in the SDG-13 unit, students planned to reduce CO₂ emissions by a certain percent and applied their knowledge to influence their local environment.

3 Methodology

3.1 Research goals

The main goal of the paper is to study the validity of the design and evaluation model presented above. The methodology is to evaluate an independently-designed EE program through a dual approach: (1) by measuring the attitudes and behavior of the participating teachers and students before and after participating in the IP and (2) by using the Environmental Attitudes and Behavior Model to identify the design features in the IP that have been shown to promote pro-environmental attitudes and behavior.

3.2 Research questions

The above research goal informs the formulation of the study's two research questions:

(1) To what extent, if any, did the IP change attitudes and behaviors of the participating science teachers and their 9th grade students, regarding the SDGs, and the importance of SDG-focused education?

(2) To what extent, if any, did the Environmental Attitudes and Behavior Model prove useful in evaluating the effectiveness of the design features in the IP in promoting pro-environmental attitudes and behavior with the participating students and teachers?

3.3 Ethics

The research received ethical approval from the Institutional Review Board, and ethical approval from the Ministry of Education (#11564). According to the protocol, teachers and students' parents signed informed consent forms for participation in the program and research.

3.4 Participants

In the first (questionnaire) phase of the study, 23 science teachers, of whom 20 (87%) were female, participated. They taught in six schools in northern Israel and participated in the study's professional development program. The ages of these teachers were between 23 and 52 years. In the second (implementation and interview) phase of the research, 6 of the 23 science teachers received approval from their school management to implement the IP in their classrooms; this group constituted the teacher sample for the classroom-based study reported here. A total of 127 9th grade students (age 14–15) of whom 76% were female, participated in this second phase. Data were collected during the 2022 academic year.

In our dual approach to evaluation, we consider that both teachers and students are learners. For teachers to effectively foster pro-environmental attitudes and behaviors among their students, they themselves should possess relevant knowledge and demonstrate such attitudes and behaviors. Therefore, both groups were included as participants in the present study.

3.5 Research tools

Two research tools were used to measure the influence of the IP on the development of teachers' and students' attitudes and behavior: (1) a pre-post questionnaire and (2) interviews.

3.5.1 Questionnaire. An attitudes questionnaire was developed for teachers and students regarding the SDGs. This is a Likert scale questionnaire that measures the attitudes, behavior, and the importance of education for SDGs (as detailed in Table 1). The questionnaire consists of 28 items, of which 20 items were based on the questionnaire that appeared in Afroz and Ilham.³⁸ The 20 items are divided into 2 categories: Attitude towards SDGs, behavior in promoting SDGs. To second category, we added 2 items related to SDG3, that was included in the IP. Six additional items were developed and added to the questionnaire to specify it for the IP: to address applying data in reasoning (3 items), COVID-19 related aspects, and the importance of the SDGs (3 items). The full questionnaire is attached in Appendix 1. Table 1 presents the categories, a representative item for each category, and the calculated Cronbach's Alpha values for each category in the student and teacher questionnaires. The category “Applying data in reasoning” represents the Intervention Program's “data-driven pedagogy,” i.e., presenting learners with data-based challenges requiring the use of mathematical literacy to address them and the subsequent construction of evidence-based scientific arguments.³⁶ The inclusion of this category allowed us to quantitatively evaluate its impact on the participating students and teachers.

Table 1 The categories of the questionnaire for teachers and students and their calculated Cronbach's Alpha

Category	A sample item	Number of items	Cronbach's alpha (students, N = 127)	Cronbach's alpha (teachers, N = 23)
1 Attitudes regarding SDGs	I believe I can contribute to environmental quality by my behavior	9	0.73	0.87
2 Behavior in promoting SDGs	I separate trash (such as plastic, glass, and paper) for recycling at home	14	0.93	0.79
3 The importance of education for SDGs	In my opinion, raising awareness of the SDGs with students is essential	4	0.83	0.89
4 Applying data in reasoning	It is important to me to give scientific explanations based on data	3	0.76	0.66

---

The readability of the items was verified by a cognitive pre-testing procedure³⁹ involving 4, 9th-grade students. During this process, the wording of the statements, their clarity, and their comprehensibility for the target population were examined to ensure that the questions were formulated clearly and comprehensibly as the researchers intended.

3.5.2 Data analysis for the quantitative data (questionnaires). Normality tests showed students' data were normally distributed. To assess changes after the IP, we used t-tests for students' attitudes and behavior. Teachers' data were not normally distributed; therefore, the Wilcoxon Signed-Rank test was used to assess changes in their attitudes and reported behavior regarding the SDGs after the IP.

3.5.3 The interviews. Interviews were given to teachers and student groups, in order to use the design model to evaluate the influence of the IP on the development of teacher and student environmental beliefs and behavior. A qualitative approach was used to investigate which model components in the program contributed to influencing participants' attitudes and behavior regarding climate change. The research tools included semi-structured interviews with teachers and students (6 teachers and 8 groups of students).⁴⁰ The interviews were conducted after the program was taught and included 5 questions:

(1) What is your opinion regarding SDG-13 climate change? Do you believe there are tools that can address the problem? Please elaborate.

(2) What differences exist between teaching the “Speak to Me in Numbers” program versus regular science teaching in your classroom? For each difference you note, please provide an example.

(3) Tell us about the activism activity for environmental change that you implemented. (For the teachers: tell us about the activism activity for environmental change that your students implemented.)

(4) What did you learn during the activism activity? (For the teachers: What did your students learn during the activism activity?)

(5) Who or what was affected by the activity? How did this change come about?

3.5.4 Data analysis for qualitative data (interviews). The analysis of the interviews was guided by the eight design features of the Environmental Attitudes and Behavior Model presented in Fig. 1. A deductive analytical approach⁴¹ was employed, whereby the interview transcripts were systematically examined for evidence corresponding to each of the eight design features. Segments of data were coded according to their alignment with the respective design feature components. In addition to these eight components, the analysis also included a search for evidence of data-driven pedagogy, in order to explore its potential contribution to the development of environmental attitudes.

The qualitative analysis was validated by two experts in science education. Throughout the validation process, discussions were held to refine the researchers' shared understanding of the essence of each design feature and the ways in which it might be manifested in the interview data, until consensus was reached. To further ensure consistency and credibility, all coding decisions were discussed and agreed upon by the authors.

A summative qualitative content analysis approach was applied to identify and rank the design features based on their frequency of mention, which served as an indicator of their salience.⁶¹ These rankings reflected the relative importance of each design feature as perceived by the participants and informed recommendations for improving program design.

4 Results

The results presented below include quantitative questionnaire findings for teachers and students, followed by qualitative interview insights for both teachers and students, organized by the design model's different features.

4.1 Quantitative results: teachers' attitudes and behavior regarding SDGs

Fig. 2 shows the effect of the IP on science teachers' attitudes and behaviors regarding SDGs, and the importance of education for fostering SDGs and for applying data in reasoning based on a Likert scale (1 to 5).

Fig. 2 The effect of the IP on science teachers' attitudes and behavior regarding SDGs, N = 23. *P < 0.05 (signed rank test).

There were no significant differences between the mean of their attitudes and behavior regarding SDGs before and after they participated in the professional development course. Teachers had the same attitude and behavior regarding SDGs (the Signed-Rank test resulted in a test statistic (S) of 55.5 and a p-value of 0.0692). However, there was a significant increase in teachers' attitudes toward the importance of education for SDGs in the pre-post comparison (the Signed-Rank test resulted in a test statistic (S) of 73 and a p-value of 0.002). In addition, the score for the category of “Applying data in reasoning” increased after the intervention. The significant difference that was found (the Signed Rank test produced a test statistic (S) of 52.5 and a p-value of 0.04), indicating an increase in teachers' perceived importance for applying a data-based pedagogy.

4.2 Quantitative results: students' attitudes and behavior regarding SDGs

Fig. 3 shows the impact of the IP on students' attitudes regarding SDGs, the importance of education regarding SDGs, the implementation of behavior that promotes SDGs in life, and applying data in reasoning in science teaching.

Fig. 3 The effect of the IP on students' attitudes and behavior regarding SDGs, N = 127. ***P < 0.001 (t-test).

There were improvements across all the assessed student categories that were examined. There was a significant effect of the program on students' attitudes and behaviors regarding SDGs, the importance of education for SDGs and regarding applying data in reasoning, based on a Likert scale (1 to 5), as shown in Fig. 3.

Students had more positive attitudes toward the SDGs after the intervention (t₁₂₆ = 6.12, p < 0.001)). In addition, a significant increase was found in students' behavior that promotes the SDGs (t₁₂₆ = 6.93, p < 0.001); a significant increase was also found regarding the importance of learning these subjects (t₁₂₆ = 6.27, p < 0.001). After participating in the program, students re-posted that they now behave more positively and think more about issues related to the SDGs.

Lastly, a significant increase was found regarding students reporting that they apply data in reasoning in science lessons (t₁₂₆ = 5.42, p < 0.001).

4.3 Qualitative results: interview data in light of the design features

The analysis of teacher and student interviews revealed which dimensions in the model salient to participants and spontaneously articulated in semi-structured interviews. The findings were organized according to the strength of each design feature in the program, as measured by the frequency with which each feature was mentioned by participating teachers and students. Table 2 presents these frequencies, including mentions of data-driven pedagogy. Specific comments and examples related to each design feature are provided in the following text, with the aim of supporting program developers in identifying and applying these features within EE programs.

Table 2 Strength level of Model's design features

Design features in the model	Mentioned by teachers (N = 6)	Mentioned by student groups (N = 8)	Strength level
(1) Social interactions	4	4	Strong
(2) Environmental knowledge	6	6
(3) Exposure to and recognition of environmental problems	6	8
(4) Intention to preserve the environment for learners	3	8
(5) Impact on emotional value and agency	5	7
(6) Diverse teaching resources	2	1	Moderate
(7) Learners' ability to effectively manage environmental resources	2	2
(8) Early childhood experiences in nature	0	0	Absent
Data-driven pedagogy	5	6	Strong

---

4.4 Design features strongly expressed in influencing student attitudes and behavior toward environmental issues in the IP

(1) Social interaction: the program includes tasks where students are asked to actively work together for the environment and to help improve the existing situation. Teachers enabled students to work in groups and conduct group discussions in a structured way. As one teacher said:

“The first skill that I noticed was most salient among the students is the social skill. This means that they focused a lot on partnership, group work and collaborative work, listening to others and sharing different opinions”.

Group activities provide students many opportunities for social interaction. During these activities, students work collaboratively, communicate with each other and carry out joint tasks. For example, one student explained:

“We acquired social skills, collaboration skills, discussion with others, how to convey our ideas and group thinking”.

(2) Environmental knowledge: The program includes teaching and learning topics related to climate change, such as the greenhouse effect and carbon cycle. One teacher noted:

“At the beginning of implementing the program, i exposed students to scientific and mathematical content related to SDG13, climate change”.

The environmental knowledge dimension constitutes a significant part of the program and takes place through a unique pedagogy of data-based learning. The content emphasized topics related to climate change, intended to help students understand the complexity and importance of environmental issues and how they affect the world. This topic was expressed in the following teacher quote:

“Students developed a deeper understanding of climate change processes, their causes and their potential impacts on the environment”.

This was also reflected in many student statements, such as:

“One piece of information that became ingrained in my mind, which is very very important, is that planting a green tree helps absorb carbon dioxide”.

Knowledge development was promoted through data analysis and connecting this analysis to issues related to the relevant environmental topics. As one teacher commented:

“In the Speak to Me in Numbers' program, the student raises questions, locates information sources, processes information and creates new knowledge relevant to their world and life, such as climate change, in the technological era, in the 21st century”.

(3) Exposure to and recognition of environmental problems: the program focuses on presenting severe environmental problems, such as the greenhouse effect leading to glacial melting and air pollution. The dimension of recognizing environmental problems is important and prominent in the program. As one student describes:

“As part of the ‘Speak to Me in Numbers’ program, we learned a lot about SDG13 about the environmental problems facing us… what contributes to the Earth's warming from various sources, including transportation and food. It was also shocking for us, because we, without noticing, negatively impact the climate”.

Recognition of environmental problems is the starting point of each unit in the program, so it's not surprising that these problems also came up in teacher interviews. For example, one teacher said:

“From the beginning of the program, I talked with the students about the human impact on the environment and about the greenhouse effect. Before we started the research work, I began presenting them with environmental phenomena and this problem of human impact on the environment”.

(4) Intention to preserve the environment for learners: in all of its tasks, the unit emphasizes the importance of taking action for the environment. The intention to preserve the environment emerged strongly in the interviews. Teachers and students expressed their desire and sense of responsibility for environmental preservation, as a result of the program. As one teacher said:

“After my participation in the ‘Speak to Me in Numbers’ program [four years], it created many changes in me, I started thinking in every place ‘What about the environment?’ We need to protect it”.

One student said: “It was shocking for us because we, without noticing, have a negative impact on the climate, and the activism task we performed was very effective in changing the existing reality and reducing carbon dioxide emissions”.

(5) Emotional value and agency: in the final task at the end of the unit, students are asked to promote actions for the environment and to contribute to improving the current situation. In other words, students are asked to be agents of change. Different teachers describe this process:

“I encourage my students to take the information and knowledge they receive and turn it into action, to intervene and be part of the solution to environmental problems they were exposed to that are related to climate change”.

Students are active in choosing their projects, as reflected in the following interview with one student:

“We chose the topic of writing a play about climate change because we live with damage, the temperature rises a lot, and there are floods, so we chose to talk about this topic because we live in it. We wrote about it to make people aware that this is enough and we must live in security”.

These comments show that agency is a very central component in the program design and was expressed very significantly in the words of students and teachers.

(6) Data-driven pedagogy: the program placed data-driven pedagogy as a central principle in teaching and learning, and this dimension was rated as strong in light of its consistent and prominent appearance in the data. Learning in the program was based on systematic work with scientific and mathematical data related to environmental issues, and in particular climate change, with the aim of understanding complex processes, drawing evidence-based conclusions, and developing informed arguments. Different teachers describe this process:

“The students worked on an activism project where they calculated how cutting meat out of their diet would affect carbon emissions. When they actually saw the results of their calculations, it really convinced them that they can make a real impact on the environment”.

The significance of the data-driven pedagogy was also reflected in the words of the students, as exemplified by the following quote:

“When we saw the huge numbers and analyzed them, we were amazed at the amount of damage that humans cause to the environment through their behavior. It made us realize that our impact on the climate is much greater than we thought, and as a result, we began to change our behavior on a daily basis”.

4.5 Design features moderate expressed in influencing student attitudes and behavior toward environmental issues in the IP

(1) Diverse teaching resources: the program is built from tasks using different teaching methods, such as experiments, videos, practical activities, and articles. Indeed, the dimension of using diverse teaching resources emerged in the teacher interviews. Combining these teaching resources allows students to approach the subject in a rich and deep way. As one teacher said:

“The program contains learning processes and content that provide good responses to students' cognitive, emotional and social needs. These responses give expression to diverse learning styles and facilitate reinforcement and identification”.

The unique use made in the program of data analysis as a basis for presenting evidence for or against a given opinion or action related to climate issues was also expressed in various interviews. For example, one teacher said:

“The program caused students to understand that there are no decisions without a scientific basis. No decisions should be made without research and an analysis of the situation. And one cannot make a decision without relying on mathematical calculations, meaning there is a need to involve all sides to understand and analyze a certain topic”.

However, in the interviews, students did not talk at all about the diversity of learning methods. Perhaps they are accustomed to diverse learning methods and therefore didn't think this characteristic was unique to the program.

(2) Learners' ability to effectively manage environmental resources: although the program includes resource management tasks, interviews indicate this dimension is weaker than others. Students developed this skill through data-based research and activism, yet teachers did not address it. Development of resource management ability is well demonstrated in the following quote from one student:

“I'm willing to pay extra for using environmentally friendly products, or offer something additional from me to my community… We must reduce the use of materials harmful to the environment and use environmentally-friendly materials”.

4.6 Design features absent from an IP

Early childhood experiences in nature: we did not find any evidence that this criterion was present in the program.

5 Discussion

In the current study, we tested the validity of the model for developing and evaluating educational programs aimed at shaping learners' environmental attitudes and behavior using a dual approach: first, we examined an educational program that showed a positive effect on students' sustainability-related attitudes based on pre-post questionnaire data; subsequently, we analyzed interview data to identify the design features that were influential in producing this effect.

The questionnaire data showed that the program improved students' attitudes and behaviors related to climate change, whereas teachers did not demonstrate a comparable change. This pattern may be partly explained by teachers' relatively high pretest scores, suggesting a possible ceiling effect, as well as voluntary participation that may have resulted in a self-selected group with already positive environmental orientations. At the same time, it is important to note that the program was primarily designed to engage students directly, while teachers' involvement focused on facilitation and guidance. Unlike students, who actively participated in hands-on and action-oriented components, including forms of environmental activism, teachers were not systematically engaged in comparable experiential or activist practices. This difference in the nature and intensity of engagement may have contributed to the differential impact observed between the two populations. Nevertheless, both students and teachers reported an increased use of data and evidence-based reasoning, as this pedagogy was innovative for the teachers and the students. The emergence of data-driven pedagogy as a strong mentioned design feature highlights the importance of working systematically with scientific and mathematical data in EE. The findings of the study are consistent with the literature and show that engaging with authentic data supports the development of evidence interpretation skills, evidence-based argumentation, and a deep understanding of the complexity of climate and sustainability issues.^42,43 Within the socio-scientific issues framework, working with scientific evidence supports not only conceptual understanding but also the development of responsibility, judgment, and agency in relation to real-world environmental issues.^44,45 Teachers' and students' statements illustrated that quantitative thinking was highly expressed in both, the teachers and the students' interviews. This finding may suggest that data-driven pedagogy could contribute to the development of students' pro-environmental attitudes when it is applied as the teaching pedagogy of climate change programs. Therefore, we suggest to add this feature to the eight design features that were reviewed in the literature, as illustrated in Fig. 4.

Fig. 4 Summary diagram of the strength level of the design features in the Environmental Attitudes and Behavior Model including the suggested design feature “data-driven pedagogy”. This summary shows how, and to what extent, the design features were present in the IP experience.

Teachers and students were included in the study since teachers must model pro-environmental attitudes and behavior to foster them in students.⁴⁶ Teachers who act in this way not only transmit knowledge, but also encourage critical thinking, evidence-based decision-making, and the ability to act intelligently on environmental issues. When teachers are seen as change agents, they are able to connect disciplines to real-world problems, and strengthen students' sense of empowerment. Our findings echo the study of Çibik and Boz-Yaman³⁷ suggested that an interdisciplinary course that connected sustainability and mathematics contributed to the development of positive attitudes toward sustainability and enhanced the abilities of future teachers to deal with complex challenges using scientific and mathematical tools.

We analyzed the interview data to determine to what degree the program was based on the 8 different design features of the proposed evaluation model. Regarding the students, we found that 5 design features were strongly represented (social interactions, environmental knowledge, exposure to and recognition of environmental problems, intention to preserve the environment for learners and impact on emotional value and agency) 2 were moderately present (diverse teaching resources and Learners' ability to effectively manage environmental resources) and 1 design feature was weakly represented showing impact without all eight design features (early childhood experiences in nature). Although the eight-dimension model emerged from a comprehensive literature review, the interview analysis indicates that additional design features may complement the model; in particular, data-driven pedagogy was frequently mentioned by participants.

Regarding the teachers, most of the design dimensions received positive recognition, though with varying levels of emphasis. In the fourth design feature, “Intention to preserve the environment for learners,” there was a gap between teachers' and students' perceptions, while students frequently highlighted this aspect, teachers mentioned it less. This discrepancy may stem from teachers perceiving themselves primarily as pedagogical facilitators rather than as directly responsible for guiding students toward environmental preservation.^47,48 Additionally, in design features such as “Diverse teaching resources” and “Learners' ability to effectively manage environmental resources,” teachers' responses were only moderately represented. These responses may indicate a more limited use of diverse didactic tools or a perspective that places the responsibility for resource management primarily on students.

Taken together, the integration of quantification of the qualitative interview findings offers an initial perspective on how observed changes in students' and teachers' attitudes and behaviors may be associated with specific configurations of the model's design features. This combined analysis highlights not only which features were present in the IP, but also how they were differentially experienced by students and teachers, with the relationships among program design features, participant perceptions, and model dimensions summarized in Fig. 4.

As mentioned, one of the proposed model, early childhood experiences in nature, was not substantively represented in the program, reflecting the focus of the intervention on secondary school learners within a formal educational setting. This finding suggests that the model should not be interpreted as a checklist in which all features are expected to be present to the same extent in every program. Rather, the model is intended to support reflection on how different design features may be combined or emphasized in ways that are responsive to learners' developmental stage and educational context. The observed changes in students' attitudes and self-reported behaviors indicate that educational programs may foster meaningful outcomes through varied configurations of design features, even when certain features are only weakly represented. From this perspective, early childhood experiences in nature may play a more foundational role for younger learners, while later educational experiences, such as the one examined here, may draw more strongly on cognitive, social, and emotional learning processes. This interpretation underscores the importance of considering both developmental timing and contextual constraints when applying the model to the design and evaluation of EE programs.

From an analysis of the interview data, we can see there is a value to using the proposed evaluation model to evaluate the IP. The model enabled us to inquire which aspects of program were consistent with the 8 design features. This process allowed us to better inquire about and understand how the program influences learners' attitudes and behavior regarding the environment.

The program's uniqueness lies in using diverse, evidence-based strategies, behavioral science, technology, community engagement, and evaluation, to promote environmental attitudes and behavior, offering a tailored and comprehensive approach to developing EE.^36,49

Our main recommendation is that evaluators of EE programs use a dual approach, i.e., evaluating the learning outcomes (via a pre-post design) as well as the presence or absence of evidence-based design features shown to positively influence learners' attitudes and behavior (via the student and teacher interviews). This approach is theoretically grounded (based on evidence about what works), practically useful (providing actionable feedback), and methodologically comprehensive (addressing both process and outcomes). Our model offers design features, clear guidance and measurable standards, aiding curriculum development and evaluation. These benefits are particularly valuable when outcomes need to be measurable and demonstrable to stakeholders.

However, we recognize its benefits come with challenges.⁵⁰ For example, design features can be problematic, particularly where local contexts and environmental issues vary significantly. Critics highlight how strict adherence to predetermined design features can lead to a ‘checkbox mentality’ that prioritizes meeting formal requirements over achieving meaningful environmental learning outcomes.⁵¹ Environmental education, by its nature, deals with complex, interconnected systems and dynamic local contexts, aspects that may be oversimplified by fixed design features.⁵²

More flexible approaches, like design-based research (DBR) and participatory design,⁵³ combine structured design with adaptability to local contexts and needs. By emphasizing iterative development and stakeholder involvement, these methods address EE complexity while adhering to broader design features.

The key is balancing structure and flexibility by combining clear design features with adaptable, participatory approaches, creating well-structured EE programs that respond to local environmental contexts and learner needs.

An additional lens for interpreting the findings is systems thinking, which has become a central framework in contemporary chemistry education (e.g., 54–57). Systems thinking in chemistry emphasizes understanding chemical phenomena as part of complex, interconnected systems that integrate molecular-level processes with macroscopic, societal, and environmental contexts. The design features identified in this study, particularly data-driven pedagogy, exposure to environmental problems, social interaction, and the development of agency, align with key elements of systems thinking, including recognizing relationships among system components, reasoning across scales, and considering human – environment interactions. The program's focus on climate-related issues required students to connect scientific knowledge with real-world data and societal implications, supporting systems-level reasoning.⁵⁸ Moreover, the emphasis on agency positions learners as active participants within these systems, consistent with calls in chemistry education to link systems thinking with action on sustainability challenges. From this perspective, the proposed model can be seen as operationalizing systems thinking through concrete design features. Making this connection explicit may further strengthen the model's relevance for guiding the design of chemistry education programs that address complex global issues.

In 2003 Nelson Mandela stated that “Education is the most powerful weapon that we can use to change the world”.⁵⁹ Today the need to lead a change related to the SDGs is urgent and greater than ever. The path to sustainability is paved by education; therefore we need to invest a lot of effort in education to achieve a sustainable situation.⁶⁰

6 Limitations of the study

The study involved a small, convenient sample of teachers and students, limiting its representation of Israel's broader student and teacher population, and constraining the generalizability of findings. While the proposed model is grounded in widely accepted theoretical perspectives on environmental attitudes and behavior, the ways in which its design features are enacted, emphasized, or interpreted may vary across cultural, educational, and national contexts. Consequently, the relative salience of specific design features, as well as their contribution to observed outcomes, may differ in other settings. Replication studies conducted in diverse cultural and national contexts could provide important opportunities to test the robustness and cross-contextual applicability of the model, thereby strengthening its empirical foundation and refining its theoretical assumptions.

In addition, the use of a single-group pre-post design without a control or comparison group limits the ability to attribute observed changes solely to the intervention. Although the program examined demonstrated a positive impact on students' sustainability-related attitudes, assessing the magnitude or causal robustness of this impact was not the primary focus of the study. Rather, the observed positive outcome served as a criterion for selecting the program as a case for analysis. The central aim of the research was to both examine the design principles underlying an educational program with demonstrated positive impact on pro-environmental attitudes and behavior as well as to explore the model applicability to evaluate the specific design features that may have contributed to program effectiveness. Furthermore, the study relied on self-reported questionnaire data, which may be subject to response biases, including social desirability effects, and may not fully capture actual changes in students' behaviors. In addition, although the questionnaire was adapted from previously validated instruments, its factor structure was not re-examined within the present sample. This may limit the strength of the conclusions regarding the instrument's construct validity in this specific context. Accordingly, alternative explanations for the observed changes, such as maturation, concurrent school-based learning experiences, or broader societal and educational discourse on climate change and environmental sustainability, cannot be ruled out. These considerations suggest that the findings should be interpreted with appropriate caution and viewed as providing preliminary insights into the potential of the proposed design-based approach. Further research conducted in diverse educational and cultural contexts, and employing comparative or controlled research designs, would help to better delineate the conditions under which the model is most applicable and to clarify the role of specific design features in shaping learners' environmental attitudes and behavior.

Conflicts of interest

There are no conflicts to declare.

Data availability

Data will be shared upon reasonable request and following approval from the Institutional Review Board (IRB).

A Appendices

A.1 Attitude questionnaire – sustainable development goals – for teachers

The questionnaire is based on (Afroz, N., & Ilham, Z. 2020). * Items that were added for the current research.

A.1.1 Applying data in reasoning.

A.1.2 Attitudes towards sustainable development goals.

A.1.3 Behavior related to sustainable development goals.

A.2 Attitude questionnaire – sustainable development goals – for students

A.2.1 Applying data in reasoning.

A.2.2 Attitudes towards sustainable development goals.

A.2.3 Behavior related to sustainable development goals.

Acknowledgements

This research was supported by the Trump Foundation, Grant 361. The authors would like to acknowledge the work of the team participants that collaborated with us to develop the “Speak to Me in Numbers” program: Shoshana Intract, Prof. Moran Bodas, Dr Sharon Geller, Pazit Nager, Dr Ayelet Odani, Dr Alex Friedlander and Dr Ira Raiman.

References

1. S. Akaygun and E. Adadan, Fostering senior primary school students' understanding of climate change in an inquiry-based learning environment, Educ. 3-13, 2021, 49(3), 330–343.
2. W.-T. Fang, A. Hassan and B. A. LePage. The Living Environmental Education: Sound Science toward a Cleaner, Safer, and Healthier Future: Springer Nature; 2023.
3. C. E. Roth. Environmental Literacy: its Roots, Evolution and Directions in the 1990s. 1992.
4. W. Fang, W. Fang. Environmental Literacy: Behavior Oriented. Envisioning Environmental Literacy: Action and Outreach. 2020, pp. 69–108.
5. C. Roth, Elements of a workable strategy for developing and maintaining nationwide environmental literacy, Nat. Stud., 1984, 37, 46–48.
6. S. Hsu and R. Roth, An assessment of environmental literacy and analysis of predictors of responsible environmental behaviour held by secondary teachers in the Hualien area of Taiwan, Environ. Educ. Res., 1998, 4(3), 229–249.
7. R. Petty and P. Briñol, Emotion and persuasion: Cognitive and meta-cognitive processes impact attitudes, Cognit. Emot., 2015, 29(1), 1–26.
8. F. X. Bogner and M. Wiseman, Adolescents' attitudes towards nature and environment: Quantifying the 2-MEV model, Environmentalist, 2006, 26(4), 247–254.
9. R. Gifford and R. Sussman. Environmental attitudes. The Oxford Handbook of Environmental and Conservation Psychology. 2012:65–80.
10. A. Kollmuss and J. Agyeman, Mind the gap: why do people act environmentally and what are the barriers to pro-environmental behavior?, Environ. Educ. Res., 2002, 8(3), 239–260.
11. J. Burgess, C. Harrison and P. Filius, Environmental communication and the cultural politics of environmental citizenship, Environ. Plan. A, 1998, 30(8), 1445–1460.
12. S. Otto and P. Pensini, Nature-based environmental education of children: Environmental knowledge and connectedness to nature, together, are related to ecological behaviour, Glob. Environ. Change, 2017, 47, 88–94.
13. S. Collado and G. W. Evans, Outcome expectancy: A key factor to understanding childhood exposure to nature and children's pro-environmental behavior, J. Environ. Psychol., 2019, 61, 30–36.
14. G. W. Evans, S. Otto and F. G. Kaiser, Childhood Origins of Young Adult Environmental Behavior, Psychol. Sci., 2018, 29(5), 679–687.
15. K. Eom, H. S. Kim and D. K. Sherman, Social class, control, and action: Socioeconomic status differences in antecedents of support for pro-environmental action, J. Exp. Soc. Psychol., 2018, 77, 60–75.
16. S. F. Rallis and K. A. Bolland, What is Program Evaluation? Generating Knowledge for Improvement, Arch. Sci., 2004, 4(1), 5–16.
17. M. Karpudewan, Z. Ismail and W.-M. Roth, The efficacy of a green chemistry laboratory-based pedagogy: Changes in environmental values of Malaysia pre-service teachers, Int. J. Sci. Math. Educ., 2012, 10, 497–529.
18. J. Sukhera, Narrative reviews: flexible, rigorous, and practical, J. Grad. Med. Educ., 2022, 14(4), 414–417.
19. A. Eagly and S. Chaiken. The Psychology of Attitudes, Harcourt brace Jovanovich college publishers, 1993.
20. S. Bamberg and G. Möser, Twenty years after Hines, Hungerford, and Tomera: A new meta-analysis of psycho-social determinants of pro-environmental behaviour, J. Environ. Psychol., 2007, 27(1), 14–25.
21. Z. Xiong, Y. Song and R. Zhu, Pedagogical Strategies for Teaching Environmental Literacy in Secondary School Education: A Systematic Review, Sustainability, 2025, 17(20), 9104.
22. N. Wells and K. Lekies, Nature and the life course: Pathways from childhood nature experiences to adult environmentalism, Child. Youth Environ., 2006, 16(1), 1–24.
23. X. Zhang, W. Jung and M. Asari, Systematic Review of Environmental Education Teaching Practices in Schools: Trends and Gaps (2015–2024), Sustainability, 2025, 17(19), 8561.
24. N. M. Ardoin, A. W. Bowers and E. Gaillard, Environmental education outcomes for conservation: A systematic review, Biol. Conserv., 2020, 241, 108224.
25. J. M. Hines, H. R. Hungerford and A. N. Tomera, Analysis and Synthesis of Research on Responsible Environmental Behavior: A Meta-Analysis, J. Environ. Educ., 1987, 18(2), 1–8.
26. P. Liu, M. Teng and C. Han, How does environmental knowledge translate into pro-environmental behaviors?: The mediating role of environmental attitudes and behavioral intentions, Sci. Total Environ., 2020, 728, 138126.
27. E. Fytopoulou, E. Karasmanaki, S. Tampakis and G. Tsantopoulos, Effects of Curriculum on Environmental Attitudes: A Comparative Analysis of Environmental and Non-Environmental Disciplines, Educ. Sci., 2023, 13(6), 554.
28. R. Gifford and A. Nilsson, Personal and social factors that influence pro-environmental concern and behaviour: A review, Int. J. Psychol., 2014, 49(3), 141–157.
29. A. Grob, A structural model of environmental attitudes and behaviour, J. Environ. Psychol., 1995, 15(3), 209–220.
30. X. Liu, A. Vedlitz and L. Shi, Examining the determinants of public environmental concern: Evidence from national public surveys, Environ. Sci. Pol., 2014, 39, 77–94.
31. M. Wiseman and F. Bogner, A higher-order model of ecological values and its relationship to personality, Pers. Indiv. Differ., 2003, 34(5), 783–794.
32. C. A. Klöckner, A comprehensive model of the psychology of environmental behaviour—A meta-analysis, Glob. Environ. Change, 2013, 23(5), 1028–1038.
33. N. D. Aldawsari, M. A. Nemt-allah and M. S. Abdellatif, Environmental Education Awareness in Light of Sustainable Development Goals and Its Relationship with Environmental Responsibility Among University Students, Sustainability, 2025, 17(21), 9393.
34. M. Burmeister and I. Eilks, An example of learning about plastics and their evaluation as a contribution to Education for Sustainable Development in secondary school chemistry teaching, Chem. Educ. Res. Pract., 2012, 13(2), 93–102.
35. R. Romeo, F. Grita, F. Parisi and L. Russo, Vulnerability of Mountain Peoples to Food Insecurity, 2020.
36. S. Rap, R. Blonder, A. Sindiani-Bsoul and S. Rosenfeld, Curriculum development for student agency on sustainability issues: An exploratory study, Front. Educ., 2022, 7, 1–17.
37. N. Çibik and B. Boz-Yaman, The Effect of a cross-curricular course on pre-service teachers' sustainable development attitudes and mathematical modeling self-efficacy beliefs, Int. J. Sci. Math. Educ., 2025, 23(4), 1033–1056.
38. N. Afroz and Z. Ilham, Assessment of knowledge, attitude and practice of university students towards Sustainable Development Goals (SDGs), J. Sustain. Dev. Plann, 2020, 1(1), 31–44.
39. S. A. Karabenick, M. E. Woolley, J. M. Friedel, B. V. Ammon, J. Blazevski and C. R. Bonney, et al., Cognitive Processing of Self-Report Items in Educational Research: Do They Think What We Mean?, Educ. Psychol., 2007, 42(3), 139–151.
40. A. Sheḳedi, Words that Try to Touch: Qualitative Research - Theory and Application, Ramot - Tel Aviv University, Tel Aviv, 2003.
41. C. Vanover, P. Mihas and J. Saldaña, Analyzing and Interpreting Qualitative Research: after the Interview, Sage Publications, 2021.
42. D. Ben-Zvi and K. Makar, International Perspectives on the Teaching and Learning of Statistics, in The Teaching and Learning of Statistics: International Perspectives, ed D. Ben-Zvi, K. Makar, Springer International Publishing, Cham, 2016, pp. 1–10.
43. J. Fagerlund, L. Palsa and P. Mertala, Exploration of domains of educational purpose in K-12 data literacy education research, Educ. Res. Rev., 2025, 46, 100663.
44. D. L. Zeidler, B. C. Herman and T. D. Sadler, New directions in socioscientific issues research, Discipl. Interdiscip. Sci. Educ. Res., 2019, 1(1), 11.
45. S. Rap and M. Bodas, Innovate for Impact: Young Adults Education and Empowerment for Climate Action, Aust. J. Environ. Educ., 2024, 40(4), 784–794.
46. A. Basheer, A. Sindiani, O. Gulacar, I. Eilks and M. Hugerat, Exploring Pre- and In-service Science Teachers' Green Chemistry and Sustainability Awareness and Their Attitudes Towards Environmental Education in ISRAEL, Int. J. Sci. Math. Educ., 2023, 21(5), 1639–1659.
47. D. Robertson, B. Padesky and C. Brock, Cultivating student agency through teachers' professional learning, Theory Into Pract., 2020, 59(2), 192–201.
48. A. Basheer, O. Gulacar, A. Sindiani and I. Eilks, The Impact of an intervention on plastics and bioplastics on Pre-Service science teachers' green chemistry and sustainability awareness and their attitudes toward environmental education, Educ. Sci., 2025, 15(3), 322.
49. M. Young, B. Klemz and J. Murphy, Enhancing learning outcomes: The effects of instructional technology, learning styles, instructional methods, and student behavior, J. Market. Educ., 2003, 25(2), 130–142.
50. C. Monroe, R. Plate, A. Oxarart, A. Bowers and W. Chaves, Identifying effective climate change education strategies: A systematic review of the research, Environ. Educ. Res., 2019, 25(6), 791–812.
51. M. Hoadley, Methodological alignment in design-based research, Educ. Psychol., 2004, 39(4), 203–212.
52. L. SauvÃ, Currents in Environmental Education: Mapping a Complex and Evolving Pedagogical Field, Canadian Journal of Environmental Education (CJEE), 2005, pp. 11–37.
53. L. Tinoca, J. Piedade, S. Santos, A. Pedro and S. Gomes, Design-based research in the educational field: A systematic literature review, Educ. Sci., 2022, 12(6), 410.
54. F. M. Ho, Turning Challenges into Opportunities for Promoting Systems Thinking through Chemistry Education, J. Chem. Educ., 2019, 96(12), 2764–2776.
55. P. G. Mahaffy, S. A. Matlin, J. M. Whalen and T. A. Holme, Integrating the Molecular Basis of Sustainability into General Chemistry through Systems Thinking, J. Chem. Educ., 2019, 96(12), 2730–2741.
56. P. Mahaffy, S. Matlin, M. Potgieter, B. Saha, A. Visa and S. Cornell, Systems Thinking and Sustainability. Converging on Chemistry's Role in the 21st Century, Chem. Int., 2021, 43(4), 6–10.
57. V. Talanquer, Some Insights into Assessing Chemical Systems Thinking, J. Chem. Educ., 2019, 96(12), 2918–2925.
58. V. Talanquer and A. R. Szozda, An Educational Framework for Teaching Chemistry Using a Systems Thinking Approach, J. Chem. Educ., 2024, 101(5), 1785–1792.
59. N. Mandela Lighting your way to a better future, Speech Delivered by Mr. Nelson R. Mandela at the Launch of Mindset Network, University of the Witwatersrand, Johannesburg South Africa., Planetarium, 2003.
60. V. Kioupi and N. Voulvoulis, Education for sustainable development: a systemic framework for connecting the SDGs to educational outcomes, Sustainability, 2019, 11(21), 6104.
61. H.-F. Hsieh and S. E. Shannon, Three Approaches to Qualitative Content Analysis, Qual. Health Res., 2005, 15(9), 1277–1288.

---