View PDF VersionPrevious ArticleNext Article
 
DOI: 10.1039/C8RP00078F (Paper) Chem. Educ. Res. Pract., 2018, 19, 1081-1095

Structuring learning processes by ladders of learning: results from an implementation study

Helena van Vorst
Duisburg-Essen University, Schuetzenbahn 70, 45127 Essen, Germany. E-mail: helena.vanvorst@uni-due.de

Received 14th March 2018 , Accepted 19th June 2018

First published on 4th July 2018

Abstract

This paper introduces Ladders of Learning (LLs) as a tool for structuring learning content and the teaching process in a transparent way for students. Learning material for a LL for Bohr's atomic model was developed by cooperation between chemistry teachers and university researchers and implemented in grade-eight chemistry classes. For evaluating the effectiveness of LLs, a mixed method research study was conducted. In a quantitative pre–post study, students’ cognitive and affective outcomes were investigated by questionnaires and compared to the results of a control group. In addition, semi-structured interviews were conducted to analyse students’ views on the LL. The results of the quantitative data analysis showed positive effects of the LL on students’ learning achievement and their interest in chemistry. The results from the qualitative data analysis confirmed these positive findings. However, students’ feedback indicated differences between the views of high-performing students and students with a lower performance in chemistry. Overall, the results of the study emphasize the relevance of structuring as a variable of teaching quality.

Introduction

Structuring learning contents and learning processes in the classroom is an essential basis for a systematic, cumulative and sustainable growth of knowledge. Students also describe the teacher's structure of a lesson and the learning content as an important aspect of teaching quality as well (Broman and Simon, 2015). However, especially in science education the traditional method of teaching science, where isolated facts are taught based on overloaded curricula, is still criticized (Gilbert, 2006). These curricula are usually built on canonical science contents (Aikenhead, 2009) from the perspective of scientists with their personal experiences as academic experts and their subjective values of important scientific concepts, which should be taught at school (Duschl et al., 2007). This proceeding hinders cumulative learning processes, as it does not consider students’ initial concepts and resources and does not present central key ideas in a coherent and transparent way for students (Duschl et al., 2007). This leads to students’ negative attitudes towards science at school as being difficult, irrelevant and boring (Lye et al., 2001; Osborne and Collins, 2001; Aikenhead, 2009; Potvin and Hasni, 2014). To address this initial situation, numerous approaches have been developed over the last few decades to overcome the traditional method of teaching isolated facts by structuring core scientific concepts and the teaching process. However, Supanc et al. (2017) noticed that only little is known about the effects of structuring on students’ outcomes.

This paper introduces Ladders of Learning (LLs) as a tool for structuring a learning content and the following learning process in a chronological and hierarchical order. As LLs are based on a graphical representation, they are visible for students in the classroom and guide them through their learning processes. Each rung of the LL (called a milestone) consists of separate elements, which represent a step in the learning process. Symbols and numbers in each element illustrate the learning activity and the level of difficulty of the task. In this paper, the basic concept and design principles of a LL are presented and the general effectiveness of this newly implemented structuring method is examined in comparison to a control group representing the traditional teaching style. Thereby, the focus is on the effects on both students’ cognitive and affective outcomes.

Theoretical framework

Since the 1950s researchers have been focusing on components to describe the quality of educational settings with the aim of identifying characteristics of effective teaching and learning at school. On this basis, numerous frameworks of teaching quality can be found with a varying number of components. One of the most frequently demanded characteristics in these frameworks refers to a clear and structured way of teaching (e.g.Bolhuis, 2003; Seidel and Shavelson, 2016; Taut and Rakoczy, 2016). In this paper, this aspect of teaching quality is summarized under the term structuring. The previous research literature does not provide a consistent conceptualization or operationalization of structuring, but presents differing terms for this aspect of teaching quality, like structuring, goal setting/orientation, classroom management, and an enormous amount of approaches for realizing a structured way of teaching and learning at school. Generally, Hospel and Galand (2016) defined structuring as “[…] the amount and the clarity of information given to students about how to satisfy teachers’ expectations and achieve the desired educational outcomes” (Hospel and Galand, 2016, p. 1). Jang et al. (2010) described three characteristics of a well-structured learning environment: (a) presenting an explicit and understandable direction for the learning process, (b) guiding students’ activities by a program of action, and (c) giving constructive feedback on students’ success. The opposite of structuring is chaos (Skinner and Belmont, 1993), where teachers do not communicate their expectations and do not provide any help or guidance, so students are working in a confusing environment without a clear objective of their learning process (Jang et al., 2010). In his meta-analysis, Hattie (2009) termed structuring elements as learning intensions. They serve as an orientation for teachers and students about the content that has to be learned. One central component of learning intentions is setting appropriate goals, which connect the current state with a future desired state in the learning process. Under goal orientation or goal setting, Seidel and Shavelson (2016) understand the “[…] investigated clarity of goals, clear and structured teaching, activation of student pre-knowledge, or use of anchors and contexts in explaining learning contents” (Seidel and Shavelson, 2016, p. 470).

There are various methods for realizing structuring in the classroom. One approach can be a reasonable content structure, which results from sequencing a learning content in a plausible way by considering students’ initial knowledge as well as theoretical and empirical research results. Additionally, a systematic teaching structure offers a further option for realizing structuring in the classroom, which for example includes the sequencing of a lesson as well as a meaningful use of teaching methods, and hence, refers to aspects of successful classroom management.

In order to make a content structure visible for students, e.g.Hattie (2009) suggested using advance organizers. This method goes back to Ausubel (1960), who invented advance organizers as a way of structuring a new learning content and presenting it to students in advance to their learning process. Thus, advance organizers are supposed to facilitate a systematic building of knowledge by activating students’ existing cognitive structure and anchoring a new content with it. In the original version, advance organizers are written texts, which summarize the key concepts of a new content. In further approaches, advance organizers were built in different variations, such as tables, graphics or concept maps. In general, these different approaches of advance organizers have in common that they are designed by the teacher in advance and are presented to the students before they start their learning processes. Concept maps are further approach for structuring knowledge during or after the learning process. A concept map can be defined as a graphical representation, which illustrates concepts and their relationships between each other (Novak and Gowin, 1984). Concepts are illustrated by labeled notes, which are connected with labeled or unlabeled, directed or undirected links (Nesbit and Adesope, 2006). Concept maps are often designed by students themselves or their teachers (Fechner, 2009) and aim in facilitating a systematic development of cognitive structures of a new learning content. Hence, concept maps are in line with a constructivist view of learning processes (Novak, 2002).

In addition to a coherent content structure as an aspect of structuring, the research literature describes a systematic teaching structure as a further facet of structuring. Bolhuis (2003) discussed this aspect under the term “learning activities”. She describes a lifelong learning process as a combination of activities from different categories, which are connected to each other and to the social environment. Consequently, it is a central teacher's task to introduce typical learning activities in a domain to students and to facilitate students’ acquisition of knowledge by appropriate activities in the classroom (Bolhuis, 2003). Seidel and Shavelson (2016) differentiated the Bolhuis’ category of learning activities in three further components: (1) social interaction/direct experiences, which include cooperative learning activities in the classroom, (2) basic information processing, referring to verbal and symbolic information, which are basic for students’ cognitive activities, and (3) domain-specific information processing, which includes typical learning activities of a specific domain. When these activities are organized in a coherent and reasonable way during a lesson, this leads to a functional classroom setting, and hence, builds a facet of successful classroom management (Rakoczy et al., 2007; Seidel and Shavelson, 2016; Taut and Rakoczy, 2016).

Although the literature provides a large number of approaches for structuring a learning content or the learning activities in the classroom, there are only rare methods for combining these two facets of structuring, and hence, for giving students an overview of their whole learning process. One method that connects a systematic content structure with an appropriate teaching structure is Ladders of Learning (LLs).

The concept of ladders of learning

Müller et al. (2015) introduced the general concept of Ladders of Learning (LLs). In their original form, LLs structure the learning content (content structure) and the ongoing learning process in the classroom (teaching structure) in a linear and hierarchical way. Fig. 1 illustrates the exemplary structure of a LL.
image file: c8rp00078f-f1.tif
Fig. 1 Exemplary structure of a LL.
Milestone. In order to structure a learning content, it is divided into different units, which are ordered one after the other. In the LL, each unit is taught within a so-called milestone (marked green in Fig. 1). A milestone represents a rung of the LL and builds its basic structure.
Element. The learning process is structured with the help of elements (marked red in Fig. 1), which have a standardized order within each milestone:

Element 1 (Introductory): students learn a new concept and its essential technical terms, guided by the teacher and supplemented by cooperative teaching methods.

Element 2 (Reinforcement): the new concept is practiced in differing social forms.

Element 3 (Evaluation): this element should show whether a student has understood the core ideas and is able to apply them.

Element 4 (Remedial work): based on the results of the evaluation, students work on their insecurities and mistakes.

Element 5 (Enrichment): students transfer their new knowledge and skills on a new context and strengthen their abilities.

Different activities support each element in a LL. They are labeled with the help of symbols. Additionally, a number codes the level of difficulty of each element. This proceeding leads to a visual representation (see e.g.Fig. 1), which is printed on a poster, and hence, functions as an advance organizer by visualizing both the content and teaching structure of a unit in the classroom. The activities and tasks of an element are explained on activity cards, which are marked by the symbol and the number of the element in the heading. This proceeding provides a goal-oriented and structured direction for students’ self-regulated work.

This original form of a LL follows a linear structure. However, the design of a LL can be adapted to differing content structures and allows the construction of different types of LLs with different layouts. In addition to a linear structure of the original LL, a systemic form comparable to a concept map can be used to visualize the complexity of a learning content and/or the activities during the learning process. Moreover, a combination of a linear and a systemic LL is an additional option. Fig. 2 illustrates the different types of LLs schematically.


image file: c8rp00078f-f2.tif
Fig. 2 Different types of LLs.

As LLs provide an overview of the structure of a new concept, students’ prior knowledge is activated, which facilitates the integration of the new concept in existing knowledge structures. Additionally, by illustrating the planned teaching structure, students get the opportunity to plan their learning activities and to find appropriate learning strategies. Thus, LLs are a tool to combine a systematic content structure with a transparent teaching structure, which guides the students through their learning process over a longer period of time. Furthermore, for teachers and curriculum developers, LLs are sufficiently flexible to adopt the content structure or the underlying teaching methods based on individual conditions or learning groups, as different elements can be added, removed, changed or rearranged.

Research results regarding structuring

Over the last few decades, numerous empirical approaches have been investigated to analyse the effects of a systematic content structure or teaching structure on students’ cognitive outcomes and their emotional states. Focusing on research on characteristics of the teaching quality in general, the results of numerous studies illustrate the relevance of structuring as an important variable for successful teaching. A meta-analysis by Wang et al. (1993) analysed general factors influencing successful learning at school. Besides variables like political, cultural and social influences of the school environment, the authors considered instructional influences and here, classroom practices in particular, including instructional techniques, rehearsal practices or sequencing of a lesson. The highest effect size was found for students’ characteristics like motivation, gender, and social behaviour. Classroom practices showed a comparably high effect size, and hence, were one of the most influential variables on students’ learning. However, the operationalization of classroom practices was very broad. It is not possible to clarify which aspect of classroom practices led to this effect.

Seidel and Shavelson (2016) had a more focused view on effective teaching characteristics and analysed the effects of teaching components visible in the classroom. They considered 112 studies from the years 1995 to 2004 for their meta-analysis. In addition to organizational and social components, like time for learning or the social context, the authors included studies on the effects of goal-setting/orientation as a variable for the content structure and execution of learning activities as a variable for the teaching structure. Looking on effects on students’ cognitive outcomes, only small effect sizes could be found for all regarded components. Besides the time for learning and differentiation, structured teaching was the most influential variable. However, structuring played only a subordinated role for students’ motivational and affective outcomes. Here, components like cooperative learning and time for learning showed the highest effect sizes. In the next step, Seidel and Shavelson (2016) differentiated between the three facets of executing learning activities (social interaction/direct experiences, basic information processing, and domain-specific information processing). The results emphasize the relevance of domain-specific information processing, which refers to typical and most adaptive learning activities in a domain, and hence, represents a variable for the teaching structure. This variable showed the highest effects on students’ cognitive and motivational outcomes in the analysis.

Focusing on content structuring, in the 1960s David Ausubel started analysing the influences of advance organizers as a method for structuring learning processes. In one of his first studies, he investigated university students’ retention from text learning in a control group design (Ausubel, 1960). In the experimental group, Ausubel introduced text-based advance organizers, containing background concepts on a more general level in advance to the learning process. In comparison, students in the control group received historical background information before the intervention. The results showed a significant positive effect for students’ retention in the experimental group. Ausubel's publication of his first results led to a wide distribution and further development of advance organizers in the next few years. Luiten et al. (1980) conducted a meta-analysis on the effects of advance organizers, considering 135 studies. Their results confirm the positive influences of advance organizers on students’ learning and retention with an increasing effect over time. In his synthesis of meta-analysis, Hattie (2009) summarized eleven meta-analyses on the effects of advance organizers and found an effect size of d = 0.41. However, the effects of advance organizers differ significantly depending on the form of presentation. Overall, it can be concluded that text-based advance organizers are less effective than non-written ones (Luiten et al., 1980; Hattie, 2009).

Nesbit and Adesope (2006) focused on the effects of concept mapping as an additional tool for content structuring. The authors included 55 studies in their meta-analyses. The focus of most of the studies was on students’ cognitive performance outcomes. Compared to alternative learning activities such as text reading or writing summaries, concept maps led to better knowledge retention and transfer. In individual learning environments, students benefited from preconstructed maps, but this result could not be transferred to cooperative learning. Additionally, a few studies indicate an advantage of preconstructed concept maps especially for students with a lower verbal proficiency and lower performance. However, only six out of 55 included studies measured the affective outcomes such as motivation, self-efficacy or satisfaction. Considering the small number of studies, Nesbit and Adesope (2006) found a positive effect of concept mapping on all affective variables. Hattie (2009) also analysed the effectiveness of concept maps by considering the effect sizes of six meta-analyses. He reported a medium effect size of d = 0.57. Willerman and Mac Harg (1991) implemented an advance organizer in the form of a concept map in eight-grade physical science classes, and hence, combined two methods of content structuring. They found a positive effect on students’ learning outcomes after a two-week intervention compared to a control group. A study of Holländer (2010) could not confirm this positive effect on students’ learning achievement in chemistry education in general. As a possible reason, she discussed that the teachers, who have realized the intervention in the classroom, have a big influence on the effect measured in her study. In total, eight teachers participated with two parallel classes in Holländer's study, one class in the intervention group and the second class in the control group. During the data analysis, it became obvious that the effects of the Advance Organizers varied significantly depending on the individual teacher. Additionally, Holländer (2010) included questionnaires regarding students’ attitudes towards chemistry in her investigation, but no influences on students’ attitudes towards chemistry were found.

Comparing the studies on the content and teaching structure overall, it becomes obvious that researchers focus on the effects of students’ cognitive outcomes predominantly, while affective variables only play a minor role. A study by Skinner and Belmont (1993) analysed the interdependence between teachers’ behaviour and students’ engagement and students’ willingness to participate in the learning process in the classroom actively, which both refer to emotional components. As a variable of teachers’ behaviour, Skinner and Belmont (1993) analysed in which way teachers provide structures, e.g. by communicating clear goals, offering necessary help and choosing appropriate teaching strategies. A path analysis showed that students’ behavioural engagement in the classroom was influenced primarily by the teacher's structure. However, students’ emotional engagement was influenced by the teacher's involvement, which describes the interpersonal relationship between the teacher and students. Structuring had no influence on students’ emotional engagement. Rakoczy and collegues (2007) analysed the effects of structuring on the cognitive and motivational aspects in mathematics education. They differentiated between a structured presentation of the learning content and a structured organization of the learning environment. As a result, the authors found a positive effect of a structured presentation of the learning content only on students’ learning achievement, while a structured organization of the learning environment influenced both students’ achievement and motivation.

A tool for realizing both aspects of structuring in the classroom is LLs. Although they are implemented in numerous schools all over the world, the effectiveness of this concept has not been evaluated empirically until now (Müller et al., 2015). Hence, the main aim of the present study is to answer the following research questions:

(1) What effect does a LL have on students’ achievement in a regular chemistry class?

(2) What effect does a LL have on affective outcomes in a regular chemistry class?

(3) What are students’ experiences of the use of LLs during regular chemistry lessons?

Methods

Design and participants

This study had a pre–post control group quasi-experimental design in order to detect the effects of the LL on students’ learning achievement and their affective outcomes. For answering research questions (1) and (2), quantitative data were collected before and after the intervention. Additionally, qualitative interviews were conducted with some of the students in order to answer research question (3). Table 1 gives an overview of the proceeding of the study.
Table 1 Proceeding of the study
Pre-test Intervention Post-test
• Cognitive ability test Intervention group: • Content knowledge test
• Demographic data • LL on Bohr's atomic model • Intrinsic motivation, content- and subject-related interest
• Content knowledge test • Semi-structured interviews • Extrinsic motivation
• Intrinsic motivation, content- and subject-related interest Control group: • Chemistry-related self-concept
• Extrinsic motivation • Regular chemistry lessons on Bohr's atomic model
• Chemistry-related self-concept

The intervention was conducted in 17 chemistry classes of five upper secondary schools (German: Gymnasium) in the German federal state of North Rhine-Westphalia during the school year 2016/2017. Five additional upper secondary schools participated as a control group. Unfortunately, one school was not able to participate in the post-test because of organizational reasons. This leads to a full sample size of 746 students (295 students in the control group and 451 in the intervention group). For data analysis, a sample of 534 students could be included, as some of the students did not participate in one of the two points of measurement.

The developed learning material dealt with the chemical content of Bohr's atomic model. As chemistry is a compulsory subject in upper secondary schools of North Rhine-Westphalia in grades 7 to 9, this topic has to be taught obligatorily to all students. To be in line with the curriculum of North Rhine-Westphalia, the study was conducted in grade 8, where the topic of Bohr's atomic model is intended to be learnt. The chemistry teachers have implemented the LL in the regular chemistry class. As the teachers of the intervention group were involved in the development of the learning material for the LL, these teachers were familiar with the unit and the central ideas of the LL. In order to get an insight into the realization of the LL in the classroom, teachers have filled in an implementation diary after every lesson. In this diary, teachers have documented how much time they have spent on the milestone, and had the opportunity to give any comments on the material and its implementation in the classroom. Based on these implementation diaries, the teachers spent 15 lessons on average on the LL. In the control group, students learned the content of Bohr's atomic model in a more traditional way, but there were no instructions regarding the content or teaching structure for the teachers of the control group. The aim of this approach was to compare the effects of the LL with the “usual” way of teaching. To ensure that students in the control group learned the same chemical concepts, their teachers were asked for a summary of the concepts that they taught during the unit about Bohr's atomic model in order to guarantee that these students had a fair chance in the content knowledge test.

Additionally 13 semi-structured interviews with 45 students from the intervention group were conducted. The aim was to analyse students’ personal experiences with the LL in comparison to their experiences during their regular chemistry lessons. For this purpose, the teachers were asked to select three students from their classes (one high-performing student, one low-performing student and one student with an average performance in chemistry) for the interviews and parents had to give written permission. The students were interviewed by a university researcher in small groups of three to six participants at school during the regular chemistry class in a separate room, where no other person could hear the students’ answers. For data analysis, the interviews were audiotaped anonymously.

Development of the learning material

The intervention of this study is based on a LL for Bohr's atomic model. According to Müller et al. (2015), LLs structure the content in separate milestones, which are based on each other and are supposed to facilitate a systematic development of knowledge. Hence, the content of Bohr's atomic model was divided into four milestones, representing the central concepts, which are described in the curriculum. The following milestones were developed:

(1) Rutherford's atomic model (Rutherford's scattering experiment, nucleus, atomic shell, and mass of an atom)

(2) Nucleus (protons, neutrons, mass number, atomic number, and isotopes)

(3) Bohr's atomic model (electrons, atomic shell, and ionization energy)

(4) Periodic table (the order of elements in the main groups and the octet rule)

In the original LL, the elements in each milestone follow a standardized five-step process structure, which was introduced before (Müller et al., 2015). For this study, the original process structure was adopted in the following way:

Element 1 (Introductory): students learn new concepts and technical terms, guided by the teacher.

Element 2 (Basic exercise): students practice basic ideas of the concept by themselves or in small groups.

Element 3 (Self-evaluation): students evaluate their understanding of the new concept with the help of a self-evaluation questionnaire, adopted from Kallweit (2015).

Element 4 (Individualized exercise): based on the results of the self-evaluation, students choose appropriate exercises for closing their knowledge gaps or for deepening and transferring their knowledge.

Element 5 (Final evaluation): teachers evaluate students’ learning outcomes by a test.

This five-step structure is repeated within each milestone, except for the final evaluation element, which is included in the LL at the end of every second milestone. Fig. 3 shows an overview of the LL with its four milestones, designed for this study. To realize a coherent learning process, a linear form of the LL was chosen as a basic structure. Additionally, a systemic sequence was integrated in the element of individualized exercises (4), where students can choose one out of five possible exercises based on the results of the self-evaluation.


image file: c8rp00078f-f3.tif
Fig. 3 LL designed for this study.

Within an element of the milestone, each activity is labeled by a symbol. An additional number refers to the number of the milestone. The central concept and technical terms of the milestone are listed in the first element. The whole LL (see Fig. 3) is printed as a visual representation on a poster, which makes the content and teaching structure of the unit visible for students in the classroom. Additionally, each student gets a copy of the poster for his or her exercise book. The respective activities and tasks are explained on activity cards, which are marked by the symbol and the number of the element in the heading. Fig. 4 shows an example of an activity card. This process facilitates students’ individual and self-regulated learning within the LL.


image file: c8rp00078f-f4.tif
Fig. 4 Example of an activity card from the LL (task for element (2) ‘basic exercise’ from the first milestone).

The learning material for the LL was developed in cooperation with ten teachers from upper secondary schools during five one-day workshops. To guarantee a standardized proceeding in the development of the learning material, teachers got a manual, which described the relevant parameters for designing the learning tasks. Furthermore, university researchers have supported the teachers during the development process. This led to a set of differentiated learning tasks, which aim to facilitate students’ learning of the central concepts on three different levels of difficulty. Fig. 5 shows an example of how the different levels of difficulty were realized for one task within the element of an individualized exercise.†


image file: c8rp00078f-f5.tif
Fig. 5 Examples of individualized exercises with different levels of difficulty: (top) an exercise with the lowest level of difficulty; (middle) an exercise with medium difficulty; and (bottom) an exercise with high difficulty.

Data collection tools

Students’ cognitive and affective variables were collected before and after the intervention by using existing questionnaires. For evaluating students’ learning outcomes, a multiple-choice single-select content knowledge test was conducted. This test relates to the chemical content of Bohr's atomic model and consists of 35 items from the studies of Dollny (2011), Holländer (2010) and Weber (in press). As each item was valued at 1 point, students could achieve a maximum of 35 points in this test. In the pre-test, the content knowledge test had a Cronbach's α reliability of 0.29. This very low value results from the fact that students did not have any prior knowledge about Bohr's atomic model, and hence, were only able to guess the right answers. In the post-test, a Cronbach's α value of 0.88 indicates a good reliability of the content knowledge test.

In order to investigate students’ affective variables, a Likert-scaled questionnaire from Fechner (2009) was used. The first section of the questionnaire asked for students’ content- and subject-related interest in chemistry and their intrinsic and extrinsic motivation on a four-point Likert scale, while the second section focused on the chemistry-related self-concept on a five-point Likert scale. The questionnaire includes 26 items in total. An exploratory factor analysis with Varimax rotation was conducted for analysing the factor structure of the pre-test questionnaire. According to the Kaiser–Meyer–Olkin measure, the sample is adequate for the factor analysis (KMO = 0.94), and all KMO values for individual items were higher than 0.615, which is above the acceptable level of 0.5 (Field, 2016). After running the analysis, five factors could be extracted with eigenvalues over 1. These factors explain 58.93% of the variance. The first factor combines the items of students’ content- and subject-related interest in chemistry as well as their intrinsic motivation. The second factor summarizes the items of students’ chemistry-related self-concept, while the third factor includes the items of students’ extrinsic motivation. The items of the fourth factor were meant to measure students’ intrinsic motivation as well, but as these items were also loaded on the first factor, they were excluded from the following analyses. The fifth factor consisted of only one item, and hence, was not considered in the data analysis as well. A factor analysis of the post-test questionnaire confirms the results of the pre-test and leads to the same three scales. Table 2 gives an overview of the reliability values and some exemplary items for the three scales on students’ affective variables.

Table 2 Reliability and exemplary items for students’ interest and self-concept in chemistry
Scale Item number Exemplary items Reliability pre-test [Cronbach's α] Reliability post-test [Cronbach's α]
Subject- and content-related interest (SCI) 11 I like working on chemical tasks 0.917 0.925
I am looking forward for the next chemistry lesson
Chemistry-related self-concept (CSC) 7 I understand the chemical content during the chemistry lessons 0.873 0.898
I actively participate in chemistry classes
Extrinsic motivation (EM) 3 I participate in chemistry classes, because I want my parents to be satisfied with me 0.591 0.619

While the reliability of the SCI and CSC is good to excellent, Cronbach's α for EM is low. The low number of items in this scale might cause this fact.

In addition to the quantitative data collection, students’ perceptions about the use of the LL in their chemistry classes were investigated during the intervention with the help of semi-structured interviews. For this purpose, the following interview guideline was used (Table 3).

Table 3 Interview guideline for semi-structured interviews
Category Question(s)
1 General information Which class do you attend?
Which mark did you have in chemistry on your last report?
2 Experiences during regular chemistry lessons (before a LL was implemented) What were your experiences with your previous chemistry lessons?
→ What did you like?
→ What didn’t you like?
→ Did you experience that there were different tasks for different students in your previous chemistry lessons?
3 Experiences with a LL What are your experiences with the LL?
→ What do you like?
→ What don’t you like?
→ Which element do you like most in a milestone?
→ How do you cope with the self-evaluation element?
How could we improve the LL?

Interview data were collected in the German language, but selected statements were translated literally into English for reporting them in this paper.

Data analysis

Analysis of the quantitative data was done with the help of the software IBM SPSS® (Version 25). Due to the big sample size, the normality of the data was verified graphically (P–P plots and histograms) instead of using the Kolmogorov–Smirnov test, because this test easily becomes significant for big sample sizes, and hence, is not sufficiently reliable for getting information about the normality of data (Field, 2016). As both P–P plots and histograms for all scales illustrate a normal distribution of the data, the parametric tests (t-tests and ANOVAs) were conducted for data analysis.

Interview data were recorded during the interviews and transcribed for analysis in MAXQDA® software. A qualitative content analysis was conducted according to Mayring (2014). Therefore, categories were deduced from theoretical assumptions about possible students’ answers and then tested and supplemented on a randomly selected sample of four interviews. In the next step, all statements were labeled according to the resulting categories by two independent coders. The reliability analysis of the interview coding was calculated using the percentage of the intercoder agreement, which was about 83%.

Results

Students’ achievement

The first research question focuses on the effect of the LL on students’ learning outcomes in the chemical content of Bohr's atomic model after the intervention in comparison to a control group. This group was taught within a more traditional classroom setting, without using LLs. Fig. 6 shows a general overview of students’ learning outcomes by comparing pre- and post-test results. To analyse the effect of the intervention, a one-way repeated-measure ANOVA was calculated, where the pre- and post-test results were mentioned as a within-subject variable, while the teaching group (intervention group [IG] or control group [CG]) was mentioned as a between-subject variable. The results confirm that students’ knowledge about Bohr's atomic model increased significantly during their chemistry lessons (F(1524) = 1948.999, p < 0.001, and ηp2 = 0.788). A significant influence of the teaching group, which was included in the analysis as a between-subject variable, indicates that students from the IG get a significantly higher learning achievement than the students in the CG (F(1524) = 144.589, p < 0.001, and ηp2 = 0.179). These results are confirmed by independent t-tests. There is no difference between CG and IG in the pre-test (t(529) = 1.008, p = 0.314, and d = 0.089), but a significant difference in the post-test with a high effect size (t(527) = 11.175, p < 0.001, and d = 0.992).
image file: c8rp00078f-f6.tif
Fig. 6 Mean scores for students’ content knowledge results in the pre- and post-tests.

Students’ affective variables

The second research question asks for the effects of the LL on students’ affective outcomes. Therefore, students’ subject-related and content-related interest (SCI), their chemistry-related self-concept (CSC) and their extrinsic motivation (EM) were surveyed with a Likert-scaled questionnaire before and after their lessons about Bohr's atomic model. The results of the paired sample t-tests, comparing the pre- and post-test results, show that students’ interest in chemistry (SCI) decreased during the unit about Bohr's atomic model within a more traditional learning environment (t(204) = −2.606, p ≤ 0.01, and d = −0.15), whereas students who have learnt with the LL showed a stable rating for SCI (t(293) = 0.269, p > 0.10, and d = 0.01). When looking at the results for EM, no significant differences between pre- and post-tests can be found for both CG (t(212) = 0.253, p > 0.10, and d = 0.02) and IG (t(304) = 0.821, p > 0.10, and d = 0.07) students. The last scale of the questionnaire focused on students’ chemistry-related self-concept (CSC). No difference can be reported for students of the CG (t(211) = −0.554, p > 0.10, and d = −0.03). In comparison, students’ CSC increases significantly (t(303) = 3.943, p < 0.001, and d = 0.18) during the intervention with the LL in the IG (Table 4).
Table 4 Mean values and standard deviations of students’ ratings of affective variables in the pre- and post-tests
Pre-test Post-test
M SD M SD
Note: SCI: subject-related and content-related interest, CSC: chemistry-related self-concept, and EM: extrinsic motivation.
CG SCI 2.65 0.64 2.55 0.72
CSC 3.38 0.79 3.35 0.82
EM 2.44 0.58 2.45 0.62
IG SCI 2.64 0.64 2.64 0.63
CSC 3.45 0.67 3.58 0.71
EM 2.51 0.57 2.52 0.59

Students’ experiences during the intervention

Semi-structured interviews were conducted with students from the IG in order to evaluate students’ experiences with the LL. The qualitative interviews were guided by two major questions:

(1) What were your experiences with your previous chemistry lessons?

(2) What are your experiences with the LL?

The students were asked for both positive and negative aspects without requesting a topic they had to focus on. Hence, the students were free to talk about any aspect of their experiences during their previous and current chemistry lessons. The first section of the interviews focused on students’ evaluation of their previous chemistry lessons. As this question was formulated as quite open, the students usually started with a general feedback, especially regarding their interest in chemistry and their understanding of chemical contents during their chemistry lessons. An additional important theme for the students was methodological issues, like conducting experiments, reading and writing texts or the teacher-fronted explanatory instruction. Overall, the students reported about a rather traditional way of learning and teaching in chemistry. Predominantly, this came along with a rather negative view on chemistry education. Students argued that their chemistry lessons were often teacher-oriented. A majority of the students characterized this teaching style as being boring. In particular, medium- or low-achieving students in chemistry criticized in this context that they had difficulties in understanding the content because teachers’ explanations were too fast. Ben's experiences are one example for this perspective, ‘To be honest, I didn’t like my chemistry course. I think, well, I don’t know, if it was because of my teacher, but chemistry was boring, because we didn’t conduct so many experiments, but he [the teacher] was always writing something on the blackboard, that he didn’t really explain and we didn’t understand anything. And when we had to write a test, I had to learn all the content at home by myself.’ As an additional aspect of difficulty, students focused on formulas and stoichiometric calculations. Asking the question about what students did not like in their previous chemistry lessons, Tim and Lea, two students from one class answered: Tim, “Too many formulas.” Lea, “Yes. We just had to memorize all the formulas.” Lisa, a student from a different class added, “Yes, and too few experiments. That was … for me it was boring. And we had to learn too many theories and a lot of … too many numbers and formulas and calculations. And at one point I decided ‘Ok, you will never understand that.’”

High-achieving students in chemistry described their classroom situation in chemistry very similarly, but had a more positive view on the more traditional way of teaching and learning. Esra's statement gives an example for this perspective, “Our previous lessons in chemistry were ok, because I like this normal way of teaching, when the teacher is standing in front of the class and then we are talking about a topic all together in class.”

When the interviewer asked students for positive aspects of their chemistry lessons, almost all students focused on experiments. In general, the role of experiments was one of the most mentioned topics in the interviews. Emma's and Jona's position is a typical view of students during the interviews on experiments in chemistry education. Emma, “Well, the lessons were very monotonous. This is why I didn’t like it. But I enjoyed the experiments. That was really fun.” Jona, “Chemistry lessons were not so bad, they were ok and sometimes, not so often, we went to the lab or we had these experiments in front of the class. That was good and very interesting.” When students reported that they have conducted experiments during their chemistry class, they rated this positively, whether or not they had a positive or negative opinion about their chemistry course in general. In contrast, when students’ reported about infrequent experiments in chemistry, they rated this negatively. However, students from only three classes criticized a lack of experiments in their chemistry courses, while the other students indicated that their teachers include experiments regularly in chemistry lessons.

Asking students for their experiences with differentiated instruction during their previous chemistry lessons, a majority of the students denied this question. Only students from three classes reported about any methods of differentiation in class. However, one of these students had attended a private school before and had these experiences there.

In the main part of the interview students focused on their experiences with the LL. Mostly, students started their feedback on the LL with a comparison to their previous chemistry lessons. A majority of the students said that they preferred the LL compared to their former chemistry lessons. Only two students preferred the more traditional method of teaching. Sarah gave the following reason for her positive attitude toward the LL “I like it [the LL] much more than the normal lessons because I understand the content much better than before and so, I would like to continue with the Ladder of Learning, but unfortunately, I think we are going to finish it soon.” As it was the case for Sarah, for most of the students their understanding of the chemical content was an important factor for their positive feedback. Students noticed this in their preparation for the tests at the end of a milestone. Nick summarized this in the following way: “Now we learn everything we need for the test during the lessons, so I don’t have to work at home so much.” In their statements, many students linked their feeling of competence and understanding with the element of individualized exercises. When the interviewer asked the students to reason their positive feedback on the LL, Samuel for example answered, “Because we get these different tasks. For example, somebody who doesn’t understand the content well, gets an easier task and the others, who are better in chemistry, they get more difficult tasks.” A majority of the students appreciated the opportunity to work on individual strengths and weaknesses. Although some students admitted that they coordinated their individualized exercises with their friend, they stressed the value of self-evaluation in particular as a tool to think about personal learning progress during the unit and not only during the preparation for a test. Samuel continued his argumentation for the LL in the following way, “[…] and the self-evaluation is also very good, because everyone can look by himself, if you got the content or if you have any difficulties and then, you can work on your problems. We have never done this in our regular chemistry lessons.” Natalie added, “Yes, I would also say that it is [the self-evaluation] very good, because if you don’t answer the questions honestly, then it's only a disadvantage for you, because it's stupid, if you don’t know something and, only because you don’t dare or don’t want to admit that you don’t understand that, at the end, you have no chance in the test.” An additional positive aspect, which was very important for students, was the use of cooperative learning methods. The students mentioned this aspect positively in nine out of the thirteen interviews. Robin, “Well, our previous chemistry lessons, well, I like the Ladder of Learning more, because now we have this structure and we can work with our partner more often, and that makes it easier for me to learn.”

Lea, a weaker student in chemistry, answered the question about what she liked about the LL with the following answer, “I like the structure a lot. Well, that we work on the elements one after the other. We start with one thing and we know where we have to finish and what's coming next and we know which task we have to work on next.” Oscar, who is one of the students with a medium achievement in chemistry, added, “And for me, it makes it easier that the Ladder of Learning guides us through the lessons.” 19 students, most of them with a low or medium achievement in chemistry, gave a positive feedback similar to Lea's and Oscar's statements. The transparent structure of the LL seems to help these students in organizing their learning process. Although teachers usually plan their lessons and have a structure of a unit in mind or in a written form, often this structure is not available for students. Medium- or low-performing students seem to perceive this as a personal disadvantage. In contrast, some high-performing students criticized the clear structure of the LL as being boring and not flexible enough. They missed the teacher's spontaneity during their lessons. Jona's statement gives an example for this point of view, “Well, this is, as you can see here [points on the LL], planned perfectly and this might be good. But sometimes, I think, when it is more spontaneous and the teacher says “let's do this” and “if you want to, we can do that”, sometimes this is better than working on the self-evaluation and then on the individualized exercises and then, we start from the beginning. I don’t like that.” Hence, when students were asked for aspects that could be improved in the LL in the future, Samuel answered for example, “Maybe making it more exciting, because you always know what's coming in the next lesson and this is boring.” Marie, who is another high-performing student in chemistry, summarized, “On the one hand it's good to see, what's coming next. On the other hand, it takes a lot of time. When you really think, “Oh, we could go on now.” It's very detailed, although I’ve understood everything, I have to work on these individualized tasks and it's always the same and then it becomes tiring.” She wished for the future, “Making it shorter that we don’t work on one topic for lessons.” One major suggestion students had about improving the LL was to include more experiments. A short conversation between Nick and his classmate Simon illustrates students’ perspectives on this issue: Nick, “Sometimes I would like to work more in the lab, some easy experiments.” Simon, “But we have said that we would not have done this anyway for this content. I think, it's not because of the Ladder of Learning. I don’t know, but I think, there are no experiments for this unit.” Although it was the aim to include practical work into the LL about Bohr's atomic model, like building models or conducting some experiments with these models, for some students this approach was not comparable to typical experiments in the school lab. As a result, students criticized a lack of practical lab work.

Discussion

Generally, the results of the study draw a positive conclusion on the implementation of LLs in chemistry classes. Students in the IG showed a significantly higher learning achievement in the post-test compared to the students in a CG, who have learned in a “usual” learning environment. This result is in line with the current research state on the effects of content structuring or structuring of the learning environment (e.g.Luiten et al., 1980; Skinner and Belmont, 1993; Rakoczy et al., 2007; Hattie, 2009; Seidel and Shavelson, 2016). Additionally, data analyses on students’ self-reported chemistry-related interest revealed a stable interest over the intervention in the IG, while students’ interest in the CG significantly decreased over time. This finding can be valued positively, because the chosen chemical content of Bohr's atomic model is considered as being rather theoretical, dry and often boring for students. The decreasing interest in the CG indicates this conclusion. As an additional affective variable, students’ chemistry-related self-concept has been investigated based on a Likert-scaled questionnaire. The results show an increase in students’ self-concept in the IG, while there was no significant change in students’ self-concept in the CG. These results are in line with students’ statements during the interviews, where they perceived a better understanding of the chemical content with the LL. Including the qualitative data of the interviews, it becomes obvious that primarily students with a low or medium performance in chemistry reported about positive attitudes towards the LL. When asking students about their experiences during their regular chemistry courses (before the LL was implemented), students painted a picture of a rather traditional method of teaching and learning in this subject. In particular, low- and medium-performing students rated this teaching style negatively. An important reason for this rating was the problems in understanding the content, as students were not able to follow the teacher's explanations. As an additional aspect, this group of students criticized a missing structure during their lessons, which they described as an obstacle for their learning. Despite all efforts for reform in the last few years, these findings confirm Gilbert's critique on traditional chemistry education, described in the Introductory section (Gilbert, 2006). There is still too little emphasis on cumulative learning, and teachers are still trying to teach too many isolated facts in too short a time. In contrast to the negative view of low- and medium-performing students on their former chemistry classes, these students reported much better attitudes towards the LL. An important reason for this rating was the understanding of the chemical content. The students related this fact with the constant opportunities for individualized practicing in the LL. Additionally, the students emphasized the role of self-evaluation as being an important anchor during their learning process for reflecting their personal learning achievement and possible knowledge gaps. In more traditional learning environments, self-evaluation is rarely used. However, students’ feedback indicates that this tool offers a good opportunity to detect individual problems during a learning process and not only during the preparation for a test. At this point, a limitation of this study becomes obvious because the elements of self-evaluation and the following individualized practicing might have caused students’ positive view on the LL. Hence, this positive effect cannot be attributed clearly to structuring. Nevertheless, students pointed out the relevance of the transparent content and teaching structure of the LL. In particular, students with a medium or low achievement in chemistry emphasized that the visible structure facilitated their learning because they always knew what they had to do and which steps would follow. This result is in line with research on general components of teaching quality and verify the importance of goal setting and a systematic structure of the learning process as one of the most important variables of teaching quality (Hattie, 2009; Seidel and Shavelson, 2016). The fact that students have mentioned the importance of structuring by themselves without any stimulation of the interviewer illustrates that this component of teaching quality does not only effect students’ learning implicitly, but students are aware of its relevance for their learning process.

Focusing on high-performing students in chemistry, a more critical view on LLs becomes obvious. Although these students also described their former chemistry lessons as being more traditional, they rated this aspect positively. A possible reason for this perspective might result from a reversed view on the critique of low- and medium-achieving students. It can be assumed that high-performing students did not have any problems in understanding the chemical content during their former lessons. Hence, these students do not benefit from the additional opportunity of individualized practicing or the self-evaluation. Although it was the aim to construct more challenging tasks for these students, some described these elements as being unnecessary. Most of the high-performing students would have preferred to continue in learning new chemical contents instead of working on additional exercises. Consequently, the individualized tasks for high-performing students within the LL have to be optimized. One approach could be to make these tasks more challenging for this group of students and to clarify the relevance of working on these additional tasks instead of continuing with the next milestone in the LL. While medium- and low-performing students also emphasized the transparent teaching structure, high-performing students perceived this aspect as being boring. They missed the presumed spontaneity of their former chemistry lessons. As one possible consequence, the individualized tasks could be adopted in a better way to meet the needs of high-performing students. Elements that offer more opportunities to follow individual interests and to deepen the knowledge of high-performing students might be an approach for a higher acceptance of the LL by this group of students. Generally, the differing feedback of high-performing students in contrast to medium- or low-performing students illustrates the importance of further in-depth analyses to clarify how structuring methods affect the outcomes of students with specific characteristics. Against the background of these findings, this aspect might be especially relevant for affective variables, which were neglected in the previous research on effects of structuring.

Limitations of the study

One major limitation of the study was the quasi-experimental design. It was not possible to assign students to the IG or CG randomly, but students had to be taught in their intact classes. Following Holländer's discussion about teachers’ influences in implementing new concepts in regular classes (Holländer, 2010), the role of the teachers in this study has to be discussed critically. It was not possible to control teacher effects or to standardize the teacher in all classes because of the amount of resources that would have been needed for realizing this. Furthermore, teachers, who were involved in the development of the learning material for the LL and in its implementation, participated voluntarily in this study over the period of a school year. It can be assumed that these teachers are engaged above average, which might also have an influence on their teaching in class. Furthermore, teachers got a manual, which gave them information about how to construct individualized tasks for the LL. Additionally, university researchers supported the teachers during the process of task development. This also might have had an impact on teachers’ professional knowledge. Additionally, it is possible that these teachers have deepened their understanding of the chemical content. Hence, these aspects lead to the question about how comparable the participating teachers were with their non-participating colleagues in the CG. A further limitation is that there is not much information available about the teaching structure in the CG because teachers were not willing to fill in the implementation diary. It can only be assumed that teachers of the control group did not use a LL because they were not familiar with this structuring tool. In order to guarantee test validity of the content knowledge test, teachers of the CG only provided an overview of the topics and contents on Bohr's atomic model, which they have taught in their chemistry classes. This proceeding was chosen to make sure that students in the control group had a comparable chance to answer the post-test questions. Furthermore, a large number of items had to be included in the pre- and post-tests to measure students’ increase in content knowledge over the whole LL. This comparably large number of items might have had an influence on the reliability of the test instrument. This aspect should be considered when evaluating the results of this study regarding students’ increase in content knowledge. In addition, there is a need for testing the generalizability of the findings across different groups of learners. The differing feedback of high-performing students in comparison to medium- or low-performing students in this study indicates this conclusion. As a whole, it has to be determined that the LL was not able to reach high-performing students optimally. Further adaptation of the individualized learning material has to be done for this specific group of students. Considering the total sample size of 746 students, it was possible to include only 534 students in the final analyses because of missing data. Although no systematic differences could be found for the excluded students based on available data, it is possible that there is an additional reason why a specific group of students did not fill in the questionnaires completely. Hence, the results have to be viewed critically against this background. As this study analyses the effects of the LL regarding the chemical content of Bohr's atomic model, additional LLs with different contents have to be constructed and evaluated. One relevant aspect might be the rare use of experiments for the chosen content, which is also the case for more traditional chemistry teaching. When students have the opportunity to work in the lab more often, this may influence the effects of the LL in comparison to more traditional chemistry education. As LLs have not been evaluated until now, further studies are necessary, which e.g. compare this structuring method with alternative structuring approaches. As it was mentioned in the discussion before, further variables like the self-evaluation and individualized exercises also might have an impact on students’ outcomes. Hence, further research is needed to locate the effects more clearly and to analyse possible interactions between structuring and other possible variables.

Implications for practitioners

Quantitative data analyses as well as qualitative data analyses reveal better student outcomes regarding both cognitive and affective variables compared to a control group and highlight the importance of implementing a systematic and transparent content and teaching structure in class. In particular, low- and medium-performing students benefit from a transparent content and teaching structure. Hence, this variable of teaching quality should be considered more during regular teaching. The results indicate that LLs are a useful tool to realize a transparent content and teaching structure in class. However, it was a lot of effort to develop the learning material. It is difficult to manage this effort as an individual teacher. This project has benefited from working together in a team of teachers from different schools over a longer period of time, where everyone could use the work of each other. A regular cooperation between teachers from different schools supported by university members has the potential to develop innovations in education and to facilitate the implementation of new concepts in teaching practice.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

This study was funded by Stiftung Mercator (project number 1344200). I would like to acknowledge the participating teachers, who have invested a lot of effort in developing and implementing the learning material. I would also like to thank my collegues, especially Prof. Dr. Elke Sumfleth, Marie-Therese Hauerstein and Martina Strübe for their support.

References

  1. Aikenhead G. S., (2009), Science education for everyday life: evidence-based practice, ways of knowing in science and mathematics series, New York, London: Teachers College Press.
  2. Ausubel D. P., (1960), The use of Advance Organizers in the learning and retention of meaningful verbal material, J. Educ. Psychol., 51, 267–272.
  3. Bolhuis S., (2003), Towards process-oriented teaching for self-directed lifelong learning: a multidimensional perspective, Learn. Instr., 13, 327–347,  DOI:10.1016/S0959-4752(02)00008-7.
  4. Broman K. and Simon S., (2015), Upper secondary school students’ choice and their ideas on how to improve chemistry education, Int. J. Sci. Math. Educ., 13, 1255–1278,  DOI:10.1007/s10763-014-9550-0.
  5. Dollny S., (2011), Entwicklung und Evaluation eines Testinstruments zur Erfassung des fachspezifischen Professionswissens von Chemielehrkräften. [Development and evaluation of a test for investigating the professional knowledge of chemistry teachers], Studien zum Physik- und Chemielernen, Berlin: Logos, vol. 127.
  6. Duschl R. A., Schweingruber H. A. and Shouse A. W., (2007), Taking science to school: learning and teaching science in grades K-8, Washington D.C.: National Academies Press.
  7. Fechner S., (2009), Effects of context oriented learning on student interest and achievement in chemistry education, Studien zum Physik- und Chemielernen, Berlin: Logos, vol. 95.
  8. Field A. P., (2016), Discovering statistics using IBM SPSS statistics: and sex and drugs and rock'n'roll, 4th edn, reprinted, Los Angeles: Sage Publications.
  9. Gilbert J. K., (2006), On the nature of “Context” in chemical education, Int. J. Sci. Educ., 28, 957–976,  DOI:10.1080/09500690600702470.
  10. Hattie J. A. C., (2009), Visible learning: a synthesis of over 800 meta-analyses relating to achievement, London, New York: Routledge.
  11. Holländer M., (2010), Effektivität des Advance Organizers als Strukturierungshilfe im Chemieunterricht der Sekundarstufe I. [Effectiveness of advance organizers as a structuring method for chemistry education in the secondary I level], Berlin: Uni-Edition.
  12. Hospel V. and Galand B., (2016), Are both classroom autonomy support and structure equally important for students' engagement? A multilevel analysis, Learn. Instr., 41, 1–10,  DOI:10.1016/j.learninstruc.2015.09.001.
  13. Jang H., Reeve J. and Deci E. L., (2010), Engaging students in learning activities: it is not autonomy support or structure but autonomy support and structure, J. Educ. Psychol., 102, 588–600,  DOI:10.1037/a0019682.
  14. Kallweit I., (2015), Effektivität des Einsatzes von Selbsteinschätzungsbögen im Chemieunterricht der Sekundarstufe I: Individuelle Förderung durch selbstreguliertes Lernen, [Effectiveness of the use of self-evaluation questionnaires in chemistry education in the secondary I level: Individual support by self-regulated learning], Studien zum Physik- und Chemielernen, Berlin: Logos Berlin, vol. 183.
  15. Luiten J., Ames W. and Ackerson G., (1980), A meta-analysis of the effects of advance organizers on learning and retention, Am. Educ. Res. J., 17, 211,  DOI:10.2307/1162483.
  16. Lye H., Fry M. and Hart C., (2001), What does it mean to teach physics ‘in context’? Aust. Sci. Teach. J., 48, 16–22.
  17. Mayring P., (2014), Qualitative content analysis: theoretical foundation, basic procedures and software solution, retrieved from http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173.
  18. Müller T., Lichtinger U. and Girg R., (2015), The MultiGradeMultiLevel-Methodology and its Global Significance: Ladders of Learning – Scientific Horizons – Teacher Education, Theory and Practice of School Pedagogy, Immenhausen, Germany: Prolog-Verlag, vol. 34.
  19. Nesbit J. C. and Adesope O. O., (2006), Learning with concept and knowledge maps: a meta-analysis, Rev. Educ. Res., 76, 413–448.
  20. Novak J. D., (2002), Meaningful learning: the essential factor for conceptual change in limited or inappropriate propositional hierarchies leading to empowerment of learners, Sci. Educ., 86, 548–571,  DOI:10.1002/sce.10032.
  21. Novak J. D. and Gowin D. B., (1984), Learning how to learn, Cambridge: Cambridge University Press.
  22. Osborne J. and Collins S., (2001), Pupils' views of the role and value of the science curriculum: a focus-group study, Int. J. Sci. Educ., 23, 441–467,  DOI:10.1080/09500690010006518.
  23. Potvin P. and Hasni A., (2014), Analysis of the decline in interest towards school science and technology from grades 5 through 11, J. Sci. Educ. Technol., 23, 784–802,  DOI:10.1007/s10956-014-9512-x.
  24. Rakoczy K., Klieme E., Drollinger-Vetter B., Lipowsky F., Pauli C. and Reusser K., (2007), Structure as a quality feature in mathematics instruction: cognitive and motivational effects of a structured organisation of the learning environment vs. a structured presentation of learning content, in Prenzel M. (ed.), Studies on the educational quality of schools. The final report on the DFG priority program, Münster: Waxmann, pp. 102–121.
  25. Seidel T. and Shavelson R. J., (2016), Teaching effectiveness research in the past decade: the role of theory and research design in disentangling meta-analysis results, Rev. Educ. Res., 77, 454–499,  DOI:10.3102/0034654307310317.
  26. Skinner E. A. and Belmont M. J., (1993), Motivation in the classroom: reciprocal effect of teacher behavior and student engagement across the school year, J. Educ. Psychol., 85, 571–581.
  27. Supanc M., Völlinger V. A. and Brunstein J. C., (2017), High-structure versus low-structure cooperative learning in introductory psychology classes for student teachers: Effects on conceptual knowledge, self-perceived competence, and subjective task values, Learn. Instr., 50, 75–84,  DOI:10.1016/j.learninstruc.2017.03.006.
  28. Taut S. and Rakoczy K., (2016), Observing instructional quality in the context of school evaluation, Learn. Instr., 46, 45–60,  DOI:10.1016/j.learninstruc.2016.08.003.
  29. Wang M. C., Haertel G. D. and Walberg H. J., (1993), Toward a knowledge base for school learning, Rev. Educ. Res., 63, 249–294.
  30. Weber K., (in press), Entwicklung und Validierung einer Learning Progression für das Konzept der chemischen Reaktion in der Sekundarstufe I. [Development and validation of a learning progress on for the concept of chemical reaction in the secondary I level].
  31. Willerman M. and Mac Harg R. A., (1991), The concept map as an advance organizer, J. Res. Sci. Teach., 28, 705–711.

Footnote

† Learning material of the LL is available in the German language after personal request via e-mail.
This journal is © The Royal Society of Chemistry 2018