L.
Szalay
*a,
Z.
Tóth
b and
R.
Borbás
c
aFaculty of Science, Institute of Chemistry, Eötvös Loránd University, Pázmány Péter sétány 1/A, H-1117 Budapest, Hungary. E-mail: luca.szalay@ttk.elte.hu
bFaculty of Science and Technology, Institute of Chemistry, University of Debrecen, Egyetem tér 1., H-4032 Debrecen, Hungary
cSzent István Secondary School, Ajtósi Dürer sor 15., 1146 Budapest, Hungary
First published on 28th July 2021
This paper reports the findings of the second and the third year of a four year longitudinal empirical research into the modification of ‘step-by-step’ instructions to practical activities which require one or more steps to be designed by the students. This method had been successfully applied for a short period for 14–15 years old students. However, the first year of the current longitudinal study, investigating the approach's effectiveness (a) for younger students and (b) over a period of time, did not produce the expected results. Therefore the research model was modified at the beginning of the second year which began in September 2017 with over 800 13–14-year-old students. Each school year they spent six lessons carrying out practical activities using worksheets we provided. The participating classes were allocated to one of three groups. Group 1 was the control group. Students simply followed the step-by-step instructions. Groups 2 and 3 were experimental groups. Group 2 students followed the same instructions, but from the beginning of the second school year their worksheets explained the principles related to the experimental design of the step-by-step experiments carried out. Group 3 students followed the same instructions, but one or more steps were incomplete and students were required to design these steps, as in the first year. However, from the second year onwards Group 3 students were taught the relevant principles of the experimental design before they started planning and carrying out the experiments. The impact of the intervention on the students’ experimental design skills and disciplinary content knowledge was measured by structured tests. After the second school year of the project it was clear that both types of instruction (used in case of Group 2 and Group 3) had a significant positive effect on the results of the students disciplinary content knowledge and experimental design skills. However, the development seemed to stop in the third year of the project. In the third year, a student's achievement was influenced mostly by their school ranking.
Seeing science as a finished (albeit impressive) building, instead of one still under construction can lead to loss of interest and motivation. This is not the only problem that may appear in students’ life. From time to time everybody needs to make informed decisions about their own or other people's lives that require a judgement as to whether a scientific-looking statement might be true or false. Therefore, people need a knowledge of science, along with technology, to make rational decisions and participate in society (e.g.Sjřberg and Schreiner, 2006; Kousa et al., 2018). Everybody should understand at least the basics of the scientific process how the scientific content knowledge is built: how hypotheses and models are created, how investigations and experiments are designed, how data and evidence are gathered and how the results are interpreted, discussed and accepted or rejected by the scientific community.
Several expressions are used in the literature to describe this dichotomy. Walker (2007) differentiates between ‘scientific knowledge (hypotheses, theory and law)’ and ‘scientific process: basic and integrated process skills (among the latter: controlling variables)’. Seery et al. (2019) interpret Kirschner's (1992) definition of the ‘substantive structure of science’ as ‘the body of knowledge making up science’; as opposed to Kirschner's term ‘syntactical structure of science’, explained by Seery et al. (2019) as ‘the habits and skills of those who practice science’. Lederman and Lederman (2012) called these two components ‘content knowledge’ and ‘knowledge of inquiry methods’. One goal of science education is to produce scientifically literate people armed with both, enabling them to make intelligent decisions and differentiate between science and pseudoscience. The PISA 2018 science framework (OECD 2019) asserts that scientific literacy requires knowledge of the concepts and theories of science, and knowledge of the common procedures and practices associated with scientific enquiry. (‘Enquiry’ and ‘inquiry’ are used interchangeably in the literature.) The terms ‘scientific literacy competencies’ or ‘scientific practices’ are also used by Cannady et al. (2019) to frame the notion of science processes as a set of core skills that supports the learning of science. They argue that the general mastery level in science practices can be conceptualized as scientific sensemaking. This requires skills and knowledge, including epistemological understandings of the process of knowledge generation and that scientific sensemaking can improve with effective science instruction. For many years the primary goal of science literacy was to improve thinking skills, including the ability to engage in research activity and planning research studies (AAAS, 1993; NRC, 1996) that are among the scientific process skills (SPS). Therefore, science content and SPS are two primary components of science education (Tosun, 2019). SPS is also referred to as scientific method, scientific thinking, critical thinking, scientific inquiry, scientific reasoning skills and thinking skills. They are used interchangeably to describe the SPS needed to be acquired by the students in the 21st century (Kambeyo, 2017). SPS include skills that can be used by a science literate person to increase the quality and the standards of life (Williams et al., 2004).
These experiences usually fail to develop a wider range of transferable skills. They lack any real-world context (Johnstone and Al-Shuaili, 2001; McDonnell et al., 2007). The development of higher-order thinking skills (Zolller and Tsaparlis, 1997), requires the integration of conceptual knowledge and procedural skills with a complex problem context (Chonkaew et al., 2016). The basic skills required in general school science curricula (Nagy et al., 2015) consist of data handling technique, identification of variables, setting research questions, hypothesis formulation, variable planning, experimental plans and making conclusions.
Such opportunities could be provided with a different approach to the practical work (Goodey and Talgar 2016). Alongside critical thinking, secondary school students should learn about experimental design and laboratory skills/techniques. These are important goals for undergraduate chemistry laboratory (Bruck et al., 2010; Bretz et al., 2013; Bruck and Towns, 2013) and it is reasonable to suggest they could be introduced, at the appropriate level, in schools. One approach is the use of pre-laboratory activities. Students may be asked to plan an experiment using provided information and criteria for a good experimental design. Critical thinking was elicited when this approach was used during a chemical inquiry practical in senior-year secondary school (Brederode et al., 2020).
Further, what could be considered then as a ‘real inquiry’ that is worth doing? According to the simplest definition, inquiry-based learning happens when students are supposed to proceed in the way scientists work (PRIMAS, 2017). Other sources are more specific, each listing activities that characterise inquiry-based education (Gott and Duggan, 1998; Phillips and Germann, 2002; Kahn and O’Rourke, 2005; Zimmerman, 2005; Mayer, 2007; Rocard Report 2007; Akkuzu and Uyulgan, 2017). Three main phases may be separated: (1) formulating research questions, (2) planning a method to get the answer and carrying out the investigation, (3) interpreting the results (Schwab,1962; Colburn, 2000; Nowak et al., 2013). These phases are applicable to four types of inquiry. (1) Open inquiry: students decide both on the question to investigate and the method they use to get to an answer. (2) Guided/bounded inquiry: the teacher provides the question, but the students decide on the test method used to answer it. (3) Structured inquiry: both the question and the method are provided to the students, but the outcome is unknown to them. (4) Confirmation inquiry: the question, the method and even the outcome are known in advance by the students (Tafoya et al., 1980; Lederman, 2004; Bell et al., 2005; Fay et al., 2007; Walker, 2007, Wenning, 2007; Banchi and Bell, 2008; Akkuzu and Uyulgan, 2017). There are other classifications. Fradd et al. (2001) defined six levels of inquiry according to the role of the teacher and the students. Domin (1999) distinguishes between the expository approach, when the students should deduce some findings, from the outcome and the discovery approach, when students need to recognise trends or pattern that explain their results. Domin (1999) calls the two approaches when the student must develop their own procedure, problem-based (requiring deduction) and inquiry-based (requiring induction). However, Duch et al. (2001) described problem-based learning as a composite of inquiry-based and context-based learning.
It is challenging to find effective ways to support students within the inquiry process and there is a fine balance between providing too little or too much guidance (Kahn and O’Rourke, 2005). However, Shiland (1999) and Domin (1999) referred to laboratory activities based on the confirmatory approach as closed or low inquiry laboratories, that Xu and Talanquer (2013) characterised as “essentially no inquiry”. According to Banchi and Bell (2008) and Chin and Chia (2006) only the guided and open inquiry develop the students’ skills at a higher level.
The second was a report on the first year of a four-year longitudinal research project that started in September 2016. The approach was similar to the first study, but with more than 900 students aged 12–13 involved. They were divided into three groups. Two groups were defined as in the first study. Group 1 was the control group following step-by-step recipes. Group 3 did the same experiments but had to design some of the steps. However, there was a third group (Group 2) that followed step-by-step recipes, but students were also given theoretical experimental design tasks. In the first year of the longitudinal study these three groups used the worksheets in six 45-minute lessons. However, for Group 3 no significant effect of the intervention was detected by the structured tests taken at the end of the first school year. It was thought that most of the students were probably still in Piaget's concrete operational stage (Cole and Cole, 2006) and that this explained the lack of effect. Padilla (1990) found that experimental skills are closely related to the formal thinking abilities described by Piaget. Nor perhaps, was the cognitive load placed on the Group 3 students managed properly (Sweller, 1988). It is also possible that some teachers of the Group 3 provided experimental design steps when time was short.
In the remaining three school years of the project, answers to these questions were sought.
RQ1: Was there any significant change in the students’ ability to design experiments (experiment design skills, EDS), in either of the experimental groups (Group 2 and Group 3) compared to the control group, because of the intervention after changing the research model?
RQ2: Did students in the experimental groups achieve significantly different scores on questions that assess disciplinary content knowledge (DCK) than control group students, because of these interventions?
RQ3: Was there a difference in EDS between students of Group 2 and Group 3?
For each group, the intervention took seven chemistry lessons both in the second (Grade 8) and third (Grade 9) school years. Teachers chose when the six lessons using the worksheets provided took place. After this students took Test 2 in Grade 8 and Test 3 in Grade 9.
When the project started in autumn 2016, 920 students were involved. All were 7th grade 12–13-year-olds. They remained in the project in the second and third years. However, if any missed one test they were no longer considered to be part of the sample.
The classes were assigned randomly to three groups in the beginning of the project (Szalay et al., 2020). In the second and third years, Group 1 continued to carry out step-by-step experiments only. However, Groups 2 and 3 (the experimental groups) were treated differently to their first year. From the beginning of the second year Group 2 continued to carry out the same step-by-step experiments as Group 1, but after doing the experiments relevant principles of experimental design, written on their student sheets, were discussed with them by their teachers. Group 3 carried out the same experiments as Group 1, but with one or more steps missing. Students were required to design the missing steps and complete the experiment (see Tables 1 and 2). Note: Students worksheets and teacher guides are numbered according to the school years, with 1–6 used in the first school year, 7–12 in the second and 13–18 in the third. Unlike year 1, relevant principles of experimental design were given in Group 3's student sheets and discussed with the students before they designed the missing steps.
No. | Topic | Experiments that Group 1 and Group 2 students had to do following step-by-step instructions, but Group 3 students had to plan before doing the experiment | What did the experimental groups learn about experimental design? (Group 2: AFTER doing the step-by-step experiments; Group 3: BEFORE designing and doing the same experiments) |
---|---|---|---|
7. | Acids, bases, indicators, neutrali-sation, chemistry in the household | Students do step-by-step experiments to observe the colour of phenolphthalein in acidic/alkaline solutions and how it changes during neutralisation. Students are given two glasses. There is the same amount of distilled water in both glasses, but one drop of white vine vinegar in one of them and two drops of the same vinegar in the other glass. The Group 3 students are asked to design an experiment to determine which of the glasses contains two drops of vinegar. | 1. Fair testing. The dependent variable (quantity of sodium hydroxide solution added to the content of both glasses until the colour of the phenolphthalein changes) depends only on the independent variable (quantity of vinegar in the glasses); all other variables are held constant (e.g. concentration of sodium hydroxide solution, the Pasteur pipette used). |
2. Quantitative analysis (the basic idea of acid–base titration). | |||
(3. The possible causes of measurement errors.) | |||
8. | Reactivity series of metals, reactions of metals | Students investigate the reactivity of magnesium, zinc and copper with cold and hot water, dilute hydrochloric acid and solutions of one another's salts by doing step-by-step experiments. They are given iron nails. Groups 1 and 2 get instructions to determine the place of the iron in the metal reactivity series. Group 3 do the same, except they are not given instructions. They must plan what to do. | 1. Fair testing. |
2. Choosing equipment and materials: considering the possible alternative solutions, depending what is available. | |||
3. Algorithmic thinking: deciding whether the order of the steps of the experiment is important or not (it is not). | |||
9. | Reactions of iron, natural waters, quality of drinking water | Students do step-by-step experiments to observe that iron(III) ions form a red compound with thiocyanate ions and the intensity of the colour depends on the concentration of the iron(III) ions (all other variable kept constant). Group 3 students have to decide whether a water sample has got too much iron(III) ions in it to make it inappropriate for drinking or not. They are given the official concentration limit issued by the authorities. | 1. Fair testing. |
2. Qualitative analysis: sensitive and characteristic reaction of iron(III) ions. | |||
3. Quantitative analysis: the properties (e.g. the colour) of a solution depend on the concentration of the solute, colorimetry, calibration series. | |||
10. | Hardness of water, water softeners, precipitation reactions | Students read about problems caused by hard water. They do step-by-step experiments to find out which cations cause the hardness of the water, and how to show their effects by measuring the height of soap foam after shaking. Using a table that shows which anions form precipitates with the cations, Group 3 students must work out which compounds could be used as water softener, and show this by experiment. | 1. Fair testing. |
2. Using data: interpreting a table about the solubility of salts. | |||
3. Qualitative analysis: precipitation reactions of calcium ions and magnesium ions. | |||
4. Semi-quantitative measurements testing the hardness of the “water samples” (solutions) before and after adding the softeners by measuring the height of the soap foam (by a ruler), formed after shaking the solutions the same time with the same intensity. | |||
11. | Reactions of calcium carbonate with acids, calcination of limestone, lime slaking | Students do step-by-step experiments to model calcination and lime slaking by holding a piece of limestone (and later a piece of eggshell) in the flame and then dropping it in distilled water containing phenolphthalein. They also model the effect of the limescale remover on limescale. Group 3 students are asked to design and do an experiment modelling how the waterbed made of limestone can neutralise acidic rain, whereas sand does not have this effect. | 1. Fair testing. |
2. Model experiments: modelling chemical reactions taking place in the industry, in the household and in the environment. | |||
3. Choosing equipment and materials: what could replace the water of the lake, the acid rain, the limestone or the sand on the bottom of the lake? How can be the changes of pH followed? | |||
4. Control experiment: how does the colour of the red cabbage indicator change without limestone/sand after adding acid to it? | |||
5. Reference material: comparing the effect of limestone and sand. | |||
12. | Nutrients (fat, carbo-hydrates, proteins, nutritional components of milk) | Students do step-by-step experiments to test for the oil in an oil-and-water emulsion, reductive sugars using Fehling's reaction, and proteins using the biuret test. Group 3 students must design and carry out experiments to test whether milk contains all those three types of nutrients. | 1. Fair testing. |
2. Choosing equipment and materials, and what is needed to test the milk sample for each nutrient | |||
3. Reference materials: comparing the extraction of oil from oil-and-water emulsion and milk; comparing the effect of Fehling reagent on glucose solution and milk; and comparing the effect of the biuret reagents on egg white solution and milk. |
No. | Topic | Experiments that Group 1 and Group 2 students had to do following step-by-step instructions, but Group 3 students had to plan before doing the experiment | What did the experimental groups learn about experimental design? (Group 2: AFTER doing the step-by-step experiments; Group 3: BEFORE designing and doing the same experiments) |
---|---|---|---|
13. | Structure of atoms, excitation of electrons, flame tests | Students do a step-by-step experiment to observe the flame test of a sodium chloride solution. The relationship of the excitation energy of the atoms of certain elements and the colour of their flame tests is explained. All students are given two solutions, each containing a metal salt. Groups 1 and 2 students do flame tests following instructions and explain the difference in excitation energies with that of sodium. Group 3 students do the same, except that they must design an experiment to identify the metal ions and decide whether their excitation energies are higher or lower than that of sodium. | 1. Qualitative analysis: flame tests. |
2. Using data: interpreting the tables containing the colours of regions of the visible spectrum, their wavelength range, and the relationship between the wavelength and energy of electromagnetic radiation. | |||
3. Understanding the steps of scientific investigations: finding a research question; collecting previous theoretical and practical knowledge; designing the experiments; collecting and evaluating data; drawing conclusion; answering the research question. | |||
14. | Structure of matter, bonding energy, amount of matter | Students are given data to calculate the volume of the water in the oceans. They are asked to determine the volume of a drop of water, and from that to calculate how many drops of water are in the oceans. Group 3 students design experiments to determine the volume of one drop of water, one drop of ethanol, and one drop of a 50% by volume ethanol–water mixture as accurately as they are able. The results must be compared and interpreted. | 1. Fair testing. |
2. Choosing equipment: considering the possible alternative solutions to measure the volume of liquids. | |||
3. Error of measurement: ways of reducing measurement errors. | |||
4. Using data: calculations of volume and mass of a drop of liquid, the amount of matter and the number of particles in it, comparing the surface tension of water, ethanol and their mixture; interpretation of the relationship of these amounts. | |||
15. | Heat of dissolution, exothermic and endothermic processes | Students do step-by-step experiments to investigate exothermic and endothermic dissolutions, modelling how the self-heating and self-cooling cups work. They read an advertisement for sodium acetate heat pad and must decide if it would work. Group 3 students are asked to design a model experiment to answer the question. | 1. Fair testing. |
2. Model experiments: differentiation of the important and unimportant aspects while modelling how products based on exothermic and endothermic dissolution work. | |||
3. Drawing conclusions from the results of the model experiment to differentiate between science and pseudoscience. | |||
16. | Rate of chemical reactions and factors influencing the reaction rate | Students watch the ‘elephant toothpaste’ demonstration to learn about influencing the rate of reaction by a catalyst (the decomposition of hydrogen peroxide catalysed by potassium iodide). They do step-by-step experiments to find out how the concentrations of sodium thiosulfate and hydrochloric acid solutions influence their reaction rates. Group 3 students must design an experiment to decide which of three hydrochloric acid solutions, of differing concentrations, the unknown solution has. Finally, the students investigate the effect of heat on the rate of reaction. | 1. Fair testing: identification of independent variable (concentration of hydrochloric acid) and dependent variable (time needed to see the formation of colloidal sulphur) and controlling variables (concentration of thiosulfate solution, temperature, volumes of solutions etc.). |
2. Semiquantitative analysis: detemination of the range of concentration of hydrochloric acid by measuring the time needed to see the formation of colloidal sulphur. | |||
17. | Strong and weak acids and bases, hydrolyses of salts, natural and artificial indicators | Students fill in a table about the colours of various indicators in acidic, neutral, and alkaline solutions using the information collected as homework. They do step-by-step experiments to differentiate between a strong and a weak acid, and a strong and a weak base by red cabbage indicator. They investigate the pH of salt solutions using universal indicator. Group 3 student must design an experiment to identify potassium hydroxide, sodium carbonate and nitric acid solutions of the same concentration. They must also design what solutions and indicators should be used to paint a Hungarian flag. | 1. Fair testing: controlling variables (concentration and volume of solutions). |
2. Qualitative analysis: identification of solutions by their pH. | |||
3. Choosing equipment and materials: pair the acidic/neutral/alkaline solutions with certain indicators to get the colours of the Hungarian flag (red, white/colourless, green). | |||
18. | Redox reactions, oxidation as giving off electron(s), reduction as gaining electron(s). | As a homework, students must balance equations describing various reactions when hydrogen peroxide is either an oxidative or a reductive agent. Group 3 students are asked to identify a pattern: when oxygen gas is among the products, then the hydrogen peroxide reduces its reaction partner. Also, they must design an experiment to prove this, using the materials and equipment available. | 1. Practicing the steps of the scientific investigations: working out patterns by observation, asking a research question, creating a hypothesis, designing experiments to test ideas, collecting data, interpreting the results and drawing conclusions. |
2. Choosing equipment and materials: what is needed to test for the presence of oxygen gas? | |||
3. Qualitative analysis: proving the production of oxygen gas by a glowing splint. |
Class sizes varied between 13 and 35 students in the second year and 14 and 35 in the third year of the project, reflecting typical class sizes in Hungarian schools. The method used to produce random samples is described in the previous study (Szalay et al., 2020). Some teachers participated with only one class, whereas others with two classes. If a teacher had two classes of students participating in the research and one class had been chosen randomly to be in Group 1, the other class would be in Group 2 or Group 3. This choice was also random.
Teachers had written permission of their school principals to participate. Also, as there were no institutional ethics committees or boards in the participating schools, only those students were involved whose parents gave their written permission for their children to participate in the research. Teachers told students that test results would not count in their school chemistry assessment, but they would be participating in work that aims to improve the teaching of chemistry.
Topics were cross-referenced to the curriculum, together with the experimental design tasks given to the Group 3 students on the student worksheets (see Table 1 for the second year and Table 2 for the third year). As in the first year, the student worksheets had an introduction intended to stimulate interest and curiosity. As with other studies (e.g.Hennah, 2019), the activities had to fit curriculum timetabling.
The student worksheets provided important DCK that was needed to solve the experimental design task and develop EDS. As in the first year of the project, each experimental design task required problem-solving skills and related to the topic of the lesson. These were guided inquiries. Group 1 and Group 2 students followed the same sequence of experiments. Unlike Group 3 students, they were provided with step-by-step instructions. This was similar to the investigation of the effect of the guided inquiry that Goodey and Talgar (2016) used in a university biochemistry laboratory course aiming to improve experimental design ability.
The difference between the first and the second/third year treatment of the Group 3 was that in the first year the students were not given help on their student sheets to design the experiments. However, in the second year and subsequently to reduce the Group 3 students’ cognitive load (Behmke and Atwood, 2013; van Merrienboer et al., 2003; Paas et al., 2003; Sweller, 2004), student worksheets were scaffolded. Group 3 students were taught the relevant principles of the experimental design before they began planning and carrying out the experiments. This is similar to advice on experiment design given in bounded inquiry modules used by Goodey and Talgar (2016) at undergraduate level. Group 3 students worked collaboratively to propose, design, and troubleshoot their own experiments, rather like the undergraduates in Boyd-Kimball and Miller's research (2018). Each series of experiments in our project had required students to gather their previous theoretical and practical knowledge, answer questions, and do step-by-step experiments. This is in accord with Crujeiras-Pérez and Jiménez-Aleixandre's (2017) suggestion that students need previous knowledge relevant to designing an experiment, as well as an opportunity to reflect on the design adopted.
Our worksheets also have similarities with the unfinished recipes prepared by Seery et al. (2019) that scaffold EDS and echoing Kirschner's (1992) suggested middle ground between expository and inquiry. As in the first year of the project, our experimental design tasks were built on the following components of the EDS defined and assessed by Csíkos et al. (2016): identification and control of variables (including the principle of ‘fair testing’, in other words ‘how to vary one thing at a time’ or ‘other things/variables held constant’); choosing equipment and materials and determination of the correct order of the steps of the experiment. The two types of analyses, qualitative and quantitative were covered. The concept of systematic and random error, introduced in the first year, were reinforced. According to Metz (1998) separating random error from true effects is not a skill that eighth grade students spontaneously engage in without scaffolding, but emerging understanding is evident (Masnick and Klahr, 2003). Since developing and using models (Lehrer and Schauble, 2000) is an important scientific skill, designing experiments modelling certain environmental, everyday, industrial or laboratory processes was also included among the tasks.
In their meta-analysis Furtak et al.’ (2012) concluded that evidence from some studies suggests that teacher-led inquiry has a greater effect on student learning than student-led inquiry. So, from the beginning of the second school year Group 2 students continued to do the same step-by-step experiments as Group 1. However, their worksheets explained the design of the step-by-step experiments. Group 2 students did paper-based experimental design tasks in their first year, but not in their second. This was done to exclude the effect of practising for the test. The approaches taken with Groups 2 and 3 resemble the use of postlab and prelab questions asked by Rodriguez and Towns (2018) to engage university undergraduate students in critical thinking.
Monetary and time expenses remained a consideration when planning activities and assessment. If these constraints were ignored, it is unlikely that the tried and tested activities would be suitable and widely used once this research project finished (Boesdorfer and Livermore, 2018). Detailed teacher guides were also provided for each student worksheet (see the links to their Hungarian versions in Supplement 1, ESI†).
The tasks needed to be different in each test for the reasons previously described (Szalay et al., 2020). Meanwhile a study published by Cannady et al. (2019) reinforced the concern that using the same instrument in a pre/post setting could invite repeated testing effects wherein students can remember correct answers. Tests 2 and 3 contained DCK and EDS tasks that could be undertaken after students had completed the tasks on the student worksheets. Only the DCK given in the National Curriculum of Hungary (2012) could be assessed on the tests Szalay et al., 2020).
EDS tasks had to be in contexts relevant to their previously gained knowledge and understanding. However, the main goal of the research remained to develop EDS that can be applied under different circumstances (Szalay et al., 2020). This also accords with the approach of Cannady et al. (2019), that tasks must integrate content with which students are familiar, and focus on the ability to apply the practices. Other authors support these views (Zimmerman, 2000 and 2007; Tosun, 2019). Cannady et al. (2019) emphasize the importance of including and supporting access to the necessary content knowledge within the assessment. This approach is also used in PISA assessments of scientific literacy competencies (OECD, 2017). It was also considered while developing the EDS tasks of the present project.
Test questions were structured according to the categories of the Cognitive Process Dimension of the Revised Bloom Taxonomy (Bloom et al., 1956; Krathwohl, 2002) as interpreted in previous publication (Szalay et al., 2020). Therefore, the different levels of the DCK are represented on the tests, and the EDS tasks require the application of higher order cognitive skills. Each test consists of eighteen compulsory items, each worth 1 mark. Nine were to assess EDS. The other nine were to assess DCK, with three items each for recall, understanding and application. By analysing the results, the effects of the different types of treatments on EDS and DCK could be investigated.
As for Tests 0 and 1, students had 40 minutes to complete Test 2 (English translation, Supplement 2, ESI†) and 40 minutes for the Test 3 (English translation, Supplement 3, ESI†). Students were coded so that teachers knew their identities and gender, but the researchers only received the anonymous data coded for statistical analysis. These codes were used throughout the project.
The following tasks were used on Tests 2 and 3 to compare the development of students’ EDS across the three groups.
Test 2; Task 2.b:
There is 6% vinegar and 20% vinegar in two identical looking plastic bottles. The labels have fallen off from both bottles. You must decide, by doing an experiment, which bottle contains the more concentrated vinegar. (Smelling does not give a satisfactory answer and you must not taste the vinegar.) The following materials and equipment are available: 2 empty glasses (you can only use both of them once), 2 spoons, 4 droppers (with a scale on them showing the volume), aqueous solution of an alkaline caustic drain cleaner, red cabbage juice. (Vinegar is acidic and red cabbage juice indicates the changes of pH by its change of colour.)
Question 1: What materials would you put into one glass and what into the other glass?
Question 2: In what order would you put the materials into one glass and into the other glass?
Question 3: How much would you put of the different materials into one glass and how much into the other glass?
Question 4: What different experiences would you expect in the case of each glass?
Question 5: Based on your experiences, how could you decide which bottle contains the more concentrated vinegar?
Test 2; Task 7:
There are three test tubes. One contains silver ions (Ag+), another aluminium ions (Al3+), and the third zinc ions (Zn2+). All are colourless aqueous solutions. (There is about 1 cm3 dilute solution in each.)
There are two labelled bottles beside them, one contains ammonia (NH3) solution and the other sodium hydroxide (NaOH) solution. The table below shows what would be seen if a little (a few drops) or much more (several cm3) ammonia solution or sodium hydroxide solution was added to the test tubes containing the solutions of the different ions.
ion | + little NH3 solution | + much NH3 solution |
Ag+ | brown precipitate | brown precipitate, dissolves |
Al3+ | white precipitate | white precipitate |
Zn2+ | white precipitate | white precipitate, dissolves |
ion | + little NaOH solution | + much NaOH solution |
Ag+ | brown precipitate | brown precipitate |
Al3+ | white precipitate | white precipitate, dissolves |
Zn2+ | white precipitate | white precipitate, dissolves |
(a) What is the fewest number of tests needed to decide which solution of metal ions is in each of the three test tubes? Explain your answer.
(b) Only one of the solutions (NH3 or NaOH) can be used to determine which ion is in which test tube. Which one? Explain your answer.
(c) How should your chosen solution (NH3 or NaOH) be added to the solutions in the test tubes?
(d) What would you see when your chosen solution (NH3 or NaOH) is added to the test tube containing the solution of aluminium ions (Al3+)?
Test 3; Task 3:
The equation of the reaction that takes place between the bromine water and the methanoic acid:
Br2 + HCOOH = 2HBr + CO2
Bromine water is yellow. The other reactant and products are colourless. Your task is to show that the rate of reaction depends on the concentrations of the starting materials. The following are available: methanoic acid solution (in a glass bottle), bromine water (in a glass bottle), distilled water (in a flask), 4 beakers (50 cm3, with a scale on their side to show the volume), 4 eyedroppers (without a scale), 4 Pasteur pipette (with a scale on their side to show the volume), 4 measuring cylinder (10 cm3, with a scale on their side to show the volume), stopwatch.
(a) What equipment and how many pieces of them are needed among the above listed ones for the experiment?
(b) Which materials and how much of them would you put into the chosen equipment at the time of the preparation of the experiment?
(c) How would you start the experiment?
(d) What experiences would you expect by the end of the experiment?
(e) Based on your experiences, how could you decide, how the rate of reaction depends on the concentrations of the starting materials?
Test 3; Task 8:
You want to identify the following three water samples, with the help of acid–base indicators:
(A) Rainwater that is only slightly acidic because of the carbon dioxide dissolved in it, and its pH is 5.6.
(B) Acid rainwater that was collected in a polluted area and its pH is 2.8.
(C) A water sample collected from the Lake Balaton that has got a pH 8.0, because the waterbed is made of alkaline rocks.
You can do no more than two experiments, and you can choose indicators from the following table. (Between the two pH values belonging to the two colours, the transition colour of the indicator could be seen that changes depending on the pH.)
Name of the indicator | One colour of the indicator |
phenolphthalein | colourless, if pH ≤ 8.2 |
bromothymol blue | yellow, if pH ≤ 6.0 |
crystal violet | green, if pH ≤ 0.8 |
litmus | red, if pH ≤ 5.0 |
methyl orange | red, if pH ≤ 3.1 |
Name of the indicator | The other colour of the indicator |
phenolphthalein | purple, if pH ≥ 10.0 |
bromothymol blue | blue, if pH ≥ 7.6 |
crystal violet | blue, if pH ≥ 2.6 |
litmus | blue, if pH ≥ 8.0 |
methyl orange | orange, if pH ≥ 4.4 |
(a) Which indicator would you use for Experiment 1? What colour change would you expect for each of the water samples?
(b) Which indicator would you use for the Experiment 2? What colour change would you expect for each of the water samples?
(c) Would false result be obtained if identical amounts of the water samples, but different amounts of indicators were used in the experiments that you have described? Explain your answer.
(d) Would you get a different result in the experiments that you described, if we replaced (as a model) the water sample “B” by 0.1 mol dm−3 acetic acid that has got a pH 2.7? Why?
Similar to Test 1, the first versions of Tests 2 and 3, including marking instructions, were made and corrected following suggestions from the university educators in the research group. Participating teachers had not seen the test before piloting the six student worksheets of the respective school years. This was to avoid tasks on Tests 2 and 3 influencing the pilot. However, to ensure test validity, Test 2 was tried with three classes/groups (N1 = 27, N2 = 12, N3 = 13, altogether 52) of 13–14 years old students not participating in the research in March 2018. For the same reason (to ensure test validity), Test 3 was tried with three classes/groups (N1 = 26, N2 = 15, N3 = 11, altogether 52) of 14–15 years old students not participating in the research in October 2018. Both tests and their marking instructions were further revised in response to results of the trial before they were filled in by the students participating in the sample.
Participating teachers marked the students’ tests, recording the marks in an Excel spreadsheet as instructed (see Supplement 2 and 3, ESI†). As there was an element of subjectivity in the grading protocol, the research group tried to standardize the grading to ensure that the application of the rubric is the same for each test, as done by Goodey and Talgar (2016). An experienced chemistry teacher reviewed the teachers’ marking and suggested modifications to the marking instructions. After discussions within the team, alterations were made. Based on these, the teachers’ marks were changed to ensure that a unified marking process, free from individual teachers’ decisions was used.
Several factors can threaten the validity of inferences made of any quasi-experimental work in chemistry education research (e.g.Shadish et al., 2002; Mack et al. 2019). Comparing outcomes of the tests written by Groups 2 and 3 with those of Group 1 (control group) was done to eliminate threats due to maturation (Shadish et al., 2002). Students’ natures and circumstances vary widely. A large cohort of students is important and, therefore, our research sample consisted of 920 students, from several schools in various parts of the country. The classes were randomly assigned to either the control group or one of the experimental groups. The content of Tests 1, 2 and 3, used to measure the outcome, differed. This was unsurprisingly unavoidable (as explained in the section written about the tests). However, construct validity (Cronbach and Meehl, 1955) was achieved, at least in part, by using the same structure for all tests, i.e. the same number of items related to the levels of the Revised Bloom Taxonomy. Statistical methods (see following section) have also been used to counterbalance the covariations.
Test | Group 1 | Group 2 | Group 3 | Σ |
---|---|---|---|---|
Test 0 | 298 | 291 | 294 | 883 |
Test 1 | 291 | 283 | 279 | 853 |
Test 2 | 272 | 258 | 282 | 812 |
Test 3 | 240 | 225 | 257 | 722 |
The following data were collected and analysed statistically:
• Student total scores for Tests 0, 1, 2 and 3 (T0 to T3).
• Student scores (marks) for EDS tasks (Tests 0, 1, 2 and 3).
• Student scores (marks) for DCK tasks (Tests 0, 1, 2 and 3).
• Gender of the student.
• School ranking. The student's school ranking amongst Hungarian secondary schools, according to the website “legjobbiskola.hu”. The participating schools were grouped into high, medium, and low ranking categories and a categorical variable was used according to these three levels (Appendix, Table 15). This allowed a statistical assessment of the impact of participating schools “quality” on the development of the students’ knowledge and skills.
• Mother's education. Two categories were formed depending on whether or not the student's mother had a degree in higher education. This categorical variable was intended to characterise the student's socioeconomic status.
• Note: Each test contained questions concerning the students’ science/chemistry grade and his or her attitude toward science or chemistry and the scientific experiments. However, the analysis of the students’ answers of the attitude questions is not included in the present study.
The results were analysed statistically. It was assumed that apart from the three types of instructional methods used during the intervention for Group 1, 2 and 3, other parameters (e.g. gender, mother's education, school ranking) and a covariate (student scores for T0 test) had also influenced the results. Therefore, the statistical analysis of data was accomplished by analysis of covariance (ANCOVA) of the SPSS Statistics software. Theobald and Freeman (2014) demonstrated how multiple linear regression techniques can distinguish treatment effects from student factors on an outcome variable across groups (Mack et al., 2019). However, if only the effects of continuous independent variables are investigated, linear regression analysis is equivalent with using ANCOVA, where the independent variables are the covariates. Nevertheless, regression analysis cannot handle nominal variables, like the ‘groups’. In this case dummy variables must be computed. In case of ANCOVA, nominal variables are the fix factors. That is the explanation, why the statistical analysis of data gained from the tests in the present research project was accomplished by ANCOVA. In the case of multiple comparisons Bonferroni correction was applied. Details of the other parameters and covariates were published earlier (Szalay et al. 2020). However, the statistical analyses which results are published in this present paper did not investigate the changes in the students’ grades and attitude, neither their effect on the development of the students’ EDS.
In the ANCOVA analysis total scores on Test 1 (T1total), Test 2 (T2total) and Test 3 (T3total) were the dependent variables, respectively. According to the preliminary analysis, the mothers’ education and the gender had a significant effect on the T0 results, but had little effect on the students’ achievement in T1, T2 and T3 tests. It might be assumed that the T0 results represent differences observed among mother's education and gender. However, over time a higher ranking school may provide more opportunities and greater incentive for development than a lower ranking school. If so, this suggests that the school ranking remains an important parameter. Therefore, at the time of the further calculations, only the group and the school ranking were parameters, and prior knowledge (represented by the T0 scores) was the covariant. According to the preliminary analysis, class size had a significant effect on the results. It is not independent of the school ranking, however, as without exception, class sizes were low (<20) in the high ranking schools and high in several of the medium and low ranking schools. For this reason, class size was not used as a parameter, but it was considered as being involved in the school ranking.
According to the Shapiro-Wilk tests the test scores are normally distributed in representative small groups (N = 24 or 48) (Appendix, Table 20). There are linear relationships between the scores on T0 and scores on T2 and T3 tests (Appendix, Table 21). The same approach was taken with the analyses of the students’ scores on the DCK tasks and on the EDS tasks on Tests 1–3 (at the end of Grade 7–9, resp.).
Group | T0total (SD) | T1total (SD) | T2total (SD) | T3total (SD) |
---|---|---|---|---|
Group 1 | 42.4% (13.2%) | 35.7% (15.7%) | 28.8% (20.3%) | 35.4% (20.6%) |
Group 2 | 41.9% (14.1%) | 43.4% (21.6%) | 40.0% (20.7%) | 34.9% (21.2%) |
Group 3 | 42.4% (12.7%) | 35.6% (18.9%) | 40.0% (22.3%) | 38.7% (21.2%) |
The ratios of the average scores of the experimental groups compared to that of the control group's in the beginning of the research project (‘Start’) and in the end of each school year (Grade 7, 8 and 9) are shown in Table 5.
Ratio | Start | Grade 7 | Grade 8 | Grade 9 |
---|---|---|---|---|
Group 2/Group 1 | 0.989 | 1.217 | 1.388 | 0.988 |
Group 3/Group 1 | 0.999 | 0.997 | 1.388 | 1.094 |
To investigate which parameters might have influenced these changes, ANCOVA analysis described in the previous section was applied for the total scores and each test. Partial Eta Squared (PES) values characterising the effect sizes are shown in Table 6.
The effects of the different instruction methods used in the cases of the different groups are shown in the row ‘Group’. According to these data, the intervention had only a weak significant effect at the end of Grade 7, but a stronger effect in the end of the following school year (Grade 8). In contrast, calculations showed no significant effect of the intervention at the end of Grade 9. However, the ranking of the students’ school already had a weak but significant effect on the students’ achievement at the beginning of the project (T0). It became stronger in the second (T2) and third (T3) years. The influence of the prior knowledge (represented by the T0total) can be explained by the fact that the higher ranking schools are able to select among many more applicants than the other schools. Unfortunately, the difference among the average achievement of the students from the different schools grew by the years. The T0total has a relatively strong effect in the second and the third year. This might be caused by the better prepared students coping more easily with the growing amount of more demanding tasks.
Looking at the average scores on the tasks intended to measure DCK, the main trends are similar to those seen in the full tests. The intervention had a weak significant positive effect on the results of Group 2 in Grade 7 and on the results of both Groups 2 and 3 in Grade 8 (Table 7). This is also shown by the relative average scores (Table 8) and the PES values (Table 9). However, in Grade 9, Group 3 performed a little better on the DCK tasks than the other groups.
Group | T0DCK (SD) | T1DCK (SD) | T2DCK (SD) | T3DCK (SD) |
---|---|---|---|---|
Group 1 | 57.6% (15.5%) | 41.1% (17.5%) | 33.7% (22.0%) | 35.6% (21.8%) |
Group 2 | 56.2% (15.8%) | 48.3% (22.6%) | 44.4% (23.7%) | 38.3% (21.5%) |
Group 3 | 58.1% (16,0%) | 38.6% (19.3%) | 45.1% (25.8%) | 41.7% (21.5%) |
Ratio | Start | Grade 7 | Grade 8 | Grade 9 |
---|---|---|---|---|
Group 2/Group 1 | 0.976 | 1.175 | 1.318 | 1.076 |
Group 3/Group 1 | 1.009 | 0.939 | 1.338 | 1.171 |
School ranking had a significant positive effect on students’ DCK scores at the beginning of the project (T0). It became stronger in the second (T2) and third (T3) years (Table 9). In contrast, prior knowledge only had a positive significant effect in Grades 8 and 9.
The raw data, prior to ANCOVA analysis, showed lower EDS scores than DCK scores (Table 10). The trends shown by the changes of the relative averages compared to that of the control groups’ (Table 11), and by the changes of the Partial Eta Squared values characterising the effect of the intervention (Table 12) are similar again to the changes of the results in the whole test.
Group | T0EDS (SD) | T1EDS (SD) | T2EDS (SD) | T3EDS (SD) |
---|---|---|---|---|
Group 1 | 27.2% (18.6%) | 30.3% (21.5%) | 24.0% (26.3%) | 35.1% (24.9%) |
Group 2 | 27.7% (18.8%) | 38.6% (24.9%) | 35.6% (26.1%) | 31.5% (26.6%) |
Group 3 | 26.7% (16.8%) | 32.6% (24.9%) | 34.9% (28.4%) | 35.6% (26.4%) |
Ratio | Start | Grade 7 | Grade 8 | Grade 9 |
---|---|---|---|---|
Group 2/Group 1 | 1.018 | 1.274 | 1.483 | 0.897 |
Group 3/Group 1 | 0.982 | 1.076 | 1.454 | 1.014 |
The intervention seemed to have a weak significant effect on the achievement of the Group 2 on the EDS tasks in the end of Grade 7. Whereas it had a somewhat stronger significant effect on the results of both experimental groups in Grade 8, with Group 2 students significantly outperforming Group 3 students. However, there was no significant difference between the average scores of the three groups on the EDS tasks in Grade 9. School ranking had a significant effect on the students’ achievement on the EDS tasks in the first year of the project. It became stronger in Grade 8 and 9, when the effect of prior knowledge also became significant.
Another possible reason for a significant increase in students’ EDS not being seen during the project's third year is that the students were less motivated by Grade 9. According to some of the participating teachers, many students had decided about their further education by this stage. If that did not need chemistry to gain a place in university, they usually put less effort in learning the subject in Grades 9 and 10.
Coincidentally, Grade 9 chemistry in Hungary requires a high level of abstract thinking relative to learning in Grades 7 and 8. This may reduce a student's enthusiasm. Perhaps they found learning chemistry hard and this also had an influence on the work they were prepared to invest. Students knew from the beginning that project test results would not count in their school chemistry assessment, perhaps enhancing these effects. This was unavoidable since teachers used different marking schemes to arrive at students’ end of semester grades. Even students intending to study chemistry further may have recognised that they do not need EDS to attend the chosen university, rightly or wrongly. The national curriculum valid from September 2020 explicitly prescribes the development of EDS, influencing textbooks and workbooks. Unfortunately, development of EDS will probably be still neglected if teachers are not convinced that EDS are important for their students’ entry to Higher Education. Assessment drives teaching and learning (e.g.Kahn and O’Rourke, 2005). Teaching EDS will only become standard practice when the final national exams include their assessment.
School ranking had a strong effect on the students’ test results Tables 6, 9 and 12). This could support both the effect of the development of the abstract thinking and the effect of the lack of motivation on Grade 9 results. It is reasonable to assume that in higher ranking schools the students’ abstract way of thinking could develop more quickly in a more challenging environment. On the other hand, those students are probably also encouraged, and often urged by their teachers and parents, to reach higher achievement in several different subjects and activities. Therefore, they might feel obliged to put more effort into writing a test, even if its result does not count at the time of their formal assessment. Snook et al. (2009), in a commentary on Hattie's book (2008), quotes the OECD volume (2005) which states that the largest source of variation in student learning is due to differences in what students bring to school – their abilities and attitudes, and family and community. Apart from these social variables the “school effects” (policies, principal, buildings, school size, curriculum, teachers) are regarded to be important and among them the teacher is considered to be the most important variable (Snook et al., 2009).
The relatively high positive development of the Group 2 students’ EDS in Grade 8 (Table 10, T2EDS) compared to Group 3 students might have been caused by the different treatments of the two groups. Group 2 students did not have to plan experiments. Therefore, those classes probably had more time to discuss with their teacher why the experiments were planned as they were (as written on the student sheets). In contrast, Group 3 students had to design experiments, in teams. Teachers facilitated each team's work and had to share their attention amongst the teams. Some teams might have successfully worked out what to do in time. But there might have been others that could not quite manage, and simply watched and followed other teams. Also, teachers might have helped those teams more to plan the experiment if they thought it was the only way to finish the lesson in time. Assuming the latter happened, it would have reduced the development of learning experimental design skills considerably compared with Group 2 students.
The answers to the research questions are as follows.
RQ1: A significant (albeit moderate) positive change was detected in the students’ ability to design experiments (EDS) in the experimental groups in contrast to the control group. This is the result of the intervention in the second year of the project (Grade 8). However, this difference disappeared by the end of the third year (Grade 9).
RQ2: Both methods used in the interventions seemed to have a significant positive effect on DCK in Grade 8. However, only a very weak effect was detected in Group 3 in Grade 9.
RQ3: Group 2 students had significantly higher average scores on the EDS tasks of Test 2 than Group 3 students. However, no significant difference was found between Groups 2 and 3 at the end of Grade 9.
Performance on any assessment is at least partially driven by the students’ motivation for success on the measure and test taking abilities (Cannady et al., 2019), and our research is not an exception. The first sentences in the beginning of each test reminded students that the research aimed to make the teaching of chemistry interesting and effective. Asking them to complete the test according to the best of their knowledge was intended as incentive, but it probably did not convince all students. Tests and student sheets were designed to make the work more interesting. Not all students might have found this to be the case. Unsurprisingly, ability scores are influenced by motivational levels and scientific sensemaking is no exception.
Despite our efforts to organise a randomized controlled trial and evaluate the results using an appropriate statistical method, the statement asserted by Mack et al. (2019) still applies to our research: “No single study can evaluate every variable and every theoretical relationship underlying an instructional model. Therefore, intervention studies should progressively build upon one another with evidence for the relationships between people, treatments, outcomes, and settings accumulating over time.”
The instruments used (40 minutes paper-based tests) could only provide a limited picture of how students benefited from the interventions. Research literature has repeatedly found that practical work does not offer a significant advantage in developing students’ scientific conceptual understanding as measured using written tests (Abrahams, 2017, Hennah, 2019). Nevertheless, this was the most likely way to ensure all students had an equal chance to complete the test successfully. This requirement would have been even more difficult to meet at a practical exam.
Finally, one interpretation of results that showed an improvement on just one measurement, could be due to statistical noise or is something peculiar to that measurement. It is possible that other instruments would have shown different effects.
However, teachers, teacher educators and educational researchers should do all they can to provide the best opportunities for each and all students to learn science.
From the test results of Group 2, it may be concluded, and suggested to teachers, that discussing experimental design with students after they have done a step-by-step experiment is a worthwhile activity. However, asking students to design experiments, even in a scaffolded way (as with Group 3) needs careful planning. This is especially important when choosing the age when it will be used. Reid and Ali (2020), discussing the implications of Piaget's and Ausubel's findings, warn that the potential to think in abstract emerges slowly during adolescence. They suggest that we can only be confident that students are capable of thinking in terms of hypotheses, a critical skill in scientific thought, by the age of 16.
The question remains whether EDS could be developed further in the two experimental groups in the last part of the present project. However, the relevance of Test 4 results will be limited, because disruptions to teaching caused by Covid-19. Therefore, a thorough enquiry about the participating teachers’ knowledge and opinion concerning the intervention might help to interpret the data.
Two aspects are important in our long-term research plans. Students seem to need even more guidance and detailed instructions on how to design experiments, which is needed even at undergraduate level (George-Williams et al., 2018). The usefulness of an experimental design template, like the one described by Cothron et al. (2000), should be investigated. Apart from providing the questions to be investigated and the basic principles of experimental design, they need to be taught in a systematic way to find the equipment and materials, the independent and dependent variables, the parameters that had to be held constant, the source of possible errors etc. A meta-analysis of studies on guided inquiry instructions concluded that more specific guidance during inquiry assignment results in higher quality learning products (Lazonder and Harmsen, 2016).
Secondly, students need motivation. Chemistry topics that impact the environment and human health are likely to be favoured by students and are congruent with augmenting their self-expression in chemistry (Ross et al., 2020). This could be accompanied by the systems thinking that is a possible way to address emerging global challenges in the context of a rapidly changing world (e.g.Mahaffy et al., 2018). Therefore, many science educators’ interest has turned recently toward this approach. The prompt used should stimulate student interest and curiosity (Cothron et al., 2000) to involve them in the learning of chemistry and systems thinking seems to be a promising approach.
Ranking | High | Medium | Low |
---|---|---|---|
a Number of students completing Test 0. | |||
School ranking | 4, 15, 17, 29, 32 | 36, 41, 72, 84, 91, 114, 170 | 198, 214, 289, 325, 416, 526 |
N | 254 | 345 | 284 |
Parameter | X 2 | df | p |
---|---|---|---|
Gender | 0.202 | 2 | 0.904 |
Mother's education | 2.333 | 2 | 0.311 |
School ranking | 4.555 | 4 | 0.336 |
T0total | 26.13 | 28 | 0.566 |
T0DCK | 13.83 | 16 | 0.612 |
T0EDS | 21.00 | 18 | 0.279 |
Mothers’ education | Group 1 | Group 2 | Group 2 | Total |
---|---|---|---|---|
No degree in higher education | 34 | 31 | 42 | 107 |
Has a degree in higher education | 126 | 129 | 118 | 373 |
Total | 160 | 160 | 160 | 480 |
Gender | Group 1 | Group 2 | Group 2 | Total |
---|---|---|---|---|
Boy | 74 | 72 | 70 | 216 |
Girl | 86 | 88 | 90 | 264 |
Total | 160 | 160 | 160 | 480 |
School ranking | Group 1 | Group 2 | Group 2 | Total |
---|---|---|---|---|
Low | 53 | 67 | 54 | 174 |
Medium | 68 | 66 | 72 | 206 |
High | 39 | 27 | 34 | 100 |
Total | 160 | 160 | 160 | 480 |
Test/DCK or EDS tasks | Statistic | df | p |
---|---|---|---|
T0 | 0.985 | 24 | 0.963 |
T1 | 0.979 | 24 | 0.877 |
T2 | 0.977 | 24 | 0.838 |
T3 | 0.960 | 24 | 0.438 |
T0DCK | 0.963 | 24 | 0.496 |
T1DCK | 0.970 | 24 | 0.673 |
T2DCK | 0.965 | 24 | 0.540 |
T3DCK | 0.960 | 24 | 0.445 |
T0EDS | 0.944 | 24 | 0.205 |
T1EDS | 0.933 | 24 | 0.113 |
T2EDS | 0.921 | 24 | 0.062 |
T3EDS | 0.936 | 24 | 0.134 |
Footnote |
† Electronic supplementary information (ESI) available. See DOI: 10.1039/d0rp00338g |
This journal is © The Royal Society of Chemistry 2021 |