A web-based ionisation energy diagnostic instrument: exploiting the affordances of technology

Abstract

The internet is prevalent in society today, and user-friendly web-based productivity tools are readily available for developing diagnostic instruments. This study sought to determine the affordances of a web-based diagnostic instrument on ionisation energy (wIEDI) based on the pen-and-paper version, the Ionisation Energy Diagnostic Instrument (IEDI) that was previously developed and reported on. The Google Forms platform was used to develop the wIEDI and it allowed a degree of personalisation such that specific second-tier options are offered in response to the student's choice of answer in the first tier. Students could choose one or more reasons in the second tier or supply their own reasons, and they were asked to indicate their confidence in their choice of answer–reason combinations. The wIEDI was administered to 274 A-level students (257 Grade 11 and 17 Grade 12), and answer–reason combinations indicating alternative conceptions were highlighted only if 5% or more students expressed confidence in them in the third-tier confidence measure. The results showed that all the possible alternative conceptions of ionisation energy reported in the previous study were also identified in the present study. Additional alternative conceptions were indicated as new reasons had to be developed for many items in the wIEDI to ensure that there were sufficient reasons for each first-tier response, and students were allowed to choose more than one reason for their answer. The wIEDI better facilitated responses reflecting the consistency of the use of specific ideas in student thinking and provided direct evidence of students’ possible manifold conceptions and thinking within each question as well as across a range of questions. It also allowed easy collation of the comments students typed in response to the ‘Others’ and ‘I do not know the answer’ options. Thus, the study makes a case for researchers and teachers using such technology in the diagnostic assessment of students.

---

Introduction

The topic of ionisation energy is important as the concepts involved also provide the foundation for the understanding of atomic structure, periodic trends and the energetics of reactions (Taber, 2003). However, it is a difficult topic to learn as it involves concepts which are highly abstract and complex. For example, to explain various trends in ionisation energy, students need to consider a variety of factors such as nuclear charge, electron–nucleus separation, the type of orbital occupied by an electron and whether the orbital is occupied by one or two electrons. To make it even more complicated, some of these factors may work in opposing directions and the students need to decide which are the dominant factors for a given situation (Tan et al., 2005a, 2005b; Taber and Tan, 2007).

Thus, it is not surprising that research has shown that students have difficulty in understanding the concepts involved in ionisation energy (Taber, 1999, 2003; Tan et al., 2005a, 2005b; Taber and Tan, 2007). When asked to explain the aspects of patterns in ionisation energies, many A-level students (16 to 18 years old) used the octet rule/full shell framework to state that atoms ‘want’ to obtain full outermost shells or octets of electrons in the outermost shell, and once they are achieved, the atoms are ‘reluctant’ to lose these electronic configurations; the students seemed to think that the sodium ion already has an octet configuration so it will not gain an electron to reform the sodium atom (Taber, 2013a). The conservation of force thinking was also commonly seen in the students’ explanations – that the attractive force arising from a charged entity will be shared amongst oppositely-charged entities. For example, students thought that if an electron left a sodium atom, the remaining 10 electrons would share the attraction from the 11 protons in the nucleus that had previously been shared among the 11 electrons. Using a two-tier multiple choice diagnostic instrument where students were required to choose an answer from the first-tier options and a reason for the answer from the second-tier options, Tan and his colleagues found that students also used the stable fully-filled or half-filled subshell conceptions (these configurations are considered to be ‘stable’ in a similar sense to ‘stable’ octets) and relation-based reasoning (considering only one or two factors instead of all the factors involved) (Driver et al., 1996) to justify their answers on the ionisation energy trend across Periods 2 and 3 of the Periodic Table. The students seemed to have difficulty in applying basic electrostatic principles that they learned in physics, if they were also taking A-level physics in the first instance, to explain the interactions between the nucleus and electrons in an atom/ion and between electrons in orbitals, and how these interactions influenced the trends of ionisation energy (Taber, 1999; Cann, 2000).

In addition, studies on students’ understanding of ionisation energy showed that students’ responses to similar items were inconsistent (Taber, 1999, 2003; Tan et al., 2005a, 2005b; Taber and Tan, 2007), with some students using “both the correct concepts and alternative modes of thinking, applying a different way of thinking to different items” (Taber and Tan, 2007, p. 386). For example, Tan et al. (2005b) reported that several students who applied the appropriate electrostatics concepts to correctly choose the answer and reason for an item on the re-formation of the sodium atom did not use similar concepts to justify their answer for an item on the stability of the species involved in the ionisation process; instead, they used the octet rule/full shell framework. Gilbert and Watts (1983) did strike a cautionary note when they stated that “it may well be that an individual's conceptions make use of several frameworks” (p. 86), and some evidence of this has been found when a student's thinking is explored in detail (Taber, 2000). Palmer (1999) was in agreement when he suggested that “different conceptions can be brought into play in response to different problem contexts” (p. 639). It has also been proposed that students can have stable conceptual resources and/or p-prims, which are interpretations of perceptual data (diSessa, 1993, 2018), that they find useful and relevant (although this may refer to pre-conscious processing) in constructing knowledge in response to a particular context at a point in time, but they do not apply these elements consistently (Taber, 2006; Taber and Tan, 2007). This ‘Knowledge in Pieces’ perspective (diSessa, 2018) is helpful in explaining why students can give the correct answers, and then show that they have alternative conceptions, even a range of different ones, for other similar questions. This discussion assumes that the elicited student's response reflects his/her “deep-rooted well-considered and significant beliefs about the world” and not “just ‘throw-away’ suggestion(s)” (Taber, 2006, p. 152) or guessing in response to questions asked during interviews or tests. Asking students to report their levels of confidence in their responses can provide researchers or teachers with more information in drawing such inferences.

When two-tier multiple choice instruments are used, another possible explanation for a student's inconsistent responses to questions involving very similar concepts is that he/she is forced to choose one reason for his/her answer even if the other reasons presented are also attractive to him/her. If the student has more than one conception for a given concept (Palmer, 1999; Taber, 2013b) or has several applicable p-prims which he/she can use to construct answers (diSessa, 1993, 2018), then allowing the student to choose more than one reason can enable him/her to display all the relevant conceptions or p-prims that he/she has for items assessing similar concepts. The researcher can track these reasons across related items instead of only one reason per item, and this could help increase the consistency of the student's responses across the related items.

Allowing several responses may seem inconsistent with the common assessment practice of offering one correct response with several incorrect distractors for an objective item. However, arguably, authentic assessment items should sometimes admit more than one correct response given that in many contexts of scientific interest (such as comparing ionisation energies between elements) there may be several relevant factors at work, and given that often the same phenomenon may have complementary explanations at several different levels of analysis (for example, in chemistry, using complementary macroscopic concepts and submicroscopic models).

Multiple choice diagnostic instruments

Various methods have been used to investigate students’ understanding of concepts so that their difficulties and alternative conceptions can be determined. These methods include interviews, multiple choice tests, concept mapping, sorting tasks, drawings, and open-ended questions. Multiple choice tests are frequently used to diagnose students’ understanding of concepts as a large number of students can be assessed in a given amount of time and they are easily administered, processed and analysed (Peterson et al., 1989; Taber, 1999; Tan and Treagust, 1999); interview data, concept maps and written responses to open-ended questions require more time and effort to analyse. In a previous study (Tan et al., 2005a, 2005b), the paper-based two-tier multiple choice Ionisation Energy Diagnostic Instrument (IEDI) was used to determine A-level students’ (Grades 11 and 12) understanding of the concepts involved in ionisation energy (an example of the items in the IEDI is given in Fig. 1). Such diagnostic instruments require students to choose an answer from the first-tier options and a reason for the answer from the second-tier options. Incorrect reasons (sometimes termed ‘distractors’) in the second tier can be derived from evidence of actual students’ alternative conceptions, usually determined from sources such as previous studies, interviews, tests, and teachers’ classroom experiences.

Fig. 1 Item 1 in the IEDI and the wIEDI.

Hasan et al. (1999) suggested the use of the Certainty of Response Index (CRI) with the items on diagnostic tests to differentiate between students’ lack of knowledge (and resorting to guessing) and possible alternative conceptions. In this protocol, respondents are requested to indicate how confident they are, on a six-point scale (0–5), of their knowledge required to answer a given question. So, if the degree of certainty is low (CRI of 0–2), it will be assumed that the answer is mainly due to guesswork, and if the degree of certainty is high (CRI of 3–5) and the answer is wrong, it will be assumed that the student has a possible alternative conception. Researchers have incorporated confidence testing into two-tier multiple choice diagnostic instruments to produce three-tier tests (Caleon and Subramaniam, 2010a; Brandriet and Bretz, 2014) and four-tier tests (Caleon and Subramaniam, 2010b; Sreenivasulu and Subramaniam, 2013; Lin, 2016; Yan and Subramaniam, 2018) to determine the students’ confidence of their answer–reason combinations or their confidence of their choices in each tier separately, and hence, whether students have genuine or spurious alternative conceptions (Caleon and Subramaniam, 2010a). Whether such an additional tier, with the extra demand on respondents, is indicated may in part depend on the purpose of an instrument. When used for classroom diagnostic assessment by teachers, the outcomes of administering such an instrument can (and should) be the basis of classroom follow-up where students can discuss their ideas further as part of a dialogue to help shift them towards thinking with the relevant scientific models.

Web-based diagnostic instrument

Computer-based assessment (CBA) has advantages over paper-based assessment. Computers appeal to many students and seem to “increase their science learning motivation when they are used in self-assessment and learning” (Nikou and Economides, 2016, p. 1246). Students can access CBA at their convenience and CBA can be programmed to provide timely feedback to large numbers of students based on their answers to the assessment items (Thelwall, 2000; Nguyen et al., 2006; Topping et al., 2007; Haigh, 2010). It also facilitates the analysis of results as it captures data electronically to enable teachers to better identify, monitor and address their students’ difficulties (Topping et al., 2007). In addition, various forms of representations can be included into the items in computer-based assessment: for example, animations as a means to determine students’ understanding of submicroscopic interactions between particles, or simulations as a means to investigate their understanding of macroscopic, real-world phenomena (Lin, 2016). Computer-based assessment can be easily administered through the internet due to its widespread use in society these days.

It is easy to create two-tier, three-tier or four-tier multiple choice diagnostic instruments using web-based productivity and assessment tools, such as Google Forms, without much technical expertise (Tan and Koh, 2008). Web-based instruments created using Google Forms have the advantages of being easily accessible to teachers and students, being readily administered to students, and allowing students’ answers to be collated automatically; the data can be downloaded as Microsoft Excel documents, allowing further processing and analyses of students’ responses to be undertaken. The Google Forms platform can inform students of their results and the answer for each item, but providing adaptive feedback, that is, diagnosing students’ difficulties if they get the items wrong and responding to these difficulties to help them attain the required learning (Devedžić, 2006), requires much more content and item development such as would be possible on platforms such as the WileyPlus ORION Adaptive Practice (Wiley, 2018).

It may be challenging to allow students to choose more than one reason in pen-and-paper tests as optical mark readers, when used to facilitate the collation of students’ answers, might not be able to process students’ choice of more than one reason per item. However, this function can be facilitated by Google Forms using a checklist format for the second-tier reasons. In addition, Google Forms can also address the issue of students treating the options in the two tiers as if they are independent of each other, allowing answer–reason combinations which are seemingly illogical and difficult to interpret (Griffard and Wandersee, 2001; Tan et al., 2002; Loh et al., 2014). This is achieved by allowing a specific set of reasons to be presented to the students based on the answer that they selected. It has to be acknowledged that a response combination that makes no sense to the researcher or teacher can be an arbitrary combination made by a student, a mistaken tick in the wrong box, or a response that makes at least as much sense to the student at that time as the other available permutations, including the orthodox ones.

Thus, the aim of the study was to determine the affordances of a web-based two-tier multiple choice diagnostic instrument on ionisation energy based on the pen-and-paper version developed in a previous study. It attempts to make a case for why researchers and teachers should consider using computer-based or web-based diagnostic instruments for the formative assessment of students.

The web-based ionisation energy diagnostic instrument (wIEDI)

The development of the previous paper-based IEDI (Tan et al., 2005a, 2005b) was guided by the procedures prescribed by Treagust (1995). It included interviews with students and inviting students to explain their choices of responses to one-tier multiple choice items; these provided the reasons for the second tier. A wide range of responses can be elicited, potentially indicating various alternative conceptions students may hold. However, it can result in too many options for a second reasoning tier than it is practical to present to students in the instrument, so less popular options have to be omitted. However, these less popular options offered a valuable resource for the present study as discussed in the next paragraph. In the first phase of the development of the web-based version, the stem and the first-tier options of all 10 items in the IEDI were converted directly into a web-based Google Forms format. Making use of the function in Google Forms which allows branching from the first tier to the second tier, changes were made in the second tiers of the items as different sets of options (branches) would be offered to students depending on their choices in the first tier. In addition to addressing ‘illogical’ answer–reason combinations, mental fatigue may be reduced in that the student is not presented with the whole list of options to consider, but only what is expected to be a relevant subset.

New options were included in the second tiers so that there were at least two options in any set of second-tier reasons (see Fig. 1 for a comparison of item 1 in the original instrument and in the web-based instrument). These additional options were derived from the second-tier options of other related items, as well as less popular responses given in the open-ended versions of the instrument and interviews from the previous study which were not adopted as options for the paper-based IEDI (where all students need to be presented with the same second-tier options regardless of their first-tier choice) as only four to five second-tier options were required for each item in the IEDI. As previously mentioned, Google Forms allowed the researchers to easily format the second tier as a checklist so that the student could choose more than one reason and even write his/her own reason if he/she wished. Again, there may be an issue here if students see the instrument as a test, and are familiar with multiple choice tests where there is always one correct response and a number of incorrect distractors. However, in a topic such as ionisation energy, explanations may involve more than one factor, for example, nuclear charge, ionic radius, the type of orbital (i.e., the value of the azimuthal quantum number) and the type of shell (i.e., the value of the principal quantum number), so students are expected to consider a range of factors at one time. Thus, it is important that students are aware that they can select several options rather than just one response option, as well as write their own reasons if necessary.

There were discussions among the researchers on the number of points to have on the confidence scale and where to place the scale – for the first tier only, the second tier only, both the first and second tiers, or the whole item (answer–reason combination). It was decided that the confidence interval would cover the whole item as the researchers wanted the students to indicate that their confidence in their answer–reason combinations. The analyses of results would focus on answer–reason combinations rather than the answer or reason(s) separately as the specific sets of reason options offered to the students were dependent on the answers chosen by them. Rather than having a multi-point confidence scale and then calculating the mean to differentiate between students’ lack of knowledge (and resorting to guessing) and possible alternative conceptions or understanding, the researchers decided to have two points only, ‘Confident’ or ‘Not confident’, so students need only consider two options. If students did not know the answer to the first tier, they could choose the ‘I do not know the answer’ option rather than guessing an answer. In this case, the confidence level tier would not be offered as it would not serve any meaningful purpose to ask if they were confident or not confident that they did not know the answer.

Ethics and ethical considerations

This research project was approved by the Nanyang Technological University Institutional Review Board (IRB-2014-12-017). Approval from the Ministry of Education, Singapore, was also granted to the researcher to work with A-level teachers and students, and collect data in schools. The schools involved in this study were selected on the basis of the willingness of the teachers in the schools to participate in the study. These teachers were known to the first author through their participation in in-service workshops conducted by the first author and/or through interactions during meetings of chemistry teachers organised by the Singapore Ministry of Education. The author emailed the teachers giving them details of the study and asked the teachers if they were interested in participating in the study. He also offered to meet them in their schools to elaborate on the study if they required further details and clarifications. Permission from the principals of the schools was sought after the teachers agreed to participate in the study.

The teachers had the freedom to select the number of classes/students to administer the instrument to. They forwarded the students involved an email from the first-named author containing a brief description of the study and the address of a website which contained more detailed information. In the website, the parents of the students were requested to give their consent to the participation of their children in the study; parents, of course, could withhold consent and the process would end with a message thanking them for taking time to visit the website. The students were also requested to give their assent to their participation in the study if their parents had given their consent. If the students declined to give assent, the process would also end with a ‘thank you’ message. The sampling method used, relying on personal contacts, and students opting in, prevents us from claiming that the results reported here are representative of the population of A-level chemistry students in Singapore. However, the main purpose of this study was not to survey a population, but to develop and trial a web-based instrument to determine its affordances.

Trials and the actual study

The first trial involved two schools with a total of 39 Grade 11 students. Four students (a pair from each school) were interviewed to determine if they had any problem with the web-based instrument and to probe the reasoning behind their responses. The instrument was revised based on the students’ responses in the quiz and interviews. The second trial involved a total of 79 Grade 11 students from two new schools and one of the schools which participated in the first trial. A pair of students from one new school was interviewed. Again, the instrument was revised based on the students’ responses in the quiz and interviews. The finalised version of the instrument, the wIEDI, is available online at https://tinyurl.com/wIEDIfinal and as the Appendix. The wIEDI was administered to Grade 11 and Grade 12 students from eight A-level institutions in Singapore from May to July 2016. In total, 274 students (257 Grade 11 and 17 Grade 12) participated, answering the questions in the web-based instrument. Student data were downloaded as Microsoft Excel documents and the Excel program was used to process the data.

Affordances of the wIEDI

As more reasons had to be included in the items of the wIEDI to cater to the branching of answers to their respective reasons, and as students are allowed to choose more than one reason for their answers in the first tier, the web-based diagnostic instrument is likely to have greater affordances over the pen-and-paper version. The wIEDI should allow the detection of more possible alternative conceptions, reveal a greater consistency of students’ answers across items and provide direct evidence of students’ manifold conceptions. Since data are captured directly in the web-based instrument, it is also easier to collate students’ responses to the ‘I do not know’ or ‘Other’ options in the instrument. These are described in the following sections.

Alternative conceptions indicated

Examples of response combinations suggesting that students held alternative conceptions are reported in Table 1. All reasons indicating alternative conceptions highlighted in the earlier study using the paper-based IEDI (Tan et al., 2005a, 2005b; Taber and Tan, 2007) were also confidently chosen by 5% or more students using the wIEDI, so the new instrument appears to successfully diagnose issues that would have been diagnosed with the earlier paper-based instrument. The lower cut-off percentage of 5% was chosen in the present study as only answer–reason combinations which students expressed confidence in were considered. Since new reasons were developed for many items to ensure that there were sufficient reasons in the second tier for each first tier response and as students were allowed to choose more than one reason for their answer, the new instrument allows the identification of additional non-canonical thinking potentially indicating alternative conceptions that would not have been diagnosed by the paper-based instrument – this affordance was to be expected (see Table 1).

Table 1 Examples of alternative conceptions indicated by reasons confidently chosen by 5% or more of Grade 11 and 12 students (n = 274) compared with that chosen by 10% or more of Grade 11 and 12 students (n′ = 979) in the previous study

Alternative conception indicated by reason	Item and reason	Percentage of students (n = 274)	Percentage of students in the previous study (n′ = 979)
Notes: * indicates that the percentage of students was less than 10% in the previous study (Tan et al., 2005a, 2005b). N.A. ** indicates that the option was not present in the previous study.
Octet rule framework (oct)
The sodium ion will not recombine with an electron to reform the sodium atom because the sodium ion has a stable/noble gas configuration, so it will not gain an electron to lose its stability.	Q1 (A2)	17	44
The Na(g) atom is a less stable system than the Na^+(g) ion and a free electron because the outermost shell of the Na^+(g) ion has achieved a stable octet/noble gas configuration.	Q3 (B2)	36	64
The second ionisation energy of sodium is greater than its first ionisation energy because removal of the second electron disrupts the stable octet structure of the Na^+ ion.	Q4 (A1)	22	16
The first ionisation energy of sodium is less than that of magnesium because sodium will achieve a stable octet configuration if its 3s electron is removed.	Q5 (B1)	7	9*
The first ionisation energy of sodium is less than that of aluminium because sodium will achieve a stable octet configuration if an electron is removed.	Q7 (B1)	5	6*
Stable fully-filled subshell conception (ffs)
The Na(g) atom is a less stable system than the Na^+(g) ion and a free electron because all subshells are filled fully with electrons for Na^+(g), whereas the Na atom has one subshell with a lone electron that gives it its instability.	Q3 (B3)	26	N.A.**
The second ionisation energy of sodium is greater than its first ionisation energy because it is more difficult to remove an electron from a fully-filled 2p subshell.	Q4 (A4)	25	N.A.**
The first ionisation energy of sodium is less than that of magnesium because magnesium has a fully-filled 3s subshell which gives it stability as paired electrons are more stable and harder to remove.	Q5 (B2)	11	13
The first ionisation energy of magnesium is greater than that of aluminium because removal of an electron will disrupt the stable completely filled 3s subshell of magnesium.	Q6 (A1)	5	6*
Stable half-filled subshell conception (hfs)
The first ionisation energy of silicon is less than that of phosphorus because the 3p subshell of phosphorus is half-filled; hence it is stable.	Q8 (B1)	8	25
The first ionisation energy of phosphorus is greater than that of sulfur because the 3p subshell of phosphorus is half-filled; hence it is stable.	Q9 (A2)	7	20
The first ionisation energy of phosphorus is greater than that of sulfur because sulfur needs to lose one electron to have a stable half-filled 3p subshell.	Q9 (A4)	8	N.A.**
Conservation of force conception (cof)
When an electron is removed from the sodium atom, the attraction of the nucleus for the ‘lost’ electron will be redistributed among the remaining electrons in the sodium ion because the number of protons in the nucleus is the same but there is one less electron to attract, so the remaining 10 electrons will experience greater attraction by the nucleus.	Q2 (A1)	37	50
When an electron is removed from the sodium atom, the attraction of the nucleus for the ‘lost’ electron will be redistributed among the remaining electrons in the sodium ion because the Na^+(g) ion has one less shell compared to the Na(g) atom.	Q2 (A2)	14	N.A.**
When an electron is removed from the sodium atom, the attraction of the nucleus for the ‘lost’ electron will be redistributed among the remaining electrons in the sodium ion because the total force of attraction by the nucleus is always divided equally between the total number of electrons.	Q2 (A3)	9	N.A.**
The Na(g) atom is a less stable system than the Na^+(g) ion and a free electron because the average force of attraction on each electron of the Na^+(g) ion is greater than that of the Na(g) atom as the attractive force of the nucleus of the Na^+(g) is shared among fewer electrons.	Q3 (B1)	6	1*
The second ionisation energy of sodium is greater than its first ionisation energy because the same number of protons in the Na^+(g) ion attract one less electron, so the attraction for the remaining electrons is stronger.	Q4 (A2)	42	18
Strongly electropositive sodium only loses electrons (pos)
The sodium ion will not recombine with an electron to reform the sodium atom because sodium is strongly electropositive, so it only loses electrons	Q1 (A1)	8	5*

---

In discussing responses to the items, we refer readers to the Appendix where each item is presented with the first-tier response options (denoted by letters) and their associated potential reasons presented (denoted by numbers). We also use some abbreviations to denote the alternative conceptions linked to particular response options:

• (oct) – octet rule framework

• (ffs) – stable fully-filled subshell conception

• (hfs) – stable half-filled subshell conception

• (cof) – conservation of force conception

• (pos) – strongly electropositive sodium only loses electrons

So, for example, for item 1 response A2 (oct) indicates a confident response option of A (on the first tier) and 2 (on the second tier), which is linked to thinking using the octet rule framework. Where responses in two or more items related to an alternative conception (e.g. the octet rule framework in items 1 and 3) are discussed, these responses may be grouped together {e.g. [Q1 (A2), Q3 (B2)] (oct)}.

The percentages in Table 1 include students who had chosen a particular option in all possible combinations as they could choose more than one reason for an answer. For example, 17% of the total number of students confidently chose the ‘octet rule framework’ reason A2 (oct) in item 1. This included students who chose option A2 only, as well as the combinations (A1, A2) only, (A1, A2, A3) only, (A1, A2, A4) only, (A2, A3) only, (A2, A3, A4) only and (A2, A4) only. There is double counting in the sense that a few of the above several combinations [(A1, A2), (A1, A2, A3), (A1, A2, A3, A4)] are also included in the computation of the percentage of students who confidently chose A1 (pos), the ‘strongly electropositive sodium only loses electrons’ option in item 1. Thus, the total percentage of students who confidently chose all possible answer–reason combinations in an item could exceed 100%.

Consistency of students’ answers

It can be seen from Table 1 that there are five entries for the ‘octet rule framework’ combinations for the current study and three entries which were highlighted in the previous study. Two entries, B1 (oct) in item 5 and B1 (oct) in item 7, were not discussed in the previous study because they were not chosen by 10% or more of the participants. It was possible that the ‘octet rule framework’ reasons were less popular or considered less likely by the students compared to the other reasons available in items 5 and 7 (see the Appendix) for the contexts in the items (comparison of the first ionisation energies of different pairs of elements), so these reasons were not chosen by many students as the sole reason they could select in the two items in the previous study. Cross-tabulation was used to study the consistency of the students’ choices across items (Tan et al., 2002) in the wIEDI (see Table 2). In the previous study, only two instances of consistency of students’ responses were found (Tan et al., 2005b), while in this study, there are 11 instances highlighted in Table 2. For example, 8% of the total number of students confidently and consistently chose the ‘octet rule framework’ options, A2 (oct) in item 1 and B2 (oct) in item 3. This figure includes students who confidently chose one of these combinations [(A2) only, (A1, A2) only, (A1, A2, A3) only, (A1, A2, A4) only, (A2, A3) only, (A2, A4) only] in item 1 as well as one of these combinations [(B2) only, (B1, B2) only, (B1, B2, B3) only, (B2, B3) only] in item 3. There were three instances of consistent choices of the ‘octet rule framework’ reasons confidently made by 5% or more of the students in the current study (there was only one instance highlighted in the previous study), but none involving B1 (oct) in item 5 or B1 (oct) in item 7 and any similar ‘octet rule framework’ reason in item 1, 3 or 4. The percentages of students choosing B1 (oct) in item 5 (7%) or B1 (oct) in item 7 (5%) were already very near the threshold percentage of 5% in the first instance, so the percentage of students confidently choosing any of the two reasons and another similar reason in item 1, 3 or 4 would likely be even lower. In addition, items 1, 3 and 4 focused on the Na⁺ ion, which had an octet of electrons in its outermost shell, so the octet rule framework might have come easily to the minds of the students in these items.

Table 2 Cross-tabulation of reasons indicating alternative conceptions confidently chosen by 5% or more students within and across the items in the wIEDI discussed in this paper

Alternative conception	Items cross-tabulated	Percentage of students (n = 274)
Octet rule framework (oct)	Q1 (A2), Q3 (B2)	8
	Q1 (A2), Q4 (A1)	5
	Q3 (B2), Q4 (A1)	10
Stable fully-filled subshell conception (ffs)	Q3 (B3), Q4 (A4)	9
Conservation of force (cof)	Q2 (A1), Q2 (A2)	13
	Q2 (A1), Q2 (A3)	9
	Q2 (A1), Q4 (A2)	24
	Q2 (A2), Q4 (A2)	8
	Q2 (A3), Q4 (A2)	6
	Q2 (A1), Q2 (A2), Q4 (A2)	8
	Q2 (A1), Q2 (A3), Q4 (A2)	5

---

The stable fully-filled subshell conception was common among students (see Table 1). Reasons B3 (ffs) of item 3 and A4 (ffs) of item 4 were created to ensure that there were sufficient reasons for the related answers in both items. These two reasons were effective distractors, each attracting about 25% of the students who were confident of their choices. The only instance of the ‘stable fully-filled subshell conception’ reasons consistently and confidently chosen by 5% or more students over two items was B3 (ffs) in item 3 and A4 (ffs) in item 4 (9% of 274 students) (see Table 2). Without these two additional reasons created for the wIEDI, there would not have been any response options consistent with the stable fully-filled subshell conception (no instance was highlighted in the previous study). The lack of consistency of students’ choices was also highlighted in the earlier study, and this was attributed to students’ holding manifold conceptions and different conceptions coming into play in response to different contexts (Gilbert and Watts, 1983; Palmer, 1999; Taber 1999, 2000). This explanation seemed to be supported in this study – even though the students were allowed to choose as many reasons as they believed supported the answer that they chose, they did not confidently and consistently choose the stable fully-filled subshell conception across all four items. As previously mentioned, items 3 and 4 focused on the Na⁺ ion, which has an octet of electrons in its outermost shell as well as fully-filled 2p subshells, so students might have been ‘directed’ to focus on the electronic configuration of the Na⁺ ion in these items when considering the reasons for their answers, but not in items 5 and 6.

Direct evidence of manifold conceptions

Apparent inconsistency across item responses in objective diagnostic instruments might be seen as suggesting that students have no stable, strongly committed, concepts informing their choices. However, when response options are based on free responses given by comparable students in interviews, such apparent inconsistences have been argued to reflect the availability of manifold conceptions/multiple frameworks such that the response given by a respondent for a particular item may not be the only viable response that students could offer for that item (Taber, 2003). However, this interpretation remains somewhat speculative as long as only one response is allowed per item.

The existence of manifold conceptions was evident in Table 3, where students could be seen choosing many reasons for an answer in the same item, even the correct reason as well as reasons indicating alternative conceptions; providing direct evidence of manifold conceptions is an important affordance of the web-based diagnostic instrument. The percentage for a given cross-tabulation in Table 3 was derived from the total number of students’ responses containing the specific answer–reason combinations. For example, in item 4, 18% of 274 students confidently chose reason A1 (oct) and the correct reason A3* in all combinations (second entry). This figure includes the students who confidently chose (A1, A3*) only as well as (A1, A2, A3*) only, (A1, A2, A3*, A4) only, (A1, A2, A3*, Other) only, (A1, A3*, A4) only, and (A1, A3*, Other) only. A number of students (5%) confidently chose all four reasons: A1 (oct), A2 (cof), A3* and A4 (ffs)! Being allowed to choose more than one reason for an answer in an item (and the students had to indicate that they were confident of their choices) in this study presented direct evidence of students’ manifold conceptions compared to the earlier study where students’ manifold conceptions were inferred through the lack of consistency of their choices across items.

Table 3 Choice of several reasons confidently chosen by 5% or more students within items 1 to 4 of the wIEDI

Item	Reasons chosen	Percentage of students (n = 274)
1	A1 (pos), A2 (oct)	5
	B1*, B2 (rxn)	6
2	A1 (cof), A2 (cof)	13
	A1 (cof), A3 (cof)	9
3	B2 (oct), B3 (ffs)	25
4	A1 (oct), A2 (cof)	9
	A1 (oct), A3*	18
	A1 (oct), A4 (ffs)	12
	A2 (cof), A3*	33
	A2 (cof), A4 (ffs)	12
	A3*, A4 (ffs)	21
	A1 (oct), A2 (cof), A3*	9
	A1 (oct), A2 (cof), A4 (ffs)	5
	A1 (oct), A3*, A4 (ffs)	10
	A2 (cof), A3*, A4 (ffs)	11
	A1 (oct), A2 (cof), A3*, A4 (ffs)	5

---

Interestingly, there were 12 instances of 5% or more students confidently choosing the ‘octet rule framework’ reasons and the ‘stable fully-filled subshell conception’ reasons within the same item and across different items (see Table 4). For example, students (5%) who confidently chose [Q3 (B2), Q4 (A1)] (oct) [Q3 (B3), Q4 (A4)] (ffs) (last entry in Table 4) decided on one of these combinations [(B1, B2, B3) only, (B2, B3) only] in item 3 as well as one of these combinations [(A1, A2, A3, A4) only, (A1, A3, A4) only, (A1, A4) only] in item 4. Tan et al. (2005b) suggested that the ‘stable fully-filled subshell’ was analogous to the ‘stable octet’, and that the latter seemed to lead ‘naturally’ to the former conception; there was no conflict between them: for example, the Na⁺ ion could have both a ‘stable octet’ of electrons as well as a ‘stable fully-filled 2p subshell’. Thus, the results showed that choices presenting both alternative conceptions were viable and attractive to the students, especially in items 3 and 4 where the Na⁺ ion was foregrounded.

Table 4 Cross-tabulation of reasons indicating the octet rule framework and stable fully-filled subshell conception confidently chosen by 5% or more students

Alternative conceptions	Items cross-tabulated	Percentage of students (n = 274)
Octet rule framework and stable fully-filled subshell conception	[Q1 (A2)] (oct), [Q3 (B3)] (ffs)	5
	[Q1 (A2)] (oct), [Q4 (A4)] (ffs)	5
	[Q3 (B2)] (oct), [Q3 (B3)] (ffs)	25
	[Q3 (B2)] (oct), [Q4 (A4)] (ffs)	12
	[Q4 (A1)] (oct), [Q3 (B2)] (ffs)	8
	[Q4 (A1)] (oct), [Q4 (A4)] (ffs)	12
	[Q1 (A2), Q3 (B2)] (oct), [Q3 (B3)] (ffs)	5
	[Q3 (B2), Q4 (A1)] (oct), [Q3 (B3)] (ffs)	8
	[Q3 (B2), Q4 (A1)] (oct), [Q4 (A4)] (ffs)	6
	[Q3 (B2)] (oct), [Q3 (B3), Q4 (A4)] (ffs)	9
	[Q4 (A1)] (oct), [Q3 (B3), Q4 (A4)] (ffs)	5
	[Q3 (B2), Q4 (A1)] (oct), [Q3 (B3), Q4 (A4)] (ffs)	5

---

Cross-tabulation of students’ choices in the wIEDI showed five instances of 5% or more students consistently and confidently choosing ‘conservation of force thinking’ reasons across items 2 and 4 (see Table 2). It could be seen that the reason was not popular in item 3 (B1, 6%) compared to the ‘octet rule framework’ (B2, 36%) and ‘stable fully-filled subshell conception’ (B3, 26%) reasons in the same item (see Table 1); hence it was not consistently chosen by the students despite them being allowed to choose more than one reason in the item. Interestingly, the students who confidently chose reason A1 (cof) in item 2 preferred B2 (oct) (16%) and B3 (ffs) (11%) to the less popular B1 (cof) (4%) in item 3, but preferred A2 (cof) (24%) to A1 (oct) (9%) and A4 (ffs) (9%) in item 4 (see Table 5). The use of the phrase “a more stable system” could have favoured the use of the octet rule framework and stable fully-filled subshell conception by students over the use of the conservation of force thinking in item 3. Likewise, the focus on the removal of the second electron from sodium in item 4 could have cued the use of the conservation of force thinking rather than the octet rule framework and stable fully-filled subshell conception.

Table 5 Cross-tabulation of reasons indicating the conservation of force thinking and the octet rule framework and/or stable fully-filled subshell conception confidently chosen by students within and across items 2, 3 and 4

Alternative conceptions	Items cross-tabulated	Percentage of students (n = 274)
Conservation of force, octet rule framework and stable fully-filled subshell conception	[Q2 (A1), Q3 (B1)] (cof)	4
	[Q2 (A1)] (cof) [Q3 (B2)] (oct)	16
	[Q2 (A1)] (cof) [Q3 (B3)] (ffs)	11
	[Q2 (A1)] (cof) [Q3 (B2)] (oct) [Q3 (B3)] (ffs)	11
	[Q2 (A1), Q4 (A2)] (cof)	24
	[Q2 (A1)] (cof) [Q4 (A1)] (oct)	9
	[Q2 (A1)] (cof) [Q4 (A4)] (ffs)	9
	[Q2 (A1)] (cof) [Q4 (A1)] (oct) [Q4 (A4)] (ffs)	4

---

‘I do not know’ and ‘Other’

Allowing the students to choose the ‘I do not know’ option for an answer reduces the likelihood of students guessing an answer, and allowing them to explain why they do not know the answer enables the researchers to be aware of the reason(s) for their difficulties. For example, 6% of the students chose the ‘I do not know the answer’ option in item 1, and several students gave the reason that they were not aware of the reverse process of ionisation. It is true that teachers do not normally address the recombination of positive ions and electrons when teaching the topic of ionisation energy (but the students would have learnt that the positively-charged ion could attract the negatively-charged electron), so students have to infer that since ionisation is an endothermic process, the positive ions and electrons lost will be at a higher energy level than the original atom or less positively-charged ion. This inference is not beyond the ability of a student taking A-level chemistry. In item 2, 8% indicated that they were not familiar with the concepts involved, or they did not understand the question. There was an increasing number of ‘blanks’ (students not providing any reason for choosing “I do not know the answer”) as the quiz progressed and those who provided reasons gave similar responses such as they were not familiar with or did not understand the concepts involved. In any case, it is better for researchers (and teachers) that students choose the ‘I do not know the answer’ option than make a guess so that there is less confusion over whether an incorrect answer–reason combination indicates a possible alternative conception or a guess, although the confidence level of the combination chosen is another check for guessing.

Allowing students to write their own reasons if they believe that reasons presented to them are inadequate also enables the researchers to gain additional insights into their thinking. In item 5, 7% of the students who were confident of their comments in the ‘Other’ section for answer B indicated that sodium and magnesium had the same number of inner shell electrons; hence the shielding effects were similar, but since the nuclear charge of magnesium was higher than that of sodium, the first ionisation energy of sodium was lesser than that of magnesium. This explanation would also work for the comparison of the first ionisation energies of sodium and aluminium, as commented by the 6% who confidently chose the ‘Other’ option for answer B in item 7. In fact, cross-tabulations indicated that nine students (3%) consistently and confidently supplied the ‘same shielding effect, different nuclear charge’ explanation to both ‘Other’ sections for answer B in items 5 and 7. However, the explanation would not work for the comparison of the first ionisation energies of magnesium and aluminium in item 6, and interestingly, no student wrote this reason in the ‘Other’ section for answer B in item 6. The authors are considering the addition of these responses to the respective items, as well as others, before the release of the web-based diagnostic instrument to teachers.

It has to be noted that although the ‘I do not know’ and ‘Other’ options can also be incorporated into pen-and-paper diagnostic instruments, the students’ comments have to be manually keyed by the researchers into a software program for further collation and analysis. Google Forms greatly facilitate this process by automatically collating the responses typed in by students. It is also possible to make it compulsory for students to type in their responses if they choose the ‘I do not know’ or ‘Other’ options before they can move on to the next question. However, the researchers elected not to do so in order to make it less tedious for students so that they would be less likely to quit the quiz; besides, they could just key in any letter or symbol in the text boxes and move on.

Limitations

Although the web-based diagnostic instrument has important affordances over the original pen-and-paper version, there are limitations associated with it.

Online administration of the diagnostic instrument

The online request for the consent and assent from parents and students and the students responding online to the quiz were supposed to lighten the administrative burden and save the curriculum time of teachers who facilitated the research in their schools. It was fine if students and parents responded that they did not wish to participate, but many did not respond at all, so the researchers could not decide when to close a quiz for a particular school and had to wait for an extended period of time before doing so. In the present study, the response rate is unknown but only five of the 279 students (2%) who responded declined to participate in the study. Thus, to ensure a higher response rate, students should attempt the web-based diagnostic instrument in class after consent and assent had been obtained either online or in the paper form. If the purpose of the research was to survey a population's conceptions, then attempting the quiz in class would ensure that the student was actually attempting the quiz himself/herself and without reference to any textbook or notes. However, their teachers must be willing to contribute the additional time and effort required to administer the quiz.

Since the focus of this study was on the affordances of the web-based instrument rather than a survey of a population's conceptions (which was done in the previous study), the required conditions were not so stringent. Teachers informed students to attempt the quiz at their convenience after studying for it and without reference to any material. The students were also reassured that the quiz had no effect on their school grades, so it was unlikely that anyone else other than the student would attempt the quiz. It was possible that some students may ‘cheat’ by checking their textbooks or other resources, and consequently shift their understanding, allowing them to give canonical responses. However, based on the results and cross-tabulations obtained which indicated student difficulties, any effect of students referring to material during the quiz on the affordances of the web-based instrument seemed to be minimal. If the instrument is adopted as a diagnostic tool for use by a teacher, these issues do not apply as the activity becomes part of normal teaching, and the teacher can decide if students are to attempt the quiz under test conditions, or as seat work or homework.

Deciding the cut-off for the identification of alternative conceptions

Similar studies using two-tier multiple choice diagnostic instruments (e.g.Tan et al., 2002; Yan and Subramaniam, 2018) usually discuss options chosen by at least 10% of the sample so that the percentage is generally above the probability of getting the item correct by chance. For example, in the original pen-and-paper IEDI, seven out of the ten items have two first-tier options (excluding the ‘I do not know the answer’ option) and four to five second-tier options; thus the chances of getting an item correct by guessing is about 10% to 12.5%. So, there is merit in discussing options chosen by at least 10% of the sample. With the wIEDI, only choices which students reported that they were confident of were taken into consideration, so it was assumed that the students were not guessing; hence it can be justified to discuss even the answer–reason combination chosen by a single student. However, having the cut-off of one student will result in a lengthy discussion of students’ possible alternative conceptions and having an unnecessarily high cut-off of 10% would eliminate 50% of the possible alternative conceptions reported in Table 1 and 73% of the cross-tabulations reported in Table 2. Thus, 5% seems to be a good compromise in deciding the results to highlight as potentially noteworthy in this report. It has to be noted that in a teaching context, even one student confidently choosing a combination indicating an alternative conception will be of concern to his/her teacher.

Conclusions

This study focuses on the affordances of a web-based diagnostic instrument on ionisation energy developed from the existing paper-based version. The results showed that all the student difficulties in the various items identified in the previous study using the paper-based IEDI were also identified in this study using the wIEDI. As expected, additional instances of similar student difficulties as well as some new ones surfaced in the present study as new reasons were included to ensure sufficient number of reasons for each answer in an item, and students were allowed to choose more than one reason for an answer. Allowing students to choose more than one reason for an answer in the wIEDI also resulted in evidence of a greater consistency of student thinking across items and provided direct evidence of students’ possible manifold conceptions. The results showed that there were many instances in which 5% or more students chose two or more reasons in the wIEDI indicating different alternative conceptions and even the correct reasons as well as one or more reasons indicating alternative conceptions. Cross-tabulation of students’ choices across items highlighted the consistency of students’ choices of reasons and the contexts in which the different conceptions came into play. Guessing by students should affect the results of the present study to a lesser extent compared to the results of the previous study in which students were not required to state their confidence in their choice of reasons. However, the low response rate of students when administering the web-based quiz at home for research purposes can be a problem if such an instrument is used for survey purposes, so students may need to answer the questions in class to achieve a better response rate.

The present study shows that a diagnostic instrument offering information about students’ understanding of a complex chemistry topic, and about students’ confidence in their understanding, can be provided in an on-line form that offers a degree of personalisation and automatic collation of responses. Teachers will likely welcome the ability to use the wIEDI, but of course the value of the wIEDI relies on teachers interpreting and applying the information it can offer to customise their classroom teaching for their particular classes.

Conflicts of interest

There are no conflicts to declare.

Appendix

Specifications of the items in the wIEDI

No.	Items	Areas of alternative conceptions
1	For Questions 1 to 4, please refer to the statement below.
	Sodium atoms are ionised to form sodium ions as follows:
	Na(g) → Na^+(g) + e
	Once the outermost electron is removed from the sodium atom, forming the sodium ion (Na^+), the sodium ion will not combine with an electron to reform the sodium atom.	Octet rule framework
	(A) True.	Meaning of electropositivity
	(B) False.	Process of ionisation
	(C) I do not know the answer.	Redox/displacement reactions
	Your Response: (A) True.
	Reason(s) (You may choose as many options as you think are correct):
	(1) Sodium is strongly electropositive, so it only loses electrons.
	(2) The Na^+(g) ion has a stable/noble gas configuration, so it will not gain an electron to lose its stability.
	(3) The process of Na(g) losing an electron is irreversible.
	(4) A very large amount of energy is required to add an electron to the Na^+(g) ion.
	Other: _________________
	How confident are you that your answer and reason(s) are correct?
	( ) Confident
	( ) Not confident
	Note: Students are asked how confident they are of their reasons for every answer that they choose except for the choice of “I do not know the answer”.
	Your Response: (B) False.
	Reason(s) (You may choose as many options as you think are correct):
	(1) The positively-charged Na^+(g) ion can attract a negatively-charged electron.
	(2) Na^+(g) can react with an anion to reform the Na atom.
	(3) Na^+(g) can undergo a displacement reaction to form Na.
	Other: _________________
	How confident are you that your answer and reason(s) are correct?
	( ) Confident
	( ) Not confident
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: B1)
2	When an electron is removed from the sodium atom, the attraction of the nucleus for the ‘lost’ electron will be redistributed among the remaining electrons in the sodium ion (Na^+).	Conservation of force conception
	(A) True.	Nuclear attraction for an electron
	(B) False.
	(C) I do not know the answer.
	Your Response: (A) True.
	Reason(s) (You may choose as many options as you think are correct):
	(1) The number of protons in the nucleus is the same but there is one less electron to attract, so each of the remaining 10 electrons will experience greater attraction by the nucleus.
	(2) The Na^+(g) ion has one less shell compared to the Na(g) atom.
	(3) The total force of attraction by the nucleus is always divided equally between the total number of electrons.
	Other: _________________
	Your Response: (B) False.
	Reason(s) (You may choose as many options as you think are correct):
	(1) The amount of attraction between an electron and the nucleus depends on the number of protons present in the nucleus and the distance of the electron from the nucleus. It does not depend on how many other electrons are present, although electrons do repel each other.
	(2) The electron which is removed will take away the attraction of the nucleus with it when it leaves the atom.
	(3) The attraction for the lost electron is very small and can be neglected compared to the attraction for the remaining 10 electrons.
	Other: _________________
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: B1)
3	In an isolated system, the Na(g) atom is a more stable system than the Na^+(g) ion and a free electron.	Conservation of force conception
	(A) True.	Octet rule framework
	(B) False.	Stable fully-filled subshell conception
	(C) I do not know the answer.	Reactions of the gaseous sodium ion
	Your Response: (A) True.
	Reason(s) (You may choose as many options as you think are correct):
	(1) Energy is required to ionise the Na(g) atom to form the Na^+(g) ion.
	(2) The Na^+(g) ion has a vacant shell which can be filled by electrons from other atoms or molecules to form the Na(g) atom.
	(3) Na^+(g) has a positive charge and will react readily with other atoms or molecules to form the Na(g) atom.
	Other: _________________
	Your Response: (B) False.
	Reason(s) (You may choose as many options as you think are correct):
	(1) The average force of attraction on each electron of the Na^+(g) ion is greater than that of the Na(g) atom because the attractive force of the nucleus of the Na^+(g) is shared among fewer electrons.
	(2) The outermost shell of Na^+(g) ion has achieved a stable octet/noble gas configuration.
	(3) All subshells are filled fully with electrons for Na^+(g), whereas the Na atom has one subshell with a lone electron that gives it its instability.
	Other: _________________
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: A1)
4	After the sodium atom is ionised (i.e. forms the Na^+ ion), more energy is required to remove a second electron (i.e. the second ionisation energy is greater than the first ionisation energy) from the Na^+ ion.	Conservation of force conception
	(A) True.	Octet rule framework
	(B) False.	Stable fully-filled subshell conception
	(C) This should not happen as the Na^+(g) ion will not lose any more electrons.	Relation-based thinking – focusing on electronic repulsion and ignoring nuclear charge
	(D) I do not know the answer.	Nuclear attraction for an electron
	Your Response: (A) True.	Process of ionisation
	Reason(s) (You may choose as many options as you think are correct):
	(1) Removal of the second electron disrupts the stable octet structure of the Na^+(g) ion.
	(2) The same number of protons in Na^+(g) attract one less electron, so the attraction for the remaining electrons is stronger.
	(3) The second electron is located in a shell which is closer to the nucleus.
	(4) It is more difficult to remove an electron from a fully filled 2p subshell.
	Other: _________________
	Your Response: (B) False.
	Reason(s) (You may choose as many options as you think are correct):
	(1) The second electron is removed from a paired 2p orbital and it experiences repulsion from the other electron in the same orbital.
	(2) The second electron to be removed experiences shielding by the other electrons.
	(3) The second electron to be removed experiences a decrease in nuclear charge because some nuclear charge is removed when the first electron leaves.
	(4) It takes the same amount of energy to remove any electron from the same atom.
	Other: _________________
	Your Response: (C) This should not happen as the Na^+(g) ion will not lose any more electrons.
	Reason(s) (You may choose as many options as you think are correct):
	(1) The Na^+(g) ion already has an octet of electrons in its outer shell.
	(2) Sodium is in Group 1 and cannot form a Na^2+(g) ion.
	(3) The inner shell electrons of the Na^+(g) ion cannot be removed as there is a very strong attraction between the positively charged ion and the remaining electrons.
	Other: _________________
	Your Response: (D) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: A3)
5	Sodium, magnesium and aluminium are in Period 3. How would you expect the first ionisation energy of sodium (1s^2 2s^2 2p^6 3s^1) to compare to that of magnesium (1s^2 2s^2 2p^6 3s^2)?	Octet rule framework
	(A) The first ionisation energy of sodium is greater than that of magnesium.	Stable fully-filled subshell conception
	(B) The first ionisation energy of sodium is less than that of magnesium.	Relation-based thinking – the more electrons, the further away they are/filling of electrons in orbitals and ignoring nuclear charge
	(C) I do not know the answer.	Nuclear attraction for an electron
	Your Response: (A) The first ionisation energy of sodium is greater than that of magnesium.
	Reason(s) (You may choose as many options as you think are correct):
	(1) The paired electrons in the 3s orbital of magnesium experience repulsion from each other, and this effect is greater than the increase in the nuclear charge in magnesium.
	(2) The 3s electrons of magnesium are further from the nucleus compared to those of sodium.
	(3) The attractive force between the valence electron and the nucleus of sodium is stronger than the attractive force between the two valence electrons and the nucleus of magnesium.
	(4) Sodium has fewer electrons compared to magnesium, so the constant nuclear charge attracts the electrons more strongly in sodium.
	Other: _________________
	Your Response: (B) The first ionisation energy of sodium is less than that of magnesium.
	Reason(s) (You may choose as many options as you think are correct):
	(1) Sodium will achieve a stable octet configuration if its 3s electron is removed.
	(2) Magnesium has a fully-filled 3s subshell which gives it stability as paired electrons are more stable and harder to remove.
	(3) In this situation, the effect of an increase in nuclear charge in magnesium is greater than the effect of an increase in the repulsion between its paired electrons in the 3s orbital.
	Other: _________________
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: B3)
6	How do you expect the first ionisation energy of magnesium (1s^2 2s^2 2p^6 3s^2) to compare to that of aluminium (1s^2 2s^2 2p^6 3s^2 3p^1)?	Stable fully-filled subshell conception
	(A) The first ionisation energy of magnesium is greater than that of aluminium.	Relation-based thinking – the more electrons, the further away they are/filling of electrons in orbitals and ignoring nuclear charge
	(B) The first ionisation energy of magnesium is less than that of aluminium.	Relation-based thinking – focusing on electronic repulsion and ignoring nuclear charge
	(C) I do not know the answer.	‘Destruction’ of orbitals when electrons are removed
	Your Response: (A) The first ionisation energy of magnesium is greater than that of aluminium.
	Reason(s) (You may choose as many options as you think are correct):
	(1) Removal of an electron will disrupt the stable completely-filled 3s subshell of magnesium.
	(2) The 3p electron of aluminium is further from the nucleus compared to the 3s electrons of magnesium.
	(3) In this situation, the effect of an increase in nuclear charge in aluminium is less than the effect of an increase in the repulsion between the electrons in its outermost shell.
	(4) The outermost electron of aluminium is removed from a 3p orbital, and 3p orbitals are higher in energy than 3s orbitals.
	(5) Removal of one electron from aluminium would result in a filled 3s subshell which is associated with greater stability.
	Other: _________________
	Your Response: (B) The first ionisation energy of magnesium is less than that of aluminium.
	Reason(s) (You may choose as many options as you think are correct):
	(1) In this situation, the effect of an increase in nuclear charge in aluminium is greater than the effect of an increase in the repulsion between the electrons in its outermost shell.
	(2) The paired electrons in the 3s orbital of magnesium experience repulsion from each other, whereas the 3p electron of aluminium is unpaired.
	(3) When an electron is removed from aluminium, the 3p orbital is destroyed. When an electron is removed from magnesium, the 3s orbital is still intact.
	Other: _________________
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: A3)
7	How do you expect the first ionisation energy of sodium (1s^2 2s^2 2p^6 3s^1) to compare to that of aluminium (1s^2 2s^2 2p^6 3s^2 3p^1)?	Octet rule framework.
	(A) The first ionisation energy of sodium is greater than that of aluminium.	Stable fully-filled subshell conception
	(B) The first ionisation energy of sodium is less than that of aluminium.	Conservation of force conception
	(C) I do not know the answer.	Relation-based thinking – the more electrons, the further away they are/filling of electrons in orbitals and ignoring nuclear charge
	Your Response: (A) The first ionisation energy of sodium is greater than that of aluminium.	Relation-based thinking – focusing on electronic repulsion/shielding and ignoring nuclear charge
	Reason(s) (You may choose as many options as you think are correct):	Trend of the first ionisation energy across Period 3
	(1) Aluminium will attain a fully-filled 3s subshell if an electron is removed.
	(2) The 3p electron of aluminium experiences greater shielding from the nucleus compared to the 3s electron of sodium.
	(3) The 3p electron of aluminium is further away from the nucleus compared to the 3s electron of sodium.
	(4) The outermost electron of aluminium is removed from a 3p orbital, and 3p orbitals are higher in energy than 3s orbitals.
	Other: _________________
	Your Response: (B) The first ionisation energy of sodium is less than that of aluminium.
	Reason(s) (You may choose as many options as you think are correct):
	(1) Sodium will achieve a stable octet configuration if an electron is removed.
	(2) In this situation, the effect of an increase in nuclear charge in aluminium is greater than the effect of an increase in the repulsion between the electrons in its outermost shell.
	(3) The first ionisation energy always increases across a period.
	Other: _________________
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: B2)
8	Silicon, phosphorus and sulfur are in Period 3. How would you expect the first ionisation energy of silicon (1s^2 2s^2 2p^6 3s^2 3p^2) to compare to that of phosphorus (1s^2 2s^2 2p^6 3s^2 3p^3)?	Stable half-filled subshell conception
	(A) The first ionisation energy of silicon is greater than that of phosphorus.	Relation-based thinking – the more electrons, the further away they are/filling of electrons in orbitals and ignoring nuclear charge
	(B) The first ionisation energy of silicon is less than that of phosphorus.	Relation-based thinking – focusing on electronic repulsion/shielding and ignoring nuclear charge
	(C) I do not know the answer.	Trend of the first ionisation energy across Period 3
	Your Response: (A) The first ionisation energy of silicon is greater than that of phosphorus.
	Reason(s) (You may choose as many options as you think are correct):
	(1) Silicon has fewer electrons than phosphorus; thus its 3p electrons face less shielding.
	(2) The 3p electrons of phosphorus are further away from the nucleus compared to those of silicon.
	(3) In this situation, the effect of an increase in nuclear charge in phosphorus is less than the effect of an increase in the repulsion between the electrons in its outermost shell.
	Other: _________________
	Your Response: (B) The first ionisation energy of silicon is less than that of phosphorus.
	Reason(s) (You may choose as many options as you think are correct):
	(1) The 3p subshell of phosphorus is half-filled; hence it is stable.
	(2) In this situation, the effect of an increase in nuclear charge in phosphorus is greater than the effect in an increase in the repulsion between its 3p electrons.
	(3) The first ionisation energy always increases across a period.
	Other: _________________
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: B2)
9	How would you expect the first ionisation energy of phosphorus (1s^2 2s^2 2p^6 3s^2 3p^3) to compare to that of sulfur (1s^2 2s^2 2p^6 3s^2 3p^4)?	Stable half-filled subshell conception
	(A) The first ionisation energy of phosphorus is greater than that of sulfur.	Relation-based thinking – the more electrons, the further away they are/filling of electrons in orbitals and ignoring nuclear charge
	(B) The first ionisation energy of phosphorus is less than that of sulfur.	Paired electrons in an orbital attract each other strongly
	(C) I do not know the answer.	Trend of the first ionisation energy across Period 3
	Your Response: (A) The first ionisation energy of phosphorus is greater than that of sulfur.
	Reason(s) (You may choose as many options as you think are correct):
	(1) 3p electrons of sulfur are further away from the nucleus compared to those of phosphorus.
	(2) The 3p subshell of phosphorus is half-filled; hence it is stable.
	(3) In this situation, the effect of an increase in nuclear charge in sulfur is less than the effect of an increase in the repulsion between its 3p electrons.
	(4) Sulfur needs to lose one electron to have a stable half-filled 3p subshell.
	Other: _________________
	Your Response: (B) The first ionisation energy of phosphorus is less than that of sulfur.
	Reason(s) (You may choose as many options as you think are correct):
	(1) More energy is required to overcome the attraction between the paired 3p electrons in sulfur.
	(2) In this situation, the effect of an increase in nuclear charge in sulfur is greater than the effect of an increase in the repulsion between its 3p electrons.
	(3) The first ionisation energy always increases across a period.
	Other: _________________
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: A3)
10	How would you expect the first ionisation energy of silicon (1s^2 2s^2 2p^6 3s^2 3p^2) to compare to that of sulfur (1s^2 2s^2 2p^6 3s^2 3p^4)?	Stable half-filled subshell conception
	(A) The first ionisation energy of silicon is greater than that of sulfur.	Conservation of force conception
	(B) The first ionisation energy of silicon is less than that of sulfur.	Relation-based thinking – the more electrons, the further away they are/filling of electrons in orbitals and ignoring nuclear charge
	(C) I do not know the answer.	Trend of the first ionisation energy across Period 3
	Your Response: (A) The first ionisation energy of silicon is greater than that of sulfur.
	Reason(s) (You may choose as many options as you think are correct):
	(1) Sulfur will have a stable half-filled 3p subshell if an electron is removed.
	(2) The 3p electrons of sulfur are further away from the nucleus compared to those of silicon.
	(3) In this situation, the effect of an increase in nuclear charge in sulfur is less than the effect of an increase in the repulsion between its 3p electrons.
	(4) The silicon atom has fewer electrons in its outer shell to share the nuclear attraction.
	Other: _________________
	Your Response: (B) The first ionisation energy of silicon is less than that of sulfur.
	Reason(s) (You may choose as many options as you think are correct):
	(1) In this situation, the effect of an increase in nuclear charge in sulfur is greater than the effect of an increase in the repulsion between its 3p electrons.
	(2) The first ionisation energy always increases across a period.
	Other: _________________
	Your Response: (C) I do not know the answer.
	Can you elaborate on why you do not know the answer or are uncertain of the answer? _____________________
	(Answer: B1)

References

1. Brandriet A. R. and Bretz S. L., (2014), Measuring meta-ignorance through the lens of confidence: examining students’ redox misconceptions about oxidation numbers, charge, and electron transfer, Chem. Educ. Res. Pract., 15(4), 729–746.
2. Caleon I. and Subramaniam R., (2010a), Development and application of a three-tier diagnostic test to assess secondary students’ understanding of waves, Int. J. Sci. Educ., 32(7), 939–961.
3. Caleon I. and Subramaniam R., (2010b), Do students know what they know and what they do not know? Using a four tier diagnostic test to assess the nature of students’ alternative conceptions, Res. Sci. Educ., 40(3), 313–337.
4. Cann P., (2000), Ionization energies, parallel spins, and the stability of half-filled shells, J. Chem. Educ., 77(8), 1056–1061.
5. Devedžić V., (2006), Semantic web and education, Boston, MA: Springer.
6. diSessa A. A., (1993), Towards an epistemology of physics. Cognit. Instruct., 10(2&3), 105–225.
7. diSessa A. A., (2018), A Friendly Introduction to “Knowledge in Pieces”: Modeling Types of Knowledge and Their Roles in Learning, in Kaiser G., Forgasz H., Graven M., Kuzniak A., Simmt E. and Xu B. (ed.), Invited Lectures from the 13th International Congress on Mathematical Education, Cham: Springer, pp. 65–84.
8. Driver R., Leach J., Millar R. and Scott P., (1996), Young People's Images of Science, Buckingham: Open University Press.
9. Gilbert J. K. and Watts D. M., (1983), Concepts, misconceptions and alternative conceptions: changing perspectives in science education, Stud. Sci. Educ., 10, 61–98.
10. Griffard P. B. and Wandersee J. H., (2001), The two-tier instrument on photosynthesis: what does it diagnose? Int. J. Sci. Educ., 23(10), 1039–1052.
11. Haigh M., (2010), Why use computer-based assessment in education? A literature review, Res. Matter., 10, 33–40, retrieved from http://www.cambridgeassessment.org.uk/our-research/all-published-resources/research-matters/rm-10/.
12. Hasan S., Bagayoko D. and Kelley, E. L., (1999), Misconceptions and the Certainty of Response Index (CRI), Phys. Educ., 34(5), 294–299.
13. Lin J.-W., (2016), Development and Evaluation of the Diagnostic Power for a Computer-Based Two-Tier Assessment, J. Sci. Educ. Technol., 25(3), 497–511.
14. Loh A. S. L., Subramaniam R. and Tan K. C. D., (2014), Exploring students’ understanding of electrochemical cells using an enhanced two-tier diagnostic instrument, Res. Sci. Technol. Educ., 32(3), 229–250.
15. Nguyen D. M., Hsieh Y. C. and Allen, G. D., (2006), The impact of web-based assessment and practice on students’ mathematics learning attitudes, J. Comput. Math. Sci. Teach., 25(3), 251–279.
16. Nikou S. A. and Economides A. A., (2016), The impact of paper-based, computer-based and mobile-based self-assessment on students’ science motivation and achievement, Comput. Hum. Behav., 55, 1241–1248.
17. Palmer D. H., (1999), Exploring the link between students’ scientific and nonscientific conceptions, Sci. Educ., 83(6), 639–653.
18. Peterson R. F., Treagust D. F. and Garnett, P., (1989), Development and application of a diagnostic instrument to evaluate grade-11 and -12 students’ concepts of covalent bonding and structure following a course of instruction, J. Res. Sci. Teach., 26(4), 301–314.
19. Sreenivasulu B. and Subramaniam R., (2013), University students’ understanding of chemical thermodynamics, Int. J. Sci. Educ., 35(4), 601–635.
20. Taber K. S., (1999), Ideas about ionisation energy: a diagnostic instrument, Sch. Sci. Rev., 81(295), 97–104.
21. Taber K. S., (2000), Multiple frameworks?: Evidence of manifold conceptions in individual cognitive structure, Int. J. Sci. Educ., 22(4), 399–417.
22. Taber K. S., (2003), Understanding ionisation energy: physical, chemical and alternative conceptions, Chem. Educ. Res. Pract., 4(2), 149–169.
23. Taber K. S., (2006), Beyond constructivism: the progressive research programme into learning science, Stud. Sci. Educ., 42(1), 125–184.
24. Taber K. S., (2013a), A common core to chemical conceptions: learners’ conceptions of chemical stability, change and bonding, in Tsaparlis G. and Sevian H. (ed.), Concepts of Matter in Science Education, Dordrecht: Springer, pp. 391–418.
25. Taber, K. S., (2013b), Modelling learners and learning in science education: developing representations of concepts, conceptual structure and conceptual change to inform teaching and research, Dordrecht: Springer.
26. Taber K. S. and Tan K. C. D., (2007), Exploring learners’ conceptual resources: Singapore A level students’ explanations in the topic of ionisation energy, Int. J. Sci. Math. Educ., 5(3), 375–392.
27. Tan K. C. D. and Koh T. S., (2008), The use of Web 2.0 technologies in school science, Sch. Sci. Rev., 90(330), 113–117.
28. Tan K. C. D. and Treagust D. F., (1999), Evaluating students’ understanding of chemical bonding, Sch. Sci. Rev., 81(294), 75–83.
29. Tan K. C. D., Goh N. K., Chia L. S. and Treagust D. F., (2002), Development and application of a two-tier diagnostic instrument to assess high school students’ understanding of inorganic chemistry qualitative analysis, J. Res. Sci. Teach., 39(4), 283–301.
30. Tan, K. C. D., Goh, N. K., Chia, L. S. and Taber, K. S., (2005a), Development of a two-tier multiple choice diagnostic instrument to determine A-level students’ understanding of ionisation energy, Singapore: National Institute of Education.
31. Tan K. C. D., Taber K. S., Goh N. K. and Chia L. S., (2005b), The ionisation energy diagnostic instrument: a two tier multiple-choice instrument to determine high school students’ understanding of ionisation energy. Chem. Educ. Res. Pract., 6(4), 180–197.
32. Thelwall M., (2000), Computer-based assessment: a versatile educational tool, Comput. Educ., 34(1), 37–49.
33. Topping K. J., Samuels J. and Paul T. (2007), Computerized assessment of independent reading: effects of implementation quality on achievement gain, Sch. Eff. Sch. Improv., 18(2), 191–208.
34. Treagust D. F., (1995), Diagnostic assessment of students’ science knowledge, in Glynn S. M. and Duit R. (ed.), Learning science in the schools: research reforming practice, Mahwah, NJ: Lawrence Erlbaum Associates, pp. 327–346.
35. Wiley, (2018), WileyPlus, https://www.wileyplus.com/wileyplus/, retrieved 4 December 2018.
36. Yan Y. K. and Subramaniam R., (2018), Using a multi-tier diagnostic test to explore the nature of students’ alternative conceptions on reaction kinetics, Chem. Educ. Res. Pract., 19(1), 213–226.

---