The coherent versus fragmented knowledge hypotheses for the structure of matter: an investigation with a robust statistical methodology

Abstract

Research in cognitive psychology and science education has acknowledged two antagonistic theoretical perspectives concerning students' knowledge. One perspective considers students' knowledge as coherent or theory-like, explaining the learning process in terms of framework theories and stable mental models, whereas the other considers it fragmented and based on the existence of knowledge elements that can be activated, interconnected and organized accordingly. Although both perspectives continue to demonstrate empirical support, a third point of view focuses on the role of the particular topic and suggests that the stability of the in situ formed conceptions varies under different circumstances being also affected by the researchers' interpretation of data. Since methodological issues are by far fundamental and crucial for the above theoretical premises, in the present research an attempt is made to test the above hypotheses by applying a robust statistical tool, the Latent Class Analysis (LCA). Data were taken from students' responses (N = 329, aged 14–15) using an instrument designed to access students' understanding of the structure of matter and its changes of states that had previously been used to identify students' progression through a series of mental models reported in the literature. However, the results did not support the hypothesis that participants would hold these as coherent mental models. The present research adds to the ongoing dialogue on such a crucial issue for its pedagogical implications by contributing to the methodological pathways for an advancement of science education research through theory building.

---

Introduction: coherent versus fragmented knowledge

The significance of students' prior ideas and their contribution to the learning process has been evident since the genesis of the constructivism. Over the past decades a great number of pieces of research had been launched in order to study these ideas. Among the findings were the significant differences between students' prior ideas and those of experts; whilst, a variety of terms has been used in order to characterize these prior ideas including conceptions, alternative conceptions, conceptual frameworks, misconceptions and so forth. In many of these studies, the nature of the students' misconceptions was investigated, drawing the conclusion that independently of the particular topic they are coherent and stable, characterizing them as theory-like (e.g., Chi, 1992, 2005; Wellman and Gelman, 1992; Vosniadou and Brewer, 1992, 1994; Ioannides and Vosniadou, 2002 ; Vosniadou, 2002). In other words, according to this perspective, students have constructed particular conceptual structures, which are explicitly considered as coherent mental models and could be used in their explanatory efforts with consistency throughout a variety of phenomena.

The acknowledgement of the differences between students' and experts' ideas and on the other hand the emphasis on their stability and coherence raised a number of questions. Smith et al. (1993) wondered for instance how the main principles of constructivism could be served by the existence of such stable misconceptions, when these should be replaced by the conceptions of experts during the learning processes. They claimed that many of the assertions of the above perspective are inconsistent with constructivism and thus, they supposed that some other productive resources may exist. Within this context, another perspective for the learning process and so for the knowledge itself was developed, typically known as knowledge in pieces or fragmented knowledge (e.g., diSessa, 1988; Smith et al., 1993; Harrison et al., 1999; diSessa et al., 2004). According to this perspective, during learning or cognitive processes some preexisting productive “implicit knowledge elements” are activated and any understanding or explanatory scheme is produced in situ when a particular phenomenon is being studied. diSessa (1993) established the term phenomenological primitives (p-prims) in order to specify such knowledge elements. These elements, within this perspective, are presented as flexible pieces of knowledge that can be evolved, modified and enlarged over time, however they could also be activated and coordinated with other pieces of knowledge into a bigger structure formulating a conception for a particular phenomenon (diSessa, 1988, 2006; Taber and García Franco, 2010).

Launching a different viewpoint, other researchers (e.g. Hammer, 1996, 2004; Taber, 2000, 2008, 2009; Taber and García Franco, 2010) argued that the two apparently antagonistic perspectives might not be as controversial as they seem to be. Taber (2008) for instance pointed that the researcher's interpretation of data might be what leads to specific evidence supporting one or the other perspective. According to Taber, there are relevant research reports on specific topics supporting the view that students have developed stable conceptual frameworks for some ideas, whereas in other cases, ideas appear to be formed in situ and varied. Thus, the question is shifted from ‘whether students' conceptions are coherent or not’ to ‘which conceptions within the context of a specific topic and under particular circumstances appear to be stable’.

It is pertinent to recall here an epistemological note on Popper's (1979) model of cognition concerning the accessibility of students' knowledge (Taber, 2008). The ‘cognitive structures’, which correspond to representation of knowledge in the ‘materiale substatum’ (brain) can only be accessed indirectly by assessing ‘conceptual structures’ portrayed in the products of human mind (talk, writing, drawing etc.). Conceptual structures are products of thoughts, or outcomes of (thinking) processes. In science education, even though the phenomena under investigation are referred to as processes, research is not designed to examine processes or mechanisms controlling changes over time. Normally, cross sectional studies are carried out and merely the outcomes of thinking processes are evaluated at one point in time and on the basis of response patterns to a specially designed instrument. The above indicate the first methodological deficiency that is expected to limit our research. Since students' responses are created by activating and processing some kind of knowledge elements or conceptual resources and given the dynamical nature of a cognitive process and its inherent nonlinearity (Stamovlasis, 2006, 2011) the notion of in situ formation seems to reasonably justify the variable consistency and stability of conceptual frameworks observed under different circumstances. Human knowledge, represented in contemporary psychological theories as networks of interrelated elements, grows, develops and increases in size and complexity by incorporating more general and/or context-specific components. Thus, a diversity of grain size of knowledge structures might exist (Smith et al., 1993), the coherence and stability of which is a core issue for conceptual change theories and it is directly interwoven with research methodology and data analysis. The notion of coherency deserves a specific attention when referring to a relatively large ‘grain size’ pieces of knowledge, e.g. the mental models, because their hypothesized stability is the origin of students' learning difficulties due to their resistance to change.

Conclusively, although a plethora of research and theoretical endeavors have added to our understanding in this area, further investigation is still needed and what becomes obvious yet is that methodological issues are involved. The majority of the studies supporting the coherent conceptual frameworks' perspective relied methodologically on interviews, small samples or weak statistical analyses. A reexamination of the issue in question suggested that suboptimal analysis and deficiency in interpretation of data can be avoided by implementing more robust statistics methods suitable to provide a more lucid picture of the nature of students' knowledge.

Methodological issues

In the psychological and educational research methodology, an essential and crucial task is often the categorization of subjects; that is, classifying students according to some criteria, such as learning outcomes, styles, ways of thinking, strategies, abilities or mental models. Nevertheless, the soundness of the classification method determines the research validly and reliability as well as the generalization of the conclusions.

The methodology implemented for the classification of tasks in mental model research is usually based on the following: for each mental model, an expected pattern of responses in a specific questionnaire is formulated, while an observed pattern is recorded. Then, the degree of correspondence between the expected and the observed response pattern is determined and the subjects are classified into groups that hypothetically possess the corresponding mental model. In such a classification, a deviation between expected and observed patterns is usually tolerated and the classification of the response is achieved with a predetermined but also arbitrary degree of agreement (e.g. 85% fit). This is known as Rule Assessment Methodology (RAM) (Siegler, 1976, 1981) and it has been followed in many research studies on mental models (e.g., Johnson, 1998a; Vosniadou and Brewer, 1992; Vosniadou, 1994). There are some weak points and disadvantages in this methodology, which are associated with the allocation of the response patterns to mental models and the statistical treatment usually implemented. First, only the predetermined mental models, i.e. the ones that the researcher proposes, can be detected; possible alternative models might be missing. Second, the assignment of response patterns to mental models is made on the basis of an arbitrary criterion of a minimum percentage of correspondence between observed and expected responses. Since the robustness of the criterion appears to be affected by the number of items used in the questionnaire, a number of alternative classifications might be obtained. A third disadvantage is that no statistical tests are implemented to provide a measure of how effectively the proposed classification fits and accounts for the data (Jansen and van der Maas, 1997).

An appropriate methodology to overcome the above disadvantages is the implementation of Latent Class Analysis (LCA), which belongs to the family of the Latent Structure Models (LSM) (Clogg, 1995; Dayton, 1998; Magidson and Vermunt, 2001; McCutcheon, 1987). In LSM, which can be used in exploratory or confirmatory analyses, a distinction between latent and manifest or observed variables is made, where the assumed underlying latent variables could be considered at different measurement levels. Latent Class Analysis is implemented when both manifest and latent variables are assumed to be categorical. LCA achieves to divide a set a response patterns into groups or clusters named latent classes (LC) where the subjects hypothetically belong to, being qualitatively different from each other. In contrast to the traditional clustering approaches, which use distances between responders, LCA implements similarity between responses and specifically probabilities of response patterns. In LCA, the assignment of cases to a certain class is based on the log of odds of being in a particular class versus another class (Magidson and Vermunt, 2001).

An advantage of LCA is that it offers quantitative statistical measures for goodness-of-fit, indicating how well the LC clusters account for the data. Ergo, LCA tests whether the mental models proposed by the researcher are statistically supported by the data and it could falsify any proposed theoretical account. However, the most important advantage of LCA is that it can detect clusters of unexpected response patterns. That is, the mental models do not have to be known in advance and therefore, unanticipated alternative classes (LCs) might be emerged from the analysis. The fit of a latent class analysis can be assessed by a number of indicators such as the number of parameters (N_par), likelihood ratio statistic (L²), Bayesian information criterion (BIC), degrees of freedom (df) and bootstrapped p-value. Non-significant p-values greater than 0.05 are desired indicating a good fit between the theoretical model and data (Vermunt and Magidson, 2002).

If the mental models of students on a particular topic are coherent, distinct and qualitative different entities responsible for certain response patterns, then the number of subjects possessing the same model, i.e. those who manifest the same response pattern, could be revealed as a distinct cluster (latent Class, LC) by LCA based on the subjects' answers to a selected questionnaire.

In LCA the emerged LCs are characterized by a number of conditional probabilities. A conditional probability is the probability of providing a certain response to an item, given that the subject belongs to a specific LC. In LCA, the classification – a division into clusters (LCs) – is realized by means of statistical criteria applied to conditional probabilities, while in usual procedures concerning mental model research, such as RAM, the classification is based on the number of correct responses. If the LCA procedure converges to a single class, then students cannot be classified into distinct clusters and their knowledge based on their response patterns cannot be classified into qualitatively different mental models. Then, the hypothesized students' mental models are not supported by the data. In contrast, when the LCA procedure converges to a limited number of distinct classes and the hypothesized (and being tested) mental-model account is correct then, the resulted classes should possess conditional probabilities consistent with these hypothesized mental-models. Note that the emerged clusters (if any) do not have to be in line with hypothesized mental-models; however, if they do, this would support a coherent mental model perspective.

Focusing on a specific topic: students' ideas on the particulate nature of matter

The particle theory of mater is a key component in natural sciences, since understanding of the material world in terms of atoms and molecules is of paramount importance. The large amount of work carried out on this topic was proved to be particularly challenging. A wide range of persisting students' misconceptions on the structure of matter has been reported in science education literature, which have a further impact on their understanding of phenomena, while they were proved to be quite beneficial in overcoming teaching and learning difficulties (Johnson, 1998b, 1998c; Papageorgiou et al., 2010). Some interesting findings are given in the following paragraphs, while a lucid review could be found elsewhere in a relevant work (Tsitsipis et al., 2010).

Students often consider matter as continuous. Even those who have developed a particulate model of matter frequently show understanding difficulties in important aspects of the particle theory. These are mostly related to the space between particles, the intrinsic motion of the particles, the relative spacing between the particles in the three states, the attractions between particles and the nature of the particles themselves (Johnson, 1998a). In the main, students have problems in conceiving the notion of ‘empty space’ among particles. Thus, many students think that the space among particles is filled with various kinds of ‘stuff’, such as air or dust, as well as with other particles or particles of the same substance (Novick and Nussbaum, 1978; Lee et al., 1993; Johnson, 1998a). Frequently, molecules are thought to be in substances like ‘blueberries in a muffin’ rather than substances to be composed of molecules (Lee et al., 1993). According to this view, ‘particles are additional to the substances’ (Johnson, 1998a, p. 399). The intrinsic motion of particles (structural units – usually molecules) was also found to be a difficult concept (Novick and Nussbaum, 1978; Lee et al., 1993; Johnson, 1998a; Papageorgiou and Johnson, 2005). In the case of the gas state, this was attributed to various causes, e.g. to low specific gravity or to the action of air (Novick and Nussbaum, 1978; Lee et al., 1993). On the other hand, in the solid state, where no motion of the substance is visible, molecules are often thought to be still (Lee et al., 1993). In accordance with the above, Dow et al. (1978) found out that, although the majority of the students in their investigation indicated particle motion in the liquid and gas states, about a third of them advocated that there was no particle movement in the solid state.

Significant conceptual difficulties are reflected in the way that the students make a distinction between states, which are not always clear or complete. Secondary students' misconceptions about the liquid state are due to the fact that they consider liquid to be merely an intermediate state between solid and gas states. In this context, students overestimate the molecular spacing in the liquid state (Dow et al., 1978). In another study, molecular spacing in the gas state has been found to be underestimated (Pereira and Pestana, 1991).

In relation to the nature of the particles themselves, students have shown a great deal of difficulty in understanding that the properties of the states of substances are due to the collective behavior of particles. They often regard a particle, e.g. a molecule, as a small quantity of a substance having all the macroscopic properties of the substance. That is, ice molecules are regarded as frozen or ‘solid molecules’, water molecules as ‘liquid molecules’, and so forth (Lee et al., 1993; Johnson, 1998a). Furthermore, molecules are described to undergo “the same changes as the visible changes in the substances. Thus, molecules start to move when ice melts into water, molecules of water are heated up and make water boil, or molecules expand, contract, melt, evaporate, condense, and so forth” (Lee et al., 1993, p. 268).

In an attempt to further investigate the various students' conceptions related to the nature of matter and organize them into conceptual categories, a longitudinal study with secondary English pupils (ages 11–14) was carried out by implementing qualitative approaches and interview methods (Johnson, 1998a). In this study, students' misconceptions were investigated thoroughly and a number of distinct models for students' classification based on the strength of their particle thinking were proposed. These distinct models are defined below:

(1) Model X: continuous substance. Particle ideas have no meaning. (2) Model A: particles in the continuous substance. The particles are additional to the substance. (3) Model B: particles are the substance, but with macroscopic character. There is nothing between the particles. Individual particles are seen as being of the same quality as the macroscopic sample – literally small bits of it. (4) Model C: particles are the substance. The properties of a state are seen as collective properties of the particles. In addition to the above, there are intermediate models. It was also suggested that these models might represent stages through which students' ideas evolve towards the science model (Johnson, 1998a).

It is worth noting that the above models are described by Johnson within the context of the mental models perspective, which corresponds to a coherent structure that a novice might develop and sustain during the learning process and before he/she attains the scientific view. The present paper reexamines the above mental model account with a confirmatory latent class analysis.

Rationale and research hypothesis

This work is part of a series of investigations aiming to understand students' ideas about the structure of matter (Tsitsipis et al., 2010, 2012; Stamovlasis et al., 2010, 2012). In this context, there is a special interest and motive to clarify the nature of this knowledge in relation to the coherence versus fragmentation debate. The issue is of paramount importance for both, the advancement of theory and the educational implications, since in practice the two perspectives suggest different pedagogical approaches.

Research in science education, as in all social science, implements a plethora of methodological approaches justified by various epistemological positions. The argument that the variables in social science are mostly categorical, and thus they are not subjected to measurement in a positivistic sense, has prevented many research studies from fostering quantitative methods. However, generalization and theory building are favored only by robust statistical methods. The latest advances in categorical data analysis have overcome many former statistical limitations and they are capable of testing sophisticated hypotheses and elucidating significant theoretical issues. Latent class analysis is one of the advanced methods that could be beneficial for the dialogue on the issue in question, and as such it was selected for the present confirmatory study.

Latent Class Analysis is the Latent Variable approach, where both manifest (observable) and latent variables are considered to be categorical. The way of accessing a latent variable, such as knowledge, is realized by the evaluation of the item responses, which are the manifest variables. If coherent mental models exist, then they are distinct entities/categories and ontologically they could be considered as categorical latent variables. These latent variables are responsible for the item responses (manifest variable), which in turn, if defined and measured at the nominal level of measurement, could be analyzed by LCA. The ‘coherence’ or ‘fragmentation’ of a latent variable itself is impossible to be directly observed by any means and thus, a researcher seeks for a representation at the observable level that could reflect the nature of knowledge as far as its coherency is concerned. Since the item responses are driven by a hypothesized coherent knowledge, the consistency of the responses in a variety of items reflects the coherency of the latent variable. Coherence is expressed by the capability of the latent variable not to contradict with itself when producing responses for a series of questions. By mapping the conditional probabilities of answering a series of question, consistently with a hypothesized mental model, one obtains a portrait reflecting the degree of coherency of the latent variable.

The main hypothesis being tested in this research is about the fragmented nature of students' knowledge on the structure of matter. Since LCA is a model based clustering analysis, it tests a hypothesized LC model and its null hypothesis (Ho) states that the model does not fit to the data; if Ho is rejected (based on the p-value), one could claim a statistically significant model. Thus, from a statistical point of view the fragmented nature of students' knowledge serves as the Ho, while the alterative hypothesis (Ha) supports the coherent mental models perspective.

Method

Subjects

The study was conducted with the participation of 329 ninth-grade junior high school Greek pupils (age 14–15), 160 of which were male and 169 female. The sample consisted of all the pupils of 18 classes, each of which belonged to a different junior high school. All the schools are located in the prefecture of Fthiotida, in central Greece; seven of them are in the capital city of Lamia, while the other 11 are dispersed at the municipalities of the prefecture. All the junior high schools of the capital and almost half of the remaining schools in the prefecture took part in the research. Pupils from different socioeconomic status and living conditions comprised the sample.

Instrument and data collection

Data were collected through a paper-and-pencil test. The instrument was synthesized by selected items/questions, which had been used to access students' knowledge on this specific domain in a number of research studies (Osborne and Cosgrove, 1983; Johnson, 1998a, 1998b, 1998c; Papageorgiou et al., 2010). Except three items, which were multiple choice questions, all the other items were open-ended questions requesting explanations also.

Before the data collection, a small pilot study (N = 25) was carried out using the particular test followed by interviews with students discussing all items in order to detect possible communication difficulties of the test. A Cronbach's alpha reliability coefficient of 0.78 was obtained for the above instrument and the present sample. The instrument is a section of the full instrument used by Tsitsipis et al. (2010) and Tsitsipis et al. (2012). It is worth noticing that pupils' responses to this test were asked without any notification almost one year after pupils had been taught the relevant subject matter. Thus, the test measures the pupils' residual knowledge. The data collection and analysis was undertaken in Greek.

The test consists of two parts covering the following topics: the particulate nature of matter (first part), a number of properties of each of the three states as a result of the collective behavior of particles (second part). A description of the instrument is presented in the Appendix.

Coding procedure

The objective of the present work is to test the coherence of students' knowledge on the structure of matter demonstrated by the consistency of their responses to a questionnaire. The mental models proposed in the literature (Johnson, 1998a), which represent the hypothetically transitional but coherent knowledge of students before they acquire the scientific view, were examined. Students' responses to each forced-choice or open-ended question were examined by the three authors in two stages. First, an agreement for an evaluation coding scheme concerning key words and expressions that were expected from students on the basis of what had been taught was achieved. Examples of such codes are: ‘continuous’, ‘particles in the substance’, ‘particles small bits of the whole’, ‘air between particles’, ‘moving particles’, ‘steady particle’, ‘gas structure’, ‘liquid structure’, ‘solid structure’, ‘liquid molecules’, ‘gas molecules’, ‘motion due to the air between molecules’, ‘sugar between molecules’, ‘molecules at different state’, etc.

In a second stage, students' answers were assessed and were allocated into different levels of understanding the particulate nature of matter, which correspond to different mental models under examination. That is, the responses in each item were linked directly to one of the mental models and they were coded as X, A, B, C or S by examining each response alone and/or in combination with other responses. For example, co-evaluating students' answers to the questions 1A, 2A and 3A (see Appendix), a student could be linked to model X or not. If not, then a co-evaluation of the answers to the questions 1B, 2B, 3B, 1C, 2C and 3C could link the student to model A or not. If not, then a co-evaluation of the answers to the questions 4 (A–C) and 5 (A, B) could link the student to model B, C or S. In the open-ended question the language used had an influential role in the evaluation. For example, a student who answered that, there are ‘molecules’ (which excludes X), ‘nothing between molecules’ (which advocates B, C, S rather than A) and ‘water molecules are in liquid state’ (which excludes C, S) was linked to model B. In contrast, a student who answered that, there are ‘molecules’ (which excludes X) and ‘water between water molecules’ (which excludes B, C, S) was linked to model A.

The four models X, A, B and C are the ones which had been introduced by Johnson (1998a, 1998b, 1998c), whereas the fifth model (S) was added in order to have a link to the complete scientific view; as also suggested by Johnson, model C does not represent a full scientific view. Subjects with model S would use the proper scientific language in their descriptions and explanations and they would be those who have undoubtedly acquired the scientific view. Finally, in case of peculiar, confused, irrelevant responses or no responses, the student was linked to another model O. In some cases where a students' response could not be undoubtedly linked to merely one mental model, alternative codes were recorded and they were used alternatively as inputs in the LCA procedure.

Some relevant examples are given below showing that students' knowledge, a latent variable that is being accessed by their responses, is not coherent and in line with a specific mental mode:

A student's responses to the questions 1A, 2A and 3A, for instance, might be different by choosing a continuous material for one state (e.g. gas state – answer which was not rare) and the particulate model for others (e.g. solid and/or liquid states); when he referred to motion of particles, he/she accepted the moving particles for the gas but not for the solid state, while talking about the space between particles he refuted the particle nature for example for the solid state, but he accepted it for the liquid state. Note that by examining similar items across physical states the greatest discrepancy in the responses became evident. Furthermore, although a student advocated model X, for instance in 4A, he discussed whether an isolated molecule of water has come from ice, liquid water or water in the gas state, respectively. Similarly, a student stated in 4B that all the three cases of water molecules (in the solid, liquid and gas state) are the same, whereas in 5B he described the molecules of sugar, water and oxygen in a context of macroscopic characteristics (e.g., a solid molecule for sugar, a liquid one for water and so on). The overall degree of inconsistency of a student's responses increases in cases where confused responses or no responses were provided.

Even though the accuracy of linking each response to a mental model might be restricted by the instrument and the students' limited expressions and language use, given that the evaluation scheme was followed strictly and consistently throughout the cases, the final portrait of LC's conditional probabilities (see Fig. 1–3) is still anticipated to reflect the actual inconsistency or consistency in students responses.

Fig. 1 Conditional probabilities for LC1 (9.2%, dominance of S-model). Items are for the solid state.

Fig. 2 Conditional probabilities for LC2 (28.6%). Items are for the solid state.

Fig. 3 Conditional probabilities for LC3 (62.2%). Items are for the solid state.

Results and discussion

Student's coded responses were introduced to LCA procedures and analyzed using Latent-GOLD software. Note that in LCA, as in any clustering procedure, the larger the number of input variables, the more unlikely is the emerging of clusters. Numerous trials were attempted with various combinations/groups of items as inputs to LCA anticipating the procedure to converge. For example, groups of items referring to the same property for all the three states of matter, e.g. motion in solid, liquid and gas states were analyzed collectively. In all these trials, LCA did not converge to any solution, but merely to one-cluster classification, indicating that latent classes did not emerge on the basis of these item responses.

Attempts to separately analyze the questions concerning the solid, liquid and gas states, respectively, lead to a successful classification for some groups of items, which are presented next. For these groups of items three clusters, i.e. three LCs, emerged, which fitted satisfactorily to the data for the solid and the gas states. Thus, in the case of these two states, a 3-cluster model was selected, while in the case of the liquid state merely one cluster emerged. The fit of such a latent-class model was assessed by the indicators already reported earlier, i.e. the N_par, BIC and bootstrapped p-values, which are presented in Tables 1 and 2 (Vermunt and Magidson, 2000).

Table 1 Results of the various LC factor models fitted to 6 items for the solid state

	L^2	BIC	Npar	df	p-value	Class. err.
a The most parsimonious and best fitting model: Npar, number of parameters; L^2, likelihood ratio statistic; BIC, Bayesian Information Criterion; df, degrees of freedom; bootstrapped p-value; classification error.
1-Cluster	526.84	2776.73	17	239	0.00	0.000
2-Cluster	400.47	2750.16	35	221	0.00	0.047
3-Cluster^a	331.17	2780.68	53	203	0.07^a	0.011
4-Cluster	289.18	2838.51	71	185	0.12	0.014
5-Cluster	260.26	2917.43	89	167	0.07	0.011
6-Cluster	240.98	2986.64	107	149	0.14	0.014

---

Table 2 Results of the various LC factor models fitted to 6 items for the gas state

	L^2	BIC	Npar	df	p-value	Class. err.
a The most parsimonious and best fitting model: Npar, number of parameters; L^2, likelihood ratio statistic; BIC, Bayesian Information Criterion; df, degrees of freedom; bootstrapped p-value; classification error.
1-Cluster	737.95	3032.73	17	239	0.00	0.0000
2-Cluster	494.51	2982.07	35	221	0.03	0.0520
3-Cluster^a	439.29	3011.00	53	203	0.15^a	0.0010
4-Cluster	401.46	3074.56	71	185	0.17	0.0154
5-Cluster	372.51	3144.19	89	167	0.17	0.0143
6-Cluster	346.02	3221.71	107	149	0.16	0.0154

---

In particular, Table 1 shows the results of the LCA for the solid state. Among all the possible cluster models (LC models), the 3-cluster model appears to be statistically significant (p = 0.07) for the solid state, the most parsimonious and the best fitting one. This LCA was based on 6 items: 1A, 1B, 1C, 4A, 4B and 4C. The corresponding three LCs account for 9.2%, 28.6% and 62.2% of the sample respectively. In each of the latent classes, a set of conditional probabilities is assigned for each item, that is, the probability for an item to be answered in line with one of the hypothesized mental models, given that the subject is a member of this latent class. Accordingly, these conditional probabilities for the three emerged latent classes for the solid state are shown in Fig. 1–3. If groups of students respond similarly and consistently with any of the hypothesized mental models then they would be allocated to the same cluster (latent class). Interestingly, the emerged clusters/latent classes, namely the LC1, LC2 and LC3 include subjects that do not respond similarly and consistently with any of the hypothesized mental models; that is, the set of conditional probabilities in each LC is not homogeneous in terms of responses in line to a certain mental model. For example, in Fig. 2 the item 4C has a probability of 0.37 to be answered in line with the A-model, a probability of 0.32 to be answered in line with the B-model, a probability of 0.21 to be answered in line with the C-model and a probability of 0.10 to be answered in line with the S-model. Thus, the emerged LCs do not coincide with any of the hypothesized mental models. Students' knowledge appears to be fragmented, not showing any coherence as the mental models perspective hypothesized. Note that LC1 (Fig. 1) tends to be the most homogeneous compared to LC2 and LC3 (Fig. 2 and 3, respectively), since students in this LC have potentially attained the scientific view (S-model).

Analogous to the solid state are the LC results for the gas state. By using the corresponding indicators for the items 3A, 3B, 3C, 4A, 4B and 4C, the 3-cluster model (see Table 2) is proved to be the most parsimonious and the best fitted (p = 0.15). The three LCs account for 24.6%, 30.3% and 45.1% of the sample respectively. In Fig. 4–6, the conditional probabilities of these three latent classes, namely the LC1, LC2 and LC3, are presented in relation to the items. All three latent classes are characterized by mixed response patterns switched among all the models (O, X, A, B, C, S), demonstrating fragmented students' knowledge.

Fig. 4 Conditional probabilities for LC1 (24.6%). Items are for the gas state.

Fig. 5 Conditional probabilities for LC2 (30.3%). Items are for the gas state.

Fig. 6 Conditional probabilities for LC3 (45.1%). Items are for the gas state.

For the liquid state, LCA did not converge to a significant classification based on the corresponding items (2A, 2B, 2C, 4A, 4B and 4C) and resulted one-cluster model, that is, for the liquid state the students' knowledge appears to possess an even higher degree of fragmentation. Moreover, although the LC analysis was also performed with various item combinations including further items such as 5A and 5B, the analysis did not reach different results.

Conclusively, LCA based on conditional probabilities did not provide evidence that any of the revealed latent classes is directly associated with the hypothesized mental models (Johnson, 1998a). The results of incoherent students' knowledge of course concern the specific mental models, since the coding was based on them. Different hypothesized mental models could be tested analogously by proper coding and the corresponding questionnaire.

In order to explicate how the LCA can serve and facilitate testing related hypotheses on coherence versus fragmented knowledge, a didactic demonstration is provided below with a semi-simulation experiment analogous to the use of a “standard sample” in chemical analysis:

In the present empirical data set an artificial segment of data was added with the following properties: all subjects' responses are in line with the mental model B for all items except item 4B, where 45% of the cases respond according to A and 55% of the cases according to X. From the ensuing analysis of the modified data (original data plus the artificial segment) four latent classes emerged. LCA has captured the artificial segment as the forth LC preserving its properties; Fig. 7 shows its conditional probability pattern, which is almost homogeneous. In this LC the conditional probabilities tend to be or are equal to the unity for mental model B and tend to be or are equal to zero for the other models in all items (except item 4B). The picture shown in Fig. 7 is the one expected when an emerged LC coincides with a coherent mental model; this is substantially different from the ones (Fig. 1 to 6) resulting from the empirical data.

Fig. 7 The artificially introduced segment (16.4%) in the sample with responses in line with model B for all items except item 4B (45% A and 55% X) was captured as LC4 preserving the conditional probabilities for each item response and its coherency. Items are for the solid state.

Prior to further discussing the findings and drawing specific conclusions for the fragmented nature of the students' knowledge about the structure of matter, it is useful at this point to make a reference to some limitations of the study of both general and specific interest. One limitation originates from the utilization of the written instrument and the coding scheme, which is based on the given responses without having the opportunity to control random errors or miscommunications by double-checking students' answers, as it is possible for instance in an interview approach (e.g. Engel and Driver, 1986). The techniques of collecting and coding data have been an issue of debate and their advantages and disadvantages have been discussed repeatedly in the past (Vosniadou et al., 2004; Straatemeier et al., 2008).

An approach widely used is the “generative method”, which implements interviews with ‘open-ended’ questions. In this approach subjects are asked to respond to certain questions by producing drawings or describing verbally whatever they have in mind. Thus, the researcher could access the subjects’ knowledge from ‘on the spot’ formation of mental representations (Vosniadou et al., 2004). However, some disadvantages associated with the interview methods, such as the prolonged processes of repeated questions and the social interference, could be sources of errors; moreover, the limited number of subjects under examination restricts generalized conclusions. This method nevertheless is particularly useful for exploratory studies, where the majority of possible mental representations could be detected, categorized and coded for a large scale research endeavor. Alternatively, the ‘forced-choice’ questionnaires were proposed as a better method for data collection (Straatemeier et al., 2008). It could be applied in large samples and it can access subjects' knowledge satisfactorily, given that the choices include all (or the most of) the possible alternatives. The advantage of a forced-choice questionnaire is that it avoids social interaction; it can be better structured and objectively assessed. In the present research the instrument included both open-ended and forced-choice items.

Conclusions

This study, aiming to shed light on the nature of students' knowledge on the structure of matter, has arrived at specific findings, which beyond the above-mentioned limitations might be restricted to the age group of 14–15; note that both Johnson's works and ours have investigated pupils' conception at the junior secondary school level. Johnson's mental models have been fostered here because his relevant work was a very well-documented one; however, it bears the disadvantages of a particular methodological choice such as those described earlier. Nevertheless, by taking a closer look at Johnson's work, one may unearth observations and clues that advocate the fragmented-knowledge perspective. Apart from the basic models (X, A, B, C), Johnson (1998a) suggested the existence of intermediate models between basic models (e.g. AB, XA, etc.) for each one of the states. Thus, in a total of 33 subjects, an increased number of models (basic and intermediate) resulted. That is, a slight different knowledge structure about ‘the structure matter’ tends to create additional models. As a result, it is expected that, when the number of subjects in a piece of research significantly increases, the possibility of maintaining a small number of mental models decreases. Thus, a robust statistical clustering procedure is needed for allocating subjects into distinct groups and this can be accomplished by LC analysis implementing conditional probabilities.

Moreover, Johnson (1998a) himself proceeded to an analysis of his findings in a two dimensional space, introducing a diagram where models were presented as territories in the ‘outcome space’ without distinct ‘boundaries’. Although each of these identifiable territories was defined with a degree of internal coherence, in that graph, the idea is in fact a move beyond the consideration of a ‘model’ in the way presented within the coherence-knowledge perspective.

Ergo, methodological issues existing behind the controversial results in searching for coherent models arise for further discussion. LCA is able to capture and overcome the above methodological complicatedness. Even though particular clusters were revealed, i.e. the null hypothesis was rejected as far the clustering procedure is concerned, the emerged latent classes do not coincide with any of the hypothesized mental models, which are supposed to be homogeneous in terms of response conditional probabilities. Thus, one may conclude that the null hypothesis (Ho) on fragmented knowledge was not rejected for the present data. The only exception might be the S-model, which expectably was shown to exhibit a degree of coherence, since students in this LC have acquired the scientific view. For the present data, where the coherence at the level of a large ‘grain-size knowledge’ – mental models – was tested, LCA supported the fragmented-knowledge perspective, opposing the mental models theory established so far for the students' knowledge on the structure of matter.

Consequently, the central message of the present work is that research in science education and cognitive science might benefit from advanced statistical modelling, getting direct and clear evidence when testing hypotheses such as the one in question. Furthermore, robust statistical modeling such as LCA can serve as a tool to investigate how knowledge-in-pieces is organised (and under what conditions) either into the scientific view in some cases or possibly into conceptual resources of varying grain-sizes in some other circumstances. To that end, longitudinal studies are needed to show how these changes are realised across ages. This work is a confirmatory one, since the coding scheme is based on hypothesized mental models. In a different methodological approach an exploratory LCA could be implemented, in which a different coding scheme could be applied and the emerged LCs could be examined in an attempt to reveal possible mental models that make sense to the theory building. This is a different analysis compared to the present case, which indicates the prospective usability of LCA. At this point, it is also worth mentioning another interesting application and data analysis with LCA, where the calculated conditional probabilities of receiving correct answers were used to demonstrate students' learning progression in various grades (Harlow et al., 2011).

In the current literature, it is apparent that new rounds of discussions have begun on the issue in question (Taber, 2008; Turcotte, 2012; Kirbulut and Beeth, 2013) and in our opinion the present approach opens a new research area that will contribute to this dialogue and ultimately to the theory building. Advancements in conceptual change theories are closely related to the present issue and the implications for education are by far crucial. Learning strategies and teaching practices are strongly dependent on our deeper understanding of the nature of students' knowledge. Thus, the theoretical frameworks in education and science education in particular have to minimize their reliance on metaphors and unfounded epistemological choices and conversely, they have to promptly adapt new research methodology advancements.

Appendix

A brief description of all the parts and their items follows:

Part 1: (The particulate nature of matter)

The first 3 items (1A, 1B, and 1C) concern the solid state.

1A. Pupils are asked to choose among five alternatives (see Fig. 8 of the appendix) the figure that best represents what they would ‘see’ if they could observe a sugar grain using a hypothetical magnifying glass enabling the view of the grain structure.

Fig. 8 The five alternatives given to the pupils. In each them a note helped to clarify the corresponding representation. A description of these notes follows: (1) continuous material, (2) molecules (of nearly spherical shape) in array, not touching each other, (3) nothing, (4) molecules (of nearly spherical shape) in a random position relatively close to each other, not touching each other, (5) molecules (of nearly spherical shape) in random position, mostly far away from each other.

1B. Pupils are asked to explain what they think exists in-between molecules, in case they chose a figure depicting molecules. Otherwise, they do not have to answer this question.

1C. Pupils are asked to answer whether or not they think that the view of the sugar structure through the hypothetical magnifying glass would remain still as the time is passing. They are also asked to explain and justify their answers.

The following 3 items (2A, 2B, and 2C) concern the liquid state.

2A. Pupils are asked to choose among five alternatives (see Fig. 8 of the appendix) the figure that best represents what they would “see” if they could observe a drop of pure (liquid) water using a hypothetical magnifying glass enabling the view of the structure of the drop.

2B. Pupils are asked to explain what they think exists in-between molecules, in case they chose a figure depicting molecules. Otherwise, they do not have to answer this question.

2C. Pupils are asked to answer whether or not they think that the view of the water structure through the hypothetical magnifying glass would remain still as the time is passing. They were also asked to explain and justify their answers.

The following 3 items (3A, 3B, and 3C) concern the gas state.

3A. Pupils are asked to choose among five alternatives (see Fig. 8 of the appendix) the figure that best represents what they would “see” if they could observe a very small quantity of oxygen, found inside a vase containing pure oxygen, a hypothetical magnifying glass enabling the view of the structure of the oxygen.

3B. Pupils are asked to explain what they think exists in-between molecules, in case they chose a figure depicting molecules. Otherwise, they do not have to answer this question.

3C. Pupils are asked to answer whether or not they think that the view of the oxygen structure through the hypothetical magnifying glass would remain still as the time is passing. They are also asked to explain and justify their answers.

At that point, pupils are prompted to circumvent the following items 4 and 5 in case they have not adopted a molecular structure of the substances in the previous items.

Part 2: (The properties of a state as a result of the collective behavior of particles)

The following 3 items (4A, 4B, and 4C) concern the same substance in three different temperatures.

4A. Pupils are prompted to make the assumption that they have isolated one single molecule from one of the following: a block of ice, some pure (liquid) water or some pure water in the gas state. They are asked whether or not they could understand if the isolated molecule has come from ice, liquid water or water in the gas state, respectively. Then, they are also asked to explain and justify their answers.

4B. Pupils are prompted to make the assumption that they have isolated one single molecule from a block of ice, another single molecule from a quantity of pure liquid water and a third single molecule from a quantity of water in the gas state. They are asked whether or not they could determine a physical state for each one of the three molecules and if yes, then, what this state is. They are also asked to justify their answers.

4C. Pupils are prompted to make the assumption that they have isolated one single molecule from a block of ice, another single molecule from a quantity of pure liquid water and a third single molecule from a quantity of water in the gas state. They are asked to compare the shape and the magnitude of the three molecules. Then, they are also asked to justify their answers.

The following 2 items (5A and 5B) concern three different substances under the same normal conditions.

Pupils are prompted to make the assumption that they have isolated one single molecule from each one of the following three substances: sugar (solid), water (liquid) and oxygen (gas).

5A. They are asked whether or not they could determine a physical state for each one of the three molecules and if yes, then, what this state is. They are also asked to justify their answers.

5B. They are asked whether they think that the three molecules are different or not. They are also asked to explain and justify their answers.

References

1. Chi M. T. H., (1992), Conceptual change in and across ontological categories: examples from learning and discovery in science, in Giere R. (ed.), Cognitive models of science, Minneapolis. MN: University of Minnesota Press, pp. 129–160.
2. Chi M. T. H., (2005), Commonsense conceptions of emergent processes: why some misconceptions are robust, J. Learn. Sci., 14(2), 161–199.
3. Clogg C. C., (1995), Latent class models, in Arminger G., Clogg C. C., and Sobel M. E. (ed.), Handbook of statistical modeling for the social and behavioral sciences. New York: Plenum, pp. 311–359.
4. Dayton C. M., (1998), Latent class scaling analysis, Thousand Oaks, CA: Sage.
5. diSessa A. A., (1988), Knowledge in pieces, in Forman G. and Pufall P. B. (ed.), Constructivism in the computer age, Hillsdale, NJ: Lawrence Erlbaum Associates, Inc, pp. 49–70.
6. diSessa A. A., (1993), Toward an epistemology of physics, Cognition Instruct., 10(2 & 3), 105–225.
7. diSessa A. A., (2006), A history of conceptual change research: threads and fault lines, in Sawyer K. (ed.), Cambridge handbook of the learning sciences, Cambridge, UK: Cambridge University Press.
8. diSessa A. A., Gillespie N. and Esterly J., (2004), Coherence versus fragmentation in the development of the concept of force, Cognitive Sci., 28, 843–900.
9. Dow W. M., Auld J. and Wilson D., (1978), Pupils' Concepts of Gases, Liquids and Solids, Dundee: Northern College of Education, Dundee Campus.
10. Engel C. E. and Driver R., (1986), A study of consistency in the use of students' conceptual frameworks across different task contexts, Sci. Educ., 70, 473–496.
11. Hammer D., (1996), Misconceptions or p-prims: how may alternative perspectives of cognitive structure influence instructional perceptions and intentions?, J. Learn. Sci., 5, 97–127.
12. Hammer D., (2004), The variability of student reasoning, lecture 3: manifold cognitive resources, in Redish E. F. and Vicentini M. (ed.), Research on physics education, Bologna/Amsterdam: Italian Physical Society/IOS Press, pp. 321–340.
13. Harlow D. B., Swanson L. H., Nylund-Gibson K. and Truxler A., (2011), Using Latent Class Analysis to Analyze Children Responses to the Question, “What is a Day?”, Sci. Educ., 95(3), 477–496.
14. Harrison A. G., Grayson D. J. and Treagust D. F., (1999), Investigating a grade 11 student's evolving conceptions of heat and temperature, J. Res. Sci. Teach., 36, 55–87.
15. Ioannides C. and Vosniadou S., (2002), The changing meanings of force, Cognit. Sci. Q., 2, 5–62.
16. Jansen B. R. J. and van der Maas H. L. J., (1997), Statistical test of the rule assessment methodology by latent class analysis, Devel. Rev., 17, 321–357.
17. Johnson P. M., (1998a), Progression in children's understanding of a ‘basic’ particle theory: a longitudinal study, Int. J. Sci. Educ., 20, 393–412.
18. Johnson P. M., (1998b), Children's understanding of changes of state involving the gas state, Part 1. Boiling water and the particle theory, Int. J. Sci. Educ., 20, 567–583.
19. Johnson P. M., (1998c), Children's understanding of state involving the gas state, Part 2. Evaporation and condensation below boiling point, Int. J. Sci. Educ., 20, 695–709.
20. Kirbulut Z. D. and Beeth M. E., (2013), Consistency of Students' ideas across Evaporation, Condensation, and Boiling, Res. Sci. Educ., 43, 209–232.
21. Lee O., Eichinger D., Anderson C., Berkheimer C. and Blakeslee T., (1993), Changing middle school students' conceptions of matter and molecules, J. Res. Sci. Teach., 30, 249–270.
22. Magidson J. and Vermunt J. K., (2001), Latent class factor and cluster models, bi-plots and related graphical displays, Sociol. Methodol., 31, 223–264.
23. McCutcheon A. L., (1987), Latent class analysis, Newbury Park, CA: Sage.
24. Novick S. and Nussbaum J., (1978), Junior high school pupils' understanding of the particulate nature of matter: an interview study, Sci. Educ., 62, 273–281.
25. Osborne R. J. and Cosgrove M. M., (1983), Children’s conceptions of the changes of state of water, J. Res. Sci. Teach., 20, 825–838..
26. Papageorgiou G. and Johnson P., (2005), Do particle ideas help or hinder pupils' understanding of phenomena?, Int. J. Sci. Educ., 27, 1299–1317.
27. Papageorgiou G., Stamovlasis, D. and Johnson P., (2010), Primary Teachers' Particle Ideas and Explanations of Physical Phenomena: the Effect of an In-Service Training Course, Int. J. Sci. Educ., 32, 629–652.
28. Pereira M. and Pestana M. E., (1991), Pupils' representations of models of water, Int. J. Sci. Educ., 13, 313–319.
29. Popper K. R., (1979), Objective knowledge: an evolutionary approach (revised edition). Oxford: Oxford University Press..
30. Siegler R. S., (1976), Three aspects of cognitive development, Cognitive Psychol., 8, 481–520.
31. Siegler R. S., (1981), Developmental sequences within and between concepts, Monogr. Soc. Res. Child, 46 (2, Serial No. 189), 1–84.
32. Smith J. P., diSessa A. A. and Roschelle, J., (1993), Misconceptions reconceived: a constructivist analysis of knowledge in transition, J. Learn. Sci., 3(2), 115–163.
33. Straatemeier M., van der Maas H. L. J. and Jansen B. R. J., (2008), Children's knowledge of the earth: a new methodological and statistical approach, J. Exp. Child Psychol., 100, 276–296.
34. Stamovlasis D., (2006), The Nonlinear Dynamical Hypothesis in Science Education Problem Solving: a Catastrophe Theory Approach, Nonlinear Dynam. Psychol. Life Sci., 10, 37–70.
35. Stamovlasis D., (2011), Nonlinear dynamics and Neo-Piagetian Theories in Problem solving: perspectives on a new Epistemology and Theory Development, Nonlinear Dynam. Psychol. Life Sci., 15, 145–173.
36. Stamovlasis D., Tsitsipis G. and Papageorgiou G., (2010), The effect of logical thinking and two cognitive styles on understanding the structure of matter: an analysis with the random walk method, Chem. Educ. Res. Pract., 11, 173–181.
37. Stamovlasis D., Tsitsipis G. and Papageorgiou G., (2012), Structural equation modeling in assessing students' understanding the state changes of matter, Chem. Educ. Res. Pract., 13, 357–368.
38. Taber K. S., (2000), Multiple frameworks?: evidence of manifold conceptions in individual cognitive structure, Int. J. Sci. Educ., 22, 399–417.
39. Taber K. S., (2008), Conceptual resources for learning science: issues of transcience and grain-size in cognition and cognitive structure, Int. J. Sci. Educ., 30, 1027–1053.
40. Taber K. S., (2009), Progressing science education: constructing the scientific research programme into the contingent nature of learning science, Dordrecht: Springer.
41. Taber, K. S. and García Franco, A., (2010), Learning Processes in Chemistry: Drawing Upon Cognitive Resources to Learn About the Particulate Structure of Matter, J. Learn. Sci., 19, 99–142.
42. Tsitsipis G., Stamovlasis D. and Papageorgiou G., (2010), The effect of three cognitive variables on students' understanding of the particulate nature of matter and its changes of state, Int. J. Sci. Educ., 32, 987–1016.
43. Tsitsipis G., Stamovlasis D. and Papageorgiou G., (2012), A probabilistic model for students' errors and misconceptions on the structure of matter in relation to three cognitive variables, Int. J. Sci. Math. Educ., 10, 777–802.
44. Turcotte S., (2012), Computer-Supported Collaborative Inquiry on Buoyancy: a Discourse Analysis Supporting the ‘‘Pieces’’ Position Conceptual Change, J. Sci. Educ. Technol., 21, 808–825.
45. Vermunt J. K. and Magidson J., (2000), Latent GOLD 2.0, User's Guide, Belmont, MA: Statistical Innovations Inc.
46. Vermunt J. K. and Magidson J., (2002), Latent class cluster analysis, in Hagenaars J. A. and McCutcheon A. L. (ed), Applied Latent Class Analysis, Cambridge University Press, pp. 89–106.
47. Vosniadou S., (1994), Universal and culture specific properties of children's mental models of earth, in Hirschfeld L. and German S. (ed), Mapping the mind: domain specific in cognition and culture, New York: Cambridge University Press, pp. 412–430.
48. Vosniadou S., (2002), Mental models in conceptual development, in Magnani L. and Nersessian N. (ed), Model-based reasoning: science, technology, values, New York: Kluwer Academic Press.
49. Vosniadou S. and Brewer W. F., (1992), Mental models of the earth, A study of conceptual change in childhood, Cognitive Psychol., 24, 535–585.
50. Vosniadou S. and Brewer W. F., (1994), Mental models of the day/night cycle, Cognitive Sci., 18, 123–183.
51. Vosniadou S., Skopeliti I. and Ikospentaki K., (2004), Modes of knowing and ways of reasoning in elementary astronomy, Cognitive Dev., 19, 203–222.
52. Wellman H. M. and Gelman S., (1992), Cognitive development: foundational theories of core domains, Annu. Rev. Psychol. 43, 337–375.

---