Analysis of text difficulty in lower-secondary chemistry textbooks

Abstract

This paper focuses on the procedure and results for analyzing text-difficulty in lower-secondary chemistry textbooks in the Czech Republic. The authors use established methodology for text-difficulty analysis by Nestler, adapted by Průcha and Pluskal by adding a second independent analyser to improve reliability. Some textbooks do not follow the expected trend of either text-difficulty coherence or increasing text-difficulty between books for the 8th and 9th grade. No trend in topic difficulty was found either. The results show that learning outcomes may differ significantly when different books are used, despite the fact that they are supposed to support the same curriculum. For this reason, the results serve to support not only teachers when selecting a textbook, but also researchers as a starting point for lesson observations.

---

Introduction

At all levels of schooling, science textbooks are often used as the primary organizer of subject matter that students are expected to master. Textbooks represent a steering instrument in education in this respect (Slavík et al., 2010, p. 10). Textbooks also provide detailed explanations of topics to be taught (Chiappetta and Fillman, 2007, pp. 1847–1848). According to research, textbooks also serve to support teachers’ preparation for classes (Stern and Roseman, 2004; Knecht and Janík, 2008). Because of the structure of the subject matter, textbooks are a significant product of a curriculum, as they represent the potentially implemented curriculum (Törnroos, 2005, p. 317). However, the textbook difficulty for one school subject by different authors varies, so students get a different opportunity to learn, which naturally influences their achievement (Haggarty and Pepin, 2002).

The process of learning in general and learning of science depends heavily on effective communication (Aydin et al., 2014, p. 384). As textbooks are one of the first resources where students encounter field-specific information, textbooks play an important role in offering credible information to students, when used appropriately. They are also a cultural artefact that mediates the process of internalising an individual's thinking (Z. Sikorová, 2008, p. 59). Textbooks are also valuable sources for learners who need to study alone at their own pace or to remember the topics taught in class (Aydin et al., 2014, p. 384). For these reasons, the text in textbooks is supposed to be comprehensible for students, as its role is to stimulate learning. As “language is a major barrier (if not the major barrier) to most students in learning science” (Wellington and Osborne, 2001, p. 2). This point stresses the interconnectedness of reading literacy and the transfer of field-specific information (Callender and McDaniel, 2009). In relation to Markow's (1988) and Nemeth's (2006) ideas, this transfer can also be seen as one of the main components in scientific literacy – mastering scientific language, i.e. using terms appropriately (see OECD, 2016, pp. 17–46). Nevertheless, as element symbols/names or formulas are field-specific representations, their meaning is not different to a word – scientific term (cf.Wellington and Osborne, 2001).

For these reasons, research on textbooks, especially their text readability, is a very important topic in teaching (Pappa and Tsaparlis, 2011). Understanding the history of science allows us to explain science education's current context, because the current context is conditioned by former initiatives (cf.Alpaslan et al., 2015, p. 208). In the same way, we need to understand current textbooks to be able to create good didactical materials.

Comprehending new information depends on both student and textual characteristics. Therefore, instructors need to assess not only their students’ comprehension ability and prior knowledge, but also the difficulty of the materials they present to their students (Pyburn and Pazicni, 2014, p. 782). Research has shown that inaccurate or incomprehensible texts can cause students’ misconceptions (Sanger and Greenbowe, 1999; Pedrosa and Dias, 2000; Bergqvist and Chang Rundgren, 2017). This can be caused either by simplifying the subject matter/concept too much or a text which is too difficult for students to comprehend. As textbooks are often a tool which determine educational content and concepts (Drechsler and Schmidt, 2005; Dávila and Talanquer, 2010), the problems of text analysis and the scope of its difficulty are of high importance. The scope of textbook analysis is vast. The papers are, for example, focused on:

• the quality of elaboration of particular topics in textbooks (Sanger and Greenbowe, 1999; Klapko, 2006; Knecht, 2007; Nakiboğlu and Yildirir, 2011),

• curricular reform indicators (Kahveci, 2010),

• comparing textbook emphasis (Kim and Park, 2008) or conception (Lee, 2012).

In the literature, several efficient textbook parameters are emphasized: lucidity, scientific accuracy, content adequacy (cf.Knecht and Weinhöfer, 2006; Mikk, 2007; M. Sikorová, 2007), topic/subject matter sequencing (Mikk, 2007), visual element clearness, and relevance (Knecht and Weinhöfer, 2006). The selected textbook parameters and criteria to evaluate them have one common intersection – subject matter/text adequacy, i.e. textbook readability (see Mikk, 2008). In the above-mentioned research focused on science textbooks, several approaches were chosen. A number of research articles focus on the structure of terms (Fitzgerald et al., 2017), the order of the subject matter (Tsaparlis, 2014) or language explicitness within one concrete topic (Taibu et al., 2015). Other authors focus on text characteristics that reflect its difficulty and comprehensibility, i.e. the use of phrases and text organization in textbooks (Biber et al., 2004; Parodi, 2010), or the use of terms and their frequency (Hsu, 2014). There are also endeavours to analyse text difficulty with a system for computing computational cohesion and coherence metrics (Pyburn and Pazicni, 2014). For the overwhelming majority of languages, there are no such systems, therefore original approaches are still being employed.

Textbook analysis has quite a tradition in Czech education research with text-difficulty with respect to semantic and syntax being one of the most frequently examined issues (Greger, 2005; Hrabí, 2007; Rusek et al., 2016). The majority of authors follow the work by K. Nestler (1974, 1982), further developed by J. Průcha (1984) and adjusted by M. Pluskal (Pluskal, 1996). The Nestler–Průcha–Pluskal method for text-difficulty analysis has been (with just small alterations) used since (see e.g.Hrabí, 2007; Weinhöfer, 2007; Rusek et al., 2016), as it offers a complex set of monitored criteria and also the possibility to compare textbook text-difficulty within one field or among other fields.

Some of the cited research focused explicitly on science textbooks. However, chemistry textbooks have not yet been given appropriate attention by researchers. This text follows the tradition, uses, and in some respects further develops the established methodology and focuses on the so far neglected field of lower-secondary chemistry education. The authors of this paper start from Beneš et al. (2009) and Banýr (1988) and from the paper by Rusek et al. (2016), who focused on the whole range of elementary school chemistry textbooks and developed the method for text-difficulty analysis by Pluskal (1996). As previous papers focused mostly on textbook text-difficulty in general, this research is more focused on semantic difficulty, comparing particular textbooks in one textbook series and the difficulty of selected topics within the textbooks.

Methods

Textbook selection for analysis

By law, a basic school in the Czech Republic is supposed to provide their students with textbooks. As a proof of content and pedagogical convenience, as well as to conform to the national curriculum, textbooks may receive an approval clause from the Ministry of Education. It is granted to a series of textbooks, i.e. compact sets of textbooks, workbooks, etc. for one school subject. Textbooks without the approval clause can be used freely (depending on the headmaster's decision), nevertheless it is considered as a certificate of suitability.

For this reason, the textbooks approved by the Ministry of Education were included in the analysis. The textbooks are listed in Table 1.

Table 1 Analysed lower-secondary chemistry textbooks

Textbook title	Published^a	Authors	Publisher
a The two records relate to two books for 8th and 9th grades.
Základy praktické chemie 1; 2	1999, 2000	Beneš, P., Pumpr, V. and Banýr, J.	Prague: Fortuna
Základy chemie 1; 2	1993	Beneš, P., Pumpr, V. and Banýr, J.	Prague: Fortuna
Chemie 8; 9	2006, 2007	Škoda, J. and Doulík, P.	Plzeň: Fraus
CHEMIE Krok za krokem; CHEMIE Na každém kroku	1999, 2000	Bílek, M. and Rychtera, J.	Pardubice: Moby Dick
Chemie 8; 9	2016, 2015	Mach, J.; Plucková, I. and Šibor, J.	Brno: Nová škola
Chemie I; II	2004, 2007	Karger, I., Pečová, D. and Peč, P.	Olomouc: Prodos

---

Topic selection

To analyse the entire book would be too time-consuming. For this reason, the methods typically used work with pre-selected chapters. Chemistry subject matter is divided between two school years. For each year there is a textbook. In order to analyse all the books, several topics were chosen.

They were chosen based on several criteria:

• fundamental topics in lower-secondary chemistry,

• all topics covered in all the analysed textbooks,

• even distribution of the analysed topics in study years,

• each analysed chapter contains at least a 200-word explanatory text in all the analysed books.

Based on these criteria, the following six topics were chosen

• Air

• Hydrogen

• Neutralization

• Alkanes

• Carboxylic acids

• Proteins

Analysis procedure

Analysed categories. To analyse the text difficulty, a method named Nestler–Průcha–Pluskal (see Průcha, 2002) was used. The research followed this procedure:

I. From each of the textbook texts, a minimum text of 200 or more words long was chosen from the explanatory part of the chapter (legends, data in tables, experiment descriptions, etc. do not count). The sample text ends with the end of the last sentence where the number of 200 characters was reached. (The number of 200 words was proved to be a sufficient sample for analysis (Průcha, 2002).)

II. Total number of words (N), sentences (from the capital letter to the full stop or other symbols) (S) and verbs in the active form (V) was counted.

III. Nouns along with substantivized verbs were identified.

IV. The nouns were sorted into categories (we used different colours): (T₁) new general terms, (T₂) new scientific terms, (T₃) geographical terms (Earth, Moon, Sun, places, states, cities, etc. and other factual names), (T₄) quantitative terms (numbers, era, percentage, mass etc.), and (T₅) repeated terms (within the selected text area).

Specifics of chemistry – symbols of elements, formulas, etc. were also considered scientific terms (cf.Markic and Childs, 2016, p. 435). Although there are other categories of terms which contribute to text difficulty (philosophical terms, mathematical symbols, historical terms, etc.), the exception in terms of geographical terms was made to reproduce the literature-known procedure where the special treatment of those terms is justified.

V. Procedures in steps II–IV were carried out by two independent researchers. The procedures in 2–4 were confirmatory, whereas 5 produced a high number of variance. Many nouns (terms) were either not identified by one or the other researcher or were categorized differently. Therefore, a third researcher analysed the particular text in order to decide (cf.Teo et al., 2014).

VI. The numbers were added to tables for each parameter. Furthermore, two levels of difficulty were counted: the syntactic difficulty (formula for N, V, and S) and the conception difficulty (formula for T₁–T₅). The sum of the latter gives the overall rate of text difficulty.

The list of abbreviations is provided in Appendix 1.

Analysed values (adopted by Pluskal, 1996). The total text difficulty – D(1) is expressed as the sum of the syntactic (D_st) and semantic (D_sm) text difficulty:

D = D_st + D_sm (1)

The syntactic text difficulty is given by the following formula (2).where N is the total number of words, V is the sum of verbs in the active form and S is the number of sentences.

The semantic text difficulty is given by the following formula (3).

where T represents the total number of terms, T₁–T₅ are particular terms (see the section above) and N is the total number of words.

Text difficulty (readability) is also influenced by the average sentence length (L) and the average length of the sentence section. Both factors are given as a fraction of the total number of words and sentences (4) or verbs in the active form (5). All the following formulas (4–13) represent the total numbers of terms in books by the publishers. They are therefore sums of the terms calculated for particular analysed topics.

Apart from text difficulty, which represents a text's readability, the content aspect needs to be considered. Except for other texts, textbook texts need to fulfil certain requirements about the used terms. For this reason, the proportion of terms (P) is measured (6).This is divided into proportions of particular terms in the total number of analysed words (7–11).The coefficient P₁ represents the proportion of new general terms, P₂ the proportion of new scientific terms, P₃ the proportion of geographical terms, P₄ the proportion of quantitative terms and P₅ the proportion of repeated terms.

Out of the five categories of terms, the sum of T₂, T₃ and T₄ represents scientific information delivered by the analysed text. Another aspect of the text's scientific value is therefore the density of scientific information (i and h) coefficients, which were also calculated (12) and (13).

The i coefficient gives the density of scientific information in the number of words, i.e. describes the overall share of scientific terms in the text. The h coefficient gives the density of scientific information in the number of terms, i.e. describes the share of new field-related terms.

An example of the term categorization is shown in Appendix 2.

Objectives of the analysis

Chemistry textbooks for 8th and 9th grades, i.e. for 14–16 year olds, were chosen as they cover lower-secondary chemistry education in the Czech Republic. To evaluate the chemistry textbooks’ text difficulty, two main foci were chosen. The first part follows results by Rusek et al. (2016) with an analysis of text difficulty where the above described values (D, D_st and D_sm) were compared. This approach provides information about the readability of a textbook, i.e. how carefully the scientific information was transformed into subject matter. Two books in one textbook series were compared. With students’ learning progression in mind, there are two logical scenarios for text difficulty development between each of the two books by the same publisher. As textbooks are supposed to guide students and develop them: (1) the authors maintain the same text difficulty, keeping each topic even or (2) the books for 9th grade are characterized by greater text difficulty as they are designed for older students with some experience in chemistry. There are topics in elementary chemistry which share a number of technical terms with everyday language (e.g. air in this analysis) and therefore may show other characteristics in text difficulty than topics without such a link to everyday language – regardless of their real difficulty. This raises the question of topics’ order in chemistry education which is out of the scope of this paper.

The second part of the analysis focused on the scientific terms used in a textbook (only D_sm, i and h are followed). In this part, the topics within each book were compared. The “scientific term load” is a reflection of the textbook authors’ pedagogical content knowledge (Shulman, 1987), i.e. the selection of terms that particular authors consider important for the students to learn in each topic. This could be the cause of the different learning opportunity that students have when using different textbooks to learn chemistry. That is why the calculated values (D_sm, i and h) were analysed within each book and between the textbooks, in order to highlight this important textbook feature.

Results and discussion

Text difficulty between two books by one publisher

The results of this analysis (Fig. 1) show either of the above-mentioned logical text difficulty scenarios. The values are of a descriptive character. Their analysis only follows certain trends. Also, there are only a limited number of values which could be processed, which limits the use of more sophisticated statistics. For this reason, no tests for significance were carried out.

Fig. 1 Semantic, syntactic and total text difficulty comparison according to school years. Note: Every two columns in shades of blue, brown and grey represent the analysed values – syntactic difficulty, semantic difficulty and overall difficulty in the books for 8th and 9th grade. The figure also offers comparison between the textbooks. The values represent sums of values calculated as a sum of the analysed text parts.

Within the books by the Moby-Dick publishing house, the most significant decline in D was found. Compared with other textbooks for the 8th grade, this textbook is the most difficult in all three calculated factors (D, D_sm and D_st). In contrast, the textbook for the 9th grade is the easiest in D_sm and D.

The decline in D (20.6) is caused by the decline of D_st (4.3) and D_sm (16.3). The decline of D_st is caused namely by the decline in sentence length (4.5). The decline of D_sm is caused by a decrease in the proportion of terms in the text (9.4%), especially influenced by a decrease in the proportion of scientific terms (4.6%). This suggests that the books for 9th graders do not bring as much scientific information as the books for 8th graders.

With respect to the development of students’ thinking and progress in scientific education, the opposite trend was expected. Easier study texts may result in deceleration of students’ development (Ainsworth, 2006).

It is possible to follow the analogous trend in books by the Fortuna publishing house (ZCH). Between all the analysed textbooks, the book for the 8th grade in this book series is the second most difficult in D_st. In the book for the 9th grade, D_sm and D are the second easiest. The decline in D (10.0) is caused by the drop in D_st (2.7) and D_sm (7.3) – this represents the second biggest drop in both values between all the analysed textbooks. The drop in D_st is caused by the drop in the number of sentences (2.14). The authors use more verbs in the active form in the sentences, therefore making the text more comprehensible. The drop in D_sm is caused namely by a lower proportion of newly introduced scientific terms (by 4.5%). However, the proportion of general terms used is higher in this case (3.8%). Despite the change in D_st, in this case, caused by a different aspect than in the case of the previous books by the Moby-Dick publishers, similar risks may occur here too.

In contrast to the previous textbooks, the most outstanding increase in D (16.0) was noted between the textbooks by the Nová škola publishing house. This is caused by the increase in D_st (6.8) and D_sm (9.2). The textbook for the 9th grade is the most difficult among all of the three parameters. The D_st value is caused by an increase in sentence length (5.0) – the greatest increase recorded. D_sm was caused by the overall proportion of terms in the text (5.4%), where the proportion of scientific terms (3.0%) and repeated terms (10.4) grows and the proportion of general terms decreases (7.5%). This increase in text difficulty could have a positive effect on scientific literacy development, and it could also affect the importance of a topic as perceived by students. On the other hand, it could result in lower text comprehensibility for students and interfere with the basic function of a textbook – subject matter transfer and learning regulation.

An increasing trend can be observed in books by the Fraus publishing house too. According to all three observed values, the books for the 8th grade by this publisher are the easiest. Among the books for the 9th grade, this book is the easiest in D_st. Among all the analysed textbooks, the D value (8.3) represents the second biggest increase given, namely by the increase in D_sm (6.8), caused by the overall increase in the proportion of terms used (4.4%) – the second biggest – and the increased proportion of repeated terms (3.8%). With textbook function in mind, this result can be considered appropriate as the text in these textbooks broadens the context and aims to repeat scientific terms.

Among all the analysed textbooks, the book series by the Prodos publishing house is the most consistent in D_sm between the book for the 8th and 9th grade (0.3) but the proportion of general terms increases (4.8) – the biggest of all the analysed books. However, a decrease in the proportions of scientific terms (1.3%) and quantitative terms (2.5%) was recorded.

Analysis of particular topics in the textbooks

The previous results were given by the variables calculated as the sum of all of the analysed texts (topics) in one book. In this chapter, the focus is put only on the D_sm, i and h values within each textbook as they are indicators of scientific content elaboration in a textbook.

The particular topic values of D_sm in the analysed books are shown in Fig. 2. It is obvious that the authors of the analysed textbooks approach the topics differently. The text's semantic difficulty seems not to be considered (see Fig. 3). This could be problematic for education's effectiveness as “students… show greater learning (comprehension) of science texts when they are highly cohesive, as determined by the use of repetition of nouns and phrases” (Hall et al., 2014, p. 79).

Fig. 2 Values of semantic text difficulty in particular chemistry textbooks. Note: The points represent the values of semantic text difficulty calculated for particular topics in the textbooks. The order of the topics is determined by their organization in each textbook.

Fig. 3 Values of semantic text difficulty and the density of scientific information in analysed chemistry textbooks. Note: The values represent sums of values calculated for particular topics in the textbooks.

A certain trend can be observed (with the exception of the topic on proteins) in the textbook by the Nová škola publishing house. In the topic on air, all the three observed values were the lowest of all the analysed topics. The highest D_sm was found in the topics on carboxylic acids (42.7) and alkanes (41.4). The highest i and h values (19.6 resp. 47.1) were found in the topic on hydrogen. Except for proteins, there is a notable trend of increasing particular topics’ D_sm. The text carries characteristics of systematic text-difficulty development. This is important for a textbook as “learners must know how a representation encodes and presents information” (Ainsworth, 2006, p. 186).

Within the textbooks with the greatest D_sm span (Moby-Dick and Fortuna (ZCH)), the greatest D_sm variance in particular topics was found. The D_sm value does not correlate with the importance of a particular topic – an example is the topic on neutralization (see below). No unifying key was found within this variation.

The biggest difference in D_sm from all the textbooks was found in the books by the Moby Dick publishing house, with distinct differences in D_sm and i among the analysed topics (D_sm 31.5 and i 15.3). The highest D_sm value (50.5) was found in the topic on neutralization, the lowest (19.0 resp. 20.0) was found in the topic on proteins, resp. alkenes. A similar trend was also noted for i. It was the highest value (23.5) in the topic on neutralization and the lowest value (8.2) in the topic on proteins. The h value also differs significantly. The topic on hydrogen (48.9) and, in contrast, the topic on proteins (22.5) show substantial differences. Regarding proteins, the authors introduce the least terms of all of the analysed textbooks. This suggests the topics in the textbook are not elaborated evenly. Despite the fact that there is no standard provided, the book authors set it themselves. Such a huge difference may seem as if neutralization is somehow a more difficult or more scientifically important topic, whereas proteins represent a considerably easier or less important topic. However, the range of D_sm in the case of neutralization between the book by Fortuna (20) and Moby Dick (50) shows that authors of the Fraus book managed to explain the topic using simpler language.

Distinct differences in D_sm and i were found in the books by the Fortuna publishing house (ZCH) too. Among other textbooks, the i values represent the biggest difference (15.6) from all the analysed books. This series also differs in D_sm compared to the other textbooks. The D_sm value is again significantly higher in the topic on neutralization (49.5) – an outlier. The i value (26.0) is distinctly higher in the same topic. The authors introduce most of the new terms between all the analysed textbooks. In contrast, the i value is lower (10.4) in the topic on proteins. In this topic, the lowest h value (22.8) was found. This suggests that the authors introduce the least new terms in this chapter.

The difference in topics can also be observed in books by the Prodos publishing house. A significantly higher difference in the h values (32.1) for the analysed topics was found. This difference is the highest between all the analysed textbooks. The D_sm value for the neutralization topic is distinctly higher (41.2), whilst the topic on air is the lowest (23.2). Neutralization is also characterised by a distinctly higher i value (26.2). In contrast, the i value in the topic on proteins (13.7) is distinctly lower. The highest h value was found within the topics of hydrogen (61.7), neutralization (60.9) and carboxylic acids (60.0), and the lowest within the topic on proteins (29.6). The fact that three of the six analysed topics are characterised by a distinctly higher h suggests that newly introduced terms are not repeated as much as in the other textbooks in this book series.

Compared to the other books, the Fortuna (PCH) and Fraus publishing house books contain significantly fewer differences in D_sm (12.2 resp. 12.1). These books carry characteristics of text difficulty equality in the analysed topics.

The analysed topics in the Fortuna publishing house (PCH) books differ in i (4.9) and h (10.9), which is the smallest difference compared to the other analysed textbooks. In this textbook series, the highest D_sm (38.5) was found in the topic on alkanes, and the lowest (26.3) in the topic on hydrogen. Alkanes are also a topic with a significantly higher i value (20.5) than the rest of the analysed topics. Alkanes with 45.2 and carboxylic acids with 46.8 represent topics with the greatest h value. In contrast, the lowest h value (35.9) was found in the topic on proteins.

In the books by the Fraus publishing house, a significantly lower i value difference (5.5) was found than in the other textbooks. In the analysed topics, opposite values for the topics on neutralization and proteins were found. Whereas D_sm (31.5) for proteins is considerably higher, D_sm (19.4) in the neutralization topic is lower. This trend is the opposite of that observed in the other analysed textbooks. A significantly higher i value (17.24) was found in the topic on alkanes and a conversely lower value (11.8) in the topic on neutralization. As far as the h value is concerned, the topic on alkanes with 47.9 is distinctly higher than the others. In contrast, the topic on hydrogen has a considerably lower (29.6) h value.

Conclusions

Textbooks represent a vital element of education. They are one of the first (if not the very first) representation of a field into a text which can influence students’ future attitude towards the field. As textbooks can be used in classrooms as well as at home, they represent a medium which requires optimal text quality. From this point of view, research on this aspect is very valuable.

The general method of textbook analysis utilized in this research was proved to be usable for chemistry textbook analysis too, thanks to the variety of term categories included in the formula. It is also not language-specific, so it can be used for other languages too. With the restrictions that the text-analysis algorithms for less frequent languages have, this method offers an interesting option.

In the analysed Czech chemistry textbooks for elementary schools (grades 8 and 9), no systematic approach towards scientific literacy and reading literacy development was found. The textbooks do not reflect the authors’ work with the theory of education in terms of introducing new terms within one topic. Nor do they reflect the authors’ strategy of the work with text syntax. With Markow's (1988) original idea further developed by Nemeth (2006) in mind, the lingual representation plays a vital role in chemistry education. Also, the fact that most students at the beginning of science study face problems understanding field-oriented texts (Callender and McDaniel, 2009) suggests how important studies such as this are. The fact discovered therefore points out the first conclusion – the syntactic as well as semantic aspect of textbooks require proper attention in future textbook writing.

The text difficulty (D) of the analysed textbooks was not as constant or progressive as expected. There were textbook series where the eighth-graders’ book was more difficult (semantically and syntactically) than the ninth-graders’ book. This also concerns the semantic text difficulty of analysed topics which confirms the results presented by Knecht (2007, p. 107). This result suggests that the term selection for particular topic chapters was accidental. This is the most obvious within the neutralization topic. With the presumption that all the elementary school chemistry topics follow the national curriculum (expected outcomes) in mind, equality within text-difficulty indicators is expected. The effort to put the educational outcomes more precisely is visible in the latest addition to the Czech national curriculum – the Educational Standards and so-called indicator tasks (see Vojíř et al., 2017). Nevertheless, the discovered inequality suggests that students who learn with different textbooks gain different in-depth information as well as quality.

The results of this study can help teachers choose a textbook which corresponds with their teaching style or their students’ needs. Also, the results can serve researchers. The method of text analysis can be replicated and used for different fields or in different countries to compare textbooks and the textbook quality and its effect on the students’ results in an international comparison (e.g. PISA). Knowledge about textbook difficulty can also be an interesting starting point for lesson observation. Last but not least, the results may serve textbook authors in textbook reeditions or new textbook creation.

The limitations of this study dwell in having only limited information gained by text-difficulty analysis. Naturally, there are many other aspects that an efficient textbook needs to contain. Therefore, it is not possible to assess a textbook solely on the basis of its text difficulty. However, the text is the most important component. For this reason the text difficulty was the first aspect analysed in the textbooks. Another will follow in order to receive a fuller picture.

In further work, the authors of this text intend to focus on the way teachers use textbooks in education, more precisely the frequency, purpose and phases in which textbooks are being used. Special attention will be given to problem tasks in the textbooks, their quality and cognitive requirements with regard to the authors’ previous work in this field (see Vojíř et al., 2017).

List of analysed textbooks

At the date of publishing, all the analysed textbooks were certified with the approval clause except for the textbooks by the Moby-Dick Publisher. These textbooks were approved by the Ministry of Education until 2011. The publishing house no longer exists, but the textbooks are still available on the market, so it is reasonable to consider they are still being used at schools. Therefore, they were left between the analysed books.

Beneš P., Pumpr V. and Banýr J., (1993), Základy chemie 1 pro 8. ročník základní školy a nižší ročníky víceletých gymnázií, Praha: Fortuna.

Beneš P., Pumpr V. and Banýr J., (1993), Základy chemie 2 pro 9. ročník základní školy a nižší ročníky víceletých gymnázií, Praha: Fortuna.

Beneš P., Pumpr V. and Banýr J., (1999), Základy praktické chemie 1 pro 8. ročník základní školy, Praha: Fortuna.

Beneš P., Pumpr V. and Banýr J., (2000), Základy praktické chemie 2 pro 9. ročník základní školy, Praha: Fortuna.

Bílek M. and Rychtera J., (1999), Chemie krok za krokem, Pardubice: Moby Dick.

Bílek M. and Rychtera J., (2000), Chemie na každém kroku, Pardubice: Moby Dick.

Karger I., Pečová D. and Peč P., (2007), Chemie I pro 8. ročník základních škol a nižší ročníky víceletých gymnázií, Olomouc: Prodos.

Mach J., Plucková I. and Šibor J., (2016), Chemie pro 8. ročník Úvod do obecné a anorganické chemie (učebnice), Brno: Nová škola.

Pečová D., Karger I. and Peč P., (2004), Chemie II pro 9. ročník základní školy a nižší ročníky víceletých gymnázií, Olomouc: Prodos.

Šibor J., Plucková I. and Mach J., (2015), Chemie pro 9. ročník Úvod do obecné a anorganické chemie, biochemie a dalších chemických oborů (učebnice), Brno: Nová škola.

Škoda J. and Doulík P., (2006), Chemie 8 učebnice pro základní školy a víceletá gymnázia, Plzeň: Fraus.

Škoda J. and Doulík P., (2007), Chemie 9 učebnice pro základní školy a víceletá gymnázia, Plzeň: Fraus.

Conflicts of interest

There are no conflicts to declare.

Appendix 1 – The list of used abbreviations

The used abbreviations:

Total text difficulty – D

Syntactic text difficulty – D_st

Semantic text difficulty – D_sm

Total number of words – N

Sentences – S

Vocabs in active form – V

Total number of terms – T

New general terms – T₁

New scientific terms – T₂

Geographical terms – T₃

Quantitative terms – T₄

Repeated terms – T₅

Density coefficients of scientific information – i, h

Average sentence length – L

Average length of sentence section – M

Proportion of terms – P

Proportion of new general terms – P₁

Proportion of new scientific terms – P₂

Proportion of geographical terms – P₃

Proportion of quantitative terms – P₄

Proportion of repeated terms – P₅

Appendix 2 – An example of the term categorization

To explain the categorization of the terms in more detail, the following paragraph was used. It was translated from Czech to English as closely to the original meaning as possible. Therefore, the language may seem a little odd.

Example 1 (Škoda and Doulík, 2006, p. 34):

Hydrogen² is the most widespread element² in the Universe⁵. 90%⁴ of all atoms² in the Universe³ are atoms³ of hydrogen³. On Earth⁵, hydrogen³ is the third most widespread element³.

The indices above the terms mark the category the term was placed into. For the meaning of the categories see chapter Analysed categories (IV). Categorization of the repeated terms is obvious. In this example Universe and Earth are not considered to be a scientific term as they represent only a reference. In geography, however, their meaning would be different.

The only difficulty is in the categorization of general and scientific terms. Its occurrence is obvious in the second example.

Example 2 (Škoda and Doulík, 2006, p. 34):

Hydrogen⁵ is a colourless gas² without taste¹ and odour¹. It produces diatomic molecules² H₂².

As seen above, chemical symbols and formulas are considered as scientific symbols.

Acknowledgements

This work has been supported by the Charles University Research Centre program No. UNCE/HUM/024. The authors hereby thank Dagmar Kotková (Stárková) and Iva Bílková Metelková for their help with collecting the data and Linda Honskusová for her help with data analysis.

References

1. Ainsworth S., (2006), DeFT: a conceptual framework for considering learning with multiple representations, Learn. Instr., 16, 183–198.
2. Alpaslan M. M., Yalvac B. and Loving C. C., (2015), Curriculum Reform Movements and Science Textbooks: A Retrospective Examination of 6th Grade Science Textbooks, Eurasia J. Math., Sci. Technol. Educ., 11(12), 207–216.
3. Aydin S., Sinha S., Izci K. and Volkmann M., (2014), Turkish, Indian, and American Chemistry Textbooks Use of Inscriptions to Represent 'Types of Chemical Reactions', Eurasia J. Math., Sci. Technol. Educ., 10(5), 383–393.
4. Banýr J., (1988), Problematika hodnocení učebnic chemie jako příspěvek k teorii učebnic. (Kandidátská disertační práce), Pedagogická fakulta, Praha: Univerzita Karlova v Praze.
5. Beneš P., Janoušek R. and Novotný M., (2009), Hodnocení obtížnosti textu středoškolských učebnic, Pedagogika, 59(3), 291–297.
6. Bergqvist A. and Chang Rundgren S.-N., (2017), The influence of textbooks on teachers’ knowledge of chemical bonding representations relative to students’ difficulties understanding, Res. Sci. Technol. Educ., 35(2), 215–237.
7. Biber D., Conrad S. and Cortes V., (2004), If you look at…: lexical bundles in university teaching and textbooks, Appl. Linguist., 25(3), 371–405,  DOI:10.1093/applin/25.3.371.
8. Chiappetta E. L. and Fillman D. A., (2007), Analysis of five high school biology textbooks used in the United States for inclusion of the nature of science, Int. J. Sci. Educ., 29(15), 1847–1868.
9. Callender A. A. and McDaniel M. A., (2009), The limited benefits of rereading educational texts, Contemp. Educ. Psychol., 34(1), 30–41.
10. Co-operation O. f. E. and Development., (2016), PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic and Financial Literacy, OECD publishing.
11. Dávila K. and Talanquer V., (2010), Classifying end-of-chapter questions and problems for selected general chemistry textbooks used in the United States, J. Chem. Educ., 87, 97–101.
12. Drechsler M. and Schmidt H.-J., (2005), Textbooks' and teachers' understanding of acid–base models used in chemistry teaching, Chem. Educ. Res. Pract., 6, 19–35.
13. Fitzgerald W. J., Elmore J., Kung M. and Stenner A. J., (2017), The Conceptual Complexity of Vocabulary in Elementary-Grades Core Science Program Textbooks, Reading Res. Q., 52(4), 417–442,  DOI:10.1002/rrq.184.
14. Greger D., (2005), Možnosti zjišťování a měření obtížnosti didaktického textu, PhD Disertační práce, Pedagogická fakulta, Praha: Univerzita Karlova v Praze.
15. Haggarty L. and Pepin B., (2002), An investigation of mathematics textbooks and their use in English, French and German classrooms: Who gets an opportunity to learn what? Br. Educ. Res. J., 28(4), 567–590.
16. Hall S., Basran J., Paterson K. B., Kowalski R., Filik R. and Maltby J., (2014), Individual differences in the effectiveness of text cohesion for science text comprehension, Learning and Individual differences, 29, 74–80.
17. Hrabí L., (2007), Náročnost textu v učebnicích přírodopisu, in Maňák J. and Knecht P. (ed.), Hodnocení učebnic (Pedagogický výzkum v teorii a praxi ed.), Brno: Paido, pp. 98–108.
18. Hsu W., (2014), Measuring the vocabulary load of engineering textbooks for EFL undergraduates, English for Specific Purposes, 33, 54–65.
19. Kahveci A., (2010), Quantitative analysis of science and chemistry textbooks for indicators of reform: a complementary perspective, Int. J. Sci. Educ., 32(11), 1495–1519,  DOI:10.1080/09500690903127649.
20. Kim H.-N. and Park D.-Y., (2008), A Comparative Study of Elementary Science Instruction between the United States and Korea, , J. Res. Curric. Instr., 12(1), 39–54.
21. Klapko D., (2006), Analýza učebnic dějepisu pro ZŠ jako evaluační nástroj efektivní kvality didaktických textů, in Maňák J. and Klapko D. (ed.), Učebnice pod lupou, Brno: Paido, pp. 53–72.
22. Knecht P., (2007), Didaktická transformace aneb od “didaktického zjednodušení” k didaktické rekonstrukci, Orbis Scholae, 2(1), 67–81.
23. Knecht P. and Janík T., (2008), Učebnice z pohledu pedagogického výzkumu, in Knecht P. and Janík T. (ed.), Učebnice z pohledu pedagogického výzkumu (Pedagogický výzkum v teorii a praxi ed.), Brno: Paido, pp. 9–17.
24. Knecht P. and Weinhöfer M., (2006), Jaká kritéria jsou důležitá pro učitele ZŠ při výběru učebnic zeměpisu? Výsledky výzkumné sondy provedené na jihomoravských základních školách, Paper presented at the Současné metodologické přístupy a strategie pedagogického výzkumu, Plzeň.
25. Lee Y.-H., (2012), A Review of Elementary Science Textbook Analysis Research Conducted over the Past Three Decades in the United States and Analysis of the Nature of Science in the Introductory Chapter of US Elementary Science Textbooks, J. Korean Elem. Sci. Educ., 31(3), 398–412.
26. Markic S. and Childs P. E., (2016), Language and the teaching and learning of chemistry, Chem. Educ. Res. Pract., 17, 434–438.
27. Markow P. G., (1988), Teaching chemistry like the foreign language it is, J. Chem. Educ., 65(4), 346.
28. Mikk J., (2007), Učebnice: budoucnost národa, in Maňák J. and Knecht P. (ed.), Hodnocení učebnic, Brno: Paido, pp. 11–23.
29. Mikk J., (2008), Sentence length for revealing the cognitive load reversal effect in text comprehension, Educ. Stud., 34(2), 119–127,  DOI:10.1080/03055690701811164.
30. Nakiboğlu C. and Yildirir H. E., (2011), Analysis of Turkish high school chemistry textbooks and teacher-generated questions about gas laws, Int. J. Sci. Math. Educ., 9(5), 1047–1071,  DOI:10.1007/s10763-010-9231-6.
31. Nemeth J. M., (2006), Translating a linguistic understanding of chemistry to outcome achievement and interdisciplinary relevance in the introductory classroom, J. Chem. Educ., 83(4), 592.
32. Nestler K., (1974), Zur Wirkung lexikalischer und syntaktischer Textmerkmale auf das Lernergebnis, Informationen zu Schulbuchfragen, 17, 63–73.
33. Nestler K., (1982), Lernen aus Texten unterschiedlecher lexikalisch – syntaktischen Komplirierthei, in Baumann M. (ed.), Lernen aus Texten und Lehrtextgestaltung, Berlin: Volk und Wiessen, pp. 144–174.
34. Pappa E. T. and Tsaparlis G., (2011), Evaluation of questions in general chemistry textbooks according to the form of the questions and the Question-Answer Relationship (QAR): the case of intra-and intermolecular chemical bonding, Chem. Educ. Res. Pract., 12(2), 262–270.
35. Parodi G., (2010), The rhetorical organization of the Textbook genre across disciplines: a 'colony-in-loops'? Discourse Stud., 12(2), 195–222,  DOI:10.1177/1461445609356500.
36. Pedrosa M. A. and Dias M. H., (2000), Chemistry textbook approaches to chemical equilibrium and student alternative conceptions, Chem. Educ. Res. Pract., 1, 227–236.
37. Pluskal M., (1996), Zdokonalení metody pro měření obtížnosti didaktických textů, Pedagogika, 45(1), 62–76.
38. Průcha J., (1984), Hodnocení obtížnosti učebnic, Praha: SNTL.
39. Průcha J., (2002), Moderní pedagogika, Portál.
40. Pyburn D. T. and Pazicni S., (2014), Applying the multilevel framework of discourse comprehension to evaluate the text characteristics of general chemistry textbooks, J. Chem. Educ., 91(6), 778–783.
41. Rusek M., Stárková D., Metelková I. and Beneš P., (2016), Hodnocení obtížnosti textu učebnic chemie pro základní školy, Chem. Listy, 110(12), 953–958.
42. Sanger M. J. and Greenbowe T. J., (1999), An analysis of college chemistry textbooks as sources of misconceptions and errors in electrochemistry, J. Chem. Educ., 76(6), 853–860.
43. Sikorová M., (2007), Návrh seznamu hodnotících kritérií pro učebnice základních a středních škol, in Maňák J. and Knecht P. (ed.), Hodnocení učebnic, Brno: Paido, pp. 31–39.
44. Sikorová Z., (2008), Role a užívání učebnic jako výzkumný problém, in Knecht P. and Janík T. (ed.), Učebnice z pohledu pedagogického výzkumu, Brno: Paido, pp. 53–62.
45. Shulman L., (1987), Knowledge and teaching: foundations of the new reform, Harv. Educ. Rev., 57(1), 1–23.
46. Slavík J., Dytrtová J. and Fulková M., (2010), Konceptová analýza tvořivých úloh jako nástroj učitelské reflexe, Pedagogika, 60(3–4), 223–241.
47. Stern L. and Roseman J. E., (2004), Can middle-school science textbooks help students learn important ideas? Findings from Project 2061's curriculum evaluation study: life science, J. Res. Sci. Teach., 41(6), 538–568.
48. Taibu R., Rudge D. and Schuster D., (2015), Textbook presentations of weight: conceptual difficulties and language ambiguities, Phys. Rev. Spec. Top.-Phys. Educ. Res., 11(1), 010117,  DOI:10.1103/PhysRevSTPER.11.010117.
49. Teo T. W., Goh M. T. and Yeo L. W., (2014), Chemistry education research trends: 2004–2013, Chem. Educ. Res. Pract., 15(4), 470–487,  10.1039/c4rp00104d.
50. Törnroos J., (2005), Mathematics textbooks, opportunity to learn and student achievement, Stud. Educ. Eval., 31(4), 315–327.
51. Tsaparlis G., (2014), The logical and psychological structure of physical chemistry and its relevance to the organization/sequencing of the major areas covered in physical chemistry textbooks, Chem. Educ. Res. Pract., 15(3), 391–401,  10.1039/c4rp00019f.
52. Vojíř K., Holec J. and Rusek M., (2017), Biological and Chemical Tasks for Lowersecondary Education and Their Methodical Comments, in Rusek M., Stárková D. and Bílková Metelková I. (ed.), Projektové vyučování v přírodovědných předmětech XIV, Praha: Univerzita Karlova, Pedagogická fakulta, pp. 221–228.
53. Weinhöfer M., (2007), Obtížnost textu vybraných učebnic zeměpisu pro základní školy, in Maňák J. and Knecht P. (ed.), Hodnocení učebnic, Brno: Paido, pp. 115–120.
54. Wellington J. and Osborne J., (2001), Language and literacy in science education, UK: McGraw-Hill Education.

---