View PDF VersionPrevious ArticleNext Article
 
DOI: 10.1039/C7RP00120G (Paper) Chem. Educ. Res. Pract., 2017, 18, 849-874

Enhancing students' HOTS in laboratory educational activity by using concept map as an alternative assessment tool

I. B. A. Ghani , N. H. Ibrahim *, N. A. Yahaya and J. Surif
Department of Educational Science, Mathematic and Multimedia Creative, Faculty of Education, Universiti Teknologi Malaysia, Johor, Malaysia. E-mail: qhurune81@gmail.com; p-norhaniza@utm.my; p-afandy@utm.my; johari_surif@utm.my

Received 21st June 2017 , Accepted 22nd August 2017

First published on 22nd August 2017

Abstract

Educational transformation in the 21st century demands in-depth knowledge and understanding in order to promote the development of higher-order thinking skills (HOTS). However, the most commonly reported problem with respect to developing a knowledge of chemistry is poor mastery of basic concepts. Chemistry laboratory educational activities are shown to be less effective in developing an optimum conceptual understanding and HOTS among students. One factor is a lack of effective assessment and evaluation tools. Therefore, the primary focus of this study is to explore concept maps as an assessment tool in order to move students' thinking skills to a higher level during laboratory learning activities. An embedded mixed method design is used in this study, which has also employed a pre-experimental research design. This design triangulates quantitative and qualitative data, which are combined to strengthen the findings. A low-directed concept mapping technique, convergence scoring method, and pre-post laboratory concept map were used in this study. An electrolysis HOTS test was used as the research instrument in order to measure the level of student achievement with respect to high-level questions. In addition, the thought process that is involved when students construct concept maps has been explored and studied in detail by utilising a think-aloud protocol. Results showed a positive development towards understanding and higher level thinking skills in students with respect to electrolysis concepts learned through chemistry laboratory activities. An investigation of the students' thinking processes showed that high-achieving students were more capable of giving a content-based explanation of electrolysis and engaged in monitoring activities more often while building a concept map. Nonetheless, all categories of students managed to show a positive increase in the activities of explanation and monitoring during the construction of concept maps after they were exposed to the assessment tool in the laboratory learning activities. In conclusion, the assessment activity using concept maps in laboratory learning activities has a positive impact on students' understanding and stimulates students to increase their HOTS.

Introduction and background

One of the learning goals for the 21st century is focused on producing workforces that exhibit global competitiveness. This can be achieved by devoting intensive effort to developing students with a profound understanding of knowledge, and the ability to properly solve a problem, think critically and creatively, and be innovative. In other words, students are encouraged to develop their thinking capabilities to higher levels. Thus, students' achievements in science and mathematics subjects are among the relevant key components to be focused on in order to achieve the educational goals of this century (Sahin et al., 2013).

The importance of mastering chemistry concepts in HOTS development

Chemistry is an important foundation for meeting learning needs across the fields of science, technology, engineering, and mathematics (STEM). The goal of chemistry education should be to focus on meaningful learning, in which students are able to properly master basic concepts of chemistry so that these can be used to solve problems in new situations. Students' ability to provide arguments and explanations on specific chemistry processes is strongly emphasised in chemistry education (Norris and Philips, 2012). Knowledge that can be developed in a structured manner is an important factor in determining a student's ability to solve a new problem (Lopez et al., 2014). However, it was observed that quality learning cannot be achieved because students have a tendency to rely on memorising chemistry facts so that they can regurgitate them in order to pass examinations (Fernandez et al., 2013).

Several studies have reported that weak mastery of basic concepts is the main problem for students learning chemistry (Cooper et al., 2010; Luxford and Bretz, 2014; Burrows and Moorings, 2015). This is alarming because successful development of the skills required to meet the educational goals of the 21st century is dependent on mastering the basic concepts of chemistry. The development of understanding in chemistry and higher level thinking skills can be improved if the problem of weak mastery of the basic concepts is addressed. This can be seen in reports on international assessments such as TIMSS and PISA, which present a decrease in students' achievement in science and mathematics subjects. General reports on the results of international assessments showed that students lack mastery of the basic concepts of science and mathematics, which leads to an inability to properly solve problems (Ministry of Education, 2012; Phang et al., 2012). Hence, an emphasis on helping students build a good understanding of basic concepts is highly relevant and needs to be extended to ensure that the development of HOTS in students can be achieved.

Various definitions have been highlighted by previous researchers associated with higher-order thinking skills. Bloom (1956), for example, defined the levels of analytical cognition, synthesis, and evaluation as higher-order thinking skills categories for exhibiting more complex cognitive activity as compared to levels of knowledge, understanding, and application. Problem-solving skills, creative and critical thinking, and the ability to ask questions about the value of higher-order thinking skills was highlighted by Zoller (1993). In addition, according to Zohar and Dori (2003), cognitive activities such as being able to compare and differentiate, build arguments, and be able to scientifically examine are examples of higher-order thinking skills. However, the definition of higher-order thinking skills according to the interpretation by Anderson et al. (2001) is slightly different; this is in accordance with the Revised Bloom's Taxonomy and is a term often used in the educational field today (Forehand, 2010).

In science education, in general, HOTS are closely associated with meaningful learning or learning through understanding, in which students are able to apply what they have learned in the context of a new situation (Novak, 2010; Hofstein and Kind, 2012; Lopez et al., 2014). For this purpose, the development of basic concepts is essential in ensuring that the development of students' knowledge and HOTS in chemistry can be achieved (Vachliotis et al., 2014). According to Krathwohl (2002), it is important to stimulate students' cognition to the highest level (creating level), but mastery of basic concepts is also necessary. This shows that the importance of mastering the basic concepts of chemistry should be emphasised.

Students' understanding of electrolysis

Electrochemistry is one of the key concepts that students need to master in order to become proficient in the science of chemistry. However, previous studies have reported that several learning problems occurred among students in mastering the concepts of electrolysis (Sia et al., 2012; Akram et al., 2014). One of the problems is that students are weak in distinguishing the final product obtained at the anode and cathode in an electrolysis process (Akram et al., 2014). They are unable to master the basic concepts that emphasise distinguishing between molten and aqueous electrolysis. This is a key reason why students cannot provide accurate assessment and justification when describing a chemical process (Schmidt et al., 2007).

These problems stemmed from weak mastery of the basic concepts required for chemical knowledge, which eventually caused students to frequently encounter difficulties in solving problems associated with electrolysis (Thompson and Soyibo, 2002; Rahayu et al., 2011; Heng et al., 2014). What is more alarming is that poor mastery of the concepts associated with one chemistry topic will hinder learning in other chemistry topics and reduce motivation to learn (Celikten et al., 2012). As a result, the objectives of students' HOTS development will not be achieved.

Learning through laboratory activities

One of the important aspects of chemistry teaching is helping students to gain in-depth understanding of a concept that can be mastered through laboratory learning activities (Lunetta et al., 2007; Ding and Harskamp, 2011). The basic goal in conducting these activities is to help students make a connection between the world of objects and events, and the world of abstract thoughts and ideas (Abrahams and Millar, 2008). Many scientists have agreed on the importance and significant impact of these events with respect to achievements in scientific knowledge (Hofstein et al., 2005; Sesen and Tarhan, 2010). These activities are also likely to provide the best platform for developing concepts and students' HOTS.

However, several issues have arisen regarding doubts whether the implementation objectives of laboratory activities are being achieved. The main issue is that many students do not appear to understand the main purpose of laboratory activities that are undertaken. They simply thought that they were merely having fun playing with laboratory apparatus and materials outwith the daily learning routine (Reid and Shah, 2007; Kim and Tan, 2011). In addition, comprehensive assessment and evaluation activities cannot be executed properly with the lab activities that have been implemented. Formative assessments are rarely considered for the purpose of improving teaching and learning methods. Summative assessments are the sole focus as these are important for grading for entering higher education institutions (Hofstein and Kind, 2012; Fernandez et al., 2013). The traditional practice of assessment and evaluation activities such as laboratory reports and quizzes on laboratory learning activities do not help the development of students' conceptual understanding (Hofstein and Lunetta, 2004; Özmen et al., 2009; Kibar et al., 2013). Active student involvement in the assessment activities to build meaningful knowledge was rarely implemented. This will only lead to slower development of students' HOTS since it cannot be provided properly (Koh et al., 2012). Therefore, it is desirable for researchers to determine the best assessment methods and approaches to laboratory and educational activities to ensure the objectives of their implementation can be optimised.

The potential of concept maps as an assessment tool for conceptual and metacognitive understanding

Among the visual tools that have been widely reported to increase students' conceptual understanding of a learned concept is the use of concept maps (Kaya, 2008; Yaman and Ayas, 2015). Proposition is an important component related to concept maps that can translate students' understanding of a concept. Proposition is a representation of two concepts that can be connected to the most appropriate linking words. The more accurate the generated proposition, the higher the level of understanding that can be achieved by an individual student. The production of an accurate concept and proposition can show that meaningful learning has occurred. This learning occurs when students are able to integrate information or new concepts with existing information or concepts, and then translate the understanding through a relationship representative that has been generated. This is closely related to the students' cognitive development at a higher level (Novak, 2010; Vachliotis et al., 2014).

Additionally, the low-directed concept mapping technique is also believed to provide students with the opportunity to develop their creative and critical thinking skills. Students are given a little information or a basic concept by the assessor or teacher in order to generate a concept map of a learning unit via the low-directed concept mapping technique. The students are challenged to create other concepts and design their understanding according to their own preferences. On the other hand, in the high-directed concept mapping technique, students' creativity has been blocked to translate the understanding gained since they have been given many concepts by the teacher/assessor. In addition, monitoring activities carried out by students while building a concept map and reflective attitudes towards the quality of concept maps that have been produced can encourage development of critical thinking skills to a higher level (Ruiz-Primo et al., 2001; Hilbert and Renkl, 2008; Kumaran and Sankar, 2013).

Concept mapping has the potential to become not only a tool for teaching and learning but also an assessment tool for the level of understanding that is reflected in students' cognitive structures. Concept maps can serve as an alternative assessment tool only if they can fulfil three essential components, namely (a) a task demand, (b) a format for students' responses, and (c) a scoring technique (Ruiz-Primo et al., 1997). A task demand is a technique or concept map task that needs to be done by the students to complete the assessment activities. For example, the assessor must determine whether to use the low or high-directed concept mapping technique in assessing the concept map. The choice depends on the purpose of assessing the concept maps that are implemented. The response format is a method of concept mapping performed with either pen and paper or ICT. Meanwhile, the scoring technique refers to the quantitative or qualitative method of scoring that is used when assessing the concept maps that are produced.

Basically, the concept maps can be assessed and evaluated by using quantitative or qualitative analyses. This depends on what is reviewed and evaluated by each examiner and the objective assessment that was constructed (Kibar et al., 2013). With quantitative analysis, much emphasis is given to calculation of the components or elements of a concept map (Ruiz-Primo and Shavelson, 1996; McClure et al., 1999). Meanwhile, qualitative analysis focuses on the explanation of the content and quality of concept maps (Kinchin et al., 2000). These two analytical methods can be evaluated by comparing with an expert concept map (Van Zele et al., 2004).

Consequently, many studies related to the use of concept maps as assessment tools have utilized scoring that emphasised accurate proposition (Ruiz-Primo and Shavelson, 1996; McClure et al., 1999; Stoddart et al., 2000; Burrows and Mooring, 2015; Yaman and Ayas, 2015). This is because many researchers classify the proposition as the component of concept maps that represents the students' understanding. Analysis of the propositions produces better reliability of the analysis of the other components of the concept map, which produces a more effective statistical analysis (Stoddart et al., 2000).

Among the scoring methods that emphasised accurate proposition is the convergence scoring method. This scoring method was found to be the most appropriate and practical to be implemented as a means of understanding the conceptual assessment of students in the class (Ifenthaler, 2010). With this scoring method, a reference concept map produced by a specialist was used as the comparison and scoring scheme for the propositions produced by the students. Several researchers such as Ifenthaler (2010) and Yaman and Ayas (2015) suggested that the application of an expert concept map is used so that the analysis scoring of a concept map is more credible and reliable. Therefore, this study used an approach involving the low-directed concept mapping technique and the convergence scoring technique based on their potential to develop an in-depth conceptual understanding. This, in turn, is expected to promote students' thinking skills at a higher level.

There are widespread reports that studied the utilisation of the concept map as a tool for teaching and learning. However, its role as an assessment tool is still untapped in many areas and needs to be improved (Kibar et al., 2013; Burrows and Mooring, 2015; Yaman and Ayas, 2015). Its validity as a tool of assessment in the teaching and learning process is still questionable. Furthermore, empirical studies relating to the nature of its ability to generate students' HOTS still need to be explored (Bramwell-Lalor and Rainford, 2014).

Some studies have been conducted on the use of concept maps as an assessment tool to understand the conceptual assessment of learning activities in the laboratory; however, it is difficult for researchers to find studies that explore and investigate aspects of the effects of visual tools on the development of students' HOTS. Moreover, in the literature, there are many studies that only focused on the effect of concept maps on the students' learning outcomes (Kaya, 2008; Özmen et al., 2009; Kibar et al., 2013); a few studies examined the thinking processes of students in concept map activities.

Furthermore, some researchers have also suggested that new research can be conducted with an emphasis on formative assessment of laboratory activities, mainly in developing students' in-depth conceptual understanding (Abrahams and Millar, 2008; Hofstein and Kind, 2012; Roberts and Johnson, 2015). This is because formative assessment and the development of conceptual understanding, as well as the metacognition of students, are seldom provided through the activities. This must be emphasised because improvement of assessment and evaluation elements have the same weighting in the interests of improving teaching and learning to achieve educational transformation in the 21st century. Therefore, the aim of this study is to meet these demands by seeking alternative approaches to assessment activities that can be carried out in chemistry laboratory activities. Emphasis is given to the potential of the assessment tool in the development of conceptual understanding in order to enhance the HOTS of students. Thus, the objectives of this study are as follows:

(a) Is the use of concept maps as an alternative assessment tool in chemistry laboratory activities able to enhance students' level of understanding of the concept of electrolysis?

(b) Is the use of concept maps as an alternative assessment tool in chemistry laboratory activities able to enhance students' level of academic achievement with respect to HOTS related to the concept of electrolysis?

(c) How do students reveal their thought processes while building a concept map in chemistry laboratory activities?

Methodology

This study used a mixed methods research design (embedded design) by Creswell and Plano Clark (2011), in which quantitative methods are run as the primary method and supported by qualitative research. One group pre-experimental (O1 × O2) by Campbell et al. (1963) was used as a quantitative method in this study. A think-aloud protocol was used to support the findings of the quantitative methods that were used.

Participants and instrumentation

Quantitative study. The use of one group pre-experimental study design (O1 × O2) by Campbell et al. (1963) is still relevant as it is still popular and widely used in the field of education (Creswell, 2012).
Participants. In order to fulfil the objectives of this study, 32 students were involved and selected from an intact class from a boarding school in Johor. Chemistry teachers, who were involved in this study, were graduates that possessed a degree in education and had been teaching chemistry for at least five years.
Concept map task. Concept map task instruments have been used to answer the first question of the study. Pre- and post-intervention concept map task instruments were used to determine the level of understanding of students revealed by the intervention study. These tasks have been designed by the researcher based on the chemical electrolysis concept and made use of low-directed concept mapping and convergence scoring techniques. Both of these techniques are selected based on their abilities to become a platform for an in-depth conceptual understanding. This, in turn, is expected to promote students' thinking skills at a higher level. The tasks consist of a list of instructions and some of the basic concepts of electrolysis that are similar to those in an expert concept map. Students are then asked to add as many new concepts related to electrolysis as possible to demonstrate the level of their understanding towards the relevant electrolysis concept.

Based on the adaptation study by Ruiz-Primo et al. (2001), an expert concept map was built after agreement was reached between the researcher and two other expert chemistry teachers. This map was developed to be used as a reference (scoring scheme) during revision of concept map assessment of students. The construction procedures of an expert concept map are summarized in Table 1 below.

Table 1 Expert concept map construction procedures for Electrolysis concept (adapted from Ruiz-Primo et al., 2001)
Steps Procedure details
Step 1 Two panel members are selected from among experts on the subject syllabus content of the electrolysis topic according to the syllabus prepared by the Ministry of Education and researchers
Step 2 All panel lists were asked to make a list of the key concepts in the domain subtopics of electrolysis to be mastered by students
Step 3 All panel members must compare and discuss the list of key concepts that they have laid out to achieve agreement on the concepts that are most important in the subtopics of electrolysis. These concepts will be relisted and referred to as a list of key concepts
Step 4 Each member of the panel is required to build a concept map based on the list of key concepts
Step 5 Discussion of the results of the three concept maps has taken place and a final concept map is created with the agreement of all three members of the panel
Step 6 The final concept map will be used as a standard concept map of electrolysis and is known as the expert concept map

Three experts validated the contents of the concept map task developed by the researchers by reviewing and giving feedback. The evaluation panel is composed of university lecturers and expert chemistry teachers who have extensive experience in the field of chemistry education. Modifications to the tasks were made based on the feedback and intensive discussions that took place between the researcher and the assessors. Next, in order to achieve reliability and feasibility of the concept map tasks, a total of 12 Form 4 students were involved in the pilot study. To evaluate the study session, an evaluation using questionnaires and revision of the concept map tasks was performed. The Cronbach's alpha value obtained through the task questionnaire is 0.968, which is an acceptable reliability value. Based on the revision results of the concept maps, researchers found that students could build a good concept map using the basic components of a concept map that had been learned. All the students were found to be able to use 50% of the concepts that were provided. This reinforces the assumption that the instrument can be accepted and implemented in the actual study (Ruiz-Primo et al., 1997).

HOTS comprehension test on electrolysis. The instrument consists of pre-test and post-test scores. The open-ended test set contains 12 items covering the four highest cognitive levels of Bloom's Taxonomy Review by Anderson et al. (2001): the levels of applying, analysing, evaluating, and creating. This test was distributed to students before and after the intervention was provided. The HOTS comprehension test, which includes the subtopics of Electrolysis Chemistry Form 4, was developed by the researcher to evaluate the effect of interventions on the development of HOTS of students.

To obtain the content validity of the test, a total of three experts composed of university professors and expert teachers of chemistry were appointed to review and comment on each test item. The assessors were selected based on their skills in composing chemistry questions according to the cognitive level of the Revised Bloom's Taxonomy. The instrument has undergone a pilot study among 31 students in order to ensure compatibility and reliability. A correlation coefficient of r = 0.815 was obtained through methods of test retest reliability. According to Leech and Barrett (2011), if the coefficient correlation obtained is more than 0.70, then the instrument can be accepted and is reliable.

Qualitative method.
Think-aloud protocol method. Generally, with this method, students were asked to say out loud what they thought during a particular task. The think-aloud protocol is a strategy that is widely popular and is used to study cognitive processes (Seng, 2007; Hilbert and Renkl, 2008). In this study, this method was carried out by asking some students to say out loud what they thought during concept mapping. When constructing a concept map for the students' thinking process, a total of 6 respondents were selected through a purposive sampling method. Yin (2009) claimed that 3 or 4 respondents are considered as an adequate number for qualitative data acquisition. These six students were selected based on the following criteria, including:

(i) 2 students who obtained a high score in the pre- and post-test on electrolysis HOTS

(ii) 2 students who obtained a moderate score in the pre- and post-test on electrolysis HOTS

(iii) 2 students who obtained a low score in the pre- and post-test on electrolysis HOTS

These six respondents agreed to voluntarily participate in this study. Additionally, the students were articulate and were nominated based on recommendations by their own chemistry teacher. This is important because the success of think-aloud protocol activity is highly dependent on the articulacy of students (Van den Haak et al., 2003). They were asked to say out loud what they thought but did not need to explain their answers. If they were silent or forgot to talk, researchers would then gently remind them to speak out loud. This process has been recorded in both audio and video format. Video footage has been taken to reduce the likelihood of doubt and bias. Furthermore, it helps to ensure saturation of the available data since audio recordings alone are not enough (Yin et al., 2005). A verbal protocol that was obtained was divided according to verbal units, which were then coded by using a category coding that was developed by Ruiz-Primo et al. (2001).

Procedure

Concept map construction training sessions were given to the students before the pre-concept map task evaluation was conducted. This aims to ensure that students have learned the basic principles of concept mapping before any other evaluation is conducted. It is based on recommendations given by many previous researchers who advised on the training that should be given to students in order to obtain the best results (Hilbert and Renkl, 2008; Hay et al., 2010; Bramwell-Lalor and Rainford, 2014). The implementation of the training session lasted for a week and emphasis was given to training in concept map construction techniques starting from a high degree of direction to a gradually lower degree of direction. This is based on a proposal by Bramwell-Lalor and Rainford (2014) to ensure that students are always motivated in building a concept map that can enhance their cognitive level.

Fig. 1 shows the flowchart of this study procedure.


image file: c7rp00120g-f1.tif
Fig. 1 Flowchart of the research.

Then, a pre-test of the HOTS Comprehension Test on Electrolysis and a concept maps pre-task were given a week before the intervention in order to identify students' existing knowledge on the concept of electrolysis. An intervention study using concept maps as an alternative assessment tool for five laboratories and educational activities proceeded for five weeks. The five weeks of laboratory learning activities covered all the electrolysis sub-topics according to the specifications of the Ministry of Education (MOE) chemistry syllabus. Table 2 shows the learning objectives for each laboratory activity.

Table 2 Learning objectives for each laboratory activity
Learning objective Laboratory learning activity phase Assessment activity tasks
(Refer to Appendix 1 for details of laboratory activities).
(1) Identifying the characteristics of electrolytes 1 A
(2) Analysing the electrolysis of molten substances 2 B
(3) (i) Analysing the electrolysis of aqueous solution through ion positioning factors in electrochemical series 3 C
(3) (ii) Analysing the electrolysis of aqueous solution through ion concentration factors in electrolytes 4 D
(iii) analysing the electrolysis of aqueous solutions through types of electrode factors 5 E

Each laboratory activity includes assessment tasks involving concept maps for pre- and post-laboratory activities. The activity of concept mapping for pre- and post-laboratory activities was believed to be a good platform for developing students' conceptual understanding (Kaya, 2008; Özmen et al., 2009; Burrows and Mooring, 2015).

Then, at the beginning of the subsequent week, a post-test that determines students' understanding of electrolysis HOTS and concept map post tasks was distributed for the purpose of assessing the impact of interventions that were carried out during laboratory learning activities. During the pre- and post-concept map evaluation tasks, think-aloud protocol procedures were performed on all six respondents.

The teaching and data collection processes in this research were carried out in the Malay language. The Malay language is the mother tongue and medium of instruction in Malaysia. According to Miles et al. (2011), the use of the native language can help the respondent to understand the instructions clearly and provide the right feedback. However, for reporting purposes, all materials used are translated into English and reviewed by an English expert to ensure that the validity of the contents is preserved.

Data analysis

Concept map. Students' concept maps were quantitatively analysed by using convergence scoring in accordance with the method proposed by Ruiz-Primo and Shavelson (1996). This scoring method is believed to be the best method of showing the development of students' understanding and knowledge and the most practical to be implemented in the classroom (Ruiz-Primo et al., 2001). This method focuses on the relationship built by the two concepts (proposition) as these elements are effective in showing students' in-depth understanding. Nonetheless, this proposition element is the most challenging element to be built by students (McClure et al., 1999).

The proposition constructed by each respondent was compared with the proposition of the expert concept map. These relationships have been given a score of either correct or incorrect using the ratio of the accurate propositions in the students' concept maps to all the propositions in the expert concept map. This is shown by the following formula:

Concept map score:

image file: c7rp00120g-t1.tif

Concept map scoring using this method was performed by the researcher and was also separately performed by one of the expert teachers involved in the construction of specialist/expert concept maps. This method is to ensure reliability (inter-rater reliability) of the score marks obtained by the student. Out of the 218 propositions that were reviewed, agreement between the two coders to 214 propositions was obtained, representing 98.17% agreement. The kappa value obtained was 0.949. Thus, the kappa value is considered to indicate excellent agreement. If there is any discrepancy, discussions take place between the two appraisers until a final agreement has been reached. The scores were analysed and the level of students' understanding was obtained based on the score comparison made between pre- and post-intervention concept maps.

The scores obtained were then analyzed to obtain the mean, percentage, and standard deviation. Inferential analysis using the paired t-test was performed to compare the mean of the pre-concept map with the mean of the post-concept map of the study that was performed.

Think-aloud protocol

The data was collected from concurrent think-aloud protocols and supported by observation, retrospective interview, and written answers by the respondents. The collected transcription protocols were given to each respondent to verify the verbal data. These protocol sets were then segmented into units of analysis (known as verbal units) and coded using the categories developed by Ruiz-Primo et al. (2001). The developed coding categories are based on cognitive processes that indicate their in-depth conceptual understanding and activities that indicate stimulation of students' HOTS. Verbal protocols obtained were qualitatively analysed using a content analysis technique (deductive method), which was then translated quantitatively through descriptive statistics, namely frequency and percentage. In order to determine the reliability of the qualitative findings, verbal unit coding according to this category was performed by two coders. The researcher and second coder coded the verbal unit on five of the twelve sets of protocols individually before comparing all the verbal unit coding. If there were differences in the coding created by these two coders, reassessment and discussion of the rationale for the selection of the coding were carried out until agreement was reached. However, if agreement was not obtained, the researcher as the first coder made judgments on the actual coding. This process is the same as the study of content analysis conducted by Hilbert and Renkl (2008), which utilises think-aloud protocol. The inter-rater reliability was determined by Cohen's kappa value κ and the value obtained was 0.912. This value is considered good according to Landis and Koch (1977) and shows that the coding performed on each verbal unit has high reliability.

Research ethics

This study emphasizes a few research ethics such as:

(i) Confidentiality agreement in terms of data, information, and identity. To maintain information confidentiality, the school's name and the participants are not mentioned in this study report. However, nicknames were used to represent the respondents for the purpose of research discussions.

(ii) Participants were informed of the purpose and their role as participants in this study. Hence, participants' involvements were based on their own will and they were free to withdraw from the study.

Results and discussion of research

Findings related to the first research question

The study found that the use of concept maps as a tool for assessing laboratory learning activities resulted in an increase in the level of students' understanding of the concept of electrolysis. This can be proved by assessment scores for students' pre- and post- concept maps as shown in Table 3 below.
Table 3 Descriptive analysis of the scores for electrolysis concept maps before and after treatment
Type of test Min score Max score Mean Standard deviation
Pre (N = 32) 3 27 13.28 6.78
Post (N = 32) 33 80 52.06 11.93

Table 3 shows an increase of 38.78% in the mean of the score for the post-concept maps compared to that of the pre-concept maps. The mean score obtained for the pre-concept maps is 13.28% and scores ranged from 3% to 27%. Meanwhile, the average score for the post-concept maps is 52.06% and scores ranged from 33% to 80%. Therefore, these findings show that the level of students' understanding improved after treatment. To strengthen the research findings, the increase in concept map scores between pre- and post-assessments can be shown through the following example of a student's concept map (Fig. 2 and 3).


image file: c7rp00120g-f2.tif
Fig. 2 Example of student's pre-concept map (refer to Appendix 2).

image file: c7rp00120g-f3.tif
Fig. 3 Example of student's post-concept map (refer to Appendix 3).

Through the analysis of the concept map, for example, through the post-concept map, students can provide more additional new concepts, using more accurate linking words and more complex concept map structures than in the pre-concept map. Analysis of the concept map reflects the increase in the students' level of understanding of the electrolysis concept that they have previously learned. The researcher believes that the students showed more complex (sophisticated) thinking and conceptual understanding after the intervention, as reflected in the post-concept maps produced after the intervention. Many previous studies, for example, Hilbert and Renkl (2008), Erdem et al. (2009), Davies (2011), and Lopez et al. (2014) emphasised the importance of proliferation of the concept and new relationships, the construction of more accurate linking words, and a more complex net structure in translating the more in-depth conceptual understanding that is acquired by the students.

Next, a paired t-test analysis was performed to identify whether there was a significant difference between the obtained mean scores. Referring to Table 4, there is a significant difference, t(31) = 26.48, p < 0.05, between the mean scores for the pre-concept map and the post-concept map for the same sample. This confirms that the mean score for the post-concept map is significantly than that for the pre-concept map after intervention. These results indicate that a given intervention can improve students' scores in evaluating the concept map for electrolysis. Next, the effect size was determined using a formula for Cohen's d and the value obtained was 4.68 (d > 0.80). The value of d (4.68) shows that the effect of the treatment on the level of students' understanding is large.

Table 4 Analysis of paired samples t-test
Type of test Mean Std deviation t Sig. (2-tailed)
Pre-test 13.28 6.78 26.48 0.000
Post-test 52.06 11.93

In conclusion, the alternative assessment of laboratory learning using concept maps has a significant impact on the level of students' understanding in learning the concept of electrolysis. This finding supports the study by Özmen et al. (2009) and Kaya (2008), who reported that students should be able to obtain a good conceptual understanding of the concepts learned through laboratory activities by using concept maps. Roberts and Johnson (2015) also confirmed that the concept map can be used as a tool to develop an understanding of laboratory activities. Through these findings, the use of concept maps as an alternative assessment tool in laboratory activities was found to have positively impacted students' conceptual understanding levels. The use of the convergence scoring method and regular construction of concept maps in the laboratory have contributed to the successful development of conceptual understanding by students.

Findings related to the second research question

Students' achievement level in HOTS test. The level of achievement in the HOTS Comprehension Test on Electrolysis was found to have increased after exposure to the intervention study. Table 5 shows the descriptive analysis that was conducted.
Table 5 Descriptive analysis of pre- and post-comprehension test on electrolysis HOTS
Type of test Min score Max score Mean Standard deviation
Pre (N = 32) 3 47 23.61 10.35
Post (N = 32) 36 92 66.75 14.18

The mean score for the post-test showed an increase by 43.14 units relative to that of the pre-test. These findings indicate that the level of students' achievement in answering HOTS questions on the electrolysis concept can be enhanced by these interventions. The researcher then conducted paired t-tests to determine whether there were significant differences between the mean scores obtained. The result of the analysis showed a significant difference, t(31) = 21.66, p < 0.05, between mean scores of pre-tests and post-tests for the same sample (Table 6). This has confirmed that the mean score for the post-test is significantly higher than that for the pre-test. These results once again prove that the use of concept maps as research interventions can increase students' achievement scores in a HOTS comprehension test on electrolysis.

Table 6 Analysis of paired samples t-test
Type of test Mean Std deviation t Sig. (2-tailed)
Post-test 23.61 10.35 21.66 0.00
Pre-test 66.75 14.18

In order to investigate whether the data was normally distributed before the T test was conducted, the researchers conducted normality testing through the Kolmogorov–Smirnov and Shapiro Wilk tests. For the HOTS pre-test data, the Shapiro Wilk test showed a significant value of 0.955, whilst the post-test data for HOTS also showed a significant value of 0.393. This shows that the HOTS pre- and post-test data have been distributed normally.

In order to identify the effect of the size of the intervention, the Cohen's d formula was used to determine the effect size, and the value obtained was 3.83 (d > 0.80). This value indicates that the effect of the intervention on generating higher-order thinking skills in students is large.

An interpretation that can be made from these findings is that the assessment activities using concept maps for laboratory learning activities can contribute to an improvement in student achievement with respect to answering higher-order questions. This finding is supported by previous studies by Bramwell-Lalor and Rainford (2014), Karakuyu (2010), and BouJaoude and Attieh (2008) on the direct effect of the use of concept maps on students' achievement in answering higher-order questions in various subject areas. However, the results of this study are found to be slightly different from those of the study by Stensvold and Wilson (1992), which found that concept maps do not give an advantage to high ability students compared to lower proficiency students. The same finding is obtained from studies by BouJaoude and Attieh (2008) and Karakuyu (2010), which found that concept maps can help students in the group of low achievers to obtain a better performance in answering higher-order questions. Furthermore, previous studies by Didis et al. (2014) have reported that many science teachers doubted the potential of concept maps in assessing and evaluating student learning outcomes at a higher cognitive level. They argued that these tools are only suitable for evaluation of basic concepts at low levels. Their findings differ from the findings of this study because the students who were involved in this study were excellent students who passed public exams and were selected by the Ministry of Education to enter a boarding school. This study shows that high-achieving students can benefit from the use of concept maps. Thus, the findings of this study may prove that the use of concept maps as an alternative assessment tool can be used to boost students' cognitive skills at higher levels.

Findings related to the third research question

Students' thinking process while building a concept map. Six students selected from the low, medium, and high achievers' categories (based on assessment scores for the HOTS comprehension test) have demonstrated a pattern in their thinking process while building a concept map through the think-aloud protocol. Table 7 below refers to the conclusions of the verbal unit coding analysis according to categories that have been developed for each respondent. Please refer to Table 9 for further explanation on the verbal coding unit.
Table 7 Percentage of verbal unit according to the category of the think-aloud protocol for each student for concept maps before and after intervention
Respondent Type of concept map assessment Percentage of verbal unit category (%)
Explanation (E) Monitoring (M) Conceptual error (CE) Inappropriate code (IC)
TF – total frequency.
A Before 53.95 22.37 21.05 2.63
After 79.14 20.14 0.00 0.72
B Before 61.36 18.18 13.64 6.82
After 77.52 19.38 2.33 0.78
C Before 52.83 18.87 15.09 13.21
After 76.99 16.81 4.42 1.77
D Before 48.00 20.00 8.00 24.00
.After 73.47 14.29 6.12 6.12
E Before 22.64 13.21 52.83 11.32
After 67.57 18.92 9.46 4.05
F Before 16.67 41.67 25.00 16.67
After 59.76 29.27 7.32 3.66

The table above shows that every student demonstrated a percentage increase in the Explanation (E) category while building a post-concept map as compared to the pre-concept map intervention. Meanwhile, it was observed that there are conflicting results for the Concept Error (CE) and Inappropriate Code (IC) categories. Each student showed a declining percentage for both categories while building a concept map after the intervention as compared to the concept map before the intervention. For the Monitoring (M) category, no pattern was obtained for the percentage of the verbal unit category. However, more detailed explanations of the findings’ subcategories such as students' Explanation (E) and Monitoring (M) can be seen in the total frequencies obtained from the analysis of verbal unit protocols for each student in Table 8 below.

Table 8 Total frequency of explanation (E) and monitoring (M) subcategories of think-aloud protocol pre- and post-concept map interventions of each respondent
Category level achievement student Respondent Type of concept map tasks evaluation Coding category and subcategory
Explanation Monitoring
E.1 E.2 E.3 TF M.1 M.2 M.3 M.4 TF
High A Before 34 7 0 41 14 0 1 2 17
After 62 30 18 110 16 9 0 3 28
B Before 22 4 1 27 5 2 0 1 8
After 66 24 10 100 18 4 0 3 25
Medium C Before 22 6 0 28 7 2 1 0 10
After 56 24 7 87 9 8 0 2 19
D Before 11 1 0 12 1 0 4 0 5
After 49 17 6 72 8 5 0 1 14
Low E Before 7 4 1 12 1 2 3 1 7
After 34 14 2 50 3 8 3 0 14
F Before 3 1 0 4 3 0 7 0 10
After 34 13 2 49 4 11 9 0 24

Based on Table 8, the total frequencies for the explanation (E) and monitoring (M) categories showed an increase in post versus pre-concept maps for almost all students. However, if scrutinised, subcategory M.3 (ineffective reflex) did not show a consistent pattern; instead, it can be seen that students E and F (group of low achievers) show higher number frequencies in the two sessions of the pre- and post-intervention concept map assessment than other students. For subcategory M.4 (reviewing the quality of a product), a pattern can be seen in the increasing frequency of all students for post-concept maps as compared to pre-intervention, except in students E and F. In fact, if scrutinised, students A and B (high achievers) show a number frequency that is much higher for the sub-category in the concept map assessment as compared to other students.

In detail, the number frequency has increased significantly in the Definition (E.1) subcategory while building the post-intervention concept maps, which clearly shows the proliferation of knowledge mastered by students. Examples of verbal communication presented by students are:

(i) While building a post-intervention concept map by high achiever students

…the electrolysis can decompose the electrolyte/this electrolyte consists of ions that are free to move, which allows the electrolyte to conduct electrical current/This electrolyte consists of molten and aqueous/example is copper(II) sulphate solution/electrolysis cannot decompose a non-electrolyte, for example naphthalene/this non-electrolyte cannot conduct electrical current

(Respondent B)

(ii) While building a post-intervention concept map by medium achiever students

…ok…electrolyte is a compound that can conduct electricity in molten or aqueous state/example … mm…a compound that is in the acidic or alkaline state, for example, HCl and NH 3 /a non-electrolyte is a compound that cannot conduct electricity

(Respondent D)

(iii) While building a post-intervention concept map by low achiever students

An electrolyte is any ionic compound that can conduct electrical current in its molten or aqueous form/a non-electrolyte is any ionic compound that cannot conduct electrical current in its molten or aqueous form.

(Respondent F)

The results from the analysis of these verbal examples show that the high achiever students can provide better explanations of the concepts of electrolytes and non-electrolytes as compared to the medium and low achievers. In fact, the low achievers are still unable to accurately explain the concept despite the fact that they have studied it. Misconceptions still occurred when student F assumed that only ionic compounds can be used as electrolytes. In fact, the student could not give an example of an electrolyte and a non-electrolyte while the high achiever students could. This proves that an explanation of the proliferation concept can expose students' in-depth understanding of a concept that has been previously studied.

The potential of concept maps to improve students' conceptual understanding through the proliferation of new concepts and more accurate proposition has been widely accepted by many researchers (Hilbert and Renkl, 2008; Burrows and Moorings, 2015; Yaman and Ayas, 2015). In addition, the tasks in a pre- and post-lab concept map assessment on five different sub-topics of the electrolysis concept have given students the opportunity to acquire a thorough understanding of the basic concepts that were involved. The periodic construction of concept maps is encouraged by many researchers in order to facilitate the development of students' conceptual understanding (Kaya, 2008; Kibar et al., 2013).

Subsequently, mastery of the content of electrolysis subtopics has helped students to compare the concepts involved according to the specifications of the task-focused instructions. This can be seen through the increase in the total frequencies in the E.2 subcategory (compare and differentiate) for all students while building a post-concept map compared to pre-intervention. For example, students should be able to compare different types of validation tests on oxygen and hydrogen gas produced at the anode or the cathode more accurately while building a concept map after the intervention. These findings support the opinions of Novak and Cañas (2008) and Chevron (2014) on the abilities of concept maps to build students' skills to better analyse information.

In addition, it is found that students are able to provide more accurate justifications of a concept or the selection of a relationship that has been learned during the construction of concept maps as compared to pre-intervention. This can be seen through the increase in the number of frequencies in subcategory E.3 (justification) on the post-intervention assessment of concept maps. Previous researchers have suggested that the use of concept maps helps students to develop good assessment skills (McMillan, 2010; Kumaran and Sankar, 2013). These skills can be developed through a process of selecting and determining the appropriate concept and associating it with the original concept involved. Kinchin (2014) also believed that concept mapping activities require good assessment skills and this can be represented by the presence of appropriate linking words in a concept map.

For the Monitoring (M) category, the students exhibited an increased frequency in all subcategories except M.3 while building a post-intervention concept map, which also contributed to the increase in the test scores of students in the HOTS Electrolysis comprehension test. The researchers assume that each mapping activity requires much reflection, which consequently helps to establish higher level cognitive skills. In fact, previous researchers such as Francisco et al. (2002), Popova-Gonci and Lamb (2012), and Cañas et al. (2012) also described the concept map as a reflection tool that is useful for assessing the construction of the cognitive structures of students with respect to concepts learned. Furthermore, the concept mapping activities require higher level reasoning skills and provide extensive opportunities for the development of students' metacognitive skills (Cañas et al., 2012; Kinchin, 2014). Thus, the periodic construction of concept maps is helpful in improving the cognitive skills of students at higher levels.

There was no consistent pattern in the M.3 subcategory for reflective activities that are not effectively exhibited by students. However, if investigated, students A and B (high achievers) show hardly any cognitive activity as a result of this treatment. These reflection activities also indicated that self-reviews by students expressed anxiety regarding building a relationship concept or a less obvious statement. However, this activity does not portray effective monitoring because it does not exhibit any action or strategy that can be used as a solution in determining a relationship. An example of a verbal communication obtained is, “I do not remember how to connect between the two concepts …”

(Respondent F)

These matters indicate that high achievers have mastered the concept of electrolysis better to allow them to provide a more effective solution on reflection activities that occur while developing a relationship between concepts. This is recognised by previous researchers such as Lopez et al. (2011), who argue that students who have mastered a concept properly can clearly and precisely solve a relationship. For clearer details, Table 9 shows explanations for the Explanation and Monitoring subcategories as well as verbal examples presented by students.

Table 9 Description of the explanation (E), and monitoring (M) subcategories as well as examples of verbal communications obtained
Category name Subcategory/code Student verbal example
Explanation Define (E.1)

(Information on options and responses of respondents)

 “Bromide ions are attracted to the anode”
Compare and differentiate (E.2)

(Information consists of a group of concepts or points that show similarities or differences in the response of the students (concept or proposition))

 “Electrodes comprise two types of cathode and anode, but the anode is the positive electrode while the cathode is the negative electrode”
Justifying (E.3)

(Information that gives reasons for students' choices and responses)

 “Molten lead(II) chloride can conduct electrical current because there are free moving ions”
Monitoring Defining or applying a strategy (M.1)

(Information specifying a strategy used during its construction)

 “I need to review the concepts that have been used”
Reflect effectively (M.2)

(Information describing self-reflection by respondents such as questioning the meaning of a word, relationships, and confirming the accuracy of options and responses that have been made)

 “I thought of changing the linking word to be more precise, from being to discharged”
Reflect ineffectively (M.3)

(Information that describes the self-review by students but does not involve any action such as anxiety expressed in developing a relationship concept, but there is no strategy that is used as a solution, or developing a less obvious statement)

 “I do not remember the relationship between these two concepts”
Reviewing the quality of a result (M.4)

(Information representing the intention to fix a relationship that has been created)

 “I want to review my concept maps to ensure all concepts have been used”

What attracted the researchers' attention is that the findings are in line with the findings for the first and second research questions. The higher the concept map and students' HOTS understanding test scores against pre- and post-assessment sessions, the higher the percentage obtained by the student in the Explanation category. These findings can be seen more clearly in a comparison of six students for three evaluation assessments, which are conducted using concept map scores, the think-aloud protocol method, and the HOTS Electrolysis comprehension test (refer to Table 10) below.

Table 10 Comparison of three assessments, which are the concept map score, the think-aloud protocol, and the HOTS Electrolysis comprehension test
Respondent Type of assessment Score concept map Category percentage (explanation) Category percentage conceptual error (CE) Score HOTS test
A Before 27 53.95 21.05 47
After 80 79.14 0.00 92
B Before 24 61.36 13.64 36
After 75 77.52 2.33 86
C Before 13 52.83 15.09 22
After 60 76.99 4.42 78
D Before 11 48.00 8.00 25
After 50 73.47 6.12 64
E Before 4 22.64 52.83 14
After 40 67.57 9.46 53
F Before 3 16.67 25.00 3
After 36 59.76 7.32 50

These findings of this study indicate that an increase in students' achievement scores in the assessment of the concept map and HOTS comprehension test is caused by the improvement in in-depth comprehension of the electrolysis concept.

Although this study was not intended to specifically identify students' misconceptions, misconceptions are closely related to an individual's understanding. For example, a student who exhibits less misconception of certain learned concepts will show a better understanding as compared to students who often showed misconceptions. The relationship between misconceptions and students' understanding was supported by previous studies by Ruiz-Primo et al. (2001) and Van Zele et al. (2004). Burrows and Moorings (2015) found that students who understood a concept better showed less misconception than students with poorer understanding.

This is also reflected in the concept map scores obtained by the students involved. The higher the score obtained for the concept maps, the lower the percentage of misconceptions exhibited by students during the think-aloud protocol. This finding is consistent with research findings presented by Burrows and Moorings (2015), who reported that students who obtained low concept map scores showed more conceptual errors than those who had high concept map scores in the think-aloud protocol interview.

In conclusion, students who have a deep understanding and are able to think at a higher level can demonstrate cognitive processes in content-based explanation and have heightened vigilance while building a concept map (Ruiz-Primo et al., 2001). Thus, as students master good understanding of the concept map, greater content-based explanation can be generated. Hilbert and Renkl (2008) also reported a close relationship between monitoring activities and the importance of producing accurate concept relationships with an increase in the yield of a learning process.

Conclusion

Overall, the main purpose of this study was to identify the extent of the impact of the use of concept maps in stimulating an increase in HOTS among students. Skills enhancement in students is expected to be generated through a deep conceptual understanding, which is obtained from laboratory and educational activities. Although it is believed that laboratory activities can provide opportunities for students to develop a deep understanding and metacognitive skills, it is a fact that the development of these skills is difficult to achieve (Glover et al., 2013; Roberts and Johnson, 2015). Manipulative skills in science are often obtained through these activities, but the development of deep conceptual understanding and enhancement of metacognitive skills in students is rarely produced (Abrahams and Millar, 2008; Hofstein and Kind, 2012; Roberts and Johnson, 2015). Thus, several proposals have been put forward by the researcher to allow new studies to be conducted with an emphasis on continuous assessment (formative) and in particular to assist the development of desired learning skills (Hofstein and Kind, 2012; Fernandez et al., 2013). Additionally, the activities of evaluation and assessment seldom receive attention from researchers in the field of chemistry education research (Miyuko et al., 2014; Teo et al., 2014; Harshman and Yezierski, 2016), while the components of the assessment are critical in ensuring the transformation of education in order to achieve the educational goals of the 21st century.

In addition, based on the researcher's knowledge and reading, studies have been conducted on the use of concept maps in laboratory learning activities, but the role of concept maps as a tool for the assessment of learning activities that take place in the laboratory, that specialise in generating students' HOTS, has not yet been reported. Studies that have been published are much more focused on generating aspects of conceptual understanding in students; hence, it is difficult for the researcher to find studies that specifically focus on the potential of concept maps as an assessment tool for stimulating the development of students' HOTS through laboratory activities. Empirical studies that focus on generating students' HOTS through the use of concept maps are still not widely recorded (Bramwell-Lalor and Rainford, 2014). Furthermore, many more studies have been conducted specifically with respect to their roles as learning and teaching tools, but research needs to be expanded to focus on their role as assessment tools (Burrows and Mooring, 2015; Yaman and Ayas, 2015). This study has provided a better understanding of how concept mapping activities can help develop conceptual understanding and thus generate students' HOTS through the exploration of their thinking processes involved in the creation of concept maps. Many studies have only focused on the impact of using concept maps on learning outcomes; however, based on the researchers’ observations, minimal research has been conducted to explore students' thinking processes while building a concept map. Thus, in realising this limitation, this study was carried out by combining elements of the application of concept maps that are said to be able to provide huge opportunities for generating high levels of thinking in students so that their potential as an alternative assessment tool in laboratory activities could be expanded.

Based on the overall research findings, the researcher would like to highlight that a higher score in the concept map assessment tasks is accompanied by a higher score in the HOTS comprehension test. Both improvements in scores are associated with increased frequencies in students' explanations and monitoring activities while building a concept map. The approach of providing a concept map construction training set, a low-directed concept mapping technique, application of a convergence scoring technique, and the construction of pre- and post-laboratory concept maps is believed by the researcher to be the most appropriate approach for assessment of laboratory learning activities for the development of students' conceptual understanding and HOTS.

In conclusion, this study agreed with the proposal submitted by Bramwell-Lalor and Rainford (2014) that providing training for students to develop concept maps gradually moved from high-directed to low-directed concept mapping techniques. This is intended as an effort to motivate students to develop low-directed concept maps that are difficult to produce. In addition, proposals by previous researchers such as Burrows and Mooring (2015), Kibar et al. (2013), and Özmen et al. (2009) regarding exposing students to the use of pre- and post-laboratory concept maps have also helped to develop their conceptual understanding of the concepts of electrolysis. In addition, the use of low-directed concept maps and convergence scoring has also helped students to gain a deep understanding as expressed by Ruiz-Primo et al. (2001) and Yin et al. (2005). Furthermore, some modifications of the concept map technique as an assessment tool using the convergence scoring method are in line with the proposal published by Yaman and Ayas (2015), which was highlighted in this study. Therefore, this study has identified a combined approach to the low-directed concept mapping technique, training given to students, the convergence scoring method, and periodic use of concept mapping activities as a significant factor in helping students to develop conceptual understanding and generate higher level thinking skills via assessment activities during laboratory learning. The potential of concept maps as an assessment tool for the development of students' HOTS via laboratory activities is expected to positively impact the science education system.

Implications of study

The study has emphasised assessment using an alternative tool, namely concept maps, which can be applied in laboratory learning activities. The use of this assessment tool is expected to contribute to the development of students' conceptual and HOTS in laboratory learning activities. The findings of this study indicate that the use of concept maps as an alternative assessment tool in laboratory activities can help teachers to develop these skills. This study can answer several questions and issues that were raised by teachers associated with the assessment approach, which is a traditional practice in schools. As discussed earlier, many teachers faced the problem of assessing students' learning during laboratory activities (Barnea et al., 2010). Chemistry teachers also lack knowledge of assessment activities that stimulate students to develop higher level thinking skills (Fernandez et al., 2013; Miyuko et al., 2014; Azraai et al., 2015). Assessment activities that have been carried out in the laboratory can develop thinking skills only at lower levels (Zoller and Pushkin, 2007; Phang et al., 2012). Additionally, these findings may also refute the findings by Didis et al. (2014), who reported that many teachers lack confidence in the use of concept maps as an assessment tool that can generate students' HOTS. The concept map construction training sessions for teachers and students that have been used in this study are expected to help teachers to be more prepared to use concept maps as an alternative assessment tool in laboratory learning activities. The application of this visual tool is not commonly used as an assessment practice at the school level. Therefore, the teacher, as a transformer of education, plays an important role in making this venture a success.

Additionally, this study is expected to contribute ideas to the Ministry of Education (MOE) in the search for an alternative assessment approach to emphasise the acquisition of in-depth knowledge to generate students' thinking skills at a higher level. Furthermore, the provision of training and professional courses related to improving the quality of teaching is an essential component of an effective teaching process. The proposal to provide teachers with comprehensive training in using concept maps as an alternative assessment tool was also supported by previous studies such as those by Kinchin (2014) and Didis et al. (2014). Most of the teachers gave negative opinions on the use of concept maps because they had less knowledge on the use of these visual tools (Didis et al., 2014). Thus, this study is expected to trigger ideas for the MOE to implement efforts to continuously improve learning assessment components in order to transform education in the 21st century.

Limitations of the study

The pre-experimental method was employed in the study without a control group. However, future research should focus on the intervention part of the experiment, which could be divided into two parts. Firstly, the intervention could emphasis scientific skills to see how students learn using concept maps. The learning analytics approach can be utilised in analysing the students' learning pattern when using the concept maps to examine their problem-solving skills in learning chemistry subjects. Besides, content analysis can be used to understand patterns in students' learning problems.

Secondly, the researchers are aware of the role of laboratory activities that emphasise the acquisition of manipulative and science processing skills. However, this study only focuses on the assessment of students' conceptual understanding that can be developed via laboratory learning activities. Nevertheless, the researchers believed that high achieving students mastered manipulative and science process skills very well. Abrahams and Millar (2008), Hofstein and Kind (2012), Roberts and Johnson (2015), and Azraai et al. (2015) also believed that manipulative skills and science processes are often acquired from the implementation of laboratory activities in schools; however, conceptual understanding is rarely developed via such activities. Moreover, many of the latest documents on the development of educational systems in countries such as the United States of America and Australia have been concerned with aspects of the change from the acquisition of manipulative skills and processes to the acquisition of conceptual knowledge (Roberts and Johnson, 2015). Hence, the researcher decided to emphasise aspects of conceptual understanding by using assessment.

Conflicts of interest

There are no conflicts to declare.

Appendix 1

Assessment task concept map in laboratory activities


image file: c7rp00120g-u1.tif

image file: c7rp00120g-u2.tif

image file: c7rp00120g-u3.tif

image file: c7rp00120g-u4.tif

image file: c7rp00120g-u5.tif

image file: c7rp00120g-u6.tif

image file: c7rp00120g-u7.tif

image file: c7rp00120g-u8.tif

image file: c7rp00120g-u9.tif

Appendix 2


image file: c7rp00120g-u10.tif

Appendix 3


image file: c7rp00120g-u11.tif

References

  1. Abrahams I. and Millar R., (2008), Does Practical Work Really Work? A study of the effectiveness of practical work as a teaching and learning method in school science, Int. J. Sci. Educ., 30(14), 1945–1969.
  2. Akram M., Johari S. and Ali M., (2014), Conceptual difficulties of secondary school students in electrochemistry, Asian Soc. Sci., 10(19), 276–281.
  3. Anderson L. W., Krathwohl D. R. and Bloom B. S., (2001), A taxonomy for learning, teaching, and assessing: a revision of Bloom's taxonomy of educational objectives, Allyn and Bacon.
  4. Barnea N., Dori Y. J. and Hofstein A., (2010), Development and implementation of inquiry-based and computerized-based laboratories: reforming high school chemistry in Israel, Chem. Educ. Res. Pract., 11(3), 218–228.
  5. Bloom B. S., (1956), Taxonomy of educational objectives. Vol. 1: Cognitive domain, New York: McKay, pp. 20–24.
  6. BouJaoude S. and Attieh M., (2008), The effect of using concept maps as study tools on achievement in chemistry, Eurasia J. Math., Sci. Technol. Educ., 4(3), 233–246.
  7. Bramwell-Lalor S. and Rainford M., (2014), The Effects of Using Concept Mapping for Improving Advanced Level Biology Students' Lower- and Higher-Order Cognitive Skills, Int. J. Sci. Educ., 36(5), 839–864.
  8. Burrows N. L. and Mooring S. R., (2015), Using concept mapping to uncover students' knowledge structures of chemical bonding concepts, Chem. Educ. Res. Pract., 16, 53–66.
  9. Campbell D. T., Stanley J. C. and Gage N. L., (1963), Experimental and quasi-experimental designs for research, Boston: Houghton Mifflin, pp. 171–246.
  10. Cañas A. J., Novak J. D. and Reiska P., (2012), Freedom vs. Restriction of Content and Structure during Concept Mapping – Possibilities and Limitations for Construction and Assessment, Concept Maps: Theory, Methodology, Technology. Proc. of the Fifth Int. Conference on Concept Mapping, 2(1), 247–257.
  11. Celikten O., Ipekcioglu S., Ertepinar H. and Geban O., (2012), The Effect of the Conceptual Change Oriented Instruction through Cooperative Learning on 4th Grade Students' Understanding of Earth and Sky Concepts, Sci. Educ. Int., 23(1), 84–96.
  12. Chevron M. P., (2014), A metacognitive tool: theoretical and operational analysis of skills exercised in structured concept maps, Perspect. Sci., 2(1), 46–54.
  13. Cooper M. M., Grove N., Underwood S. M. and Klymkowsky M. W., (2010), Lost in Lewis structures: an investigation of student difficulties in developing representational competence, J. Chem. Educ., 87(8), 869–874.
  14. Creswell J. W., (2012), Educational Research: Planning, Conducting and Evaluating Quantitative and Qualitative Research, 4th edn, Pearson New International Edition.
  15. Creswell J. W. and Clark V. L. P., (2011), Designing and conducting mixed methods Research, 2nd edn, Thousand Oaks, California: SAGE Publications.
  16. Davies M., (2011), Concept mapping, mind mapping and argument mapping: what are the differences and do they matter? High. Educ., 62(3), 279–301.
  17. Didis N., Özcan Ö. and Azar A., (2014), What do pre-service physics teachers know and think about concept mapping? Eurasia J. Math., Sci. Technol. Educ., 10(2), 77–87.
  18. Ding N. and Harskamp E. G., (2011), Collaboration and Peer Tutoring in Chemistry Laboratory Education, Int. J. Sci. Educ., 33(6), 839–863.
  19. Erdem E., Yılmaz A. and Oskay Ö. Ö., (2009), The effect of concept mapping on meaningful learning of atom and bonding, Procedia, Soc. Behav. Sci., 1(1), 1586–1590.
  20. Fernandez C., Holbrook J., Mamlok-Naaman R. and Coll R. K., (2013), How to teach science in emerging and developing environments, in Teaching chemistry – A studybook, Sense Publishers, pp. 299–326.
  21. Forehand M., (2010), Bloom's taxonomy, Emerging perspectives on learning, teaching, and technology, pp. 41–47.
  22. Francisco J. S., Nakhleh M. B., Nurrenbern S. C. and Miller M. L., (2002), Assessing Student Understanding of General Chemistry with Concept Mapping, J. Chem. Educ., 79(2), 248–257.
  23. Glover S. R., Sewry J. D., Bromley C. L., Davies-Coleman M. T. and Hlengwa A., (2013), The Implementation of a Service-Learning Component in an Organic Chemistry Laboratory Course, J. Chem. Educ., 90(5), 578–583.
  24. Harshman J. and Yezierski E., (2016), Characterizing high school chemistry teachers' use of assessment data via latent class analysis, Chem. Educ. Res. Pract., 17(2), 296–308.
  25. Hay D. B., Tan P. L. and Whaites E., (2010), Non-traditional learners in higher education: comparison of a traditional MCQ examination with concept mapping to assess learning in a dental radiological science course, Assess. Eval. High. Educ., 35(5), 577–595.
  26. Heng L. L., Surif J. and Seng C. H., (2014), Individual Versus Group Argumentation: Student's Performance in a Malaysian Context, Int. Educ. Stud., 7(7), 109.
  27. Hilbert T. S. and Renkl A., (2008), Concept mapping as a follow-up strategy to learning from texts: what characterizes good and poor mappers? Instruct. Sci., 36(1), 53–73.
  28. Hofstein A. and Kind P. M., (2012), Learning in and from science laboratories, in Second international handbook of science education, Springer: Netherlands, pp. 189–207.
  29. Hofstein A. and Lunetta V. N., (2004), The laboratory in science education: foundations for the twenty-first century, Sci. Educ., 88(1), 28–54.
  30. Hofstein A., Navon O., Kipnis M. and Mamlok-Naaman R., (2005), Developing students' ability to ask more and better questions resulting from inquiry-type chemistry laboratories, J. Res. Sci. Teach., 42(7), 791–806.
  31. Ifenthaler D., (2010), Relational, structural, and semantic analysis of graphical representations and concept maps, Educ. Technol. Res. Dev., 58(1), 81–97.
  32. Karakuyu Y., (2010), The effect of concept mapping on attitude and achievement in a physics course, Int. J. Phys. Sci., 5, 724–737.
  33. Kaya O. N., (2008), A Student-centred Approach: Assessing the Changes in Prospective Science Teachers' Conceptual Understanding by Concept Mapping in a General Chemistry Laboratory, Res. Sci. Educ., 38(1), 91–110.
  34. Kibar Z. B., Yaman F. and Ayas A., (2013), Assessing prospective chemistry teachers' understanding of gases through qualitative and quantitative analyses of their concept maps, Chem. Educ. Res. Pract., 14(4), 542–554.
  35. Kim M. and Tan A., (2011), Rethinking Difficulties of Teaching Inquiry-Based Practical Work: Stories from elementary pre-service teachers, Int. J. Sci. Educ., 33(4), 465–486.
  36. Kinchin I. M., (2014), Concept Mapping as a Learning Tool in Higher Education: A Critical Analysis of Recent Reviews, J. Contin. High. Educ., 62(February), 39–49.
  37. Kinchin I. M., Hay D. B. and Adams A., (2000), How a qualitative approach to concept map analysis can be used to aid learning by illustrating patterns of conceptual development, Educ. Res., 42(1), 43–57.
  38. Koh K. H., Tan C. and Ng P. T., (2012), Creating thinking schools through authentic assessment: the case in Singapore, Educ. Assess., Eval. Acc., 24(2), 135–149.
  39. Krathwohl D. R., (2002), A revision of Bloom's taxonomy: an overview, Theor. Pract., 41(4), 212–218.
  40. Kumaran V. S. and Sankar A., (2013), An Automated Assessment of Students' Learning in e-Learning Using Concept Map and Ontology, in Advances in Web-Based Learning – ICWL 2013, pp. 274–283.
  41. Landis J. R. and Koch G. G., (1977), The Measurement of Observer Agreement for Categorical Data, Society, 33(1), 159–174.
  42. Leech N. L. and Barrett K. C., (2011), IBM SPSS for Intermediate Statistics Use and Interpretation, 4th edn, New York: Taylor and Francis Ltd.
  43. Lopez E., Kim J., Nandagopal K., Cardin N., Shavelson R. J. and Penn J. H., (2011), Validating the use of concept-mapping as a diagnostic assessment tool in organic chemistry: implications for teaching, Chem. Educ. Res. Pract., 12(2), 133–141.
  44. Lopez E. J., Shavelson R. J., Nandagopal K., Szu E. and Penn J., (2014), Ethnically diverse students' knowledge structures in first-semester organic chemistry, J. Res. Sci. Teach., 51(6), 741–758.
  45. Lunetta V. N., Hofstein A. and Clough M. P., (2007), Learning and teaching in the school science laboratory: an analysis of research, theory, and practice, in Handbook of research on science education, pp. 393–441.
  46. Luxford C. J. and Bretz S. L., (2014), Development of the bonding representations inventory to identify student misconceptions about covalent and ionic bonding representations, J. Chem. Educ., 91(3), 312–320.
  47. McClure J. R., Sonak B. and Suen H. K., (1999), Concept map assessment of classroom learning: reliability, validity, and logistical practicality, J. Res. Sci. Teach., 36(4), 475–492.
  48. McMillan W. J., (2010), Teaching for clinical reasoning–helping students make the conceptual links, Med. Teach., 32(10), e436–e442.
  49. Miles T., Marianne C. and Jimmy B., (2011), The first language in science class: a quasi-experimental study in late Freanch Immersion, Mod. Lang. J., 95, 182–198.
  50. Ministry of Education Malaysia, (2012), Preliminary Report Malaysia Education Blueprint 2013–2025, Ministry of Education Malaysia, pp. 1–268.
  51. Miyuko M., Tacoshi A. and Fernandez C., (2014), Knowledge of Assessment: an Important Component in the Pck of Chemistry Teachers, Problems of Education in the 20th Century, 62, 124–147.
  52. Norris S. P. and Phillips L. M., (2012), Reading science: how a naive view of reading hinders so much else, in Metacognition in Science Education, Springer: Netherlands, pp. 37–56.
  53. Novak J. D., (2010), Learning, Creating, and Using Knowledge: Concept maps as facilitative tools in schools and corporations, J. E-Learn. Knowl. Soc., 6, 21–30.
  54. Novak J. D. and Cañas A. J., (2008), The theory underlying concept maps and how to construct and use them. Florida Institute for Human and Machine Cognition Pensacola Fl, 284, 16, http://www.ihmc.us. http://cmap.ihmc.us/Publications/ResearchPapers/TheoryCmaps/TheoryUnderlyingConceptMaps.htm.
  55. Othman A., Talib O. and Ibrahim D. A., (2015), Analisis Dokumen Silibus Kimia Organik Matrikulasi Berdasarkan Taksonomi Bloom, Jurnal Kurikulum and Pengajaran Asia Pasifik, Bil, 3(3), 20–31.
  56. Özmen H., DemİrcİoĞlu G. and Coll R. K., (2009), A comparative study of the effects of a concept mapping enhanced laboratory experience on Turkish high school students' understanding of acid–base chemistry, Int. J. Sci. Math. Educ., 7(1), 1–24.
  57. Phang F. A., Abu M. S., Ali M. B. and Salleh S., (2012), Faktor penyumbang kepada kemerosostan pelajar dalam aliran sains: satu analisis sorotan tesis. In eprints utm (p. 17), retrieved from http://www.medc.com.my/medc/seminar_medc/fromCD/pd….
  58. Popova-Gonci V. and Lamb M. C., (2012), Assessment of Integrated Learning: Suggested Application of Concept Mapping to Prior Learning Assessment Practices, J. Contin. High. Educ., 60, 186–191.
  59. Rahayu S., Treagust D. F., Chandrasegaran A. L., Kita M. and Ibnu S., (2011), Assessment of electrochemical concepts: a comparative study involving senior high-school students in Indonesia and Japan, Res. Sci. Technol. Educ., 29(2), 169–188.
  60. Reid N. and Shah I., (2007), The role of laboratory work in university chemistry, Chem. Educ. Res. Pract., 8(2), 172.
  61. Roberts R. and Johnson P., (2015), Understanding the quality of data: a concept map for the thinking behind the doing in scientific practice, Curriculum J., 26, 345–369.
  62. Ruiz-Primo M. A. and Shavelson R. J., (1996), Problems and issues in the use of concept maps in science assessment, J. Res. Sci. Teach., 33(6), 569–600.
  63. Ruiz-Primo M. A., Shavelson R. J. and Schultz S. E., (1997), On the validity of concept map-base assessment interpretations: an experiment testing the assumption of hierarchical concept maps in science, National Center for Research on Evaluation, Standards, and Student Testing (CRESST), Center for the Study of Evaluation (CSE), Graduate School of Education and Information Studies, University of California, Los Angeles.
  64. Ruiz-Primo M. A., Shavelson R. J., Li M. and Schultz S. E., (2001), On the Validity of Cognitive Interpretations of Scores From Alternative Concept-Mapping Techniques, Educ. Assess., 7(2), 99–141.
  65. Sahin A., Ayar M. C. and Adiguzel T., (2013), STEM Related After-School Program Activities and Associated Outcomes on Student Learning, Educ. Sci.: Theor. Pract., 14(1), 309–322.
  66. Schmidt H. J., Marohn A. and Harrison A. G., (2007), Factors that prevent learning in electrochemistry, J. Res. Sci. Teach., 44(2), 258–283.
  67. Seng G. H., (2007), The Effects Of Think-Aloud In A Collaborative Environment To Improve Comprehension Of L2 Texts, Reading Matrix: An International Online Journal, 7(2), 2–45.
  68. Sesen B. A. and Tarhan L., (2010), Promoting active learning in high school chemistry: learning achievement and attitude, Procedia, Soc. Behav. Sci., 2(2), 2625–2630.
  69. Sia D. T., Treagust D. F. and Chandrasegaran A. L., (2012), High School Students' proficiency And Confidence Levels In Displaying Their Understanding Of Basic Electrolysis Concepts, Int. J. Sci. Math. Educ., 10(6), 1325–1345.
  70. Stensvold M. and Wilson J. T., (1992), Using concept map as a tool to apply chemistry concepts to laboratory activities. J. Chem. Educ., 69(3), 230–232.
  71. Stoddart T., Abrams R., Gasper E. and Canaday D., (2000), Concept maps as assessment in science inquiry learning – a report of methodology, Int. J. Sci. Educ., 22(12), 1221–1246.
  72. Teo T. W., Goh M. T. and Yeo L. W., (2014), Chemistry education research trends: 2004–2013, Chem. Educ. Res. Pract., 15(4), 470–487.
  73. Thompson J. and Soyibo K., (2002), Effects of lecture, teacher demonstrations, discussion and practical work on 10th graders' attitudes to chemistry and understanding of electrolysis, Res. Sci. Technol. Educ., 20(1), 25–37.
  74. Vachliotis T., Salta K. and Tzougraki C., (2014), Meaningful Understanding and Systems Thinking in Organic Chemistry: Validating Measurement and Exploring Relationships, Res. Sci. Educ., 44(2), 239–266.
  75. Van den Haak M., De Jong M. and Jan Schellens P., (2003), Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue, Behav. Inf. Technol., 22(5), 339–351.
  76. Van Zele E., Lenaerts J. and Wieme W., (2004), Improving the usefulness of concept maps as a research tool for science education, Int. J. Sci. Educ., 26(9), 1043–1064.
  77. Yaman F. and Ayas A., (2015), Assessing changes in high school students' conceptual understanding through concept maps before and after the computer-based predict–observe–explain (CB-POE) tasks on acid–base chemistry at the secondary level, Chem. Educ. Res. Pract., 16(4), 843–855.
  78. Yin Y., Vanides J., Ruiz-Primo M. A., Ayala C. C. and Shavelson R. J., (2005), Comparison of two concept-mapping techniques: Implications for scoring, interpretation, and use, J. Res. Sci. Teach., 42(2), 166–184.
  79. Yin R. K., (2009), Case study research, design and methods, 4th edn, Los Angeles: Sage.
  80. Zohar A. and Dori Y. J., (2003), Higher order thinking skills and low-achieving students: are they mutually exclusive? J. Learn. Sci., 12(2), 145–181.
  81. Zoller U., (1993), Are lecture and learning compatible? Maybe for LOCS: Unlikely for HOCS, J. Chem. Educ., 70(3), 195.
  82. Zoller U. and Pushkin D. (2007), Matching Higher-Order Cognitive Skills (HOCS) promotion goals with problem-based laboratory practice in a freshman organic chemistry course, Chem. Educ. Res. Pract., 8(2), 153.
This journal is © The Royal Society of Chemistry 2017