Nicolás
Veiga
a,
Florencia
Luzardo
a,
Kenneth
Irving
b,
María N.
Rodríguez-Ayán
c and
Julia
Torres
*a
aÁrea Química Inorgánica, Departamento Estrella Campos, Facultad de Química, Universidad de la República, Montevideo, Uruguay. E-mail: jtorres@fq.edu.uy
bDETEMA, Facultad de Química, Universidad de la República, Montevideo, Uruguay
cUNADEQ, Facultad de Química, Universidad de la República, Montevideo, Uruguay
First published on 24th October 2018
As a fundamental part of their chemical education, first-year undergraduate students are substantially involved in laboratory activities. Despite the specific teaching staff choices on the main laboratory aims, students normally receive a vast amount of information during these activities. Apart from understanding theoretical content, fundamental skills such as manipulation, data collection and interpretation should be developed. In this context, learners could feel overwhelmed since they can only process a few pieces of information at a time. Indeed, our experience at the Universidad de la República (Uruguayan public university) shows that many first-year students are in fact not able to cope with all the information they receive during laboratory activities. As a result, many of them only follow the experimental protocol automatically, without gaining significant knowledge or developing the necessary skills. In this work, we assessed the use of new online interactive pre-laboratory activities implemented for 252 first-year university students enrolled in a 12-module General Chemistry laboratory course. The student choice of interactive versus more traditional material was evaluated together with observed preferences regarding the different interactive tools offered. Besides, an online pre-laboratory discussion forum was also implemented and assessed. Both the interactive material and the discussion forum were chosen freely by the majority of students (61% and 79%, respectively). Interestingly, the choice was to some extent modulated by student previous performance. Interactive pre-laboratory material was more frequently chosen by low previous performance students, whereas pre-laboratory forum was preferentially used by high previous performance students. Finally, the influence of these new materials on student laboratory performance was statistically analyzed. Other personal and academic variables were also taken into account. Interactive material access was positively correlated with the final laboratory marks for medium previous performance learners. On the other hand, for lower previous performance students, the academic discussion between teachers and partners promoted by the online forum was positively correlated with their academic performance.
At the Universidad de la República, Uruguay, our experience shows that many students enter the University with poor previous chemistry knowledge and scarce laboratory skills. General Chemistry II course is located in the second semester, but it is the first chemistry university course including a laboratory module. The main specific teaching goals for laboratory learning in this course are to improve theoretical understanding on chemical reactions and to develop basic practical skills. To enroll General Chemistry II, students must have approved General Chemistry I, a theoretical course on Atomic Structure and Chemical Bond in the first semester. These subjects are very formative but not directly related to General Chemistry II main concepts (chemical reactions, kinetics, thermodynamics, equilibrium, redox processes, etc.). So, most students lack a sound background. Besides, since this is the first chemistry laboratory course they have after a very scarce experience in High School, very few practical skills are already developed by students. In line with that, many students can be overwhelmed during laboratory, only focusing on the immediate practical tasks.
Many reports show that pre-laboratory activities focusing on understanding the laboratory session beforehand positively influence the student information processing and their mental engagement (Agustian and Seery, 2017). Any incoming information during the laboratory class is normally integrated into the learning process more easily if it is related to pre-existing knowledge, avoiding the risk of overloading the students working memory and favoring well-constructed long-term connections between practical activities and associated theory (Bodner et al., 2001). In this scenario, pre-laboratory activities allow better theory understanding and experimental planning. This can increase both learning and performance in the class, highlighting the relevant content over the accessory information and involving greater familiarization of students with the experiment and the objectives of the task (Byrne, 1990; Johnstone, 1997; Johnstone and Letton, 1999; Nicholls, 1999; McKelvy, 2000; Koehler and Orvis, 2003; Tasker et al., 2003). This strategy can be especially relevant for students that have comparatively poor theoretical background, since pre-laboratory work offers a scaffold to support new information, minimizing working memory overload during the class (Schmid and Yeung, 2005).
Before this intervention, pre-laboratory work in General Chemistry II course implied that students must read the theory and the specific protocol beforehand. Then, during the class and after a short expositive guidance including some demonstrations carried out by the instructor, they followed the protocol in order to observe and gather experimental evidence from different phenomena. This is in fact a well-known approach (Olson and Loucks-Horsley, 2000; Rollnick et al., 2001; Hofstein and Mamlok-Naaman, 2007). Of course, this strategy was not effective for all students since many of them just focused their efforts on the particular protocol and could hardly understand the underlying theory or develop the necessary skills including observation, comparison of results and connection to the theoretical background. Indeed, our observations are in line with previous experiences showing that students must be fully engaged in the learning process, and not just following a practical technique, in order to gain significant knowledge and develop laboratory skills (Olson and Loucks-Horsley, 2000; Johnstone and Al-Shuaili, 2001; Hofstein and Mamlok-Naaman, 2007).
The traditional pre-laboratory approach comprises indeed sessions such as short lecture presentations and demonstration of procedures (Abraham et al., 1997). But, pre-laboratory work can also be presented in different innovative forms (Rollnick et al., 2001; Lyle and Robinson, 2002), among which activities based on educational software have proved to be very useful (Limniou et al., 2007; Winberg and Berg, 2007). Nowadays, virtual learning environments (VLEs) offer the possibility of developing materials including high-resolution images, videos or animations, helping students to understand concepts or to familiarize themselves with materials and procedures. Communication technologies increase the effectiveness, versatility and interactivity of pre-laboratory work (Nicholls, 1999). These tools can be used by learners at any time and can be repeated, paused and shared with partners to discuss. Besides, through instructional design, a cognitive theory of multimedia learning that combines visual and verbal formats, very effective multimedia products can be developed (Mayer and Moreno, 2002). Another advantage of VLEs is that student participation can be tracked by teachers to ensure assessment and participation follow-up (Vician and Charlesworth, 2003; Bunce et al., 2006). Furthermore, incorporated digital self-assessment tools can provide instantaneous feedback, encouraging students to check their responses and to monitor their own learning (Black and Wiliam, 1998; Heap et al., 2004). In this way, students have a more active role in the process since the flexibility of the learning environment allows them to prepare themselves for the practical work focusing on their special needs at their own pace (George, 2001; Govindasamy, 2001; Hall et al., 2003; Koehler and Orvis, 2003; Mercer-Chalmers et al., 2004; Santally and Raverdy, 2006; Saleh, 2008; Limniou and Whitehead, 2010).
According to the expectancy-value theory of motivation, student choice, persistence, enthusiasm and performance depend on the expectancy and the value given by students to each achievement task. In turn, both the expectancy and the value are modulated by student personal ability perception, individual goals, self-schema, affective memory and perceived difficulty of the task (Wigfield and Eccles, 2000). So, a careful selection of tasks can lead to the development of a comprehensive curricular platform based on VLEs that can completely change the didactic strategic approach. As a matter of fact, in flip teaching strategy for example, a curricular platform involves and motivates students by using various strategies and tools to engage them in self-directed constructive learning outside the classroom and before face-to-face meeting with teachers in the class (Teo et al., 2014). Previous results show that students develop in that case a better understanding of the theory undergirding the procedures before they perform the practical module, also diminishing the concomitant anxiety (Teo et al., 2014).
In this work we examined new pre-laboratory tools developed for a complete General Chemistry laboratory semester course for undergraduate first-year students in Uruguay. These new materials were produced with the goal of promoting the acquisition of significant content knowledge and enhance the development of laboratory skills along the course, especially for medium and low performance students having the lowest previous content background. To be effective, the implemented tools must be preferred and effectively used by students. In fact, they must engage students in learning, enhancing the impact of the actual undemanding routines (Tiberghien et al., 2001; Montes and Rockley, 2002; McNally, 2006) and promoting the linkage to the theory they learn (Hart et al., 2000; Tiberghien et al., 2001; Séré, 2002). In this sense, exploring the student choices reflecting their preferred materials for pre-laboratory work becomes central. Accordingly, the student choice of offered pre-laboratory tools was assessed, especially comparing the newly implemented interactive material (IM, and the different tools included) with the more traditional material (TM, black and white printable laboratory guide). Furthermore, the student perception on the usefulness of each incorporated tool was also assessed. In addition, the student participation in the created online laboratory discussion forum was also evaluated. Finally, the influence of the implemented tools, as well as other many variables, on the laboratory final marks was also analyzed and discussed. These marks reflect student performance in content knowledge and practical skills. The central research questions are:
• What are student choices concerning materials for pre-laboratory work? Is that choice dependent on the academic background of the student?
• What is the impact of the interactive tools and the participation on the online pre-laboratory forum on final laboratory marks? Is that modulated by personal characteristics? Does that impact depend on the academic background of the student?
All laboratory sessions conditions and instructors (usually one teaching assistant per 20–25 students) were the same for all students. Each week, when students arrive at the 3 hour laboratory class, they have a brief 10 minutes multiple-choice test. These marks account for 33% of the final laboratory mark (FLM) and comprise theoretical content (50%) and experimental procedure (50%) previous knowledge. Then, the instructor in charge gives a ca. 15–20 minutes expositive guidance on the objectives of the work, the laboratory materials to be used, and the experimental tasks involved. This guidance can occasionally include short demonstrations. During the laboratory, the instructors answer practical questions posed by students and also spark the group discussion by posing attention-grabbing questions. However, the students themselves are expected to set up and carry out the experiments, as well as to interpret the data and arrive to final conclusions. Each student may or may not reach the expected final result (chemical product, observations, data, etc.), depending on their in-lab performance, which in turn is also modulated by student skills and pre-laboratory knowledge. The experimental results obtained by students are summarized in a brief final report they must produce after each laboratory. Apart from already mentioned previous knowledge assessment accounting for 33% of FLM, the remaining 67% of FLM comprises the instructor summative evaluation on laboratory skills based on in-lab work and quality of the obtained product. This means assessing quantity and quality of obtained chemical products but also evaluating the data collection and treatment, plot production, arriving to adequate conclusions, etc. In order to assess the reliability and robustness of this subjective evaluation, specific practical goals and precise matching criteria are pre-identified both in general and for each laboratory class. Weekly discussions among instructors promote negotiation and full clarification of the assessment procedure. Besides, ca. 15% of laboratory classes are independently evaluated by 2 instructors. In summary, the instructors evaluate performance and reports in a standardized way throughout all the laboratory groups. The total final laboratory mark (FLM), gathering information both on previous knowledge and laboratory skills, was used as an indicator of student performance during data analysis (range values: 0–20).
Theoretical and practical details of each laboratory exercise were presented on IM in different ways including audio, text, images, videos, external links, interactive fields, etc. Each type of tool was represented inside the IM file by a specific graphic form (icon, colored pin, etc.), which was the same for the 12 practical exercises to help rapid identification. Fig. 1 shows some examples. To enhance active processing within the auditory-verbal and visual-pictorial channels, the delivered materials were produced using a spatial integration of images/videos/schemes and audio/written text (Mayer and Moreno, 2002). Each practical interactive module comprises different sections among which students are allowed to freely navigate back and forwards, while information is given sequentially. The IM sections are the following:
(a) Theoretical background: it is designed to integrate each laboratory theory with the student previous knowledge. The theory can be read in normal text and/or listened to. The audio version can be alternatively downloaded in order to be used in other devices. Some user-activated suggestions of further work or calculations to reinforce some complex issues are embedded in the written text. Furthermore, attention-grabbing information is also included. This is presented as very brief texts and eye-catching images that appear randomly located in the theoretical background section. They contain useful information connecting the practical view of the concepts involved in each laboratory class, including for example industrial or medical applications, real-life curiosities, etc.
(b) Materials, equipment and experimental protocol: this section contains full descriptions aided by interactive photos of each reagent or glass material. They are available as drop-down images that simultaneously show brief texts with the chemical formula or some useful notice on how to use them. Besides, short videos or animations on how to use equipment are also at disposal. The videos focus on practical details such as connecting or using unfamiliar equipment or glassware with simultaneous brief texts emphasizing the most important aspects of the procedures. The protocol is a written experimental technique with the detailed procedure to be carried out. Some complex techniques are especially highlighted in the text. Users can click them to see sequences of images with integrating texts explaining the detailed correct experimental aspects.
(c) Safety section: links to the security sheet of each reagent are included.
(d) Self-assessment: multiple-choice questions including instantaneous feedback are incorporated at the end of the pre-laboratory interactive material. When the answer is selected by the user, automatic feedback containing either an explanation on why one answer is incorrect or more information on the correct answer is delivered instantaneously.
(e) Final report: a printable section is presented with some fields that can be filled previously to the laboratory class. Some interactive suggestions to help with the calculations necessary to fill the final report are also included.
During the first three weeks from the start of term, each delivered interactive material was uploaded on VLE together with a very brief preliminary survey (voluntarily filled by some of them). The objective of this short survey was to rapidly gather student opinions as users, in order to improve the produced modules practically as they were created. For instance, the large attached audio files made the interactive materials difficult to download and handle by some PCs. Consequently, in the final version of IM the audio was linked and not embedded, being now also downloadable as a separate file to be used in other devices. More detailed data collection of student access and use of the materials and further analysis was carried out from the 4th week on (described below).
Alternatively, students were also allowed to use the previously existing traditional materials, TM, which contain exactly the same theoretical background text (without the audio version or the interactive suggestions), the same written protocol (without photos or videos to show the materials and equipment) and the same final report section to be filled by students (not online but manually during the class after printing the material). Even though TM were also available online, they did not include any interactive tools, attention-grabbing information or colored texts. In summary, the traditional materials, represented by TMx (x = 1–12, according to the practical exercises on Table 4 in the Appendix 2), contain identical theoretical content and the same practical protocol and final report texts as the IMx, but the information is delivered in a very different way. So, the main difference between IM and TM is that the latter consists of just a written text that does not promote Mayer's brain integration of visual, verbal and audio channels (Mayer and Moreno, 2002).
Then, in order to evaluate the real use of IM, and not only the fact that students accessed a certain IMx or TMx, a general survey was carried out to all students asking if IM was used (always, very often, seldom or never). Even though this survey was not mandatory, it was completed by all students present during Lab 12 week (218 students). This explanatory variable concerning the use of IM is called interactive material use and denoted as IMU. In order to make IMU a two-category variable, the frequency the students declared to have used the materials was associated with two values: Yes (always, very often) or No (seldom, never). Since 99% of the students for whom IMU = No (39% of the whole sample) declared to have used the TM more frequently than the IM, IMU = yes entails those students that either used only the IM or both IM and TM. Therefore, IMU will properly describe the frequent use of IM, regardless of the fact that students can always resort to TM. The same procedure for making two-category variables was applied to the declared use of videos/images (VU variable) and audios (AU variable) available only in the IM.
Forum discussion participation was also followed by Moodle's tools. This variable is named as laboratory forum access and symbolized LFA. Both active participation in the discussion either asking or answering questions and passive participation just entering the forum to read previous posts are included as a positive value in this categorical variable.
Finally, we incorporated in the survey a section concerning the usefulness of each interactive tool included, in order to evaluate the value students give to it. Statements saying that each tool was useful to study had to be agreed or disagreed (I totally agree, I agree, I don’t agree or disagree, I disagree, I totally disagree). We also included an open answer field to gather the student feedback as to why they believed the different pre-lab tools were useful for them.
Both PCA and PCR methods were chosen because they are widely used to perform a descriptive multivariate analysis, being the methods of choice when the meaningful information (covariance and correlations between the variables, the difference between the samples, etc.) is hidden behind a large amount of data that are too complex to be easily interpreted (Jolliffe, 1982; Abdi and Williams, 2010; Bro and Smilde, 2014). The use of both techniques allowed us to correlate the variations in the FLM to the variations of several predictors (selected variables), while handling particularly well the collinearity. The statistically meaningful correlations and the significant variables were identified using the Martens uncertainty test (Martens and Martens, 2000).
Further statistical analyses were conducted on the samples divided into three thirds, according to their acquired theoretical skills during the previous course General Chemistry I (GChem I). To do so, final marks on GChem I were used to divide the students into the highest, medium and lowest marks, all of them though having approved General Chemistry I course or the General Chemistry I global exam to be able to enroll in the course under study.
IM site access (IMA) is expected to be in line with the already discussed declared use in the survey, IMU. Indeed, a positive correlation was found for IMU and IMAx, even though some dispersion of data was observed especially for lower x. The latter deserves a special description. The initial number of total students accessing both IM and TM (IMA + TMA) was much higher than the total number of students. This was probably caused by the initial curiosity promoted by the intervention. However, as the course went by, both TM and IM access figures went down and stabilized. Finally, by the end of the course, a final slight decrease was observed in both IM and TM access probably due to the fact that at that point (last week) many students had already reached the necessary points to get the minimum required marks, so some of them did not prepare themselves for the last laboratory class. Interestingly, all along the course, IMA was generally slightly higher than TMA (see Appendix 3, Fig. 4), in line with declared use of interactive materials (IMU). This tendency did not show any clear dependence on the type of laboratory or the difficulty of theoretical concepts involved.
Among interactive tools embedded in IM for pre-laboratory work, self-assessment questions were valued as the most useful tools (see Table 1). Indeed 90% of students agreed to their usefulness, according to the fact that since these self-assessment questions prepare the students for the initial multiple-choice test, they are expected to improve the personal success in individual academic goals according to the expectancy-value theory (Wigfield and Eccles, 2000). A similar behavior can be expected for calculation suggestions, which are designed to practice calculations to be done during the laboratory in order to produce the final report. Indeed, calculation suggestions were considered useful by 83% of students. As a matter of fact, in the open answer field of the survey, 29% of the individuals admitted having used the IM more frequently because the pre-lab tools allow them, in advance, to better understand the topics in order to be well prepared to deal with the laboratory tasks. Other preferred tools in terms of usefulness were those based on the spatial combination of images and brief texts, as recommended by Mayer's multimedia work (Mayer and Moreno, 2002). Indeed, drop-down pictures of material and equipment, videos demonstrating experimental procedures or explaining the use of equipment and attention-grabbing texts were valued as useful by a very high percentage of students (86%, 71% and 65%, respectively). In this sense, 38% of the students who answered the survey claimed, in the open answer field, that they used the IM more frequently because they provide a much better user experience, characterized by an entertaining and/or interactive user-friendly design that highlights the relevance and/or application of the concepts. Finally, the safety links, audio versions of theoretical background and editable final report were considered useful by lower percentages (38%, 38% and 39% of students, respectively). The influence of student previous performance on these choices can be seen in Table 1. Following the general trend observed for the interactive materials use in general, with the exception of self-assessment questions having a high influence on student assigned individual goals expectancy-value and considered more useful by the upper previous performance students, all incorporated pre-laboratory activities were considered more useful by lower or medium performance students.
Item | Total % | Previous performance | ||
---|---|---|---|---|
Upper third (%) | Medium third (%) | Lower third (%) | ||
Self-assessment questions | 90.4 | 30.3 | 31.8 | 28.3 |
Drop-down pictures of material and equipment | 86.2 | 26.2 | 29.2 | 30.8 |
Calculation suggestions | 82.7 | 26.5 | 27.6 | 28.6 |
Videos | 70.8 | 21.7 | 24.8 | 24.2 |
Attention-grabbing texts | 65.3 | 21.1 | 22.6 | 21.6 |
Safety links | 38.1 | 12.5 | 11.4 | 14.2 |
Audios | 38.0 | 11.1 | 15.2 | 11.7 |
Editable final report | 29.8 | 8.9 | 10.1 | 10.7 |
With regard to the declared use of linked audiovisual tools, results show that students who declared in the survey to use more frequently the interactive material, also stated to frequently employ the videos/images and audio tools available. For example, 70% of students using frequently IM also used audio or video tools frequently, whereas only 10% of students using TM more frequently did. Notwithstanding, the use of audio and video was less frequent than the use of all the other interactive activities included in the IM. In fact, only 39% of all the surveyed students declared having regularly watched the videos and only 28% declared having habitually listened to the audio. The low percentage obtained for audio version of theoretical background use is in line with the low usefulness value assigned to this tool. Some differences according to previous academic performance appear for the video tools use: 34, 40 and 44% of the students within upper, medium and lower previous General Chemistry I marks declared to have often watched the videos. On the other hand, medium previous performance students chose to listen to the attached audio files containing the undergirding theory more frequently (37%) than those who had performed worse (26%) or better (22%) during General Chemistry I. It is also worth mentioning that among students using the audio, only 5% declared to have used the audio before reading the material whereas most of them listened to it while they were reading (40%) or after having read the material to revisit it before the test (55%).
The laboratory forum was accessed by 79% of students. This is also a high percentage, considering that the participation was voluntary and no credits were given for reading or writing in the forum. In average, 20 to 60% of the students accessed to each discussed topic as the course went by. It is worth mentioning that within students accessing the laboratory forum, 50% accessed to 1 to 5 topics, 26% to 6 to 10 topics and 24% to more topics during the course. However, these figures cannot be directly correlated with real active work on laboratory issues. Besides, forum laboratory access seems to have a different correlation with previous performance in Chemistry. In this intervention in particular, students with higher previous performance made use of this new tool more actively: 85, 77 and 75% accessed the forum within upper, medium and lower previous General Chemistry I marks. Of course, the uncertainty of those values are unknown, making it is impossible to know whether the correlation is statistically significant. Finally, it is worth mentioning that most questions are posted by students in the forum. Consequently, the level of discussion is defined at least partly by the students, giving the chance to the high performance students to enhance the individual goals expectancy for the activity (Wigfield and Eccles, 2000).
Predictors | Designation | Variable type | Range |
---|---|---|---|
Definition variables | |||
University matriculation year | UMY | Continuous | 2010–2015 |
Gender | G | Category | M/F |
Number of approved courses | NAC | Continuous | 2–29 |
Number of failed courses | NFC | Continuous | 0–15 |
Previous General Chemistry I final marks | GChem I | Continuous (out of 60) | 8–54 |
Explanatory variables | |||
Interactive materials real access | IMAx | Category | Yes/No |
Traditional materials real access | TMAx | Category | Yes/No |
Laboratory forum real access | LFA | Category | Yes/No |
Interactive materials use (survey) | IMU | Category | Yes/No |
Use of videos/image (survey) | VU | Category | Yes/No |
Use of audio (survey) | AU | Category | Yes/No |
Final total marks of the laboratory course (FLM) were employed as a performance evaluation output (response variable). As already explained, the final marks comprise both pre-lab knowledge assessments (33%) and in-lab performance (67%). The latter is mainly modulated by laboratory skills acquisition, but also by pre-lab knowledge that could promote students success preventing cognitive overload according to Johnstone's theory framework (Johnstone et al., 1994; Johnstone, 1997). In this regard, FLM can be a composite measure. An interesting correlation was found when we separated the final laboratory mark in its components. Pre-lab knowledge (evaluated by marks on multiple-choice test) and in-lab performance (assessed by the instructors during laboratory) showed a positive correlation in PCA co-linearity test (correlation coefficient was 0.773), indicating that pre-laboratory learning has a marked correlation with the in-lab developed skills. This might be related to the predicted decrease in cognitive overload caused by the higher previous knowledge demonstrated by the higher multiple-choice test marks (Johnstone et al., 1994; Johnstone, 1997). Of course, the in-lab performance can also be modulated by the instructor actions. However, the instructor influence is expected to be relatively small, since their role is highly standardized throughout the groups and only entails giving a general guidance throughout the work (by an initial expositive 15–20 minutes talk and occasional individual surveillance, making questions or giving practical advices).
The PCR results are shown in Fig. 2 and Table 3. The optimum number of principal components is 6, explaining 38% of the total FLM variance (r2 = 0.38). The standardized coefficients for the selected predictors are also listed in Table 3. According to the positive values of the correlation coefficients, the University matriculation year (UMY) and the General Chemistry I marks are positively correlated with FLM. On the contrary, the number of failed courses (NFC) shows a negative correlation with FLM (bearing negative correlation coefficients). These trends can also be derived from the loadings plot (Fig. 2a), where it can be seen that UMY, GChem I and FLM are located in the same region of the correlation loadings plot. This means that they are described by similar values of the first three principal components. Conversely, NFC (or NAC) and FLM are located far apart in the diagram. Consequently, as it is expected, those students who recently enrolled at the University and had the best previous academic performance during the first semester tend to earn the highest final marks in the laboratory course.
All students | Highest GChem I marks | Medium GChem I marks | Lowest GChem I marks | |
---|---|---|---|---|
a x stands for the number of each one of the 12 interactive material associated with the 12 practical exercises described on Table 4 (Appendix 2). b Statistical significance of 95% according to Martens’ uncertainty test. | ||||
Number of students | 252 | 84 | 84 | 84 |
Number of PC | 6 | 3 | 1 | 8 |
Explained variance | 38% | 21% | 23% | 45% |
r 2 | 0.38 | 0.27 | 0.28 | 0.53 |
Variables | Standardized coefficients | |||
Gender | 0.474b | 0.026 | −0.060 | −0.197 |
University matriculation year | 0.437b | 0.274b | 0.031 | 0.467b |
Number of approved courses | −0.230 | 0.141 | −0.012 | −0.303 |
Number of failed courses | −0.358b | −0.290b | −0.024 | −0.334 |
Previous General Chemistry I final marks | 0.619b | 0.212b | −0.014 | 0.089 |
Interactive materials use (survey) | 0.084 | 0.032 | 0.171b | −0.073 |
Use of videos/image (survey) | −0.038 | 0.038 | 0.131b | −0.048 |
Use of audio (survey) | −0.218 | −0.037 | 0.087b | 0.023 |
Laboratory forum access | 0.328 | 0.050 | 0.099b | 0.894b |
x | All students | Highest GChem I marks | Medium GChem I marks | Lowest GChem I marks | ||||
---|---|---|---|---|---|---|---|---|
IMA | TMA | IMA | TMA | IMA | TMA | IMA | TMA | |
4 | 0.278 | 0.163 | −0.002 | −0.018 | 0.158b | 0.154b | 0.405 | −0.007 |
5 | 0.175 | 0.036 | 0.045 | −0.050 | 0.165b | 0.118b | 0.121 | 0.035 |
6 | 0.272b | 0.124 | −0.063 | −0.052 | 0.162b | 0.131b | 0.574 | 0.295 |
7 | 0.223 | 0.204b | −0.019 | 0.018 | 0.151b | 0.094b | 0.610b | 0.432b |
8 | 0.227b | 0.267b | 0.009 | 0.058 | 0.127b | 0.101b | 0.240 | 0.304 |
9 | 0.137 | 0.159b | 0.101b | 0.115b | 0.168b | 0.105b | 0.253 | 0.366 |
10 | 0.149 | 0.052 | 0.078 | 0.056 | 0.179b | 0.114b | 0.257 | 0.308 |
11 | 0.114 | 0.138 | 0.133b | 0.004 | 0.184b | 0.112b | 0.145 | −0.057 |
12 | 0.083 | 0.076 | 0.141b | 0.058 | 0.157b | 0.133b | −0.237 | 0.200 |
Class | Description of the experimental activity | Laboratory main skills to be developed |
---|---|---|
1 | Synthesis of potassium nitrate. Chemical reactions. | General laboratory knowledge. Weighing, measuring volumes, vacuum filtration. Qualitative analysis. |
2 | Solubility determination at various temperatures. Identification of a sample. | Heating, measuring variables, plotting. Qualitative analysis. |
3 | Preparation and titration of aqueous solutions. | Manipulation of volumetric glassware. Quantitative analysis. Data treatment. Significant figures. |
4 | Synthesis of alum. | Working in the hood. Safety manipulation of strong acids and bases. Calculation of yield. |
5 | Synthesis of aspirin. | Microscale manipulation including filtration in a syringe. Drying with solvents. |
6 | General equilibrium observation. | Working with gases. Use of solvents. |
7 | Kinetics. | Complex interpretation of data. Use of a spectrometer. |
8 | Acid–base equilibrium. | Potentiometric titration. Analysis of data. |
9 | Solubility equilibrium. | Determination of solubility product. Quali- and quantitative interpretation of data. |
10 | Thermochemistry. | Building and use of calorimeter. Analysis of results in comparison to tabulated data. |
11 | Galvanic cells. | Electrochemical cells construction. Use of different electrodes and voltmeter. |
12 | Electrodeposition and redox systems. | Electrolytic cell construction. Current efficiency calculation. Interpretation of a complex redox system. |
Final laboratory marks are also positively correlated with the student gender (see Table 3). That trend means that the FLM are, in average, higher for male students (who account for 30% of total students). This influence seems to be accounted for the third PC (PC3, Fig. 2a) for which G and FLM have positive PC3 values. However, the correlation should not be that strong, since PC3 only explains 8% of the total FLM variance. Indeed, male student average marks are slightly higher than the average obtained by female students (13.43 ± 0.82 and 12.84 ± 0.53, respectively).
The average influence of other predictors concerning the use of implemented tools on FLM can be analyzed by further looking at Fig. 2a and Table 3. Even though it is not statistically significant, the laboratory forum access (LFA) seems to have a positive correlation with FLM, both sharing positive PC1 values. Therefore, students who accessed the laboratory forum tend to obtain higher final marks. This can also be appreciated in the scores plot of Fig. 2b, where those students who have accessed the online forum (in red) appear towards the area with principal components values close to those for FLM (positive PC1 and small PC2 values). Laboratory forum access promotes discussion and sharing with partners and teachers. Prior knowledge helps students making connections promoting significant learning. So, these findings are in agreement with previous reports (Tobias, 1994; Zuhrieh, 2009).
In addition, the final laboratory mark appears very near to the avowed use of the interactive tools (IMU, VU and AU) in the correlation leadings plot, suggesting a positive correlation among them (see Fig. 2a). Indeed, the scores plots depicted in Fig. 3a and b show the distribution of students according to the interactive material use, IMU, or the video tools use, VU. From them it can be inferred that students who prepared their experimental activities using these interactive pre-laboratory tools (in red) obtained, in average, higher laboratory marks than those who did not (in blue).
Fig. 4 Number of students accessing the interactive or traditional materials along the course (IMA and TMA variables, respectively). |
Lastly, real access to the interactive pre-laboratory materials (IMAx) is also positively correlated to FLM (Fig. 2a). Of course, a positive correlation is also found for the access to the traditional materials (TMAx), though to a lower extent (TMAx are farther apart from FLM in Fig. 2a and show, in general, smaller standardized coefficients in Table 3). A more detailed study on the influence of individual modules (for laboratory exercises 4–12) is presented in the Appendix 5 (see Fig. 6). In general terms, despite some initial dispersion due to the fact that students accessed both types of materials at the beginning of the intervention, as the laboratory course went by, the access to IM became especially frequent for those students with higher final laboratory marks, whereas this was not observed for TM access.
Fig. 6 Scores plotted for the first two PCs of the statistical model. The access to the individual module is depicted with different colors: red (access), blue (no access). |
As it happened to LFA, IMU, VU and AU, even though a correlation was evidenced in the scores plot for IMAx and FLM, this was not statistically significant (see Table 3). So, a more detailed study on this issue was carried out, considering the previous performance of students (see the next section).
Then, for students having obtained the lowest marks in General Chemistry I course (though having approved this course or a global exam), a similar high correlation is observed in the case of the date of matriculation to the university, as expected. Besides, there is no general trend for the influence of IM or TM (either evaluated from access or declared use in the survey) on laboratory final marks. However, for this group of students, which are expected to have the lowest performance based on their previous marks and also bear the lowest use frequency (75%), the impact of the laboratory forum access is statistically significant (95% confidence level), being very positive according to its high standardized coefficient (Table 3). This is probably due to the fact that the previous discussion with teachers and partners on the laboratory subjects once they have accessed either material especially helps this set of students to diminish the gap between their previous knowledge and all the information they must absorb and use during the laboratory exercise.
Finally, the results for students with medium General Chemistry I marks are very interesting. The use of IM, audiovisual tools and laboratory forum access are positively correlated to final laboratory marks, within 95% confidence. Furthermore, the individual access to the different modules IMx is also highly correlated to final performance. The influence of online access to IM is in all cases higher than the influence of access to TM.
The main limitations of the study are the limited size of the sample which also shows some inhomogeneity and the fact that students were allowed (and not randomly forced) to choose among different options when preparing the laboratory. Another point to be noticed is that the correlation data is based on students self-report of use of the pre-laboratory materials and not their actual use. Notwithstanding, the declared use and the real electronic access are highly correlated. Since students were not forced or qualified for entering the sites, the observed access is probably a good estimation of real use. In any case, for all tools access measurements, the correlation with real active brains-on activity is by no means straightforward. Finally, another limitation concerning instructors can arise. Even though in the present intervention teaching assistants only gave a similar standardized initial guidance and students were expected to work on their own during laboratory, some inevitable differences arise from different people working as instructors for the different groups.
Proposed predictors | Designation | Variable type | Range |
---|---|---|---|
Definition variables | |||
Birth year | BY | Continuous | 1985–1997 |
High School graduation year | HGY | Continuous | 2004–2015 |
University matriculation year | UMY | Continuous | 2010–2015 |
Gender | G | Category | M/F |
Number of approved courses | NAC | Continuous | 2–29 |
Number of failed courses | NFC | Continuous | 0–15 |
Number of approved exams | NAE | Continuous | 0–22 |
Number of failed exams | NFE | Continuous | 0–14 |
Previous General Chemistry I final marks | GChem I | Continuous (out of 60) | 8–54 |
Mathematics I course approbation | Maths I | Category | Pass/Fail |
Mathematics I exam approbation | Maths IE | Category | Pass/Fail |
Explanatory variables | |||
Interactive materials real access | IMAx | Category | Yes/No |
Traditional materials real access | TMAx | Category | Yes/No |
Laboratory forum real access | LFA | Category | Yes/No |
Interactive materials use (survey) | IMU | Category | Yes/No |
Use of videos/image (survey) | VU | Category | Yes/No |
Use of audio (survey) | AU | Category | Yes/No |
Only the independent predictors were included in the final X variable set. As a first approximation, the collinearity among the continuous variables was assessed and the results are listed in Table 6. It can be seen that BY, HGY and UMY are positively correlated. Besides, NFE is negatively correlated with UMY, as expected. Therefore, we employed UMY as the independent variable representing this subset of definition variables. Similarly, since NAC and NAE are highly correlated, we used only NAC as predictor of the student previous academic performance. Finally, General Chemistry I final marks and NFC were included in the statistical analysis, for they are linearly independent of the other selected variables.
BY | HGY | UMY | NAC | NFC | NAE | NFE | GChem I | |
---|---|---|---|---|---|---|---|---|
BY | 1 | 0.71 | 0.581 | −0.18 | −0.489 | −0.105 | −0.312 | 0.12 |
HGY | 0.71 | 1 | 0.656 | −0.268 | −0.56 | −0.206 | −0.437 | 0.157 |
UMY | 0.581 | 0.656 | 1 | −0.504 | −0.853 | −0.337 | −0.647 | 0.276 |
NAC | −0.18 | −0.268 | −0.504 | 1 | 0.164 | 0.839 | 0.215 | −0.0489 |
NFC | −0.489 | −0.56 | −0.853 | 0.164 | 1 | 0.0492 | 0.642 | −0.251 |
NAE | −0.105 | −0.206 | −0.337 | 0.839 | 0.0492 | 1 | −0.0324 | 0.268 |
NFE | −0.312 | −0.437 | −0.647 | 0.215 | 0.642 | −0.0324 | 1 | −0.342 |
GChem I | 0.12 | 0.157 | 0.276 | −0.0489 | −0.251 | 0.268 | −0.342 | 1 |
The whole set of proposed predictors (including continuous variables, Table 6) were analyzed using PCA in the conditions described in the Methodology section. The results indicate that the optimum model has 11 PC, explaining 53% of the data variation. The correlation loadings (depicted in Fig. 5) support the above pre-selection made for the continuous variables. In addition, it is possible to identify the independent category variables to include in the model. They are G, IMU, VU, AU and LFA. Finally, IMAx and TMAx descriptors are also included, even though they show a correlation with IMU, since they encompass significant information on the student access and use of interactive and traditional materials, and the PCR method is able to reasonably deal with this type of collinearity.
Overall, Table 7 shows the final set of independent predictors employed throughout the statistical analysis.
Predictors | Designation |
---|---|
Definition variables | |
University matriculation year | UMY |
Gender | G |
Number of approved courses | NAC |
Number of failed courses | NFC |
Previous General Chemistry I final marks | GChem I |
Explanatory variables | |
Interactive materials real access | IMAx |
Traditional materials real access | TMAx |
Laboratory forum real access | LFA |
Interactive materials use (survey) | IMU |
Use of videos/image (survey) | VU |
Use of audio (survey) | AU |
This journal is © The Royal Society of Chemistry 2019 |