Stephen R.
George-Williams
*,
Jue T.
Soo
,
Angela L.
Ziebell
,
Christopher D.
Thompson
and
Tina L.
Overton
School of Chemistry, Monash University, Victoria, 3800, Australia. E-mail: Stephen.george@monash.edu
First published on 2nd March 2018
Many examples exist in the chemical education literature of individual experiments, whole courses or even entire year levels that have been completely renewed under the tenets of context-based, inquiry-based or problem-based learning. The benefits of these changes are well documented and include higher student engagement, broader skill development and better perceived preparation for the workforce. However, no examples appear to have been reported in which an entire school's teaching laboratory programme has been significantly redesigned with these concepts in mind. Transforming Laboratory Learning (TLL) is a programme at Monash University that sought to incorporate industry inspired context-based, inquiry-based and problem-based learning into all the laboratory components of the School of Chemistry. One of the ways in which the effect of the programme was evaluated was through the use of an exit survey delivered to students at the completion of seven experiments that existed before the TLL programme as well as seven that were generated directly by the TLL programme. The survey consisted of 27 closed questions alongside three open questions. Overall, students found the new experiments more challenging but recognised that they were more contextualised and that they allowed students to make decisions. The students noted the lack of detailed guidance in the new laboratory manuals but raised the challenge, context and opportunity to undertake experimental design as reasons for enjoying the new experiments. Students' perceptions of their skill development shifted to reflect skills associated with experimental design when undertaking the more investigation driven experiments. These results are consistent with other literature and indicate the large scale potential success of the TLL programme, which is potentially developing graduates who are better prepared for the modern workforce.
Several studies have suggested that chemistry graduates lack (or are unable to articulate) many transferable skills that are desired by employers, such as time management, independent learning and team-working (Hanson and Overton, 2010; Sarkar et al., 2016). Even students who continue into research positions have been found to be lacking an appreciation of scientific methodology or experimental design as ‘virtually no attention is given to the planning of the investigation or to interpreting the results’ (Domin, 1999).
The skills agenda has gained prominence within Australia and is well exemplified by the 2016 governmental report (Norton and Cakitaki, 2016) which found that many undergraduates struggled to find work within four months of graduation, with science graduates faring less well than arts graduates. Monash University offers many internal programmes designed to enhance the employability of undergraduate students either through attempts to broaden skill development or work placements. However, until recently these have been largely extracurricular.
The TLL programme sought to enable undergraduate students to develop the skills they needed to obtain employment through a redesigned laboratory programme. In common with many other institutions, the original Monash University laboratory programme relied heavily on traditional expository (or recipe-based) experiments. These types of laboratory activities (i.e. heavily prescriptive ones) are generally utilised to consume minimal resources whether these be time, space, equipment, or personnel (Lagowski, 1990). Whilst consumption is a major issue, it has been noted that students will often proceed through these experiences with little to no thought about the reasoning behind the procedures (Woolfolk, 2005). Furthermore, these experiences achieve little in the way of developing a wider range of transferable skills and usually lack any real-world context (Johnstone and Al-Shuaili, 2001; McDonnell et al., 2007). To address this, a variety of different delivery methods have been attempted.
It should be noted that these same outcomes, i.e. high engagement with real industry inspired examples, could be achieved through the use of industrial placements or work-integrated learning (Cooper et al., 2010). However, when cohort sizes are very large industrial placements are simply not a practical means to achieve this contextualisation. Hence, the inclusion of some work-integrated learning into the undergraduate teaching laboratories may bridge this gap.
‘Inquiry lab students valued more authentic science exposure but acknowledged that experiencing the complexity and frustrations faced by practicing scientists was challenging, and may explain the widespread reported student resistance to inquiry curricula’. (Gormally et al., 2009)
Through this survey, the effect of the TLL programme on a range of areas could be monitored. This survey was used in order to measure whether the programme was truly better at preparing the students for the workforce by creating a more engaging, industry-focused laboratory programme that allowed students to develop a wider range of transferable skills. This survey was designed to measure:
(1) The level of inquiry and contextualisation of the experiments, as noted by the students.
(2) The reported underlying motivations of the students.
(3) The overall perception of learning.
(4) Student perception of the skills developed in a given experiment.
(5) Student identified issues associated with the experiments.
(6) The level of (and reason for) the student enjoyment of the experiments.
Alongside the inclusion of industrially relevant experiments, an increase in the level of inquiry was achieved by removing excessive guidance in the laboratory manual (which was replaced by prompting questions or multi-directional flowcharts), obscuring the experimental outcome that the students might achieve or through simply instructing the students to devise their own means by which to complete the experiment. All experiments generated from the programme were designed with the principles of problem-based learning in mind, either by providing a context or through ensuring at least a minimal level of inquiry.
A survey more suited to our purpose was found in the work of Russel (2008), which contained closed questions designed to monitor the effect of the inclusion of more research-based experiments in their undergraduate teaching laboratories. Many questions overlapped with the aims of TLL, especially those around context, the level of guidance within the lab manual or the underlying motivation or engagement of the students. This survey had been validated and was considered an ideal starting point for the final survey used in this study.
The Russel (2008) survey was modified over several iterations. The formatting was changed to be consistent with other studies being undertaken and any mention of ‘chemistry course’ was changed to ‘lab’. Three questions were removed as they were considered to unlikely to be altered by a single laboratory experience (e.g. This lab experience has made me more interested in earning a Doctoral degree (PhD) or Master degree in a science field). Eight items were altered to their negative version (e.g. ‘This lab experience made me learn’ became ‘This lab experience did not make me learn’) to avoid students agreeing to every item, i.e. to avoid acquiesce bias (Watson, 1992). Six new closed questions were also added to capture the students’ perceptions of the ease, challenge, contextualisation, openness or level of interest in the experiments. Finally, three open questions were added to further probe the students’ perceptions. These questions were ‘What skills did you develop throughout today's experience?’, ‘Was there anything that could be improved about today's lab?’ and ‘Did you enjoy today's lab? Why/why not?’ The final version of the survey consisted of 27 closed questions, one distractor question (‘Please select agree and disagree to this question’) and three open questions.
The number of students varied between the courses, from approximately 100 to 1200+. In many cases where the student cohort was large, only a subset of the students were surveyed. For example, for the analysis of a new experiment in first year that ran twice a day for five days of the week, students were invited to complete the survey at the completion of the experiment on Tuesday and Thursday morning. Hence, not all students were surveyed but the sample was large and representative of the cohort.
Experiment name | Number of responses | % of students | |
---|---|---|---|
Pre TLL | Rearrangement | 51 | 49 |
EAS | 40 | 38 | |
Anthracene oxidation | 91 | 76 | |
Macrocycles | 37 | 46 | |
Isomerisation | 49 | 61 | |
Proteins (2016) | 70 | 58 | |
Panacetin | 184 | 74 | |
Post TLL | Proteins (2017) | 138 | 69 |
Sunscreen | 51 | 49 | |
Pseudenol | 65 | 63 | |
Electronic waste | 160 | 59 | |
Nylon | 69 | 58 | |
Food project | 97 | 49 | |
Enzymes | 28 | 47 |
Experiment name | Year level – course focus | Method type | Context (real life or industry) | Scientific content | Additional notes | |
---|---|---|---|---|---|---|
Pre TLL | Rearrangement | 2nd – Inorganic and organic chemistry | Expository | None noted | Carbocation rearrangement | One four hour session. Historical methods used (e.g. hydrazone wet test). Students worked in pairs. |
EAS | 2nd – Inorganic and organic chemistry | Expository | None noted | Electrophilic aromatic substitution | One four hour session. Typically completed within 2 hours. Students worked in pairs. | |
Anthracene oxidation | 3rd – Medicinal chemistry | Expository | Enzyme mimics | Oxidation using vanadium catalysts | One four hour session. Contains a long wait time of 2 hours. Underutilised context. Students worked in pairs. | |
Macrocycles | 3rd – Advanced inorganic chemistry | Expository (mimics literature) | None noted | Synthesis of macrocyclic cage complexes | One four hour session. Required students to obtain method from literature sources. Students worked in pairs. | |
Isomerisation | 3rd – Advanced inorganic chemistry | Expository | None noted | Kinetics of ligand isomerisation | One four hour session. Utilises kinetics/physical chemistry in student perceived synthetically focused course. Students worked in pairs. | |
Proteins (2016) | 2nd – Food chemistry | Expository | Food proteins | Protein detection and measurement | One four hour session. Context limited by use of non-commercially available defatted soy protein. Students worked in groups of 4. | |
Panacetin | 2nd – Inorganic and organic chemistry | Expository | Black market pharmaceuticals | Solute/solvent portioning | One four hour session. Typically completed within 2 hours (out of the 4 assigned). Students worked in pairs. | |
Post TLL | Proteins (2017) | 2nd – Food chemistry | Expository | Food proteins | Protein detection and measurement | One four hour session. Commercially available milks used. Required students to obtain method from literature sources. Students worked in groups of 4. |
Sunscreen | 2nd – Inorganic and organic chemistry | Expository (mimics literature) | Sunscreens and UV-active materials | Aldol condensation | One four hour session. Traditional synthesis with students aiming to make the best sunscreen. Students worked in groups of 3. | |
Pseudenol (previously Panacetin) | 2nd – Inorganic and organic chemistry | Flowchart/student directed | Black market pharmaceuticals | Solute/solvent portioning | One four hour session. Non-stepwise method, prompting questions used. Context strengthened. Students worked in pairs. | |
Electronic waste | 1st – Introductory chemistry | Flowchart/student directed | Metal wastes from electronic goods | Transition metals, complexes and colour | One three hour session. Non-stepwise method, students follow multi-directional flowchart. Students worked individually. | |
Nylon | 3rd – Materials chemistry | Inquiry/investigation | Production of Nylon | Step-growth polymerisation | One four hour session. Very simple method. Students allowed to investigate anything available. Students worked individually. | |
Food project | 2nd – Food chemistry | Inquiry/investigation | Nutritional components of foods | Methods used for non-ideal food samples. | 2 four hour sessions, 1 week apart. Methods utilised in prior experiments of same unit. Students worked in groups of 4. | |
Enzymes | 2nd – Biological chemistry | Expository then Inquiry/investigation | Commercially available digestive supplements | Enzyme degradation of complex sugars | 2 four hour sessions, 1 week apart. First week traditional method followed by inquiry-based second week. Students worked in pairs. |
For the quantitative analysis, the responses were combined into two major groups, the largely expository, unchanged experiments (Pre-TLL) and those generated or revised during the TLL programme (Post-TLL). Significant differences between student responses to each of the 27 closed questions where measured through the Wilcoxon Signed Rank test, which presumed dependence between students responding to both the Pre-TLL and Post-TLL surveys. For questions that showed a statistical difference to a 95% confidence interval, the resulting Z values (obtained from SPSS) were divided by the square root of the respective sample size of respondents to generate a measure of the effect size (r). The cut-offs for the ‘size’ of the effects were determined through the work of Hattie (2008), which was later extended (Fritz et al., 2012; Lenhard and Lenhard, 2016) to r values. The ranges were defined by:
(1) 0 ≤ r ≤ 0.1. ‘Student effect size’. This refers to the natural variation in any group of students. For example, a more motivated student may respond more positively than a less motivated student.
(2) 0.1 < r ≤ 0.2. ‘Teacher effect size’. This refers to the effect of a particularly motivated teacher over the course of a single year (i.e. this effect size could be achieved given time/motivation).
(3) r > 0.2. ‘Zone of desired effect’. This refers to interventions that have an immediate impact and are where educators should typically focus their efforts.
The qualitative data was analysed for emerging themes in an inductive manner. Themes were generated through several rereads of the responses during which recurring themes were identified. These themes were studied in order to identify any redundancies and each theme was given a code. Themes extracted from the data and their codes were given to two other researchers who attempted to code a portion of the qualitative data for each of the three open questions. If needed, themes and codes were refined and these final themes were then used to recode the rest of the responses. This data was then expressed as a percentage of participants who raised a particular theme.
This can be further considered through comparison to the work of Barlett et al. (2001) in which they provided acceptable minimal sample sizes for varying populations. However, this is complicated by the fact that the researchers only provided acceptable sample sizes for either continuous or categorical data, whilst this study utilises ordinal data. Even presuming the lower acceptable sample sizes for the continuous data set apply for an alpha of 0.05 (i.e. a 5% error), six of the seven sample sizes are considered below the required levels. This is simply due to the smaller cohorts enrolled in those courses. However, when the datasets are combined into Pre-TLL and Post-TLL, the number of responses (522 and 608 respectively) are considerably above the required number (106). Hence, whilst some individual experiments may not be fully represented through this analysis, the overall Pre-TLL vs. Post-TLL comparison has sufficient statistical power.
Additionally, as students may have filled out the survey on multiple occasions, it is possible that they simply became accustomed to the survey and simply answered the questions in a repetitive manner. The inclusion of negative stems and a distractor question hopefully forced students to stop and reread questions but this cannot be confirmed at this time.
Another potential limitation was the experiments that were chosen to represent the Pre-TLL dataset. These experiments were selected for practical concerns with timetabling and the delivery of other questionnaires not related to this study. Hence, some experiments that did not form part of this study may have affected the results obtained (i.e. if all teaching experiments in all chemistry courses were surveyed, different results may have been obtained).
It is also possible that the previous background of the students, in terms of encounters with other teaching laboratories could influence their responses. This cannot be completely discounted but, during the first year at Monash University, all students undertake multiple inquiry-based laboratories (called IDEA experiments) during semester one and two. Consequently, all students in higher years have at least some experience with inquiry experiments which would potentially mitigate this issue.
Beyond the performance of a Cronbach's alpha test, no further measurements of reliability or validity were performed. Hence, there may be issues regarding the validity or reliability of the instrument. However, the inclusion of qualitative data through the open questions and sourcing the bulk of the survey from a pre-validated and thoroughly tested source was believed to be sufficient to counter this concern.
Finally, the method of interpretation could be a source of error. However, it is believed that the iterative nature of the theme generation negated this issue to a significant degree, particularly through the use of multiple coders.
The Wilcoxon Signed Rank test showed that 18 of the 27 closed questions were answered in a significantly different way. All questions alongside the p, Z and r values are shown in Table 3. Of this, only eight showed an effect size within the ‘zone of desired effect’ (Hattie, 2008; Lenhard and Lenhard, 2016). Fig. 1 shows the responses to these eight questions.
Question | N | p | Z | r |
---|---|---|---|---|
This lab experience was worthwhile. | 680 | 0.95 | −0.1 | — |
This lab experience was interesting. | 680 | 0.79 | −0.3 | — |
This lab experience helped me better understand chemistry, in general, as a result of completing the chemistry lab. | 1122 | 0.55 | −0.6 | — |
In my life, I will not use skills I’ve learned in this chemistry lab. | 1098 | 0.44 | −0.8 | — |
This lab experience did not make me learn. | 1118 | 0.43 | −0.8 | — |
Having the opportunity to use chemistry instruments helped me learn course topics. | 1115 | 0.40 | −0.8 | — |
Even if I don’t end up working in a science related job, the laboratory experience will still benefit me. | 1111 | 0.33 | −1.0 | — |
This lab experience has made me more interested in a science career. | 1108 | 0.28 | −1.1 | — |
This lab experience made me realize I could do science research in a real science laboratory (for instance at a university, or with a pharmaceutical company). | 1114 | 0.076 | −1.8 | — |
Having the opportunity to use chemistry instruments made this course less interesting for me. | 1114 | 0.051 | −1.9 | — |
This lab experience has made me less interested in science. | 1117 | 0.045 | −2.0 | −0.06 |
This lab experience has made me more interested in chemistry. | 1118 | 0.008 | −2.7 | −0.08 |
This lab experience helped me understand how the topics that are covered in chemistry lecture are connected to real research. | 1119 | 0.003 | −3.0 | −0.09 |
This lab experience gave me a better understanding of the process of scientific research as a result of this experiment. | 1106 | 0.002 | −3.1 | −0.09 |
Finding answers to real research questions motivated me to do well in the chemistry lab. | 1115 | 0.001 | −3.3 | −0.10 |
Finding answers to real world questions motivated me to do well in the chemistry lab. | 1106 | <5 × 10−4 | −3.5 | −0.10 |
This lab experience presented real science to students, similar to what scientists do in real research labs. | 1123 | <5 × 10−4 | −3.9 | −0.12 |
This lab experience was not very similar to real research. | 1120 | <5 × 10−4 | −4.3 | −0.13 |
This lab experience was not well organized. | 1112 | <5 × 10−4 | −5.1 | −0.20 |
This lab experience was open enough to allow me to make decisions. | 678 | <5 × 10−4 | −5.4 | −0.16 |
This lab experience was easy. | 594 | <5 × 10−4 | −6.0 | −0.25 |
In this lab, the instructional materials did not provide me with explicit instructions about my experiment. | 679 | <5 × 10−4 | −6.5 | −0.25 |
This lab experience was well contextualised to real life or the workforce. | 681 | <5 × 10−4 | −6.5 | −0.25 |
This lab experience was challenging. | 679 | <5 × 10−4 | −8.6 | −0.26 |
In this lab, I did not repeat experiments to check results. | 1121 | <5 × 10−4 | −8.6 | −0.25 |
In this lab, the instructional materials provided me with sufficient guidance for me to carry out the experiments. | 1117 | <5 × 10−4 | −9.3 | −0.28 |
In this lab, I can be successful by simply following the procedures in the lab manual. | 1118 | <5 × 10−4 | −9.3 | −0.28 |
Fig. 1 Horizontal stacked bar charts for the eight questions showing an effect size within the ‘zone of desired effect’ Pre and Post TLL. |
It is worth noting that this analysis does not take into account the variation in sample sizes for the different experiments, which range from 28–184. To investigate the effect of this, 28 random responses were chosen from each data set (e.g. only 28 of the responses to the electronic waste experiment and so on for all of the other experiments). Following a Wilcoxon Signed Rank test, only one new question showed a significant difference (which focused on general chemistry understanding) but still exhibited a small effect size. Six other questions no longer showed a significant difference, although they previously exhibited effect sizes r < 0.14 and were considered irrelevant. One new question rose above the r = 0.2 threshold (which focused on student perception of the organisation of the experiment) whilst the effect size of the original eight questions increased by 0.02–0.08. Hence, at worst, combining the data for analysis causes the overall effect sizes to be underestimated for the original eight questions and these were considered to be the items most affected by the TLL programme. The full data for this analysis is present in the Appendix.
Fig. 1 indicates the direction of the change in the students’ responses to any of the eight questions that showed a change within the ‘zone of desired effect’. For example, the top two horizontal stacked bars showed that the students’ responses shifted right for the Post-TLL experiments compared to the Pre-TLL experiments. Hence, students were more likely to select neutral, agree or strongly agree to the concept that the experiment was open enough for them to make decisions in the Post-TLL experiments. With the exception of the responses to the level of the contextualisation within the experiment (which saw a large decrease in neutral and a rise in agree and strongly agree), the shift of the horizontal stacked bars provides a simple approximation of the shift in the students’ responses.
Overall, students responded in such a manner to imply that they found the Post-TLL experiments less easy whilst more challenging, contextualised, and open (i.e. they could make more decisions themselves). They were also more likely to repeat results and found that the laboratory manual offered less guidance and could not be relied upon in order to complete the experiment with no additional materials or aids. Overall, these results are considered very positive outcomes for the TLL programme as removing dependence on the laboratory manual was a key goal (as it was perceived that students over-relied upon it for guidance). This indicates a move away from expository recipe-style manuals and an increase in inquiry. Furthermore, the recognition of increased openness, challenge and contextualisation were also considered positive results.
It is also of interest to note the questions that showed little to no significant difference after the modified experiments. Questions referring to overall motivation (e.g. finding answers to real research questions/real world questions motivated me to do well), interest (e.g. This lab experience made me more interested in a science career or This lab experience was interesting/worthwhile) or even overall learning (e.g. This lab experience did not make me learn) showed little to no significant difference. It would seem that either even more significant changes to the laboratory programme would be required to effect these items or that the students were unlikely to become more positive. In this case, approximately 80% of students always stated that the lab was interesting or worthwhile so there was very little room for improvement no matter what changes were made. It is also possible that these inherently intrinsic factors (such as interest and motivation) are simply too innate to a given student and are unlikely to be influenced by external factors such as the type of laboratory experienced.
That being said, the closed questions, and the subsequent quantitative analysis, can only provide a surface analysis of the impact of the new experiments. Hence, discussion of the responses to the 3 open questions is required.
Experiment | Top three skills developed | Percentage of respondents (%) |
---|---|---|
Rearrangement | Practical skills | 88 |
Transferable skills | 25 | |
Theory | 13 | |
EAS | Practical skills | 76 |
Theory | 32 | |
Transferable skills | 10 | |
Anthracene oxidation | Practical skills | 76 |
Patience | 17 | |
Theory | 16 | |
Macrocycles | Practical skills | 69 |
Following a literature method | 16 | |
None | 11 | |
Isomerisation | Practical skills | 84 |
Transferable skills | 49 | |
Theory | 19 | |
Proteins (2016) | Practical skills | 84 |
Transferable skills | 35 | |
Teamwork | 16 |
Experiment | Top three skills developed | Percentage of respondents (%) |
---|---|---|
Proteins (2017) | Practical skills | 80 |
Transferable skills | 35 | |
Teamwork | 25 | |
Sunscreen | Practical skills | 76 |
Transferable skills | 43 | |
Time management | 29 | |
Pseudenol | Practical skills | 71 |
Transferable skills | 36 | |
Critical thinking | 10 | |
Electronic waste | Transferable skills | 45 |
Observational skills | 40 | |
Note taking | 22 | |
Nylon | Transferable skills | 42 |
Experimental design | 39 | |
Practical skills | 35 | |
Food project | Transferable skills | 76 |
Teamwork | 49 | |
Experimental design/practical skills | 29 | |
Enzymes | Experimental design | 56 |
Practical skills/transferable skills | 33 | |
Teamwork | 19 |
The Pre-TLL results in Table 4 show several notable features. Firstly, the development of practical skills (student examples include ‘Use of the glassware’ and ‘Liquid–liquid extraction. TLC and how to interpret it. How to use pKA and separate solution mixture. Saw HNMR machine being used’) was a major theme for all six Pre-TLL experiments raised by 69–88% of the respondents. Secondly, many students stated a greater understanding of particular theoretical concepts (student examples include ‘Improved my mechanism understanding’ and ‘Understanding of how a catalyst works’) as a skill that they had developed, ranging from 13 to 32% of the responses. Thirdly, even though these experiments were predominately expository, students often raised a range of transferable skills, including, but not limited to, time management, teamwork, critical thinking and communication (student examples include ‘Teamwork skills and communication skills’ and ‘Critically thinking about instructions’). Individual transferable skills were generally not raised by more than 10% of the cohort, hence an overarching theme was generated that subsumed all transferable skills and was raised in four of the six Pre-TLL cases by 16–49% of respondents.
Deviations from the above observations can be explained by the nature of the experiments themselves – patience was raised in the anthracene oxidation experiment that involved a two hour reflux whilst following a literature method was raised in the macrocycles experiment as students were expected to find the method in the literature before arriving to laboratory session. Overall, the Pre-TLL results appear focused on the development of practical skills, limited transferable skills and regularly focus on theoretical understanding. As already noted, these are not unique but highlight the success of the survey at detecting these students’ perceptions.
Analysis of the Post-TLL responses in Table 5 indicates a range of similarities and differences when compared to the Pre-TLL results. Three of the Post-TLL experiments (Proteins (2017), Sunscreen and Pseudenol) show very similar results to the Pre-TLL experiments. This is to be expected as these new experiments, whilst contextualised, were still focused on the development of new practical techniques and could be considered largely expository, albeit with more inquiry focus than many Pre-TLL experiments as the overall outcome of the experiments were discovered by the students. However, it is worth noting that in all three cases an individual transferable skill (teamwork, time management and critical thinking respectively) was now raised enough by the students to become one of the top three aims. Hence, these Post-TLL experiments were still being recognised as opportunities to develop practical skills whilst raising recognition of several transferable skills. This is likely to be a result of the increased connection to the students’ daily lives and/or potential career paths when experiencing a conceptualised laboratory.
The Post-TLL experiments that allowed students to undertake experiments of their own design (Nylon, Food Project and Enzymes) showed common responses to one another. In all three cases, the development of experimental design skills (student examples include ‘Creating methods’ and ‘How to develop experiments to achieve certain aims or outcome’) were recognised by 29–56% of the students. The development of transferable skills also become much more prominent in the responses, with the extreme result of 76% of students raising them in the Food Project experiment. The development of practical skills was still raised in all of these experiments, but to a much lower degree (29–35%). This is likely an artefact of the students now raising a much larger breadth of developed skills.
The only case in which the development of practical skills was not a significant theme was the electronic waste experiment. This experiment involved the dropwise addition of metal ion solutions to a range of reagents and was, therefore, practically quite simple. Hence, other skills were raised by the students such as taking observations and making detailed notes.
Lastly, an increase in theoretical understanding was no longer raised as a common theme in the Post-TLL experiments. This would appear to contradict the quantitative results in which there was no notable difference in the students reported level of chemistry understanding or overall learning. However, it is possible that the new experiences simply provided a richer environment for skill development, which resulted in students identifying a broader range of skills that reduced the extent to which they viewed developing a deeper theoretical understanding as a skill that had been developed. Hence, it appeared to a lesser extent in the open answers but remained unchanged in the closed responses. This is particularly positive as the TLL experiments were designed to incorporate a larger diversity of learning experiences, rather than simply providing a chance to study a given theoretical principle. Overall, the Post-TLL results showed a larger range of skill development, particularly incorporating more transferable skills and experimental design. This was achieved without sacrificing the development of practical skills in the more expository experiences.
Experiment | Top three improvements raised by ≥10% of respondents | Percentage of respondents (%) |
---|---|---|
Rearrangement | Greater guidance | 42 |
No changes required | 21 | |
Better time management | 19 | |
EAS | No changes required | 43 |
Procedural issues | 14 | |
Greater guidance | 14 | |
Anthracene oxidation | Less waiting time | 48 |
Greater guidance | 35 | |
No changes required | 12 | |
Macrocycles | No changes required | 31 |
Greater guidance | 26 | |
More explanation of theory/better time management | 14 | |
Isomerisation | Greater guidance | 55 |
Better teaching associates/greater context required | 15 | |
Procedural issues | 10 | |
Proteins (2016) | No changes required | 48 |
Greater guidance | 25 | |
Better time management | 15 |
Experiment | Top three improvements | Percentage of respondents (%) |
---|---|---|
Proteins (2017) | Greater guidance | 47 |
No changes required | 28 | |
Procedural issues | 17 | |
Sunscreen | Greater guidance | 40 |
Better time management | 33 | |
No changes required | 13 | |
Pseudenol | Greater guidance | 64 |
Better time management | 20 | |
No changes required | 11 | |
Electronic waste | Greater guidance | 50 |
No changes required | 15 | |
Introduce group or team work | 11 | |
Nylon | Greater guidance | 62 |
Better time management | 22 | |
No changes required | 15 | |
Food project | Greater guidance | 52 |
Pre-assignment to groups | 21 | |
— | ||
Enzymes | No changes required | 29 |
Procedural issues | 21 | |
Greater guidance | 18 |
Table 6 shows that the improvements asked for with the Pre-TLL experiments were quite varied. The themes ranged from better guidance (student examples include ‘More guidance’ and ‘The instructions in the lab manual were vague’), better use of time (student examples include ‘Ran very close to time, organisation could be better’ and ‘Could maybe find something to make it last closer for the 4 hours, as opposed to finishing at 4:30 [a 90 minute early leaving time]’), better Teaching Associates (student examples include ‘Lab demonstrators gave barely any info as to the theory, explaining kinetics (4102) or any of my data’ and ‘Having better TA’), fixed procedural issues (student examples include ‘Flask was not sufficient to capture all solid after recrystallisation’ and ‘Filtration, most solid fell through into the flask’) or even calls for no changes at all (student examples include ‘Not that I can think of’ and ‘Not really, exercise 3 is pretty well organised and went smoothly’). The only universal issue noted was that students routinely called for a greater amount of guidance in every Pre-TLL experiment.
The Post-TLL results (Table 7) show a similar range of themes to the Pre-TLL results, albeit with a much different focus. The desire for greater guidance was now the main issue raised in six of the seven Post-TLL experiments. This shift is in good agreement with the quantitative data that highlighted that the students no longer felt that the laboratory manual provided sufficient guidance to complete the experiment. The strong desire for guidance is also a logical extension of the Pre-TLL results, as guidance was already a perceived issue for the students and the TLL programme deliberately sought to remove the recipe-like approach. This backlash is a likely result of students already being accustomed to expository experiments and the stepwise instructions normally provided. Hence, this result is considered positive as students will need to learn to deal with limited guidance throughout their future careers. This is the first step in acclimatising students to the uncertainties of a real workplace.
Calls for better time management in three of the cases (20–33%) were increased compared to pre-TLL (14–19%). This is most likely due to the longer, and more challenging, experiments generated through the TLL programme. Additionally, the call for group/team work in the electronic waste experiment was simply due to the requirement for students to work individually in a course where they typically worked in groups. Finally, the desire for pre-assignment to groups was a response to the fact that students were given a topic to investigate, rather than being allowed to choose from a list. Overall, many of the issues raised appeared to be responses from students speaking out against the new, more challenging, less prescriptive programme. This situation could possibly be ameliorated through conversations with students about the aims of the programme. That being said, these teething problems are common in cases where inquiry or problem-based learning has been implemented (Bruck and Towns, 2009; Gormally et al., 2009) and may subside over time.
Experiment | Respondents who enjoyed the laboratory (%) |
---|---|
Rearrangement | 95 |
EAS | 87 |
Anthracene oxidation | 57 |
Macrocycles | 70 |
Isomerisation | 74 |
Proteins (2016) | 84 |
Average | 81 |
Standard deviation | 14 |
Experiment | Respondents who enjoyed the laboratory (%) |
---|---|
Proteins (2017) | 82 |
Sunscreen | 86 |
Pseudenol | 78 |
Electronic waste | 92 |
Nylon | 78 |
Food project | 70 |
Enzymes | 78 |
Average | 80 |
Standard deviation | 7 |
In both sets the average percentage of students stating that they enjoyed the experience was quite high (≥80%) and the average values were the same within one standard deviation. Hence, the new laboratory experiments had no measurable impact (at least by the survey utilised) on the reported enjoyment by the students. This would appear to be in contrast to the results of many others (Gormally et al., 2009) who noted that students were ‘resistant’ to such changes in the curriculum. Potentially, this could imply that the new experiments were better received than originally anticipated. However, another reading of the data is that the students enjoyed the old expository laboratory experiments just as much as the new ones. This implies that enjoyment may not be the best measure by which to judge the effectiveness of any particular teaching intervention.
The reasons behind their enjoyment were very informative. The top three reasons (raised by ≥10% of the students) for enjoying any experiment are shown in Tables 10 and 11.
Experiment | Top three reasons for enjoying the laboratory | Percentage of respondents (%) |
---|---|---|
Rearrangement | Good teaching associate | 32 |
Good practical skills | 22 | |
Interesting, worthwhile or fun | 20 | |
EAS | Short experiment | 53 |
Easy experiment | 30 | |
Good teaching associate | 20 | |
Interesting, worthwhile or fun | 20 | |
Anthracene oxidation | Interesting, worthwhile or fun | 35 |
Easy experiment | 15 | |
Interesting theory | 13 | |
Macrocycles | Interesting, worthwhile or fun | 51 |
Good time management | 14 | |
Good practical skills | 11 | |
Isomerisation | Interesting, worthwhile or fun | 58 |
Good time management | 17 | |
Good teaching associate | 17 | |
— | — | |
Proteins (2016) | Interesting, worthwhile or fun | 32 |
Good guidance | 23 | |
Good teaching associate | 17 |
Experiment | Top three reasons for enjoying the laboratory | Percentage of respondents (%) |
---|---|---|
Proteins (2017) | Strong context | 36 |
Interesting, worthwhile or fun | 31 | |
Good time management | 14 | |
Sunscreen | Interesting, worthwhile or fun | 26 |
Strong context | 23 | |
Good challenge | 21 | |
Pseudenol | Strong context | 31 |
Interesting, worthwhile or fun | 28 | |
Good challenge | 15 | |
Electronic waste | Interesting, worthwhile or fun | 48 |
Easy experiment | 15 | |
Strong context | 10 | |
Nylon | Interesting, worthwhile or fun | 51 |
Strong context | 34 | |
Chance to development method or undertake investigation | 26 | |
Food project | Chance to development method or undertake investigation | 24 |
Good teamwork or team | 23 | |
Strong context | 14 | |
Enzymes | Chance to develop method or undertake investigation | 39 |
Strong context | 10 | |
— | — |
Throughout the Pre-TLL experiments (Table 10), the most common reason raised (20–58%) for enjoying an experiment was that they were either interesting, worthwhile or fun (student examples include ‘Yes, it was interesting’ and ‘It was fun and pretty’). Typically, students did not state why the experiment was any of these particular descriptions. The importance of a good Teaching Associate was another major theme (student examples include ‘TA is nice and helpful. She makes the practical go very smoothly’ and ‘engaging demos and interesting end results’), appearing in the responses to four of the six experiments (17–32%). Whilst it is pleasing to hear that those particular Teaching Associates were well received, it is concerning that the students associated a large amount of the success of the experience to a small number of staff. This dependence on the Teaching Associate is a significant area of research (e.g. note the work of Velasco et al. (2016)), particularly in their training and development (Flaherty et al., 2017), and not overly surprising to see come through in this case.
Outside of these main themes, students reported enjoying easy (student examples include ‘It was simple and instructions are clear’ and ‘b/c it was easier than previous labs’) or short experiments (student examples include ‘it was only 2 hours’ and ‘it was quick’). This result is in good agreement with the work of DeKorver and Towns (2015), which showed that students often focus on simply completing the experiment as quickly as possible in order to achieve the highest mark possible.
Interesting practical skills (student examples include ‘Learning/practicing interesting techniques’ and ‘The techniques were consistent and satisfying’) and significant guidance (student examples include ‘because clear instructions were given’ and ‘very clear instructions’) were also raised, but only in response to single experiments. Overall, no mention was made of context or inquiry, which is to be expected, both from the nature of the experiments and the quantitative data discussed earlier.
The Post-TLL responses shown in Table 11 indicate a very different set of responses to the Pre-TLL responses. Firstly, whilst the students still routinely raised that the experiments were either interesting, worthwhile or fun, they were also far more likely to raise the context of the experiment as a reason for this (student examples include ‘the context was interesting’ and ‘interesting as an investigative exercise similar to industry processes’). In fact, enjoyment due to the context of the experiment was a notable theme in all seven Post-TLL experiments (10–36%). This effect, i.e. the raising of context as a reason for enjoyment, is common in other implementations of context-based learning (Pringle and Henderleiter, 1999).
It is also interesting that whilst the Sunscreen and Pseudenol experiments were known to be difficult, a number of students (15–23%) raised the challenge as a reason for their enjoyment of the lab (student examples include ‘Yes, quite challenging’ and ‘Was a good thinking and practical challenge in chem and science principles’). Themes relating to the ease of the experiment were only noted in the first year experiment, electronic waste, which is reasonable considering the year level involved and the simple practical skills utilised.
For the experiments that included a significant component of inquiry or experimental design (nylon, food project and enzymes), this was directly stated by the students (24–39%) as a reason they enjoyed the experience (student examples include ‘make some polymers and test the properties within own design’ and ‘It was interesting to design our experiments’). Only one theme was unique to an experiment, which was good teamwork or tea’ in the Food Project experiment (student examples include ‘team was good’ and ‘I really enjoyed the team dynamic’). Overall, this increase in enjoyment as a direct result of increased inquiry or problem based learning is a well-known artefact of these types of teaching laboratories that was raised earlier in this article (Weaver et al., 2008).
It is worth noting that whilst the post TLL experiments were never considered short and only rarely easy, there appeared to be no notable negative impact on student enjoyment. Overall, these results are promising for the TLL programme. The students appear to enjoy the experiments for the same reasons that they were created – to incorporate more industry contexualised, inquiry/problem based experiments. This is in spite of the reported issues with guidance as shown in the quantitative data and the desired improvements raised by the students.
Furthermore, students routinely recognised that the experiments were more contextualised and more open (i.e. they were more able to make decisions). A large amount of effort was undertaken to incorporate more student control and greater real world context so this is a welcome result.
The students were also more likely to state that they repeated experiments, indicating an increase in a simple scientific practice – that of reproducibility. The students regularly reported (in both the closed and open questions) that the laboratory manual no longer provided enough information on its own to guide them through the experiment. As a central aim of TLL was to remove excessive guidance and encourage scientific practices (e.g. repetition), these were seen as favourable outcomes. However, it worth noting that there is always room for improvement with regards to student guidance and it is likely that the clarity of the student instructions could be further improved.
Students were more likely in the Post-TLL experiments to raise the development of transferable skills and skills associated with experimental design. This was accompanied by a decreased focus on development of theoretical understanding.
The proportion of students stating that they enjoyed the experiment did not change after the TLL programme. However, in the new experiments, students raised the strong context and open design of the experiments as reasons for their enjoyment – themes that were absent from the Pre-TLL data.
Overall, this research shows that the advantages gained by both contextualisation and inquiry/problem based learning persist when incorporated into a large, complex, multi-year undergraduate program. Whether or not these changes will have persistent, long term effects on the students understanding and articulation of their transferable skills will be determined through future research.
Question | N | p | Z | r |
---|---|---|---|---|
This lab experience was worthwhile. | 652 | 1.00 | 0.00 | — |
This lab experience was interesting. | 665 | 0.95 | −0.06 | — |
This lab experience helped me better understand chemistry, in general, as a result of completing the chemistry lab. | 660 | 0.70 | −0.39 | — |
In my life, I will not use skills I’ve learned in this chemistry lab. | 662 | 0.62 | −0.49 | — |
This lab experience did not make me learn. | 666 | 0.57 | −0.57 | — |
Having the opportunity to use chemistry instruments helped me learn course topics. | 663 | 0.49 | −0.68 | — |
Even if I don’t end up working in a science related job, the laboratory experience will still benefit me. | 500 | 0.39 | −0.86 | — |
This lab experience has made me more interested in a science career. | 662 | 0.32 | −0.99 | — |
This lab experience made me realize I could do science research in a real science laboratory (for instance at a university, or with a pharmaceutical company). | 663 | 0.32 | −0.99 | — |
Having the opportunity to use chemistry instruments made this course less interesting for me. | 664 | 0.31 | −1.01 | — |
This lab experience has made me less interested in science. | 659 | 0.30 | −1.03 | — |
This lab experience has made me more interested in chemistry. | 661 | 0.078 | −1.76 | — |
This lab experience helped me understand how the topics that are covered in chemistry lecture are connected to real research. | 655 | 0.068 | −1.83 | — |
This lab experience gave me a better understanding of the process of scientific research as a result of this experiment. | 667 | 0.064 | −1.85 | — |
Finding answers to real research questions motivated me to do well in the chemistry lab. | 500 | 0.055 | −1.92 | — |
Finding answers to real world questions motivated me to do well in the chemistry lab. | 667 | 0.034 | −2.12 | −0.12 |
This lab experience presented real science to students, similar to what scientists do in real research labs. | 665 | 0.018 | −2.37 | −0.17 |
This lab experience was not very similar to real research. | 498 | 0.016 | −2.42 | −0.23 |
This lab experience was not well organized. | 665 | 0.008 | −2.67 | −0.10 |
This lab experience was open enough to allow me to make decisions. | 661 | <0.0005 | −4.45 | −0.34 |
This lab experience was easy. | 665 | <0.0005 | −4.83 | −0.36 |
In this lab, the instructional materials did not provide me with explicit instructions about my experiment. | 662 | <0.0005 | −5.00 | −0.21 |
This lab experience was well contextualised to real life or the workforce. | 502 | <0.0005 | −5.05 | −0.29 |
This lab experience was challenging. | 415 | <0.0005 | −5.05 | −0.30 |
In this lab, I did not repeat experiments to check results. | 499 | <0.0005 | −5.08 | −0.29 |
In this lab, the instructional materials provided me with sufficient guidance for me to carry out the experiments. | 664 | <0.0005 | −5.33 | −0.36 |
In this lab, I can be successful by simply following the procedures in the lab manual. | 666 | <0.0005 | −5.75 | −0.29 |
This journal is © The Royal Society of Chemistry 2018 |