View PDF VersionPrevious ArticleNext Article
 
DOI: 10.1039/C5RP00099H (Paper) Chem. Educ. Res. Pract., 2015, 16, 929-938

The influence of PBL on students' self-efficacy beliefs in chemistry

Lloyd M. Mataka a and Megan Grunert Kowalske *b
aBerea College, Berea KY, 40403, USA
bWestern Michigan University, Kalamazoo, MI 49008, USA. E-mail: megan.kowalske@wmich.edu

Received 27th May 2015 , Accepted 11th September 2015

First published on 11th September 2015

Abstract

A convergent mixed methods research study was used to investigate whether or not undergraduate students who participated in a problem-based learning (PBL) laboratory environment improved their self-efficacy beliefs in chemistry. The Chemistry Attitude and Experience Questionnaire (CAEQ) was used as a pre- and post-test to determine changes in students' self-efficacy beliefs in general chemistry laboratories at a Midwestern University in the USA. Interviews were used to augment the quantitative data. Paired sample t-tests were used to determine the difference in means between pre- and post-tests. Analysis of variance was used to determine the influence of confounding variables. Thematic analysis was used to interpret the interview data. There was an observed improvement in students' self-efficacy beliefs using both qualitative and quantitative data. Interviews with the participants indicated that students had higher self-efficacy beliefs in chemistry for conducting chemistry experiments and for participating in undergraduate research after the PBL laboratory experience than they had before it. Students felt that PBL provided them with autonomy while exploring phenomena and allowed them to take more responsibility for their own learning. This study is significant in that it adds knowledge to research on the effects of PBL instruction and strengthens existing information on the relationship between PBL and the affective domain. Results from this study may also be informative to chemistry laboratory instructors on what PBL has to offer with regard to student outcomes.

Introduction

Although the major purpose of learning is to enable students to master specific skills and content, ignoring the affective domain can greatly reduce the efficiency at which these skills are mastered (Griffith and Nguyen, 2006). According to Adkins (2004), most educational institutions focus on teaching cognitive skills at the expense of the affect. However, Adkins believes that developing cognitive skills without considering the affective domain is forgetting the whole purpose of education. There is enough evidence to suggest that affective factors correlate with performance in the cognitive domain (Bloom, 1979). For instance, affective factors such as attitude, self-efficacy beliefs, motivation, and anxiety have been shown to influence students' learning (Kan and Akbaş, 2006; Evans, 2007; Mogane, 2010). Science education needs students who are resilient and motivated to perform difficult and challenging activities (Fairbrother, 2000; Zimmerman and Lebeau, 2000). Therefore, science education must aim to improve students' self-efficacy beliefs to reap the benefits mentioned above. Further, students with higher self-efficacy beliefs (SEBs) put more effort into their education because they are highly motivated (Bandura, 1997; Zimmerman and Kitsantas, 1997; Chowdhury and Shahabuddin, 2007).

Theoretical framework

Self-efficacy beliefs (SEBs) are “people's judgments of their capabilities to organize and execute courses of action required to attain designated types of performances” (Bandura, 1986, p. 391). SEBs are domain- and task-specific, and thus can vary both between and within individuals. SEBs are not necessarily related to the skills individuals actually possess, but how they perceive their own capabilities for a specific task. Accordingly, people with higher perceptions of their capabilities set up standards against which to evaluate their performance and strive to realize those goals (Bandura, 1977). As a result, people with higher SEBs are more likely to perform difficult tasks than their counterparts with low SEBs (Bandura and Schunk, 1981; Fairbrother, 2000). Due to this, students with high self-efficacy beliefs have higher academic motivation, put forth more effort, and are more persistent when doing classroom activities than students with lower SEBs (Zimmerman and Lebeau, 2000). Higher SEBs also correlate with improved academic performance (Nicolidau and Philippou, 2004; Chowdhury and Shahabuddin, 2007; Jungert and Rosander, 2010).

Bandura (1977, 1986, 1997) proposed four theoretical sources of self-efficacy beliefs: mastery experiences, vicarious experiences, social persuasion, and physiological/affective state. There is a growing body of research examining the factors that improve students' self-efficacy beliefs and their relation to these four sources (Keller, 1987; Moos and Azevedo, 2009; Jungert and Rosander, 2010). From these studies, constructive classroom environments such as problem-based learning have been amongst the top contributors to improved affect (Myers and Fouts, 1992; Nolen, 2003). Problem-based learning (PBL) is an instructional technique where students are required to solve ill-structured real world problems (Barrows, 1988; Belt et al., 2002; Hmelo-Silver, 2004). Usually students work cooperatively (in groups) to solve these ill-structured problems (Ram, 1999). In most instances, instructors provide learning materials that students read to develop their problem statements. With guidance from the instructor, students develop researchable questions and the method to research the questions (Ram, 1999).

PBL is an instructional technique that conforms to the tenets of constructivism in education. Constructivists assert that people acquire knowledge by building meaning based on their experiences with the environment (Piaget and Inhelder, 1958; Vygotsky, 1978; Richardson, 2003). Constructivism is a philosophical view of human understanding characterized by three primary propositions (Savery and Duffy, 1995). The three propositions are as follows:

• Understanding develops through interactions with the environment,

• Cognitive conflict or puzzlement is the stimulus for learning and determines the organization and the nature of what is learned, and

• Knowledge evolves through social negotiation and through the evaluation of the viability of individual understandings.

Problem-based learning fulfills the three constructivist propositions suggested by Savery and Duffy (1995) in that students are involved in cognitive conflict through solving problems, social negotiation through group interactions, and interaction with the environment through hands on experiences, resource searches, and other activities.

Bodner (1986, p. 873) summarizes the constructivist approach in education as an approach where “knowledge is constructed in the mind of the learner”. Constructivist approaches encourage students to be actively involved in their own learning by solving problems (e.g., problem-based learning) or answering deep cognitive questions (inquiry). PBL also aligns with Bodner's summary of the constructivist learning environment in that students take more responsibility for their learning. That is, students develop research questions and methods of investigations, and actually conduct the research. This effectively alters the roles of instructors and students. The instructors merely guide the instructional process while the students do the bulk of the work on their own.

Why do we need problem-based learning?

PBL is advantageous in that students are actively involved in their own learning. Learning by doing is much more effective than passive reception of information because active participation allows students to experience the challenges involved in solving the problems (Chin and Chia, 2006). Furthermore, science education must provide authentic learning environments to students (Savery and Duffy, 1995). Problem-based learning easily fulfills this role because by solving real world problems, students have an idea about how real scientists conduct themselves, what challenges they encounter, and how they approach these challenges. Engel (1991) asserts that PBL has the potential to improve life-long competencies that include:

• Dealing with problems and making reasoned decisions in unfamiliar situations,

• Reasoning critically and creatively, and

• Adopting a more universal or holistic approach.

The PBL environment has the potential to foster positive SEBs because students set their goals and develop realistic plans to accomplish them (Keller, 1987). Furthermore, the activities that students encounter in PBL enable them to explore their abilities towards solving problems and applying classroom work to real world situations (Lohman and Finkelstein, 2002; Yalcin et al., 2006; Gabr and Mohamed, 2011). Liu et al. (2006) found that computer-enhanced PBL significantly improved (F(1,246) = 12.61, p < 0.001) students' computer SEBs scores. Similar results were also obtained by Dunlop (2005) in computer SEBs and Rajab (2007) in biology SEBs. However, there has not much done to explore the effects of PBL on chemistry students' SEBs. Therefore, this study investigates the changes in SEBs that occurred when students participated in a PBL chemistry laboratory environment. The study answered the research question, “What changes in self-efficacy beliefs in chemistry occur when students participate in a PBL laboratory unit?”

Methodology

Research design

The overall research design for this study is a pre-experiment (one group pretest/posttest design) with mixed data sources (Borg and Gall, 1983, p. 657; Creswell, 2002). The study seeks to determine changes in students' self-efficacy beliefs in chemistry after learning through PBL laboratory units. A concurrent (convergent) mixed methods design was used as the research methodology (Fetters et al., 2013). This design involves collecting and analyzing both quantitative and qualitative data within a single study to obtain a deeper understanding of the problem (Creswell, 2002). This design is based on the assumption that neither quantitative nor qualitative designs give a full picture of the problem and its analysis, hence the need to combine both to complement each other (Greene et al., 1989; Tashakkori and Teddlie, 1998). The qualitative data in this study were collected after the quantitative data only for the purpose of convenience, but could as well have been collected concurrently. The mixing was done only at the interpretation stage to ensure triangulation of the data sources (Greene et al., 1989; Creswell et al., 2003).

Ideally, concurrent mixed methods recommends equal weighting between quantitative and qualitative data; in practice, one data collection method is usually given more weight than the other (Creswell et al., 2003). In this study, quantitative data formed the core and the qualitative section provided data for triangulation (Creswell et al., 2003). The role of the quantitative section was to provide information about the changes in students SEBs after participating in the PBL environment. Data from the qualitative phase has been used to provide a deeper understanding of students' SEBs and the effects of the PBL environment, because we had a chance to converse with the participants and also triangulate with the quantitative data. Convenience sampling has been used in this study because the participants were in a specific course with a specific teaching method.

Research context

This study was undertaken in general chemistry laboratories (general chemistry lab I and II) at a Midwestern University in the USA. General chemistry lab I (referred to as CHEM 1) activities were conducted in the Fall 2012 semester while general chemistry lab II (CHEM 2) activities were conducted in the Spring 2013 semester. CHEM 1 and 2 had 87 and 93 participants, respectively. CHEM 1 had five sections of approximately 18 students each while CHEM 2 had 6 sections of approximately 16 students each. Self-reported data of student GPA, ACT score, major, and number of chemistry courses were collected. Interviews were conducted during the Spring 13 semester. All students in CHEM 2 were invited to participate in interviews, with six students volunteering and completing the interviews. Table 1 shows the biographical information of the participants.
Table 1 Characteristics of the PBL groups
Student characteristics Fall 12 Spring 13
GPA range 3.4–4.0 2.2–4.0
ACT score range 24–32 15–34
Student major Biomedical sciences, Biology, Biochemistry, Chemistry, and Engineering Biomedical sciences, Biology, Biochemistry, Chemistry, and Engineering
No. of chemistry courses 1–5 1–5
No of GTAs Joseph, Norma, John, Jeff Agatha, Norma, Jeff
Gender Male: 53 Male: 57
Female: 34 Female: 36
No. of sections 5 6
Laboratory class Gen CHEM I Gen CHEM II
PBL topic Sensors Biodiesel
Interviewees — Cherie, Jacob, Lucas, Daniel, Isabel, William

Four graduate teaching assistants (GTAs) taught CHEM 1. We gave them the pseudonyms Joseph, Norma, John, and Jeff. One faculty member supervised the instruction process. CHEM 2 sections were taught by 3 GTAs. Two CHEM 1 GTAs, Norma and Jeff, were GTAs for CHEM 2, with the addition of Agatha (also a pseudonym). These GTAs received training with inquiry and PBL instruction through weekly discussions before and after each laboratory class leading to and during the PBL units.

The instruction process

Students worked in groups of three to four, with each group developing and selecting one research question or hypothesis to address. In CHEM 1, students optimized conditions for the detection of three pesticides on organic and non-organic fruits and vegetables using a color sensor. In CHEM 2, students were asked to optimize the laboratory-scale production of biodiesel. Both courses followed the same PBL instructional process, with the CHEM 2 process referred to here. During the first day of instruction, students read handouts containing information on general factors that affect the production of biodiesel such as temperature, concentration of the reactants, and type of catalyst. Their role was to establish optimum conditions of temperature, concentration, and type catalyst for the production of Biodiesel. Students read through the handouts and each group developed one question they would research, following the GTA's approval, over the next three weeks of PBL activities. During the second day of PBL, groups investigated their questions experimentally and produced initial results. Meanwhile, students reflected upon their experimental results and determined ways to improve upon them in the next lab. During the third day, groups repeated their experiments with the necessary improvements or to confirm their earlier results. The fourth day of PBL was for presentation of the results.

Data collection

The Chemistry Attitude and Experiences Questionnaire (CAEQ) developed by Dalgety et al. (2003) was used to measure students' self-efficacy beliefs in chemistry. This instrument contains three sections focusing on aspects of student affect. The first section on perception is a seven-point semantic differential, and the third section on experiences is a five-point Likert scale survey. The second section on confidence is a seven-point Likert scale, with students being asked how confident they are (from 1 = not confident to 7 = totally confident) with respect to completing chemistry-related tasks (such as checking the accuracy of experimental data or interpreting the main points of a chemistry research article).

In this study, the self-efficacy items from the confidence section of the instrument were used to measure students' self-efficacy beliefs. Some items from this section were removed because they were not relevant to our study group. The instrument has been previously validated with a Cronbach's alpha of 0.82 (Dalgety et al., 2003). We revalidated the self-efficacy section with our participants and found a Cronbach's alpha of 0.89. Both of these values are regarded as good (George and Mallery, 2003). The instrument was administered on the first day of the PBL unit before any instruction and the last day of PBL after the students' presentations. The study was conducted in accordance with Institutional Review Board (IRB) procedures, and students' consent was obtained to use the survey responses for research purposes.

We developed a semi-structured interview protocol through a review of literature and discussions amongst the authors. The protocol contained 15 questions, three of which addressed aspects of SEBs. The remaining interview questions addressed other areas of interest in the study, such as students' attitudes towards chemistry, experiences in the PBL laboratories, and suggestions for improvement. The semi-structured protocol allowed the interviewer to ask additional questions and clarify responses to understand students' experiences in the PBL laboratories. The three guiding interview questions focused on SEBs are as follows:

(1) Do you feel more or less confident about your abilities in chemistry as a result of completing the PBL laboratory unit? Why?

(2) How did the PBL unit affect your ability to plan and conduct experiments?

(3) How did the PBL unit affect your interest and/or confidence in participating or conducting undergraduate research?

Interviews were integral to this study because they provided a deeper understanding of students' SEBs. They are a two-way conversation that enables participants to clarify their responses and interviewers their questions; therefore, we were able to obtain subtle and detailed information that surveys could not capture.

Quantitative data analysis

SPSS® was used for quantitative data analysis. Analysis for data normality was done using the Kolmogorov–Smirnoff's test. Paired sample t-tests were used to determine the mean differences between the pre- and post-test scores on the CAEQ self-efficacy component. This test needed each participant to have both pre- and post-test scores. Due to this, some students were dropped from the analysis because they had only one score. Cohen's d values for the differences were calculated from the t-test parameters. This test was used to determine the practical sense of the differences in mean scores.

Before we conducted major significance tests, we investigated the influence of confounding variables: GPA, GTA, ACT scores, students' major, and number of chemistry courses taken (including high school, community college, or college courses). Table 2 shows the ANOVA results for these variables. These were selected because studies have shown that factors like high school GPA and ACT scores influence college admission (Geiser and Santelices, 2007). Research has also shown that SAT (similar to ACT), college major, and gender influence student affect (Sabot and Wakeman-Linn, 1991; Folsom-Meek and Nearing, 1999).

Table 2 ANOVA of self-efficacy results
Variable CHEM 1 CHEM 2
F-values F-values
N Pre Post N Pre Post
Significant at *p = 0.05; **p = 0.01.
ACT 84 1.36 0.75 82 0.91 2.24
Courses 81 3.62* 4.72* 87 2.78* 1.65
Major 84 1.72 1.63 75 0.92 1.53
GTA 85 2.90* 3.20* 93 8.27** 1.77
GPA 85 0.52 0.67 87 1.16 1.36

Qualitative data analysis

Interview respondents' demographic data such as gender, number of chemistry courses, and majors were recorded to help with data analysis, as shown in Table 5. The interviews were transcribed in Microsoft Word®, and identifiers were used instead of names to protect participants' identities. The interview transcripts were transferred to HyperResearch® software for coding. HyperResearch® is a qualitative analysis software program that allows for the identification and organization of codes in text, audio, and video files. For this study, the text files were imported into the program and segments of text were highlighted and labeled with codes from the coding scheme. The program also allows for sorting and arranging codes into themes and code families, which is helpful when working through qualitative data.

Grounded theory qualitative data analysis methods were used for the coding process as proposed by Glaser and Strauss (1967). Rather than using a priori codes, codes and themes were developed by the researchers. The participants' transcripts were thoroughly read and data carefully reviewed line by line while assigning codes to emergent concepts (Glaser, 1992; Bradley et al., 2007). Two researchers read two of the interview transcripts and coded them together while discussing the codes and emergent themes. With further review and discussion using the constant comparison method, codes were refined and assigned to text. Codes were assigned to any text that fit the code definition, regardless of the interview question. All codes addressing similar issues were categorized into themes (Creswell, 1998). Interpretation of these themes provided information about students' SEBs.

In qualitative studies, researchers need to establish their trustworthiness and that of the data being presented (Lincoln and Guba, 1985; Eisner, 1991). Therefore, to establish credibility of the qualitative data, after the codes were generated and consensus was reached with the codebook, each researcher coded four transcripts independently. The codes from these four transcripts were used to calculate the intercoder agreement. The codes were mutually exclusive, hence the intercoder Kappa of 0.83 was found using SPSS®, indicating strong agreement between the coders (Di Eugenio, 2000; Stemler, 2001).

Results

Table 2 shows the ANOVA results of the CAEQ SEBs variables. The asterisks show significant differences for each individual test. ACT scores, GPA, and student major did not influence SEBs in either chemistry laboratory course. The number of chemistry courses influenced students' SEBs scores in both pre- and post-test of CHEM 1 and the pre-test of CHEM 2. Post hoc Tukey's test showed that students with four or more chemistry courses had significantly higher SEBs scores. The GTA also influenced SEBs results in both CHEM 1 pre- and post-test and CHEM 2 pre-test. Post hoc analysis indicated no difference in mean scores amongst students taught by different GTAs in CHEM 1 pre-test, but a difference in individual SEBs means for GTAs in both CHEM 1 post-test and CHEM 2 pre-test. The post hoc analysis in CHEM 1 indicated that mean SEBs scores from Jeff's class were significantly lower than that from John's classes for the post-test only. In CHEM 2 pre-test scores, Jeff's students had significantly lower scores than Norma's, however, Jeff's students had the highest improvement in SEBs scores that enabled them catch up with students from the other sections.

Gender influenced the pre-SEBs (t = 3.038, p = 0.003) scores in CHEM 1, but this influence did not persist in the post-test (t = 1.922, p = 0.058). Furthermore, gender did not significantly influence (p > 0.05) SEBs scores in CHEM 2. The influence of gender was only significant in the CHEM 1 pre-test and not in any subsequent tests for CHEM 1 or CHEM 2.

To determine the influence of PBL on SEBs scores, a paired sample t-test was conducted. Results of the paired sample t-test between pre-test and post-test SEBs scores are shown in Table 3. The trend from the means indicates that participating in the PBL learning environment improved the students' CHEM 1 SEBs scores from 4.63 to 5.01. The increase was statistically significant (t = 4.97, p = 0.000). Similarly, the mean CHEM 2 SEBs score improved from 5.03 to 5.34, and the increase was also statistically significant (t = 4.70, p = 0.000). Many students from Chem 1 continued on to Chem 2 in this study, which allows for semi-longitudinal tracking of students' SEBs. From the pre- and post-test SEB data, there is virtually no difference from the post-test Chem 1 and pre-test Chem 2 values (5.01 at the end of Chem 1 to 5.03 at the start of the Chem 2 PBL unit). Students in Chem 2 participated in five non-PBL laboratories prior to taking the pre-test. This indicates that there was no gain during the first five laboratories of Chem 2, yet there is a significant gain between the pre- and post-test for Chem 2, as was seen in Chem 1. This supports the finding that increases in SEBs results from participating in the PBL units specifically. More importantly, these gains appear to be stable and long lasting, as they did not decrease during the first half of Chem 2.

Table 3 Paired sample t-tests of SEBs data
CHEM Test N Mean score t p Δm SDpooled d
1 Pre-test 85 4.63 4.97 0.000 0.375 0.697 0.54
Post-test 5.01
2 Pre-test 93 5.03 4.70 0.000 0.312 0.640 0.49
Post-test 5.34

Table 3 shows the effect sizes, Cohen's d values for CHEM 1 (d = 0.54), and CHEM 2 (d = 0.49). The effect size provides information about how many standard deviations difference there is between the pre- and post-test. In simple terms, it asks the question, “Is the difference big enough to be considered useful for a specific purpose?” Both Cohen's d values are within the moderate range, indicating that the improvement was not only statistically but also practically significant (Valentine and Cooper, 2003). This shows that PBL has a significantly positive relationship with students' SEBs. The Cohen's d values imply that in CHEM 1, the students changed their self-efficacy beliefs by 0.54 standard deviation on average, while in CHEM 2, they changed by a standard deviation of 0.49. This indicates that the improvement is adequate for educational purposes.

We conducted Wilcoxon Sign Rank tests for individual items to determine which items influenced the change in the mean SEBs scores. Table 4 shows results of individual item analysis. In both chemistry courses, significant improvements were observed on item 1, Reading the procedures for an experiment and conducting the experiment without supervision; item 3, Proposing a meaningful question that could be answered experimentally; item 5, Converting the data obtained in a chemistry experiment into results; and item 11, Writing up the results section in a laboratory report. In addition to the 4 items shown above, SEBs scores for CHEM 2 increased on three other items (2, 6, 7, and 10).

Table 4 Individual item t-test analysis for SEBs
No. Items CHEM
1 2
— Not significant at p = 0.05.
1 Reading the procedures for an experiment and conducting the experiment without supervision p < 0.01 p < 0.01
2 Ensuring that data obtained from an experiment is accurate — p < 0.05
3 Proposing a meaningful question that could be answered experimentally p < 0.01 p < 0.01
5 Converting the data obtained in a chemistry experiment into results p < 0.01 p < 0.01
6 After reading an article about a chemistry experiment, writing a summary of the main points — p < 0.05
7 Designing and conducting a chemistry experiment — p < 0.01
10 Applying theory learned in a lecture for a laboratory experiment — p < 0.05
11 Writing up the results section in a laboratory report p < 0.01 p < 0.01

Qualitative data

The qualitative interviews covered three major questions related to SEB:

(1) Do you feel more or less confident about your abilities in chemistry as a result of completing the PBL laboratory unit? Why?

(2) How did the PBL unit affect your ability to plan and conduct experiments?

(3) How did the PBL unit affect your interest and/or confidence in participating or conducting undergraduate research?

Representative participant quotes are used to show the themes and findings for each question. While data was analyzed as a whole for emergent themes, it is presented here by question for ease of discussion because there was limited code overlap between these questions. Overarching themes from the SEB portion of the interviews included mastery experiences increasing confidence in chemistry, increased confidence in planning and conducting experiments due to overcoming challenges in the PBL setting, increased interest in enrolling in future chemistry courses and participating in undergraduate research, and increased confidence in their ability to conduct research in the future due to mastery experiences. These themes clearly align with the interview questions, which is why responses have been grouped by question. Table 5 shows interview participant demographic data, including their pseudonyms, GPA, ACT score, number of high school chemistry courses taken, number of college chemistry courses taken (including current enrollment), and college major.

Table 5 Interview participant demographic data
Pseudonym GPA ACT High school CHEM courses College CHEM courses Major
Lucas 4.0 31 1 2 Undecided
William 2.75 29 2 2 ChemEng
Jacob 3.0 27 1 2 MechEng
Isabel 4.0 31 2 2 Elementary education
Cherie 3.75 24 1 2 Biology
Daniel 3.6 32 3 1 Chemistry

Do you feel more or less confident about your abilities in chemistry as a result of completing the PBL laboratory unit? Why?

For this question, the interviewees felt that PBL generally improved their confidence in chemistry because of the mastery experiences it provided. For instance, Jacob felt that his confidence improved because the PBL lab provided a new experience, “I do feel a lot more confident just because it gives you that experience you haven't had before in chemistry”. Similarly, William felt that being taught the ability to determine what you are finding out using the given equations improved his confidence in chemistry. William said, “Mainly because you have to. You have the equations that you are using, but you have to know what you are finding, then where to put it in the equation, where most labs are, ok you found this value, this is where it goes”. He contrasted the PBL and traditional laboratory activities in terms of understanding the equations he was using.

Lucas also felt that he was more confident in his abilities to set up his own experiment after undergoing the PBL instruction, “I definitely feel more confident and say now I can, with some time, set up a good experiment and figure out what's gonna work and what's not gonna work next time”. To Cherie, the confidence arose because she felt that the PBL activities provided a better understanding of chemistry and its applications. She said, “I think it (PBL) just gave me a better general understanding of chemistry and how you apply it so, hopefully, I think that will help me in the future chemistry classes”.

Daniel, however, felt that he was already confident about lab processes and the lab did not affect his confidence at all. Daniel felt that the PBL lab set up was confusing and harder to deal with. Nevertheless, he acknowledges that the PBL labs were generally better than the usual labs. Daniel said, “Well I'm pretty confident myself in lab processes. I think that lab set up, it almost discourages a little bit just because of the confusion and it's a little harder to approach… This was more of you are doing this and try and do this, but with all, it overwhelms that we put in there, it really wasn't a great experience. It was better than the regular chemistry labs that we were doing before”.

The statements above show that students were confident in their abilities because of the mastery experiences involved in the PBL process where students had to figure out activities on their own and apply chemistry knowledge to their research.

How did the PBL unit affect your ability to plan and conduct experiments?

The students generally felt that the PBL environment improved their ability to plan and conduct chemistry experiments. The students felt that PBL provided in-depth instruction, it helped with organization for laboratory activities and timing of the experiments, and provided them with more responsibilities as students. These factors raised students' confidence in planning and conducting chemistry experiments. For instance, Jacob felt that PBL improved his ability to plan for the experiment. In this case, he felt that PBL gave him an opportunity to plan and determine what variables to deal with before the actual experiment. He said, “Well, I had the experience for experiment planning before with, like, physics lab and other chem. labs. PBL was a lot more in-depth and it helped me to really be able to find all the different variables and account for them before you actually did the experiment. So if you get some unfamiliar results, you would definitely have to run back through all your variables and say, oh yeah, that's where I was wrong, I should probably fix that for next rounds”.

William also agreed that PBL improved his confidence in planning and conducting chemistry experiments. He felt that PBL helped him understand that through proper organization and teamwork, experiments can become a lot easier. William said, “I think it helped [improve my confidence] quite a bit. I mean, the first time that we ran the experiment, we were disorganized. It took us four or three hours and even then we were rushing for time and trying to get done. Then the second time we were more organized and had a more clear procedure, we knew what we were supposed to be doing, worked more as a team, and kind of got done a lot faster”.

Lucas felt that PBL helped improve his confidence in his ability to plan and conduct chemistry experiments because he recognized that with time and experience he could improve upon his experiments and conduct the activities within a shorter period of time. He felt that his ability to recognize the mistakes and improve upon them improved his confidence to plan and conduct chemistry experiments. Lucas said, “Over the weeks it helped me learn to be more precise and helped me to conduct experiments in a much shorter amount of time. I recognized, you know, mistakes that I was making, I was able to correct them so that when we did the final experiment we did it quickly and it all went pretty well”. Cherie felt that being given more responsibility than normal improved her ability to plan and conduct experiments. She said, “It gave us a little bit more responsibility than normal so we learned how to plan and guide ourselves in doing the experiment more than the former structured labs that we started with”.

However, Daniel felt that the PBL activities had no effect on his confidence in planning and conducting chemistry experiments because of having limited trials on which to improve on. He felt that just two days of experiments did not give the students sufficient time to try out some variables. He explained, “I don't think this one did as much just because we were limited on the trials and the fact that we can do one trial per day made it so that the first day we had to get our data and the second day we really, we could've changed some things to make our process better, but anything that we will change will make our data inconsistent between the two days. So it was really the struggle between whether we change and try to improve or do we have consistency of our data”.

This section shows that having experienced challenges and successes of a real life research project improved students' SEBs for planning and conducting experiments because students were responsible for most of their work in this learning environment. This shows how mastery experiences influence students' SEBs.

How did the PBL unit affect your interest and/or confidence in participating or conducting undergraduate research?

In terms of conducting undergraduate research, generally students felt that PBL had an effect on their confidence because PBL was an informative experience that made them become more independent and provided them with good practice for undergraduate research. William felt that PBL made him realize that he could do more in undergraduate research. Although he had been interested in undergraduate research before, he never had the opportunity to do it until the PBL activity. He said, “I have been really interested in doing undergrad research. This is the first one I've ever done and it was pretty fun, pretty informative, and a good experience. I think I can do more”.

Lucas felt that PBL was frustrating, but at the end made him feel more confident about his ability to conduct independent research. He said, “I would say that it initially kind of frustrated me because it was difficult, but it makes me more confident because I know that I can be independent and that also I can work with someone”. In addition, Isabel felt that PBL made them practice doing research, and this improved her confidence in conducting undergraduate research. She explained, “Just the practice and the experience doing the lab and creating the questions and determining all values and variables that change, I think on the whole I just needed practice doing that”. Cherie believed that PBL enhanced her understanding of the experimental process, which improved her confidence in conducting undergraduate research. Cherie said, “It gave me a little bit more confidence because I have a better understanding of what you would do to start and end an entire experiment”.

Clearly the students here felt that the mastery experiences they got from these activities, including the frustrations of research, made them more confident about their abilities to conduct undergraduate research. Increasing students' confidence in their abilities to conduct scientific research and better understand the nature of scientific research is an important outcome for undergraduate science education and encouraging students to pursue undergraduate research experiences.

Discussion

The assertion from this study is that 'participating in a PBL laboratory unit increased students' self-efficacy beliefs in chemistry primarily through mastery experiences'. The purpose of this study was to investigate whether or not participating in PBL activities increases students' SEBs in chemistry. The students participated in two PBL activities from two different chemistry laboratory classes: CHEM 1 and 2. We collected quantitative data using the self-efficacy section of the Chemistry Attitude and Experiences Questionnaire and qualitative data through interviews. This section presents a discussion of this assertion, starting with the influence of the confounding variables, quantitative data, and qualitative data.

The influence of confounding factors such as ACT scores, student major, GPA, number of chemistry courses, and gender was determined for each individual test. Generally, the confounding factors such as ACT, student major, and GPA did not influence SEBs of each individual test. This indicates that these confounding variables may not be good predictors of students' SEBs in chemistry. The range of the number of chemistry courses, both high school and college, previously taken was between one and five. Therefore, ANOVA was conducted to determine the influence of this number of previous chemistry courses to students SEBs. The influence of the number of chemistry course was observed in SEBs scores from the CAEQ. Previous experience with chemistry might have improved the SEBs of students with four or more chemistry courses more than the rest of the group. However, during the post-test of CHEM 2 students, this advantage was not present. It is possible that through the course of two college courses, students achieved equivalent SEBs regardless of the number of previous chemistry courses through the accumulation of mastery experiences. Further, students might have had better group interactions in CHEM 2, as referenced during the interviews. This is an example of vicarious experiences or social persuasion, which might have encouraged or helped students with fewer chemistry courses to achieve equivalent SEBs to their counterparts with more chemistry courses. We observed a significant influence of gender only in the CHEM 1 pre-test and not in any subsequent tests for CHEM 1 or CHEM 2. While there is not enough data to explain this, it is possible that instruction through PBL closed the observed gender gap between male and female participants' SEBs scores.

In CHEM 1 the four GTAs were Joseph, Norma, John, and Jeff, while in CHEM 2 the GTAs were Agatha, Norma, and Jeff. The influence of GTAs was observed in the SEBs scores of both pre- and post-tests in CHEM 1 and the pre-test of CHEM 2. A post hoc analysis was done to determine what mean scores were responsible for the observed differences. This post hoc analysis showed that in all the cases, the mean scores from Jeff's lab classes were consistently lower than those from the other GTAs. However, this difference disappeared in the post-test of CHEM 2. It is possible that the other GTAs provided the students with better opportunities for mastery experiences or had better verbal and social persuasion skills or Jeff may have taken longer to feel comfortable in the PBL instructional setting than other GTAs. Some GTAs had more difficulty adjusting to instruction in the PBL setting than others, which could have affected the SEBs of students in their classes (Current and Grunert, 2014).

Results from the SEBs data have shown that there was a significant increase in mean scores on the self-efficacy scale between pre- and post-tests. This indicates that the PBL learning environment has the potential to improve students' SEBs in chemistry. This is not surprising, as Jungert and Rosander (2010) and Senay (2010) claimed that a conducive teaching/learning environment can foster the development of students' self-efficacy beliefs. PBL instruction is one of those conducive learning environments, as proposed by Keller (1987), and as was also shown by Moos and Azevedo (2009), and Dunlop (2005). Furthermore, PBL embodies the tenets of constructivism in that it (1) encourages an authentic task, (2) designs the learning environment to support and challenge the learner's thinking, and (3) supports the learner in developing ownership for the overall problem or task (Bodner, 1986; Savery and Duffy, 1995). This environment may be responsible for the improvement of students' SEBs, primarily by providing students with authentic mastery experiences. The results from this study corroborate findings from Liu et al. (2006) and Dunlop (2005), who found that computer-enhanced PBL significantly improved students computer SEBs, and Rajab (2007), who found that PBL improved SEBs in biology.

It is not surprising that students improved their SEBs on the item “Reading the procedures for an experiment and conducting the experiment without supervision” because the activities in PBL require them to do literature readings to understand their activities. Similarly, proposing a meaningful question is part of PBL activities (Hmelo-Silver, 2004), hence their improvement in this area is not surprising. Surprisingly, CHEM 1 data shows that there was no improvement in students' confidence in their ability to design and conduct chemistry experiments. One of the most important aims of PBL is to enable students identify a problem and design a process to solve this problem, thus we expected an improvement of their SEBs on this item. It is possible that the students in CHEM 1 were not confident in their abilities to plan and carry out an experiment because this was the first college chemistry laboratory course for most of them.

In CHEM 2, students felt that they were more confident in applying theory learned in class to laboratory experiments, as well as designing and conducting experiments. This was not the case with the CHEM 1 students. The CHEM 2 class did a PBL biodiesel unit. These activities had more application of theory from reaction kinetics such as the roles of catalysts, temperature, and amounts of substances, which was more related to course content than to the activities from the sensor unit. This experience provided the CHEM 2 students with an opportunity to apply their knowledge about reaction kinetics. Without direct ties to General Chemistry I content, CHEM 1 students had limited opportunities to apply course content to the sensors lab.

Contrary to the CHEM 1 data, students in CHEM 2 had an improved confidence in designing and conducting a chemistry experiment. This finding matches with one of the major aims of PBL, which is to be able to design and conduct research to solve an ill-structured problem (Belt et al., 2002). The feeling of confidence that the students displayed here is very important because Bandura and others have shown that confidence in an individual's ability to perform tasks influences the accomplishment of those tasks (Bandura et al., 1977; Bandura et al., 1980). Therefore, these results indicate that students will be more likely to accomplish the tasks mentioned above due to their improved SEBs.

Gender did not influence students' SEBs in chemistry in this study. Results from this study agree with Rose (2003) and Smist (1993), who also found no difference between the SEBs scores of male and female students in college chemistry except on lab skills, where male participants scored significantly higher than females. These results contradict those of Busch (1995), who found a difference in mean SEBs scores between male and female participants doing computer sciences. Using the same CAEQ on freshman chemistry students in New Zealand, Dalgety et al. (2003) found that male participants scored significantly higher on most self-efficacy items than female participants. These results disagree with those by Majere et al., (2012) who found that form four female students in Kenyan secondary schools (probable age range: 16–19) had significantly better self-efficacy beliefs in chemistry than boys.

The qualitative data corroborated the findings from the quantitative section on the relationship between PBL and students' SEBs. The students reported that PBL improved their SEBs in chemistry. In this case, PBL improved their confidence in chemistry in general, in conducting chemistry experiments, and in conducting undergraduate research in particular. These results corroborated the quantitative data, which also indicated an improvement in the students' SEBs. We predict that the experience of solving an ill-structured problem that models real life experiences provided our study group with an appropriate context for real life chemistry experiments and experience with undergraduate research. This improvement in SEBs may also come from improved student intrinsic motivation as reported by Hmelo-Silver (2004). Solving problems with practical relevance may have improved students' motivation to do chemistry experiments and undergraduate research, which consequently may have improved their confidence in doing these activities.

Implication for teaching

Results from this study have indicated that students who participated in the PBL laboratory units improved their SEBs and that these improvements were stable and long-lasting. Bandura (1986, 1993) has argued that SEBs have a positive effect on students' cognitive development. These students may also put more effort in their education endeavors because high SEBs motivate students to work hard and take on challenging tasks (Bandura, 1997; Zimmerman and Kitsantas, 1997; Chowdhury and Shahabuddin, 2007). Furthermore, research found that students with higher SEBs have improved conceptual understanding and classroom performance compared to students with lower SEBs (Andrew, 1998; Ferla et al., 2009). Educators, therefore, must recognize that PBL has the potential to improve students' motivation and classroom performance due to improved students' SEBs.

Limitations

This study was exploratory and was not designed to have a control group. In future studies, however, we plan to add a control group for comparison purposes. Further, more interview participants would have strengthened the study; however, we were unable to recruit more students. Both the voluntary nature of participation and the timing of the interviews, which occurred at the end of the semester, contributed to fewer interview participants than we would have liked. Despite these limitations, responses from the majority of students who participated in the interviews corroborated results from the quantitative data.

Acknowledgements

We would like to acknowledge the work of two chemistry department research groups at Western Michigan University for using their work as the foundation for the PBL units. We would also like to thank the Chemistry Department and the Mallinson Institute for Science Education at Western Michigan University for their support of this research.

References

  1. Adkins S., (2004), Beneath the tip of the iceberg, Am. Soc. Train. Dev., 58(2), 28–33.
  2. Andrew S., (1998), Self-efficacy as a predictor of academic performance in science, J. Adv. Nurs., 27, 596–603.
  3. Bandura A., (1977), Self-efficacy: toward a unifying theory of behavioral change, Psychol. Rev., 84(2), 191–215.
  4. Bandura A., (1986), Social foundations of thought and action: a social cognitive theory, Englewood Cliffs, NJ: Prentice-Hall.
  5. Bandura A., (1997), Self-efficacy: the exercise of control, New York, NY: Freeman.
  6. Bandura A. and Schunk D. H., (1981), Cultivating competence, self-efficacy, and intrinsic interest through proximal self motivation, J. Personality Soc. Psychol., 41(3), 586–598.
  7. Bandura A., Adams N. E. and Beyer J., (1977), Cognitive processes mediating behavioral change, J. Personality Soc. Psychol., 35, 125–139.
  8. Bandura A., Adams N. E., Hardy A. B. and Howells G. N., (1980), Tests of the generality of self-efficacy theory, Cognit. Ther. Res., 4, 39–66.
  9. Barrows H. S., (1988), The tutorial process, Springfield, IL: Southern Illinois School of Medicine.
  10. Belt S. T., Evans E. H., McCreedy T., Overton T. L. and Summerfield S., (2002), A problem based learning approach to analytical and applied chemistry, Univ. Chem. Educ., 6, 65–72.
  11. Bloom B. S., (1979), New views of the learner: implications for instruction and curriculum, Child. Educ., 56(1), 4–11.
  12. Bodner G. M., (1986), Constructivism: a theory of knowledge, J. Chem. Educ., 873–878.
  13. Borg W. R. and Gall M. D., (1983), Education research: an introduction, 4th edn, New York, NY: Longman Inc.
  14. Bradley E. H., Curry L. A. and Devers, K. J., (2007), Qualitative data analysis for health services research: developing taxonomy, themes, and theory, BMC Health Serv. Res., 42(4), 1758–1772.
  15. Busch T., (1995), Gender differences in self efficacy and attitude towards computer, J. Educ. Comput. Res., 12, 147–158.
  16. Chin C. and Chia L. G., (2006), Problem-based learning: using ill-structured problems in biology project work, Sci. Educ., 90(1), 44–67.
  17. Chowdhury M. and Shahabuddin A., (2007), Self-efficacy, motivation and their relationship to academic performance of Bangladesh college students, Coll. Q., 10(1), 1–9.
  18. Creswell J. W., (1998), Qualitative inquiry and research design: choosing among five traditions, London: Sage.
  19. Creswell J. W., (2002), Research design: Qualitative, quantitative, and mixed methods approaches, 2nd edn, Thousand Oaks, CA: Sage Publications.
  20. Creswell J. W., Clark V. L. P., Gutmann M. and Hanson W., (2003), Advanced mixed methods research designs, in Tashakkori A. and Teddlie C. (ed.), Handbook of mixed methods in social & behavioral research, Thousand Oaks, CA: Sage, pp. 209–240.
  21. Current, K. M. and Grunert, M. L., (2014), The classroom discourse of chemistry GTAs, Intl. Conf. Chem. Educ., Toronto, ON.
  22. Dalgety J., Coll R. K. and Jones, A., (2003), Development of Chemistry Attitudes and Experiences Questionnaire (CAEQ), J. Res. Sci. Teach., 40(7), 649–668.
  23. Di Eugenio B., (2000), On the usage of kappa to evaluate agreement on coding tasks, in Proceedings of LREC, vol. 1, pp. 441–444, retrieved from http://www.nlp.cs.uic.edu/PS-papers/lrec00.pdf.
  24. Dunlop J. C., (2005). Problem-based learning and self-efficacy: how a capstone course prepares students for a profession, Educ. Tech. Res. Dev., 53(1), 65–85.
  25. Eisner E. W., (1991), The enlightened eye: qualitative inquiry and the enhancement of educational practice, New York, NY: Macmillan.
  26. Engel C. E., (1991), Not just a method but a way of learning, in Boud D. and Feletti G., (ed.), The challenge of problem-based learning, London: Kogan, pp. 22–33.
  27. Evans B., (2007), Student attitudes, conceptions, and achievement in introductory undergraduate college statistics, The Math. Educator, 17(2), 24–30.
  28. Fairbrother R., (2000), Strategies for learning, in Monk M. and Osborne J. (ed.), Good practice in science teaching. What research has to say, Philadelphia, PA: Open University Press, pp. 7–24.
  29. Ferla J., Valcke M. and Cai Y., (2009), Academic self-efficacy and academic selfconcept: reconsidering structural relationships, Learn. Indiv. Differ., 19, 499–505.
  30. Fetters M. D., Curry L. A. and Creswell J. W., (2013), Achieving integration in mixed methods designs-principles and practices, Health Res. Educ., 48(6), 2134–2156.
  31. Folsom-Meek S. and Nearing M. R. J., (1999), Effects of academic major, gender, and hands-on experience on attitudes of preservice professionals, Adapted Phys. Act. Quart., 16.388–402.
  32. Gabr H. and Mohamed N., (2011), Effect of problem-based learning on undergraduate nursing students enrolled in nursing, Int. J. Acad. Res., 3(1), 154–162.
  33. Geiser S. and Santelices M. V., (2007), Validity of high-school grades in predicting student success beyond the freshman year: high-school record vs standardized tests as indicators of four-year college outcomes, Center for Studies in Higher Education, University of California, Berkeley, Berkeley, CA.
  34. George D. and Mallery P., (2003), SPSS for Windows step by step: A simple guide and reference, 4th edn, Boston: Allyn & Bacon.
  35. Glaser B. G., (1992), Emergence versus forcing basics of grounded theory analysis, Mill Valley, CA: Sociology Press.
  36. Glaser B. G. and Strauss A. L., (1967), The discovery of grounded research: strategies for qualitative research, New York, NY: Aldine De Gruyter.
  37. Greene J. C., Caracelli V. J. and Graham W. F., (1989), Toward a conceptual framework for mixed-method evaluation designs, Educ. Eval. Pol. Anal., 11(3), 255–274.
  38. Griffith K. G. and Nguyen A. D., (2006), Are educators prepared to affect the affective domain? Nat. Forum Teach. Educ. J., 16(3), 1–4.
  39. Hmelo-Silver C. E., (2004), Problem-based learning: what and how do students learn? Educ. Psychol. Rev., 16(3), 235–266.
  40. Jungert T. and Rosander M., (2010), Self-efficacy and strategies to influence the study environment, Teach. High. Educ., 15(6), 647–659.
  41. Kan A. and Akbaş A., (2006), Affective factors that influence chemistry achievement (attitude and self efficacy) and the power of these factors to predict chemistry achievement, J. Turkish Sci. Educ., 3(1), 76–85.
  42. Keller J. M., (1987), Development and use of the ARCS model of instructional design, J. Instr. Dev., 10(3), 2–10.
  43. Lincoln Y. S. and Guba E. G., (1985), Naturalistic Inquiry, Beverly Hills, CA: Sage.
  44. Liu M., Hsieh P., Cho Y. J. and Schallert D. L., (2006), Middle school students' self-efficacy, attitudes, and achievement in a computer-enhanced problem-based learning environment, J. Interact. Learn. Res., 17(3), 225–242.
  45. Lohman M. C. and Finkelstein M., (2002), Designing cases in problem-based learning to foster problem-solving skills, Eur. J. Dental Educ., 6(3), 121–127.
  46. Majere I. S., Role E. and Makewa L. N., (2012), Gender disparities in self-concept, attitude and perception in physics and chemistry, Atlas J. Sci. Educ., 2(1), 61–69.
  47. Mogane M. G., (2010), Relationship between the attitude of first year medical students towards chemistry and their learning outcomes. (Master's thesis). University of South Africa. Retrieved from http://uir.unisa.ac.za/bitstream/handle/10500/3266/dissertation_mogane_m.pdf?sequence=1.
  48. Moos D. C. and Azevedo R., (2009), Learning with computer-based learning environments: a literature review of computer self-efficacy, Rev. Educ. Res., 79(2), 576–600.
  49. Myers R. E. and Fouts J. T., (1992), A cluster analysis of high school science classroom environments and attitude toward science, J. Res. Sci. Teach., 29(9), 929–937.
  50. Nicolidau M. and Philippou G., (2004), Attitudes towards mathematics, self-efficacy and achievement in problem-solving, Euro. Res. Math. Educ. III, Thematic Group 2, pp. 1–11.
  51. Nolen S. B., (2003), Learning environment, motivation, and achievement in high school science, J. Res. Sci. Teach., 40(4), 347–368.
  52. Piaget J. and Inhelder B., (1958), The growth of logical thinking, from childhood to adolescence, London: Routledge & Kegan Paul.
  53. Rajab A. M., (2007), The effects of problem-based learning on the self efficacy and attitudes of beginning biology majors, Doctoral dissertation, University of California, Los Angeles.
  54. Ram P., (1999), Problem-based learning in undergraduate instruction. A sophomore chemistry laboratory, J. Chem. Educ., 76(8), 1122–1126.
  55. Richardson V., (2003), Constructivist pedagogy, Teach. Coll. Rec., 9, 1623–1640.
  56. Rose W. D. L., (2003), Student self-efficacy in college science: an investigation of gender, age, and academic achievement, Master's thesis, University of Wisconsin-Stout.
  57. Sabot R. and Wakeman-Linn J., (1991), Determinants of performance in introductory courses in economics and seven other disciplines, retrieved from http://sites.williams.edu/wpehe/files/2011/06/DP-7.pdf.
  58. Savery J. R. and Duffy T. M., (1995), Problem-based learning: An instructional model and its constructivist framework, Educ. Tech., 35(5), 31–38.
  59. Senay A., (2010), The contribution of chemistry self-efficacy and goal orientations to eleventh grade students' chemistry achievement, (Master's thesis), Middle East Technical University, retrieved from http://https://etd.lib.metu.edu.tr/upload/12612573/index.pdf.
  60. Smist J. M., (1993), General chemistry and self-efficacy, (Report No. SE054247), Massachusetts, ERIC Document Reproduction Service No. ED368558.
  61. Stemler S., (2001), An overview of content analysis, Practical Assess. Res. Eval., 7(17), retrieved May 13, 2014 http://from http://PAREonline.net/getvn.asp?v=7&n=17.
  62. Tashakkori A. and Teddlie C., (1998), Mixed methodology: combining qualitative and quantitative approaches, Appl. Soc. Res. Meth. Series, vol. 46, Thousand Oaks, CA: Sage Publications.
  63. Valentine J. C. and Cooper H., (2003), Effect size substantive interpretation guidelines: Issues in the interpretation of effect sizes, Washington, DC: What Works Clearinghouse.
  64. Vygotsky L. S., (1978), Mind in Society: The development of higher psychological processes, Cambridge, MA: Harvard University Presspp. 79–91.
  65. Yalcin B. M., Karahan T. F., Karadenizlie D. and Melih E., (2006), Short-term effects of problem-based learning curriculum on students' self-directed skills development, Croation Med. J., 47(3), 491–498.
  66. Zimmerman B. J. and Kitsantas A., (1997), Developmental phases in self-regulation: Shifting from process goals to outcome goals, J. Educ. Psych., 89, 29–36.
  67. Zimmerman B. J. and Lebeau R. B., (2000), A commentary of self-directed learning, in Evensen, D. H. and Hmelo-Silver, C. E. (ed.), Problem-based learning: A research perspective on learning interactions, Mahwah, NJ: Lawrence Erlbaum Associates, pp. 299–313.
This journal is © The Royal Society of Chemistry 2015