Pharmacology students’ perceptions of creating multimodal digital explanations

Abstract

Students can now digitally construct their own representations of scientific concepts using a variety of modes including writing, diagrams, 2-D and 3-D models, images or speech, all of which communicate meaning. In this study, final-year chemistry students studying a pharmacology subject created a “blended media” digital product as an assignment to summarize an independently prepared technical literature review on a current research topic in pharmacology for a non-expert audience. A blended media is a simplified way for students to combine a variety of modes to complement a narration to explain a concept to others. In this study, the students learned how to create a blended media during a one-hour workshop, and used the technique to create the representation as an assessment task. The research question that guided the study was, “What are the students’ perceptions of making a digital product such as blended media and how did these shape their multimodal awareness?” We draw from theoretical perspectives in multimodalities, representations and meaning making. Data included interviews at three points of the semester, the literature review and the digital media product. We present three case studies with volunteering students, who demonstrated a strong awareness of effective communications techniques as they attended to the audience. Making a blended media is a creative way for chemistry students to summarize complex scientific information and as a task may help to focus their multimodal awareness and developing communications skills.

---

Introduction

Chemistry education and science education more generally have long recognized the value of having students interpret and/or generate single or multiple representations (Ainsworth, 1999, 2006; Ferk et al., 2003; Goldman, 2003; Kozma, 2003; Seufert, 2003; van der Meij and de Jong, 2006; Cheng and Gilbert, 2009; Gilbert and Treagust, 2009; Davidowitz et al., 2010; Adandan, 2014). There is a new and growing body of research in science learning that demonstrates the value of students constructing their own representations of science concepts using a range of modes (Prain and Waldrip, 2006; Waldrip et al., 2006; Hand et al., 2009; Tytler et al., 2013; Cooper et al., 2015).

Researchers in language and literacy describe a mode as a “meaning-making system in order to articulate the meanings demanded by the social requirements of particular communities” (Kress et al., 2001, p. 43). Specific to chemistry learning, there are a number of forms or modes of meaning making including writing (Prain, 2006; Hand and Choi, 2010; Ritchie et al., 2010); diagrams (Prain and Waldrip, 2006; Subramaniam and Padalkar, 2009; Hubber et al., 2010; Carlisle et al., 2015); flow charts (Davidowitz et al., 2005); visualisations (Gilbert, 2007; de Berg, 2012); physical models (Prins et al., 2008); and powerpoint presentations (Waldrip et al., 2006). Each of these has a role in communicating scientific knowledge. When students represent their own ideas through generating writing, images, models and voice, or, a combination of modes (Lemke, 2000; Prain, 2006; Prain and Waldrip, 2006), they make meaning.

Meaning is made when learners create their own representations because they ‘translate’ information into different representational forms. Further, when creating multiple representations there is an added benefit for learning: “multiple representations refers to the practice of re-representing the same concept through different forms, including verbal, graphic and numerical modes, as well as repeated student exposures to the same concept” (Prain and Waldrip, 2006, p. 1844). Thus, when students re-represent information by creating additional representational forms using different modes, they may enhance their understanding of the science concepts. According to Yore and Hand (2010), “The transformation among multimodal representations has the greatest potential in promoting learning and depth of processing” (p. 96). While this translational work is important, it may be limited in upper-level university chemistry programs or subjects due to the heavy content load and dependence on written assignments and examinations.

In the contemporary context of university chemistry education in Australia, Learning and Teaching Academic Standards (Jones et al., 2011) specify that tertiary science graduates should be effective communicators of science. More particularly, the standards expect science graduates to “engage with their audience and be able to convey their message in a clear and understandable manner” (p. 14). The audience could be peers, scientific non-experts or the general community. Thus, effective communication for a range of audiences and purposes, and represented in a variety of modes including new media, is an important graduate outcome for university chemistry programs.

Theoretical background

In order to better understand how students make meaning when they develop multiple representations, we draw from the field of semiotics, which is about sign systems. When learners interpret information in a representation generated elsewhere, such as in a diagram or piece of text, they must compare their own interpretation of the expert-generated representation to the referent or object being represented (Peirce, 1931/1955). The dynamic interplay between the representation (the sign), the referent (the concept/content being represented) and the meaning made (personal interpretation) creates an important triadic relationship for meaning making:

“A sign, or representamen, is something which stands to somebody for something in some respect or capacity. It addresses somebody, that is, creates in the mind of that person an equivalent sign, or perhaps a more developed sign. That sign which it creates I call the interpretant of the first sign. The sign stands for something, its object. It stands for that object, not in all respects, but in reference to a sort of idea…that is to have a like content…. The triadic relationship is genuine, that is its three members are bound together by it in a way that does not consist of any complexus of dyadic relations.” (italics in original) (Peirce, 1931/1955, pp. 99–100).

Peirce diagrammed this dynamic relationship between the representation, the referent and the meaning or idea generated in his triadic model of a semiotic system, as shown in Fig. 1. Science educator Jay Lemke (1998) built on Peirce's (1931/1955) model and explained the interplay of three influences on the semiotic system:

Fig. 1 Peirce's (1931/1955) triadic model of a semiotic system.

“When we construct meaning we always simultaneously construct a ‘presentation’ of some state-of-affairs, orient to this presentation and orient it to others and in doing so create an organized structure of related elements…. How much more is this true when we take into account the co-deployment of visual, gestural, graphical, mathematical and other resource systems!” (Lemke, 1998, pp. 91–92).

Semiotic theory helps us understand why it is important to have learners interpret representations made by experts and then generate their own. Thus, the field still holds currency for science learning, even though Peirce's work originated early in the 20th century. According to Waldrip et al. (2010), “with any topic in science, students’ understandings will change as they seek to clarify relationships between their intended meanings, key conceptual meanings within the subject matter, their referents to the world, and ways to express these meanings” (p. 67).

If university students are given opportunity to design and make their own representations or “signs” of scientific concepts, they must make decisions about the relationship between the content (referent), the construction (representation) and the most appropriate mode/s to use. Hoban and Nielsen (2012) showed that it is important for pre-service teachers to consider which mode to use—written, auditory or visual—as the most suitable for a particular concept and for a particular representation. The wide range of possible modes means that students now have creative opportunity for multimodal representation in the context of their disciplinary science learning in university.

Representations that include more than one mode are ‘multimodal’ and student awareness of modal choices helps to create a coherent product, as noted by Jewitt (2009): “semiotic mode[s] combine as resources for meaning making…as the resources of different modes are combined, meanings are corresponding, complementary or dissonant as they harmonize in an integrated whole” (p. 301). Royce (2007) called this ‘intersemiotic complementarity,’ where modal choices are made in the design process so that representations are complementary. Chemistry, as a science discipline in universities, is heavily dependent on limited modes of communication such as writing and sketches as exemplified in written examinations and laboratory reports. However, digital technologies enable students to not only use more modes to communicate, but also to combine them in interesting ways. Attending to the need for complementarity across a range of representational forms enhances meaning making and may provide opportunity for chemistry students to therefore develop communications skills if they have to explain the concepts to someone else. In our digital world, students have increasing opportunity to be multimodal designers and can utilize a wide range of easily accessible digital media forms, where novel multimodal assessment tasks become possible.

Creating representations with Web 2.0 technology

In countries such as Australia, mobile digital devices are ubiquitous. Virtually every university student has a smartphone, tablet and/or laptop. Many science learners use several of these devices concurrently (Moll and Nielsen, 2016) and Gen Y learners are assumed to be highly proficient in their use. Since Prensky (2001) labelled them ‘digital natives,’ learners have gained unparalleled access to technological devices such as smartphones, digital still cameras, video cameras and tablets, supported by Web 2.0 affordances. The ease of access means that students can learn from technology by accessing information, but this access also enables them to learn with technology by working with this information to generate different forms of representations across multiple modes to illustrate their own knowledge (Jonassen et al., 1996; Kim and Reeves, 2007; Willett, 2007). As a result, new media is increasingly being used in science teaching at all levels and provides exciting opportunities for learning: “manipulation of computer-generated texts, where students integrate imagery, sound, mathematical symbols, diagrams, and writing, alters the role of written language as the major or dominant medium of learning” (Prain, 2006, p. 180).

Teachers in schools have found it challenging to integrate the use of new technology forms in their classroom practice (Howard et al., 2015; Howard and Gigliotti, 2016) and academics similarly struggle to meaningfully integrate contemporary educational technologies into their teaching (Martinovic and Zhang, 2012; Bennett et al., 2015). Part of the challenge of integrating digital technologies involves pedagogies and strategies to have students use the technologies for learning. For example, having students create media forms, such as animations (Sperling et al., 2003; Yang et al., 2003) or simulations, has previously involved expensive or complicated software, and thus, these media forms were difficult for students to produce in the classroom. A very simple form of student-generated animation is now available. Hoban and colleagues (Hoban, 2007, 2009; Hoban and Nielsen, 2010, 2012; Hoban et al., 2011) developed slowmation as a stop-motion animation technique where the creator manually moves the objects whilst taking digital still photographs at predetermined intervals. This technique overcomes the limitations of complicated software programs in animating objects and has been used with a wide range of learners, including pre-schoolers (Fleer and Hoban, 2012); preservice primary teachers (Hoban and Nielsen, 2012); and preservice secondary science teachers (Kidman et al., 2013; Amos, 2015; Keast and Cooper, 2016; Paige et al., 2016).

There are even simpler techniques for students to create other forms of digital representations. A podcast is an audio file, recorded using a microphone attached to a computer or using the built-in microphone on a smartphone. Bartle and colleagues (Bartle et al., 2011; Pegrum et al., 2014; Bartle, 2016) introduced a podcast assignment in their undergraduate chemistry course. The task was constrained to a three-minute recording based on one of two key chemical concepts: ‘acid and base’ or ‘oxidation and reduction.’ Students created a wide range of podcasts, many of which were simple reproductions of facts about the types of reactions. However, some students developed highly contextualized and creative podcasts where metaphors or analogies were used to explain the concept. The entertaining and informative media products developed by students in pairs represent high levels of student knowledge as well as deep engagement.

A more complex example of student-generated media is offered by Vanderlelie (2013), who has biochemistry students create a ‘musical explanation’ Vanderlelie (2016) to explain a biochemical pathway. She reports that students create video products that are often highly engaging and very creative and use a range of representational forms within the video to complement music, images, diagrams and labels. A recent edited book by Hoban et al. (2016b) includes many more examples of a variety of student-generated media products, clearly demonstrating the range of strategies in current use at universities around the world to engage science learners. As students become designers of media, they ‘think like teachers’ which promotes multiple ways to make meaning:

“The process of articulating their knowledge about the domain…forced them to reflect on that knowledge in new and meaningful ways. We have all stated at one time or another that the quickest way to learn about subject matter is to have to teach (design) it…when learners actively construct knowledge from multimedia, they acquire cognitive, metacognitive and motivational advantage over learners who attempt to absorb knowledge” (Jonassen et al., 1996, pp. 95–97).

Drawing from the fields of student-generated media in science learning, representations, multimodalities and semiotics, this study explores the experiences of chemistry students taking a pharmacology subject and making a blended media explanation. We ask the following research question: What are the pharmacology students’ perceptions of making a digital product such as blended media and does this help them develop communication skills and multimodal awareness?

Method

Participants and context

A class of 40 final year students were enrolled in a 13-week subject, Principles of Pharmacology, which is required in the Medicinal Chemistry degree and a final year elective in other Bachelor of Science degrees at the University of Wollongong in Australia. Principles of Pharmacology is an introductory subject for chemistry students covering basic pharmacological concepts, such as the molecular basis of drug action, including receptors, pharmacokinetics, bioavailability and factors affecting drug activity. The subject involves a laboratory component that includes computer simulations, but for the purposes of this paper, the key tasks are a literature review (15% of the subject mark, due in Week 10) on a contemporary area of investigation in pharmacology that is complementary to lecture material and a culminating presentation (5% of the subject mark, due in Week 12) on the findings of the literature review that is presented as a ‘blended media’ (Hoban et al., 2016a). Students choose three preferred topics from an instructor-supplied list that frames the inquiry as a question. Examples of questions include: Is ferroquine a novel anti-malarial? What therapeutic options are available for treating Crohn's Disease? And, Could the spice Curcumin be a useful drug in the clinic? The list also includes a leading journal article for each question as a starting point for the literature review. Students ‘bid’ on topics during Week 2, where randomly, students are called on to nominate their preference among the remaining topics. Students also had a one-hour training session on using tools such as Scopus and SciFinder to search the chemical literature and a one-hour workshop where they learned about developing a blended media as an explanation. The literature review was specified as a scientific report of length 8–10 pages that the students had eight weeks to prepare.

Chemistry students are increasingly expected to engage with primary literature (see for example, Forest and Rayne, 2009; Ma, 2015; Shea et al., 2016) and tasks such as a literature review offer students a means to synthesise such reading. The 3–5 minute blended media presentation was to outline the literature review and answer the guiding question for a non-expert audience, which gives the students the opportunity to work with additional representational forms (of their choosing) to recontextualise the information – a key element of multimodal learning and effective communication of complex information. A mini ‘film festival’ was held in Week 12, which was two weeks following submission of the literature review and each blended media was played for the class. While the literature review task is not the focus of the current article, it is important to acknowledge the relationship between this task and the blended media product.

Participants in the current study include three students (two male, one female) who volunteered to take part in this research. The three case students were among seven who were studying either Medicinal Chemistry or Medical Health Science degrees and volunteered to participate in the research. Each of these students constitutes a case (Stake, 1995; Merriam, 2007; Yin, 2014). The three examples were chosen for this paper because the range exemplifies the wider data set and the participants used very different styles in their presentations. The research was conducted according to protocols reviewed by the University of Wollongong's Human Research Ethics Committee.

Data gathering methods

The current study used several qualitative methods to gather data for subsequent analysis:

Interviews with participating students. The first and second authors conducted three interviews with each participant at three points in the term. A preliminary interview was conducted immediately after students chose the topic in Week 2. These interviews probed the students’ prior knowledge on the chosen topic; reasons for choosing that topic; how they planned to conduct the literature review; and their experience in making digital media products. The second interview took place right after the students submitted the literature review task. Questions asked students to summarise the topic, including what they had learned about it and further probed how they had conducted the literature review. They were also asked about their general approaches to learning new and complex information and some basic demographic questions. The final interview occurred in the two weeks following submission of the blended media assignment and focused on their perceptions of the task, and included decision making during construction.

Interview with the subject instructor (author 3). The instructor was interviewed early in the term and questioned about his goals for the task; the students’ responses to it, including how they engage with the parts of the task; the intentions behind the parts of the literature review and blended media tasks; and his experience with different kinds of student presentation formats.

Collection of artifacts. The literature review and blended media were collected as artifacts. All interviews were transcribed verbatim and scripts for the blended media products were collected where available or generated by a research assistant. In this study, no data were gathered during the time that the students were actually constructing their blended media products. Instead, interview data provide a window into students’ perceptions of the tasks. Their design choices are explored as possible indicators of multimodal awareness and knowledge of effective communications strategies.

Data analysis

The data were analysed qualitatively for: (i) students’ perceptions about their experiences of making the blended media products, including how this developed out of the literature review part of the overall task; and (ii) the students’ design choices in making the blended media. We were assisted in our analysis through a specialised software program called Multimodal Analysis [MMA] (O’Halloran, 2013). This software allows for fined-grained coding and synthesis of video (e.g. blended media) and text-based data (e.g. interview transcripts). In our coding, we used a framework of ‘intermodality’ (Painter et al., 2013) to focus on how meaning is communicated through the modal choices made in the blended media presentation. Intermodality refers to the relations between meaning portrayed in text type, in this case the blended media, and what is represented using various modes such as text, labels, diagrams, video or narration. To process each blended media product in the data set, we segmented the digital media into clauses and with the MMA software, identified the mode used in each segment along with a time stamp for each transition across the explanation. This served to summarise the participants’ modal choices as represented in the blended media, but also to highlight interesting and creative depictions that subsequently served as a form of stimulated recall in the final interviews.

Interview data were analysed thematically with a focus on the students’ design intentions, in particular, for communicating the scientific information. For example, during the final interview, episodes identified in the MMA analysis were the focus of interview questions that asked about the reasons for the design choice. Thus, in the MMA analysis we identified semiotic resources that appeared to support the explanation, and through the final interviews, we interpret how the students’ decisions to utilise particular resources supported meaning making through construction of the blended media product.

Results

Results are presented as case studies of the three students. Each case begins with background about each student and the question that was the focus of the literature review task. All participating students had some experience with making digital media products prior to this subject, although none had made such a product as a task in any subject during their degree programs. During the literature review task, the students primarily read journal articles but also reviewed notes and texts from the subject. The blended media summarised the literature review for a non-expert audience.

Case study 1. David: the therapeutic potential of TRPV receptors

David was studying a Medical Health Science degree and was interested in the medical effects of drugs and how they can be used to improve health. He chose the following question for his literature review and blended media presentation: What are the therapeutic possibilities for targeting vanilloid (TRPV1) receptors and what problems are associated with this drug target?

During the first interview, David indicated that he chose the topic because he had never heard of TRPV receptors and so was curious to learn about them and what they did. David knew that receptors in general were “important in terms of cell-to-cell communication and hormone action…and vastly important for biological processes” (first interview, March 16, 2015). During the second interview (May 14, 2015), which followed the submission of the literature review assignment, he was asked what he had learned about TRPV receptors, but also about his process for learning about the topic. Through reading a number of research articles while preparing the literature review, he came to understand that TRPV receptors play an important role in pain detection, relaying information to the central nervous system. A response can be evoked by both agonists and antagonists. Further, TRPV receptors are also involved in temperature regulation, but the mechanisms are unclear.

Representations and perceptions of making a blended media. In David's third interview, which occurred after the blended media assignment was submitted, we asked him about his process of making the blended media. David explained that he used images or videos found via a Google search to portray aspects of his literature review, including animated molecular models; still images taken from the internet; hand-drawn posters depicting therapeutic pathways and effects using highlighter pens to add colour to labels; and an analogy of a competition between two contemporary rap music artists to compare activation by antagonists and agonists.

David used a 3-D computer-generated rotating colour model of the TRPV receptor depicting all of the subunits and binding sites while the voiceover narration described the molecule and its activity in the body: “The TRPV1 receptor is a serpentine receptor consisting of six trans-membrane helices and one core domain between helices five and six.” The short video clip showed the molecule rotating first in the horizontal plane and then in the vertical plane, which displayed the membrane helices and peripheral projections. To illustrate three mechanisms for activating the molecule, David presented a hand-drawn poster (shown in Fig. 2) that added colour highlights as the words were stated in the narration.

Fig. 2 David's hand-drawn poster from his blended media.

At 01:09 of the blended media, an 18-second animation of a generalised cell membrane ion channel opens as activators attach. Molecules enter the channel as the voiceover says, “In regards to the therapeutic potential of targeting TRPV1 receptors, both agonists and antagonists have proven beneficial at reducing pain and other disease symptoms, but which one is more useful in a clinical context?” While the channel depicted was not actually the TRPV receptor (because a Google search did not produce any), David aimed to illustrate a molecular passage in the body, which helped to set the context for the viewer to understand the functioning of TRPV receptors in the body.

Drawing labels and adding colours was also part of his blended media, which reflected his own style of learning, as he noted during the third interview: “I write all my study notes and I go through with a highlighter…I do like to visualise things, especially in regards to pathways.” He added colour highlights to the labels for the activity of the TRPV receptor, using the same colour pattern consistently throughout the blended media. His choice to include similar modal choices in his blended media reflected his own way of learning: “if I was to read ‘it binds here and it does this and it activates this which goes to the brain’, that's a lot harder for me to interpret as opposed to seeing a diagram of it. So yeah, that's my personal way of learning put into there” (third interview).

David used an analogy of a competition between the two rappers Kanye West and Dr Dre to personify (Lakoff and Johnson, 2003) the differing pathways for agonist and antagonist activation: “they’re like music artists that I like….and they both kind of compete; it's like who is the most popular rapper?” During the third interview, David spoke about using the analogy of two popular rappers to introduce an element of humour, but also to help the viewer understand the nature of the competition in agonist and antagonist pathways for pain relief. Other aspects of the explanation for the agonist pathway were emphasised with still images of, for example, topical analgesic creams such as Zostrix and Capzasin-P and an image of a red hot chilli pepper. David's presentation in the blended media also included other hand-drawn lists of diseases and symptoms eased through agonist action on the TRPV receptor to decrease the sensation of pain.

David scripted the narration for his blended media using an iPad and organised the presentation according to the headings in his literature review. The flow of the blended media was similar to the literature review, not in terms of complexity of content represented, but in sequence of sections: both tasks began with a definition and the context for where the receptor functions, but while the literature review moved to a technical comparison of the agonist and antagonist functions of TRPV, the blended media introduced the analogy of the rap artists. He recorded the narration as the last step in his production process. In the follow-up interview, David noted that he had not made a ‘blended media’ previously in the degree program, but had much experience with PowerPoint presentations, writing essays and lab reports. David noted in the third interview that the blended media task afforded a different kind of learning experience: “I was able to learn about as opposed to just writing something in a Word document.” The more typical kinds of assessment tasks in science degrees rely on working with written texts and then generating a text-based product. Working with multiple modes for the blended media provided a context for using a range of visualisations, which makes for interesting viewing, but can also help learning, according to David.

Case study 2. Ruby: the potential of interleukin-21

Ruby was studying Medicinal Chemistry and was also interested in the medical effects of drugs and how they can be used to improve health. Her question for the literature review and blended media presentation was: What is the potential for targeting Interleukin-21 (IL-21) in immune-related diseases? Even as Ruby had chosen the topic, she admitted that she did not know much about it and was interested in the specificity of the topic rather than a more general question that other students seemed to favour: “so it was very specific, talking about one receptor…and I thought it's targeting lots of different autoimmune diseases and that’ll give me lots of different things to talk about” (first interview, March 12, 2015). In choosing the topic, she had quickly searched the internet with the search term ‘Interleukin-21’ and found many references to rheumatoid arthritis and autoimmune diseases, which she interpreted favourably because it meant that she could readily find information about the topic. Consistent with the task to generate a literature review to answer the guiding question, she made a plan to begin with the structure of the receptors, then learn how they work while considering treatments for different diseases.

Ruby had some prior experience with digital media production, having previously helped a PhD student with research on slowmation. Her work on that project involved making a ‘claymation’ of a process from analytic chemistry, so she was familiar with the technology tools and software to use in generating a media product.

Representations and perceptions of making a blended media. Ruby used the free version of VideoScribe to make her blended media, which ran for 05:43 and used a combination of modes, including VideoScribe objects; still images; animations; diagrams of the JAK-STAT pathway; text; labels; and a narration. She chose VideoScribe because of its collection of digital objects including pictures, cartoon-like characters and arrows, all of which can be easily manipulated: “you just put them wherever you want them to go” (third interview). The user can also import pictures or videos from elsewhere and easily record an audio track. Ruby prepared a script for the narration, made an overall plan similar to a storyboard and then assembled images and objects into the blended media. Finally, she adjusted timing within VideoScribe until she was satisfied with the overall alignment.

The blended media begins with a cartoon of a castle protected by ninja warriors (see Fig. 3). The analogy with IL-21 was developed through the cartoon imagery, and explained the system in general terms, including the action of cytokines. The media product moves on to the more specific description of interleukin-21, including where it is produced and the roles it plays in the body controlling the activity of T cells, B cells and NK cells.

Fig. 3 Ruby's castle diagram.

At 01:26 of the blended media, the voiceover describes how Il-21 binds to its receptor. Ruby used objects from within the VideoScribe program and animated them to simulate operation of the immune response as a cytokine binds and activates the JAK-STAT pathway. Further imagery is shown in animated form to illustrate the hypothesised role of interleukin-21 in immune-mediated diseases such as rheumatoid arthritis and lupus, using the ninja analogy introduced at the beginning.

During the first interview, Ruby was asked to consider the upcoming blended media task in this final-year chemistry subject. She noted: “I think it's an interesting approach.”

However, writing was her preferred way of learning: “I’ll probably learn best from writing the actual scientific article, from reading the things and then writing and trying to understand it that way as opposed to actually doing a movie.” On some level, she felt that reading and writing were more efficient for learning.

During the second interview, which followed the submission of the literature review portion of the task, we queried what she now understood about the topic. She offered a detailed explanation starting with what interleukins are and how they function as cell signals when binding to a specific receptor, and admitted: “it was definitely difficult to condense a very difficult topic into something more generally for a wider science-based audience as opposed to a very specific immunologist.” The difficulty was a result of the nature of the complexity of the topic and the need to include specific immune cells and responses.

During the third interview, she was asked to reflect on the value of the blended media task for her learning:

“I don’t know if it necessarily benefits me any more than doing the literature review because I feel I’m just doing it because [that is] the assessment task that I needed to do, whereas I’d done all of the learning and the research and all of that kind of thing in the paper.”

Ruby felt confident with a more standard format of presentation to demonstrate her learning. She did, however, admit during the final interview that there was a different sort of communication element to the blended media task:

“They are definitely based more on the audience than say the topic or anything like that or how much knowledge you have beforehand….I tried to make it look a little bit nicer having little images and a castle and having cute little bugs and stuff like that so it isn’t just text. I try not to have a lot of text in it because that is really boring to look at.”

Ruby is clearly aware of the value of using different media forms and modes to represent information so that the product is interesting for the viewer. Attending to the viewer is a key aspect of effective communications and the final case is about Parker, who developed an interesting storyline in his blended media.

Case study 3. Parker: organometallics

Like Ruby, Parker was also studying Medicinal Chemistry. He was interested in the history of drugs and knew that early therapeutic approaches involved metals and metal compounds. He chose the following question for the literature review and blended media presentation: Organometallic anti-cancer compounds: Can metals save the day? This question had not been his first choice for the task, although he was interested in new discoveries in the field, which made the topic appealing to him.

Representations and perceptions of making a blended media. Parker's blended media runs for 05:27 and involves a role play with three characters: The President, The Surgeon General and The Scientist, all played by Parker, including costumes, hats, glasses and props (see Fig. 4). The storyline involves The Surgeon General imploring The President to start a new battle on cancer, and then calls in The Scientist who makes a presentation about three organometallics and their potential as anti-cancer drugs. In the three-way dialogue, The Scientist makes a pitch for funding to continue the research, which The President handsomely rewards along with a team of top-notch graduate students.

Fig. 4 Parker as the surgeon general.

Parker prepared his blended media using iMovie. For labels or text, he wrote on a piece of white poster paper with a black marking pen. The Surgeon General is dressed as a general complete with military cap and uniform and refers to a hand-drawn graph posted on a backdrop of a bed sheet secured to the brick wall behind him. The graph depicts “Battles lost” in the war on cancer while introducing the nature of cancer and disease progression. The camera cuts away to The President, in business suit and tie looking thoughtful and authoritative. The President sits in front of a red curtain, agrees that current treatments are not working and calls off-camera to “Bring in the Scientist!”

The Scientist (Parker), dressed in lab coat and safety glasses, nervously addresses The President and gives a presentation that describes three different organometallic drugs as possible treatments for cancer. The Scientist uses props of hand-drawn diagrams of a generic organometallic molecule, diagrams of each molecule and pathways for each of the molecules as he describes the reaction pathways and how each compound might be utilised in cancer treatment.

Parker chose to make the presentation as a video instead of stop-motion techniques because he had found them time-consuming and ‘fidgety’ in his previous experience with the technique. In the final interview, he was asked about preparing the presentation. He noted that he attended to the audience by considering the kinds of questions they may ask and through this, “realised that there was actually a lot of gaps in the papers on the basic chemistry of it all so I had to just go through and come up with all different sort of formulations and mechanism.” For example, the journal article that dealt with rhenium did not talk about how the radical species actually react with tissue. Parker's diagrams of chemical pathways were originally going to be more complicated, but he realised that to explain them would require much more time, so instead, he provided a diagrammatic overview.

Parker's presentation used humour effectively and this was intentional as he commented during the third interview:

“I have a tendency to be quite official when I’m giving a presentation; I guess just to the point and saying all that information, having really planned slides, easy to understand…but with this, especially the way it was introduced to us, it said to me ‘You can do this funny and you won’t get punished for it,’ like, go crazy, do whatever, be creative.”

His comments reflect a sense that this was a different kind of presentation that allowed for some creative license in a way that a standard presentation does not. He chose the storyline on purpose: “I felt it was kind of important to, rather just giving information…have a story that people can follow and then putting humour factors makes people laugh so keep them interested.”

Blended media as presentation of the literature review

Written tasks are a common part of assessment tasks in chemistry subjects and science degrees. In previous iterations of the subject at the centre of this research, the literature review task included an oral presentation, which had become “a bit of a stale task for everybody”, according to the instructor. Further, the subject had grown from 15 to 40+ students over time, with future increases expected, so that a 15 minute presentation for each student had become untenable. While valuing both the literature review and presentation parts of the task, the instructor sought a way to adapt the presentation portion of the task. He also valued other aspects of the task, as noted during the interview:

“We found that our students, especially the ones coming in to PhDs are very good at their specialised topic and talking about that and we wanted to create students that could communicate more broadly, so using other media forms and also talking to a broader audience because communicating science and health topics is extremely important…so that communication bridge is a big picture kind of thing.”

While the focus of the current paper is not on evaluating the quality of the blended media products that the students made, the tasks were marked in class during a final class meeting of the semester. The instructor used a rubric identifying categories including “Organization” (2 marks: logical sequence, references cited appropriately); “Content” (4 marks: attention-grabbing introduction, terms defined, accuracy, relevant material, balance of detail and generality); “Presentation” (4 marks: clear, high quality digital media, conveys material to a general audience, suitable length). As a 5% assignment, some students felt that the amount of work involved was inconsistent, however, all three case students in the study earned 5 out of 5, while the overall class average was 3.85.

Our case participants reported minimal technical challenges in producing their blended media. While some had much prior experience with a range of digital production techniques, most had not. That they attempted new techniques and produced interesting and engaging blended media presentations suggests that any challenges faced were manageable.

Discussion

Each of the students in this study produced a digital product using blended media to explain a pharmacology topic that was an extension of an earlier literature review task where they answered a guiding question about the topic. The main finding from this study is that in making a digital explanation for non-experts, each of these final-year chemistry students generated a scientific explanation and demonstrated an awareness of how to represent scientific content in a variety of ways using a range of modes that were integrated in a digital media product intended for a non-expert audience.

Furthermore, the students used modes of communication that they had not previously incorporated into assignments, which makes blended media a different kind of assignment. With affordances beyond producing a presentation in PowerPoint, generating a digital explanation uses a combination of modes while using personal technology tools (smartphone, digital camera, everyday materials and software on their own laptop computers). Today's students have easy access to such technology and are fluent users. With a minimum of guidance, they can utilise these tools to generate digital presentations of complex scientific information. This makes their use in assessment tasks relatively straightforward. Importantly, such tasks go beyond concept development to include communications skills and multimodal awareness. The task can also provide creative opportunity as it engages students in media making or translating other expert-generated representations for use in the blended media. The study also adds to the body of science education literature on the range of modal forms of representations and forms of communication beyond diagrams (Prain and Waldrip, 2006; Subramaniam and Padalkar, 2009; Hubber et al., 2010); flow charts (Davidowitz et al., 2005); models (Prins et al., 2008); PowerPoint presentations (Waldrip et al., 2006); and slowmation (Hoban, 2005, 2007; Hoban and Nielsen, 2013; Hoban et al., 2016b).

In the current study, the chemistry students used knowledge they developed through the literature review component of the task and then developed a digital explanation to communicate this knowledge for a non-expert audience. In developing a digital explanation, the students made many decisions about how to best represent the science concept(s), which involved the use of different combinations of modes. For example, David used an analogy to support and personify the illustration of the agonist and antagonist activity of TRPV1, making it memorable. Further, he had decided to take on the topic of TRPV1 receptors because he did not know very much about them. As a final year medical health student, he sought to develop his own knowledge through the task. Parker was focused on generating an interesting and humorous presentation through the digital media and believed that he had license to do so within the format. The result was an engaging and interesting presentation.

Some of our participating students commented on the length of time it took to prepare the blended media. The fact that it was a very small percentage of the overall subject mark (5%) meant that some students felt their efforts were not adequately rewarded. However, we see a key benefit of this strategy in the fact that students need to attend to the audience as they construct the blended media. By doing so, they consider not only the information represented through the modes used, but also how to use the different modes to make the messages and information engaging and interesting. We suggest that this is a manifestation of multimodal awareness that is enabled by virtue of the task, seemingly affording a creative license beyond a traditional PowerPoint presentation, which students acknowledged had not been occasioned in their degree programs before.

Implications

From a theoretical perspective, when chemistry students make a digital explanation, they must think about the concept they are trying to represent (the referent); which representation(s) to use; and which modes can be used in a complementary manner to provide a coherent explanation of a scientific concept (Peirce, 1931/1955; Lemke, (1998); Kress et al., 2001). For example, David used several modes in his blended media (diagrams, text, photos of real objects, images of 2-D and 3-D models, voiceover and an analogy) to explain the activity of the TRPV1 receptor; Ruby used digital objects found in the VideoScribe program in addition to images and video she found on the internet; and Parker told a story through a role play and used a range of props. The challenge for the chemistry students is to re-represent scientific content by using technology to choose and/or match different modes so that the voiceover of the narration complements the visual mode of images, video or other modes to clearly explain the science concept. In essence, the task required the students to design the blended media as a multimodal representation with ‘intersemiotic complementarity’ (Royce, 2007, p. 63).

There is value in creating blended media presentations that will be shared. Because a blended media is a digital product that can be displayed in class or shared online, the sharing is a potentially powerful teaching and learning tool for others to use. As a teaching and learning strategy in chemistry, blended media makes the students’ representations of their content knowledge ‘public’ and demonstrates their attention to the audience and hence, their communicative skill. We believe that blended media provides important potential for chemistry students to develop communications skills through a focus on multimodal awareness by creating a digital media product. By translating information for a non-expert audience, the students considered how to best represent the content using different modes, then create or find the actual modes, integrating them using technology into a digital product.

Conclusions

Science education literature emphasises the value of students constructing their own multimodal representations of science concepts. Translating these into other representational forms holds additional possibilities for both learning the content and developing communications skills. The creation process for blended media is accessible and relatively simple, which makes it appealing for a presentation task in chemistry or other science disciplines. Learners use their own technology (digital still or video camera, computer), free software and everyday materials to construct models or other representations and are motivated to work with multiple modes and accurately present relevant information about the topic in a way that is appealing and informative for the audience. The content is also expressed in the form of a multimodal digital representation that is easily shared, which opens up other possibilities for critique by peers or instructors.

A limitation of this study is that our methods did not capture the actual creation process as students developed their blended media products. We did, however, gather students’ perceptions of this task and we are able to make reasonable judgements on how effectively the blended media product communicates scientific knowledge to a non-expert audience. When language is clear, images are aligned to the narration and an engaging storyline underpins the blended media, we can justifiably claim that the creators are aware of the affordances of multiple modes to effectively communicate meaning.

In this research, we have relied on interviews with participating students (and the instructor) to gather perceptions of the blended media task. The interviews occurred at three points over the semester in relation to the parts of the assessment tasks, which were also iterative. Data gathering built from earlier steps in task production, thus, taken together, these offer a form of triangulation for our analyses and a reasonable means to deal with threats to validity in this qualitative research project.

Further research on blended media is needed. According to Yore and Treagust (2006), there is need to investigate the “enhanced cognition that occurs during the transformation from one representation to another representation or one mode to another” (p. 308). There is also scope to more deeply explore the use of analogies or metaphors by the creators of the blended media products. Following Orgill and Bodner (2004), who argued that chemistry instructors could be more effective in how they use analogies, students likewise could likely benefit from understanding about how to use analogies as tools for reasoning, particularly for abstract concepts. It is our aim, following this small case study, to better understand the significance of the multimodal transformations that happen as part of the design process to create a blended media. And, media-making technologies are ever more accessible to learners and with the benefits seen for these chemistry students, blended media is a useful form of meaning making activity that is accessible to all.

Acknowledgements

This research was funded by a faculty grant from the University. The authors would also like to thank the students who participated in this study and the anonymous reviewers whose feedback helped to sharpen the paper's focus.

Notes and references

1. Adandan E., (2014), Investigating the influence of pre-service chemistry teachers’ understanding of the particle nature of matter on their conceptual understanding of solution chemistry, Chem. Educ. Res. Pract., 15, 219–238.
2. Ainsworth S., (1999), The functions of multiple representations, Comput. Educ., 33, 131–152.
3. Ainsworth S., (2006), A conceptual framework for considering learning with multiple representations, Learn. Inst., 16(3), 183–198.
4. Amos R., (2015), Using technologies to support learning science, in Toplis R. (ed.), Learning to teach in the secondary school, Abingdon, UK: Routledge, pp. 204–219.
5. Bartle E., (2016), Creative podcasting in chemistry: a case study, in Hoban G., Nielsen W. and Shepherd A. (ed.), Student- generated digital media in science education: learning, explaining and communicating science content, London: Routledge, pp. 16–28.
6. Bartle E., Longnecker N. and Pegrum N., (2011), Collaboration, contextualisation and communication using new media: introducing podcasting into an undergraduate chemistry class, Int. J. Innov. Sci., 19(1), 16–28.
7. Bennett S., Agostinho S. and Lockyer L., (2015), Technology tools to support learning design: implications derived from an investigation of university teachers’ design practices, Comput. Educ., 81, 211–220.
8. Carlisle D., Tyson J. and Nieswandt M., (2015), Fostering spatial skill acquisition by general chemistry students, Chem. Educ. Res. Pract., 16, 478–517.
9. Cheng M. and Gilbert J. K., (2009), Towards a better utilization of diagrams in research into the use of representative levels in chemical education, in Gilbert J. K. and Treagust D. (ed.), Multiple representations in chemical education, Dordrecht: Springer, pp. 55–73.
10. Cooper M. M., Williams L. C. and Underwood S. M., (2015), Student understanding of molecular forces: a multimodal story, J. Chem. Educ., 92(8), 1288–1298.
11. Davidowitz B., Rollnick M. and Fakudze C., (2005), Development and application of a rubric for analysis of novice students’ laboratory flow diagrams, Int. J. Sci. Educ., 27(1), 43–59.
12. Davidowitz B., Chittleborough G. and Murray E., (2010), Student-generated submicro diagrams: a useful tool for teaching and learning chemical equations and stoichiometry, Chem. Educ. Res. Pract., 11, 154–164.
13. de Berg K., (2012), A study of first-year chemistry students’ understanding of solution concentration at the tertiary level, Chem. Educ. Res. Pract., 13, 8–16.
14. Fleer M. and Hoban, G., (2012), Using ‘slowmation’ for intentional teaching in early childhood centres: possibilities and imaginings, Austr. J. Early Child., 37(3), 61–70.
15. Ferk V., Vrtacnik M., Blejec A. and Gril A., (2003), Students’ understanding of molecular structure representations, Int. J. Sci. Educ., 25(10), 1227–1245.
16. Forest K. and Rayne S., (2009), Incorporating primary literature summary projects into a first-year chemistry curriculum, J. Chem. Educ., 86, 592–594.
17. Gilbert J., (2007), Visualization in science education, Dordrecht: Springer.
18. Gilbert, J. and Treagust, D., (2009), Multiple representations in chemical education, Dordrecht: Springer.
19. Goldman S., (2003), Learning in complex domains: when and why do multiple representations help? Learn. Instr., 13, 239–244.
20. Hand B. and Choi A., (2010), Examining the impact of student use of multiple modal representations in constructing arguments in organic chemistry laboratory classes, Res. Sci. Educ., 40(1), 29–44.
21. Hand B., Gunel M. and Ulu C., (2009), Sequencing embedded multimodal representations in a writing to learn approach to the teaching of electricity, J. Res. Sci. Teach., 46(3), 225–247.
22. Hoban G., (2005), From Claymation to slowmation: a teaching procedure to develop science knowledge, Teach. Sci., 51, 26–30.
23. Hoban G., (2007), Using slowmation to engage preservice teachers with science, Contemp. Iss. Techn., 7, 1–9.
24. Hoban G., (2009), Facilitating learner-generated animations with slowmation, in Lockyer L., Bennett S., Agostinho S. and Harper B., (ed.), Handbook of research on learning design and learning objects: issues, applications, and technologies, Hershey, PA: IGI Global, pp. 313–330.
25. Hoban G. and Nielsen W., (2010), The 5 Rs: student-generated animations of science concepts, Teach. Sci., 56, 33–37.
26. Hoban G. and Nielsen W., (2012), Encouraging preservice primary teachers to create multimodal representations of science concepts with ‘Slowmation’, Res. Sci. Educ., 42, 1101–1119.
27. Hoban G. and Nielsen W., (2013), Learning science through creating a ‘Slowmation’: a case study of preservice primary teachers, Int. J. Sci. Educ., 35, 119–146.
28. Hoban G., Loughran J. and Nielsen W., (2011), Slowmation: preservice primary teachers representing science knowledge through creating multimodal digital animations, J. Res. Sci. Teach., 48, 985–1009.
29. Hoban G., Nielsen W. and Hyland C., (2016a), Blended media: student-generated mash-ups to promote engagement with science content, Int. J. Mob. Blend. Media, 8, 35–48.
30. Hoban G., Nielsen W. and Shepherd A., (2016b) (ed.), Student-Generated digital media in science education: engaging students in learning, explaining and communicating content, London: Routledge.
31. Howard S. K. and Gigliotti A., (2016), Having a go: looking at teachers’ experience of risk-taking in technology integration, Educ. Info. Techn., 21(5), 1351–1366.
32. Howard S. K., Chan A., Mozejko A. and Caputi P., (2015), Technology practices: confirmatory factor analysis and exploration of teachers’ technology integration in subject areas, Comput. Educ., 90, 24–35.
33. Hubber, P., Tytler, R., and Haslam, F., (2010), Teaching and learning about force with a representational focus: pedagogy and teacher change, Res. Sci. Educ., 40(1), 5–28.
34. Jewitt, C., (2009), The Routledge handbook of multimodal analysis, London: Routledge.
35. Jonassen D., Myers J. M. and McKillop A. M., (1996), From constructivism to constructionism: learning with hypermedia/multimedia rather than from it, in Wilson B. G. (ed.), Constructivist learning environments, Englewood Cliffs, NJ: Educational Technology Publications, pp. 93–106.
36. Jones S., Yates B. and Kelder J., (2011), Learning and teaching academic standards statement, Strawberry Hills, NSW: Australia Learning and Teaching Council.
37. Keast S. and Cooper R., (2016), Developing pedagogical content knowledge of pre-service teachers using slowmation as a shared experience, in Hoban G., Nielsen W. and Shepherd A. (ed.), Student-generated digital media in science education: learning, explaining and communicating content, London: Routledge, pp. 151–165.
38. Kidman, G., Keast, S., and Cooper, R., (2013), Enhancing pre-service teacher learning through slowmation animation, Int. J. Eng. Educ., 29(4), 846–855.
39. Kim, B., and Reeves, T., (2007), Reframing research on learning with technology: in search of the meaning of cognitive tools, Instr. Sci., 35, 207–256.
40. Kozma, R., (2003), The material features of multiple representations and their cognitive affordances for science understanding, Learn. Instr., 13, 205–226.
41. Kress G., Jewitt C., Ogborn J. and Tsatsarelis C., (2001), Multimodal teaching and learning: rhetorics of the science classroom, London: Continuum.
42. Lakoff G. and Johnson M., (2003), Metaphors we live by. Chicago: University of Chicago.
43. Lemke J., (1998), Multiplying meaning: visual and verbal semiotics in scientific text, in Martin J. R. and Veel R. (ed.), Reading science: critical and functional perspectives in discourses of science, New York: Routledge, pp. 87–113.
44. Lemke, J., (2000), Across the scales of time: artifacts, activities, and meanings in ecosocial systems, Mind, Cult. Act., 7(4), 273–290.
45. Ma, J., (2015), Incorporating research-based problems from primary literature into a large-scale organic structure analysis course, J. Chem. Educ., 92, 2176–2181.
46. Martinovic, D., and Zhang, Z., (2012), Situating ICT in the teacher education program: overcoming challenges, fulfilling expectations, Teach. Teach. Educ., 28, 461–469.
47. Merriam, S., (2007), Qualitative research and case study applications in education, San Francisco: Jossey-Bass.
48. Moll, R. F., and Nielsen, W., (2016), Development and validation of a social media and science learning survey, Int. J. Sci. Educ., DOI: http://10.1080/21548455.2016.1161255.
49. O’Halloran, K., (2013), Multimodal analysis (software package), available: http://www.multimodal-analysis.com.
50. Orgill, M., and Bodner, G. (2004), What research tells us about using analogies to teach chemistry, Chem. Educ. Res. Pract., 5, 15–32.
51. Paige, K., Bentley, B., & Dobson, S., (2016), Slowmation: an innovative twenty-first century teaching and learning tool for science and mathematics pre-service teachers, Aust. J. Teach. Educ., 41(2), DOI: http://10.14221/ajte.2016v41n2.1.
52. Painter, C., Martin, J. R., and Unsworth, L., (2013), Reading visual narratives, Sheffield, UK: Equinox.
53. Peirce, C., (1931/1955), Logic as semiotic: the theory of signs, in Justus B. (ed.), Philosophical writings of Peirce (1893-1910), New York: Dover, pp. 98–119.
54. Pegrum, M., Bartle, E., and Longnecker, N., (2014), Can creative podcasting promote deep learning? The use of podcasting for learning content in an undergraduate science unit, Brit. J. Educ. Technol., 46(1), 142–152.
55. Prain, V., (2006), Learning from writing in secondary science: some theoretical and practical implications, Int. J. Sci. Educ., 28(2-3), 179–201.
56. Prain, V., and Waldrip, B., (2006), An exploratory study of teachers’ and students’ use of multi-modal representations of concepts in primary science, Int. J. Sci. Educ., 28(15), 1843–1866.
57. Prensky, M., (2001), Digital natives, digital immigrants, On Horiz., 9(5), 1–6.
58. Prins, G., Bulte, A., van Driel, J., and Pilot, A., (2008), Selection of authentic modelling practices as contexts for chemistry education, Int. J. Sci. Educ., 30, 1867–1890.
59. Ritchie, S., Tomas, L., and Tones, M., (2010), Writing stories to enhance scientific literacy, Int. J. Sci. Educ., 33(5), 685–707.
60. Royce T. D., (2007), Intersemiotic complementarity: a framework for multimodal discourse analysis, in Royce T. D. and Bowcher W. L. (ed.), New directions in the analysis of multimodal discourse, Mahwah, NJ: Erlbaum, pp. 63–109.
61. Seufert, T., (2003), Supporting coherence formation in learning from multiple representations, Learn. Instr., 13(2), 227–237.
62. Shea, K. M., Gorin, D. J., and Buck, M. E., (2016), Literature-based problems for introductory organic chemistry quizzes and exams, J. Chem. Educ., 93, 886–890.
63. Sperling, R., Seyedmonir, M., Aleksic, M., and Meadows, G., (2003), Animations as learning tools in authentic science materials, Int. J. Instr. Media, 30(2), 213–221.
64. Stake R., (1995), The art of case study research, Thousand Oaks, CA: Sage.
65. Subramaniam, K., and Padalkar, S., (2009), Visualisation and reasoning in explaining the phases of the moon, Int. J. Sci. Educ., 31, 395–417.
66. Tytler R., Prain V., Hubber P. and Waldrip B., (2013), Constructing representations to learn science, Amsterdam: Sense Publishers.
67. Vanderlelie, J., (2013). Improving the student experience of learning and teaching in second year biochemistry: assessment to foster a creative application of biochemical concepts, Int. J. Innov. Sci. Math. Educ., 21(4), 46–57.
68. Vanderlelie J., (2016), Musical explanations for learning biochemistry, in Hoban G., Nielsen W. and Shepherd A. (ed.), Student-generated digital media in science education: learning, explaining and communicating content, London: Routledge, pp. 225–238.
69. van der Meij, J., and de Jong, T., (2006), Supporting students’ learning with multiple representations in a dynamic simulation-based learning environment, Learn. Instr., 16, 199–212.
70. Waldrip, B., Prain, V., and Carolan, J., (2006), Learning junior secondary science through multi-modal representations, Elect. J. Sci. Educ., 11(1).
71. Waldrip, B., Prain, V., and Carolan, J., (2010), Using multi-modal representations to improve learning in junior secondary science, Res. Sci. Educ., 40, 65–80.
72. Willett, R., (2007), Technology, pedagogy and digital production: a case study of children learning new media skills, Learn. Med. Techn., 32(2), 167–181.
73. Yang, E., Andre, T., Greenbowe, T., and Tibell, L., (2003), Spatial ability and the impact of visualization/animation on learning electrochemistry, Int. J. Sci. Educ., 25, 329–349.
74. Yin R., (2014), Case study research: design and methods, Thousand Oaks, CA: Sage.
75. Yore, L. D., & Hand, B., (2010), Epilogue: plotting a research agenda for multiple representations, multiple modality and multimodal representational competency, Res. Sci. Educ., 40, 93–101.
76. Yore, L. D., & Treagust, D. F., (2006), Current realities and future possibilities: language and science literacy—empowering research and informing instruction, Int. J. Sci. Educ., 28(2–3), 291–314.

---