Grace Lee Yuan
Ping
,
Chang
Lok
,
Tan
Wei Yeat
,
Tan Jie Ying
Cherynn
and
Emelyn Sue Qing
Tan
*
Department of Chemistry, National University of Singapore, 3 Science Drive 3, Singapore 117543, Singapore. E-mail: emelyntan@nus.edu.sg; Tel: +65 6516 2674
First published on 14th September 2017
Three internally developed mobile apps, “3D Sym Op”, “SM2 Chem” and “ARMolVis”, available for free on both the Apple App Store and Google Play Store were evaluated in seven studies. Each study was a systematic process of Pre-Test, In-lecture App Demo, App Assisted Interactive Tutorials (AAITs) and/or Independent App Usage (IAU), followed by a Survey and Post-Test. Overall, the mobile apps were effective evident by the higher Post-Test vs. Pre-Test % increase for those who used the app more frequently compared to those who used the app rarely. Apps were most effective when used in AAITs with the Blended Learning approach. This approach requires the physical presence of both teacher and student, but with some element of student free play such as using the app to complete worksheets in pairs or groups.
Connectivism has been proposed as a learning theory from the prism of Blended Learning (Vranes et al., 2015). Connectivism, a learning theory for a digital age, has these principles among others, where learning occurs by connecting specialised nodes or information sources; nurturing and maintaining connections is needed to facilitate continual learning and the ability to see connections between fields, ideas, and concepts is a core skill (Siemens, 2005). Knowledge emerges from an individual's learning network when the individual is able to perceive connections between different perspectives and concepts (Dunaway, 2011). In course design, educators should scaffold learning by providing an extensive array of resources, on top of the standard three which are lecture notes and recommended textbooks, tutorial questions and solutions, and hands-on experiments that are used in lectures, tutorials and labs, respectively. The challenge for the technology enhanced platforms is how to inextricably integrate them to fill the voids, and allow the learner to create connections and yet not duplicate what these three sets of resources already offer.
Dual Coding Theory (Paivio, 1969) postulates that nonverbal (animation using images seen by eyes) and verbal (narration using spoken words heard by ears) stimuli are processed differently and along distinct channels in the human mind, creating separate models, pictorial and verbal, and subsequently integrating both with prior knowledge if any. With the ability to code a stimulus in two different ways, the chances of retrieving and recalling it is faster and higher, compared to if the stimulus was only coded one way (Paivio, 1969). Drawing on this cognitive theory, learning using multimedia presentations has proven to aid recall, understanding and application of information especially concepts, facts, or principles that are highly imaginable (Ardac and Akaygun, 2004; Jones, 2013), improve problem solving skills (Kelly and Jones, 2008) and promote learner understanding (Mayer and Moreno, 2002). Technology enhanced platforms available presently are not only able to provide better nonverbal and verbal media resulting in more comprehensive models, but are also able to venture into the other sensory memories of touch, smell and in not too long taste, activating all five senses in immersive learning experiences.
In particular, the study of Chemistry involves thought at multiple levels – macro, symbolic and sub-micro, commonly depicted as Johnstone's Triangle (Johnstone, 1982). Macro refers to observable changes in matter such as colour changes, bubbles or smell, typically occurring in the laboratory. Sub-micro is at the molecular level where there are dynamic changes in structures or processes imperceptible to the human eye. These two levels are linked by the symbolic level and are exemplified qualitatively and/or quantitatively. Qualitatively refers to the use of symbols, notations or diagrams while quantitatively refers to the use of mathematical equations or graphs. One major advantage that technology enhanced platforms bring to Chemistry education is the ability to provide better molecular level representation of reactions and processes; and to allow learners to move seamlessly between these three thinking levels.
Blended Learning (Graham, 2006; Center for Education Reform, 2011; Staker and Horn, 2012) with technology enhanced platforms and activities are beneficial when firstly, they connect inextricably with and compliment the standard resources; secondly, they provide better options for nonverbal and verbal stimuli that tap into more senses; and thirdly, specifically for Chemistry, they are able to help learners move seamlessly between the macro, symbolic and sub-micro thinking levels.
In this paper, three internally developed mobile apps are evaluated in seven studies (Table 1), each with a systematic combination of Pre-Test, In-lecture App Demo, App Assisted Interactive Tutorials (AAITs) and/or Independent App Usage (IAU), followed by a Survey and Post-Test. The hypothesis is that if the mobile apps are indeed effective, the Post-Test vs. Pre-Test % increase for those who used the app more frequently (Very Frequently, Frequently and Occasionally) should be higher compared to those who use the app rarely (Rarely, Very Rarely and Do not use/Do not have). In the case that the hypothesis is true, the limitation to the hypothesis is those who use the app more frequently have a higher % increase because these students are more engaged, self-driven higher performers, as determined by their summative assessment results. The limitation was addressed using the largest set of data, for the “SM2 Chem” app (Table 1), which showed that the summative assessment result mean of those who used the app more frequently compared to those who used it rarely, were similar (p-value of 0.19), indicating that the more frequent app users were not made up by higher performers.
Mobile app | Academic year | N | Year of study | Pre-Test | In-lecture app demo | Email blast with app details | App Assisted Interactive Tutorials (AAITs) | Independent App Usage (IAU) | Survey | Post-Test | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | “3D Sym Op” | Sem 2 AY15/16 | 29 | 76 | Year 2 UG | ✓ | ✓ | ✓ | ✓ (2 AAITs) | ✓ | ✓ | ✓ |
2 | “3D Sym Op” | Sem 1 AY16/17 | 28 | ✓ | ✓ | ✓ | ✓ (2 AAITs) | ✓ | ✓ | ✓ | ||
3 | “3D Sym Op” | Sem 2 AY16/17 | 19 | ✓ | ✓ | ✓ (2 AAITs) | ✓ | ✓ | ✓ | |||
4 | “SM2 Chem” | Sem 2 AY14/15 | 88 | 269 | Pre-U | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
5 | “SM2 Chem” | Sem 2 AY15/16 | 89 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |||
6 | “SM2 Chem” | Sem 2 AY16/17 | 92 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |||
7 | “ARMolVis” | 2016 | 137 | Pre-U | ✓ | ✓ (1 AAIT) | ✓ |
In our literature review, there are several journal papers and online articles with descriptions of Chemistry related apps and their advantages to solve a particular problem (Libman and Huang, 2013). However, the uniqueness of this work is the systematic app evaluations using Pre-Tests and Post-Tests, to conclusively show that apps are useful, rather than taking it as an assumption. Additionally, the seven app evaluation studies use three internally developed, customised module-specific mobile apps, which cannot be said of other studies which report on third party apps.
Mobile app | Earliest publish date on Apple App and Google Play Stores | Total downloads (at 21 April 2017) | Downloads per year |
---|---|---|---|
“3D Sym Op” | 18 December 2015 | 14476 | 10783 |
“SM2 Chem” | 12 December 2013 | 2465 | 734 |
“ARMolVis” | 20 March 2016 | 1387 | 1275 |
Visualisation of symmetry operations, movement according to symmetry, within a molecule and in one's mind, is a challenge for certain students (Raiyn and Rayan, 2015). Hence, this app was developed that has movable, scalable 3D models together with symmetry operations that are performed on touch. Once the symmetry elements are identified, the point group is determined via a flow chart, and for each point group, a Character Table is available within the app. Details in the Character Table are required in Molecular Orbital Theory and interpretation of Infrared and Raman Spectra, which are taught in undergraduate modules such as CM2111 – Inorganic Chemistry 2.
The effectiveness of the “3D Sym Op” was assessed systematically using the timeline in Fig. 1(a). The registration email that was sent to the CM2111 – Inorganic Chemistry 2 class had no indication that this educational study is technology enhanced or app related to avoid recruiting participants with such bias. Instead, the email invited opt-in volunteers to participate in an educational study on Molecular Symmetry, with the incentive of $15 vouchers, and withdrawal was possible at any stage. Details in this email followed the university's requirements for an ethics review exemption. The 30 minute Pre-Test and Post-Test (Fig. 2) were identical with 12 questions extracted from past year examinations of CM2111, in order to constructively align this app evaluation study with their eventual final assessment. The Point Group Flowchart and Character Tables were included in both tests. At the first of two 45 minute App Assisted Interactive Tutorials (AAITs), which were by registration and additional to the standard CM2111 lectures and tutorials, participants working in groups of 2–3 used the app to visualise the symmetry elements and operations (movements) in 3D and drew in 2D all the symmetry elements on the four molecules (Fig. 2). The next AAIT had 2–3 grouped participants visualising the symmetry elements and operations within the app, together with the provided Point Group Flowchart, assigning the point group to four molecules (Fig. 2). The task was also reversed, given the point group Oh, all symmetry elements had to be visualised using the app and listed on the worksheet. Prior to the Post-Test, a survey collecting self-professed frequency of app use, perception of app's usefulness and preferences compared to physical models was completed. The survey crafted with nine convergent and divergent questions to test consistency and reduce survey fatigue (National Research Center Inc., 2016) had the format of multiple choice, Likert scale and open text.
Fig. 1 Typical timelines for the (a) three “3D Sym Op” app evaluation studies, (b) three “SM2 Chem” app evaluation studies and (c) “ARMolVis” app evaluation study. |
Fig. 2 Selected questions and tasks extracted from the Pre-Test and Post-Test, and AAIT 1 and 2 worksheets of the “3D Sym Op” app evaluations. |
The hypothesis for the app evaluations is that if the mobile apps are indeed effective, the Post-Test vs. Pre-Test % increase for those who used the app more frequently (Very Frequently, Frequently and Occasionally) should be higher compared to those who use the app rarely (Rarely, Very Rarely and Do not use/Do not have). The Post-Test vs. Pre-Test % increase was calculated as [(Post-Test − Pre-Test)/Pre-Test] × 100%, where 100% equates to doubling and 200% is for tripling of the score. Dixon Q test at 95% confidence level was performed on % increase values to exclude outliers. The mean % increase for those who responded in the survey that they used the app more frequently (Very Frequently, Frequently and Occasionally) and mean % increase for those who responded that they used the app rarely (Rarely, Very Rarely and Do not use/Do not have) were then calculated. Subsequently, was reported. The number of respondents in each frequency category varied and to deduce each category's contribution to the % increase, a weighted % increase for each and then grand average was calculated as .
For statistical analysis, initially, a two-sample F-test for variances was carried out to determine if the variance of the % increase for those who responded that they used the app more frequently and variance of the % increase for those who responded that they used the app rarely, differed. Following that, the corresponding two-sample t-test was carried out using the same data with the alternative hypothesis that Post-Test vs. Pre-Test % increase for those who used the app more frequently is higher compared to those who use the app rarely, μfrequently > μrare. The p-value and Cohen's D for effect size were reported. These statistical tests were performed on the moderately normal distributed data, which was determined by calculating the skew of the Pre-Tests and Post-Test scores, giving skew values between 0.5 and 0.7.
Knowledge and understanding of scientific language is a necessary factor for understanding Chemistry as a subject. For most students, scientific language represents a new, foreign-seeming language (Markic, 2015). This difficulty is essentially doubled for English as a Second Language (ESL) speakers who are learning Chemistry terms in English. Having learnt Chemistry in Mandarin throughout all previous school years, now having to learn it in English is not an easy feat, particularly because of the vast number of unfamiliar Chemistry terms. Hence, the “SM2 Chem” app was created using recommended approaches to teaching Chemistry to ESL speakers, including using vocabulary and technology, where vocabulary lists are accompanied with pronunciation and images, and incorporated into conversations and writing (Kimbrough and Cooper, 2009).
The 20 minute Pre-Test consisted of 30 multiple choice questions – 10 questions gave the Chinese word and definition and the student had to choose the corresponding English word from four choices, the next 10 were reversed, giving the English word and definition and asking the student to choose the corresponding Chinese word from four choices and the last 10 were to match 10 Chinese words and definitions by selecting from a list of 20 English terms. After establishing this proficiency baseline, an App Demo was performed prior to the first lecture and students were shown and instructed to use the Syllabus option within the app, which presents the terms by the 17 Chapters in chronological teaching order of the course. Prior to any lecture, they were instructed to read the lecture notes and extract the Chinese terms and/or definitions from that Chapter within the app and write them in their lecture notes. Eight weeks later, a survey collecting their self-professed frequency of app usage and perceptions, followed by the identical 20 minute Post-Test was carried out (Fig. 1(b)). The Pre-Test and Post-Test were done together with the continual assessment tests for the entire SM2 Chemistry class.
Data processing and statistical analysis was identical to “3D Sym Op” as discussed in the previous section. The skew of the Pre-Tests and Post-Test scores were between −0.1 and 0.1 indicating symmetric normal distributions.
The mobile app “ARMolVis” was used in a 30 minute activity within the department's outreach programmes, such as Junior College (JC) Afternoons (Fig. 1(c)). The goal of the AAITs was for participants to appreciate that Chemistry is everywhere and using trendy technology like augmented reality enhanced the learning experience. The activity begun with a 5 minute Pre-Test, during which the participants, individually or in pairs, filled in a table with the Major Chemical/s, or the most distinctive chemical, in each of 6, 10 or 20 Everyday Products, to the best of their knowledge (Fig. 3). On the AAIT Worksheet that was subsequently handed out, 5–20 minutes, depending on the number of everyday products, was allocated for participants to use the app to scan the 2D photos and Common Name, printed on their AAIT worksheet, and use the information that appears on their device (chemical name, formula and 3D molecule) to complete the worksheet with chemical name, formula and 3D structure. For the Post-Test, without referring to their AAIT worksheet, participants, individually or in pairs, filled in the Post-Test which is identical to the Pre-Test.
Fig. 3 Pre-Test for 20 everyday products, selected questions extracted from AAIT worksheet and Post-Test for 20 everyday products of the “ARMolVis” app evaluation. |
After all the Pre-Tests and Post-Tests were marked and scored, the Pre-Tests and Post-Tests for 6, 10 and 20 everyday products were separated. The mean and standard deviation for the Pre-Test for 6 everyday products and that for the identical Post-Test were deduced and plotted. The difference between the Post-Test mean and Pre-Test mean was calculated. These data processing steps were repeated for the tests with 10 and 20 everyday products.
Positive perceptions of the “3D Sym Op” app were quantified in a series of questions including a 4.4 out of 5 score agreeing that the app was overall effective and 85% preferring the app over physical molecular models (Table 3). Most quoted reasons for the preference were that the app showed the symmetry elements for molecules (27%), was convenient and easily accessible (24%) and the user was able to watch the movement (symmetry operation) (16%).
Fig. 5 Pre-Test and Post-Test Scores before and after the AAIT using the Worksheet with 6, 10 or 20 Everyday Products and the “ARMolVis” App. |
Of the 137 participants, only two (1%) showed lower Post-Test vs. Pre-Test scores. Apart from good recall, the advantages of learning in an AAIT rather than a traditional lecture, where participants hear content and view images, chemical names and structures on slides, are that the “teachers” role is minimal and “students” self-drive and self-pace their learning.
The activity, AAIT, in which the app was used and its goal, which was for participants to appreciate that Chemistry is everywhere and using trendy technology like augmented reality enhanced the learning experience, directed the capabilities of the app. If it was used in a class where the teacher would teach these students again, rather than this one-off outreach activity, one could go deeper into the chemical and physical properties of each of the molecules and then link it to why it is used in the everyday product, for deeper chemical intuition and representational competence.
In our opinion and shown with this work, apps are most effective when used in AAITs with the Blended Learning approach. This approach requires the physical presence of both teacher and students, but with some element of student free play such as using the app to complete worksheets in pairs or groups.
This journal is © The Royal Society of Chemistry 2018 |