Technology in Pedagogy, No. 10, August 2012
Written by Kiruthika Ragupathi
Recent pedagogical movements have demonstrated the value of mastering multiple literacies, asking students to become knowledgeable not only in their analysis of the written word, but also in other forms of visual media ranging from advertisements to photojournalism to cinema. However, while approaches to literacy have become increasingly “multimodal”, student outputs have remained largely “unimodal”, with the written word being privileged for its ability to convey a level of complexity supposedly outside the purview of other communication forms.
Research indicates that students who incorporate multimodal forms and approaches to their learning are better engaged with the content than those who employ traditional approaches, thereby enhancing their thinking and learning process. It is possible for students to convey their ideas that is critically engaged through the use of multimodal forms, says Dr Jasmine Nadua Trice, a lecturer in the Ideas and Exposition programme, a multidisciplinary critical thinking and writing programme at the National University of Singapore. Her background in Film and Media studies with a PhD in Communication Culture and her interest in teaching film studies, public speaking and film productions lead her to trying out the use of multimodal communications in her modules.
In this session, Dr Trice shared her experience teaching a General Education Module (GEM) that essentially employs multimodal communications focusing not on technology but on the content (Emergent Media), and more importantly on the multimodal forms that the assignments took place in. Using her class as a case study, she examined the potential usefulness of multimodal communications for undergraduate level criticism, asking what kinds of critical pedagogies such an approach to student inquiry might enable.
Multimodal communications: An overview
Dr Trice provided a brief overview of multimodal forms of communication and highlighted some examples of scholarly work that inspired the proposal of a new course.
Multimodal communication is a form of communication that uses a combination of written, audio and visual forms to convey an idea and works in tandem with media literacy movements. Gunther R. Kress, a Professor of Semiotics & Education at the University of London points out that “in this ‘new media age’ the screen has replaced the book as the dominant medium of communication and this dramatic change has made image, rather than writing, the center of communication.” Multimodal literacy, therefore, is an established field and it is apparent that it is possible to understand critical ideas and academic analysis through multimodal forms in an undergraduate classroom.
Multimodal scholarship
In the recent years, multimodal scholarship is stronger in the fields of media studies and digital humanities. The multimodal scholarships sometimes take the form of web-based or interactive text forms and at other times use video essays or screencasts.
Dr Trice showcased some examples of scholarly work that were of particular interest to her:
- Vectors, Journal of Culture and Technology in a Dynamic Vernacular, USC: “Vectors is realized in multimedia, melding form and content to enact a second-order examination of the mediation of everyday life.”
- International Journal of Learning and Media, MIT: “Rich media contributions representing key research findings that exceed the boundaries of the printed page.”
- Kairos, A Journal of Rhetoric, Technology, & Pedagogy: “publish scholarship that examines digital and multimodal composing practices, promoting work that enacts its scholarly argument through rhetorical and innovative uses of new media.”
- Alliance for Networking Visual Culture: “creates scholarly contexts for the use of digital media in film, media and visual studies.”
She highlighted the examples that she would use in the module – those that her students need to read or watch. The examples helped students to visualize complex concepts and emphasizes the fact that creativity is the most important factor in using multimodal communications effectively.
- Alexandra Juhasz, Learning from YouTube (MIT Press, 2010):
“Learning from YouTube, the first video—book published investigates questions with a series of more than 200 texts and videos, also known as “texteos.” This video-book, an example of web-based or interactive text mode, integrates the news clips based on the interviews that Dr Juhasz had with CNN, her book and the assignments created by her students when she taught the module on “Learning from YouTube”. Students in her class used YouTube as the media to do their assignments.
Dr Trice highlighted that when she tried using this as an example in her module, the students were disconcerted with the format and also required that a steep learning curve was necessary for her students to use such interactive text. Hence this reading, she said was avoided in the current semester.
- Richard Langley, “American Un-Frontiers: Universality and Apocalypse Blockbusters”:
This example showcases the use of visual elements and the usage of text in a video essay to underscore the idea that the author is getting across. This example of video essay integrates icons, text and archival footage in interesting ways employing a screencast method that employs a linear way of presenting the video essay. (http://vimeo.com/32288942) - David Gauntlett, “Making is Connecting” (www.artlab.org.uk / www.theory.org.uk)
An example of what one can do with basic screencast software (http://www.youtube.com/watch?v=nF4OBfVQmCI&feature=relmfu)
This example highlights:- The tone that is used in the video he uses when oral voice-over is added much more casual – and that the casual tone does not make the video any less professional but more importantly how the tone has to match the medium;
- the use of basic information design for presentation of ideas;
- gives a context of what is being talked about;
- gives a literature review with citing secondary sources that is used;
- also visualize the quotes from others.
- Images from a graphic design books. E.g., visualizing content – Europe; Corriette Schoenaerts, for her fashion spread on countries and borders, in Robert Klanten et al., Eds. Data Flow: Visualizing Information in Graphic Design (Berlin: Gestalten, 2010), 189; Christoph Niemann, Sleep Agony Chart, in Robert Klanten et al., Eds. Data Flow: Visualizing Information in Graphic Design (Berlin: Gestalten, 2010), 107; C.P.G. Grey, “The True Cost of the Royal Family Explained”
Dr Trice emphasized to her class that the content skills, conceptual skills and practical skills learned in the module would be integrated in producing the assignments. She reassured her students that it was not necessary for them to have high levels of technical ability and skills for doing well in the module and what mattered most was the CREATIVITY.
Proposal for a new module using multimodal communications
A workshop at CDTL that introduced her to the screencasting options available at NUS (Camtasia Relay and Ink2Go), her background in Film and Media studies coupled with her interests in teaching film studies, public speaking and film productions inspired her to propose the new module on “Emergent Media and Multimodal Communications”. This enabled her to combine all her interests to explore a more productive approach to teaching. The module was developed so as to provide students with a broad understanding of transitional media and culture not only through engagement with module content, but also through developing written, oral, and visual communication strategies.
To get her students to understand and better appreciate the use of multimodality in the module, she introduced the idea of multimodality to her students by probing them on:
- what the idea of “modality” entails,
- what the idea of “multimodality” entails,
- how different is multimodal from unimodal communications, and
- how the written word is still the dominant mode employed in most of the University assignments.
She briefed her class on how things would be different in the module where they (her students) would be involved in producing assignments that employ different forms of modality. The students on the first day of the class were also encouraged to contemplate on what they would gain and/or lose when moving from the written mode to multimodal approach of critical ideas. Students were then asked to reflect upon if it was possible for them to convey critical ideas and academic analysis through multimodal forms. She emphasized the idea of critical thinking and whether it was possible to convey their ideas that is critically engaged and analytically rigorous using images, audio and the written or spoken word.
The module had three units with each culminating in an assignment that require students to use one or more of the written, oral, or visual communicative modes. The assignment tasks were designed to cultivate the practical comprehension of media by allowing students to convey ideas about class content using multiple forms of communication, both residual and emergent. The tasks enabled her students to:
- combine video, still images, audio, and text to convey complex, academic investigations in a clear and creative manner, and
- convey critical ideas in an unconventional form.
However, she also emphasized that the main focus of the assignments were on thinking about the ideas and the video essays using Screencast and not on the technology itself.
The first assignment was to use multimodal essays which were posted on the class Facebook page with peers providing reviews and comments on the essay. The second assignment involved the use of Screencast videos. And the final assignment was an oral presentation in groups.
Assignment 1 – Multimodal Essay:
The multimodal essay assignment is not about testing the multimedia skills but on the usage of the visual parts of the essay. The students were advised against the use of pre-made or readily available templates, as it was important to create something original that is visually and aesthetically compelling. The output was a 575-600 word multimodal essay. The assignments were graded in such a way that the three quarters of the grade was for the content analysis and on how they would visualize the theoretical concepts (75% for analysis; 15% for multimodal aspects and 10% for writing style and structure). All students were required to post their assignments on a class Facebook page that has to be accompanied with an explanation as to why they used a certain approach (assignments were uploaded on SCRIBD, an online PDF environment). This allowed students to justify their visual process/approach taken. Their classmates were then required to comment and critique on their peers’ work. Dr Trice felt that this was extremely helpful for her to understand the student’s thought process, particularly when it is difficult to understand the execution.
The common approach was that students used the evolution approach (e.g., from a book to iPAD). One student used a newspaper format and provided a wider context with the use of news splashes. Listed below are some samples from her students’ work:
 |
 |
Assignment 2 – Screencast/ Video essay
Students will create screencast videos or video essays, each of which should be 6 minute long clip. Again, the grading’s focus was on the analysis with 75% marks assigned for content and 25% assigned for the multimodal aspect. Dr Trice briefed and showed samples on how the students’ video essays should focus on multimodal scholarship and information design; use of videos, moving and still images; slides, on editing and juxtaposition, the voice-over narration, the use of on-screen text and symbols, and the use of music.
Students produced a variety of video essays: videos with no voice – so text heavy slides, with interesting use of on-screen text, good visualization of core ideas, and visuals inspired by RSAnimate series.
Assignment 3: Group oral presentation
The focus of the oral presentations was on: visual aids employed in the presentation; audiences and informative strategies; the vocal and physical modes of delivery; and on preparing for questions.
Assessment/Grading criteria
Overall, the assignments were assessed based on the following grading criteria:
|
Analysis (75%) |
Multimodal aspects (25%) |
|
|
Pedagogical potentials of multimodal communications
- Enrich and empower student learning. Providing learners an opportunity to create a shared representation of language – textual form, visual form and an auditory form— proves to be cognitively and pedagogically valuable. The usage of multimodal communication in their assignments help students transfer ideas from writing into multiple ways of communicating, offering them greater opportunities for meaning making. It helps them convey their ideas in critically engaged and analytical rigorous ways. With the changed and changing communication, the use of multimodality in the assignments will enable students to enter the workplace confident of their own potentials.
- Engages peers and promotes reflection. The multimodal components provides a greater opportunity for students to engage with their peers as it allows them to present their arguments in multiple ways through written, spoken, and visual texts. When students view, comment and critique the work of the peers, it aids in reflection after the assignment task and promotes overall learning. These appeal to students’ interest and motivate them to be engaged learners.
- Enhance writing and communication skills. Making the multimodal essays, video essays and screencast helped students to hone their writing and communication skills.
Reflections and future directions
Dr Trice reflected upon the planning of assignments and indicated that she would change the way she did the oral presentation assignment and would consider the use of other criteria for assessing multimodal forms based on the work by (Ball, 2012). Ball (2012) identifies items that need to be considered when assessing such multimodal forms of assignments and could also be used by students when developing their assignments and while peer reviewing other’s work. Some items to consider include: (i) the project’s structural or formal elements must serve its conceptual core; (ii) the design decisions made must be deliberative, controlled, and defensible; (iii) the project should have distinguishable and significant goals that are different from what be achieved on paper; (iv) the design should enact the argument; and importantly it is important for students to have thought of a visual metaphor for the argument.
Summary of Feedback/ Suggestions from the Discussion
Dr Trice welcomed ideas and ways that participants have employed multimodality in their classroom. A lively discussion followed and participants discussed on:
- To what degree is it possible for undergraduates to convey critical analysis through multimodal forms?
- How to develop class content and/or assignments that would allow students to employ multimodal approach?
- How should the grading criteria be designed to effectively assess such multimodal forms of assignments?
There were other participants who had used such multimodal forms of assignments in their modules. They also agreed with Dr Trice that the technical skill of students was never a problem as it was “super easy” to edit and create movies (e.g., FinalCut Pro, Windows movie maker, Adobe premiere). They also pointed out that when students start working in groups, they tend to help each other. One participant felt that once a student’s work is uploaded, and a high bar is set, then all the other students try to outdo each other, and in the process also teach other.
Another participant indicated that for his module, the students were free to choose their own platform based on what they were comfortable with – YouTube, videos, multimodal or essay. Based on his experience, students submitting written essays tend to go deeper in their analysis. However, he used an assessment criterion that awarded 60% for content and 40% for presenting ideas and also had two different sets of criteria for the multimodal form and the written essay. However, he found that it was difficult to follow two standards and he also felt that this might not be fair.
Q & A Session
Listed below are some questions from the subsequent Q & A session:
| Q: | Did you have lessons that taught students the necessary technical skills for creating such assignments? |
| JT: | Drag-drop editing necessary to create these videos does not need background knowledge on technology. I created a tutorial using Screencast – options include the use of camtasia, ink2go, & imovie. Students can also meet with me for consultations if they needed help. Only those students who were super enthusiastic used the consultation sessions—and usually the huge proportion of focus was on content. |
| Q: | Do students with technical skills/technology background have an edge/advantage over the others? |
| JT: | This was something that I put a considerable amount of thought into and that was the reason for having the grading criteria place a greater emphasis on the content rather than on technology. I also got students to get their preliminary sketches and to have discussions with their peers before submitting the assignment. I also got them work in groups and discuss on what they were planning to do, and that also helped, I think. |
| Q: | How do you measure if this new method is more effective than your old method? |
| JT: | I don’t think it is very different from writing an essay, it is pretty similar structurally and in terms of the ideas that they get across if they are doing a voice-over in particular. It is interesting for collaboration, and students will find it easy for peers to watch it rather than reading the peer’s essay. It is also interesting for public dissemination– making the materials available beyond the classroom. So it might be good to a website in addition to the Facebook page. In term of whether it is better for critical thinking – I think a lot of it is same when compared to writing essays but this form is more novel, and students like it for its novelty.I also felt that students were seeing each other’s work and were benefitting from it. I required them to comment on at least 2 of their peer’s work. But students generally went beyond that and comments comment on more students’ work. Since it is in the students’ social space, appearing on their FB timeline. I also discussed with them on providing constructive critique and how they could improve on their comments. |
| Q: | Do you spend time to talk to students about copyright and plagiarism (fair use of information)? |
| JT: | During my first lesson, I talk to them on the fair use of information. I told them to make the link to their essays private as I was not sure if the references made were appropriate. I did not spend too much time on that aspect as the presentations were not made public. And also since this was mainly for educational purpose, I guess it is fair use. I also informed that whatever they used cannot be pre-fabricated and that the components made has to be original. I also gave them open source websites, creative commons site from where they could get the images, photos and music. Students also need to provide a worksite page, and which would have the references and links. Personally, I would spend more time on it when I teach this course again. |
| Q: | If we want to incorporate this type of teaching, as a teacher what skills do I need to have? |
| JT: | CDTL’s workshops like Screencast, Breeze, Ink2Go, Moviemaker would be a good starting point. I also researched and explored online on the things one can do with Screencast. It is also important to get a lot of examples and showcased them to the students and engaged them in discussion during the class. Most of these applications are intuitive. |
| Q: | Have you wondered about how different a traditional essay by the same student would be? What are the implicit and explicit assumptions that are more pronounced in multimodal forms vs a traditional essay? Are students prone to making assumptions when make a video due to addition of music, tonality, songs, etc. How do contrast the both? |
| JT: | That’s right, rather than explicitly spelling out exactly what they are trying to get across, they would present to you with some kind of multimodal image, sound combination and expect those to the work, and the meaning is more ambiguous.But this is something that I have not looked at it very rigorously which I should look at. But one of the things that I try to do is to include a lot of opportunity for students to discuss their design elements and they would need to include justifications for their decisions in terms of the multimodal form. And since for most of them this was a first attempt, most of them were reading scripts from probably an essay format. There should be probably ways to study if there is difference between traditional essay & video essay. But definitely this is something that I would like to work on in the future. |
References
Cheryl E. Ball, (2012). “Assessing Scholarly Multimedia: A Rhetorical Genre Studies Approach,” Technical Communication Quarterly, 21: 61-77, 2012.