Sadaf Ansari
Ridge View Residential College (RVRC)
Ansari, S. (2024). Leveraging AI tools for engagement and higher learning outcomes: An exploratory study [Paper presentation]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-s-ansari/
SUB-THEME
Opportunities from Generative AI
KEYWORDS
Higher-order learning, AI, interdisciplinarity
CATEGORY
Paper Presentation
EXTENDED ABSTRACT
The release of open-access Generative artificial intelligence (GenAI) has made its use in higher education inevitable (Mollick, 2024). Educators’ initial concerns, which centred on unreliability and plagiarism, have since been replaced by universities embracing the technology, with a focus on the positive potential of AI in learning (NUS AI COP, 2023; CTI Cornell, 2023). Emerging literature on the use of open-access GenAI indicates the possibilities for enhancing student learning and transfer of knowledge when paired with specific inquiry-related skills such as critical thinking and ideation (Gregersen, 2023; Mollick, 2024). This is rapidly shifting the pedagogic focus in using GenAI from ‘what works’ to the possibilities offered by ‘what if?” (Ross, 2023). Set within such a context, this exploratory case study evaluated the impact of integrating GenAI on student engagement and higher-order learning outcomes for an interdisciplinary course.
BACKGROUND
Recent studies indicate that the use of GenAI can positively impact transfer; once students have gained foundational knowledge, GenAI can provide a time-efficient mode to test multiple ideas for complex problems, beyond traditional modes (Eapen, 2023; Mollick, 2022). This can enhance student engagement by simplifying the process of knowledge application, and build their proficiency in cognitively-demanding higher learning outcomes (Kirschner et al., 2022)
PROJECT AIMS AND SCOPE
The study was conducted with 32 students reading an interdisciplinary NUS General Education course—RVSS1000 “Exploring Sense of Place”—taught in small classes of 14-18 students using a seminar-style format. The class composition was multidisciplinary; it included students in their first and second year of undergraduate studies from six different NUS faculties. The course instructor’s observations from teaching the course across several semesters highlighted that students often lacked engagement and struggled with the cognitive demands of ideation—a higher learning outcome—while working on the final project. This was compounded by the inherent challenge of integrating creative ideas within complex spatial parameters of a real site and communicating these ideas effectively to stakeholders. End-of-semester time constraints further exacerbated these challenges for students. The course instructor evaluated the impact on these three interconnected problems encountered during ideation, by enabling students to automate the process of visualisation using GenAI – Microsoft CoPilot.
METHOD
The impact of integrating GenAI was evaluated using three types of data – a) the course instructor’s observations of student engagement with ideating project solutions using GenAI, b) analysis of students’ visualisations for the project using GenAI, and c) a student perception survey about using GenAI. An exploratory case study method was adopted for data collection and evaluation, due to the fast-evolving context-specific nature of GenAI capabilities.
KEY FINDINGS
Overall, findings from the study demonstrated the amelioration of the three interconnected problems encountered by students. Firstly, GenAI helped integrate students’ ideas within the complex spatial parameters of any real-world site by drawing on its vast database of site-specific photographs. Secondly, GenAI produced realistic three-dimensional visualisations that enhanced effective communication of students’ creative ideas to external partners and stakeholders. Lastly, automating the visualisations freed students’ time to focus instead on the higher cognitive demand of generating creative ideas for the project and iterative refinements.
The course instructor’s observational data indicated an increase in positive student engagement during project discussions and consultations; compared to previous cohorts, students experimented more with complex creative ideas, provided meaningful peer feedback, and undertook effective improvements of their projects to incorporate peer and instructor feedback. During the formal presentation, the use of GenAI also enabled students to fully explain their creative ideas and engage stakeholders in discussions about implementation, a limitation faced by previous cohorts.
Students’ use of GenAI for the project demonstrated their ability to undertake the cognitively-demanding task of synthesising creative ideas anchored by robust interdisciplinary analysis. This was evidenced in the use of GenAI to scale the spaces correctly and connect site-specific details about materials, landscape, biodiversity, stakeholder priorities, and function. In some exemplary projects, students used GenAI to further elaborate spatial features and cited relevant literature justifying their ideas for implementation.
The student perception survey conducted at the end of the course included questions about the impact of using GenAI on the three challenges identified by previous cohorts, using a Likert scale. Students strongly agreed about the interconnected benefits of using GenAI for enabling the cognitively demanding task of ideation by automating the process of visualisation and iterative refinement, with scores for the six questions ranging from 73%-79%.
While it is difficult to establish causation, data from NUS student feedback also showed minor improvements in students’ expected grades and course difficulty levels with the use of GenAI. Similar improvements were noted for ‘teacher’s ability to enhance interest in the subject’ and ‘overall effectiveness’. The addition of a new AI-related question in the NUS Student Feedback Survey in AY2024/25 offers an opportunity to clarify this further.
Despite the learning affordances of using GenAI identified by students, survey data about their recommendation for its use by future cohorts showed mixed results, with only 56% supporting it. 24% decided against using GenAI due to a perceived steep learning curve. 21% were unsure due to their own struggles with using GenAI, as they discovered that prompt engineering was not an intuitive skill. Hence, the study concluded that decisions about the continued integrative use of GenAI in this course (and perhaps others) must also address such valid concerns and avoid creating new digital disparities in student learning (NUS CTLT, 2024).
REFERENCES
CU Committee Report: Generative Artificial Intelligence for Education and Pedagogy | Center for Teaching Innovation. (n.d.). Retrieved December 14, 2023, from https://teaching.cornell.edu/generative-artificial-intelligence/cu-committee-report-generative-artificial-intelligence-education
Eapen, T. T., Finkenstadt, D. J., Folk, J., & Venkataswamy, L. (2023, July 1). How generative AI can augment human creativity. Harvard Business Review. https://hbr.org/2023/07/how-generative-ai-can-augment-human-creativity.
Gregersen, H., & Bianzino, N. M. (2023, May 26). AI can help you ask better questions-and solve bigger problems. Harvard Business Review. https://hbr.org/2023/05/ai-can-help-you-ask-better-questions-and-solve-bigger-problems
Jensen, T., Dede, C., Tsiwah, F., & Thompson, K. (2023, July 27). Who Does the Thinking: The Role of Generative AI in Higher Education [Video]. YouTube. International Association of Universities. Retrieved December 12, 2023, from https://www.youtube.com/watch?v=gE_GKsdTPAs.
Kirschner, P. A., Hendrick, C., & Heal, J. (2022). How Teaching Happens: Seminal Works in Teaching and Teacher Effectiveness and What They Mean in Practice. Routledge.
Mollick, E. (2024). Co-intelligence: Living and working with AI. Portfolio/Penguin.
Mollick, E.R., and Mollick, L., (December 13, 2022). New Modes of Learning Enabled by AI Chatbots: Three Methods and Assignments. Available at SSRN: https://ssrn.com/abstract=4300783 or http://dx.doi.org/10.2139/ssrn.4300783
NUS CTLT (2024, August 7). Policy for Use of AI in Teaching and Learning. Retrieved August 25, 2024, from https://ctlt.nus.edu.sg/wp-content/uploads/2024/08/Policy-for-Use-of-AI-in-Teaching-and-Learning
Ross, J. (2023). Digital Futures for Learning: Speculative Methods and Pedagogies. Routledge.