Teaching Augmentative Uses of ChatGPT and Other Generative AI Tools

Jonathan Y. H. SIM
Department of Philosophy, Faculty of Arts and Social Sciences (FASS)

jyhsim@nus.edu.sg

 

Sim, J. Y. H. (2023). Teaching augmentative uses of ChatGPT and other generative AI tools [Paper presentation]. In Higher Education Campus Conference (HECC) 2023, 7 December, National University of Singapore. https://blog.nus.edu.sg/hecc2023proceedings/teaching-augmentative-uses-of-chatgpt-and-other-generative-ai-tools /  

SUB-THEME

AI and Education 

 

KEYWORDS

ChatGPT, generative AI, philosophy of technology, AI augmentation

 

CATEGORY

Paper Presentation 

 

ABSTRACT

Since the rise of generative artificial intelligence (GenAI) like ChatGPT, educators have expressed concerns that students may misuse these tools by growing too reliant on them or use it to take shortcuts in their learning, thus undermining important learning objectives that we set for them.

 

Such concerns are not new in the history of technology. Socrates was one of the first to voice concerns about how the invention of writing would be detrimental to people’s memories:

“[Writing] will implant forgetfulness in their souls; they will cease to exercise memory because they rely on that which is written, calling things to remembrance no longer from within themselves, but by means of external marks.” (Phaedrus, 274b-277a)

 

Common to these complaints is the fear that new technologies will replace existing human processes—as a substitutive tool—leading to a deterioration or loss of certain human abilities. This is not the only approach to technology—we can also use these tools in an augmentative way to enhance existing human abilities and processes (Eors Szathmary et al, 2018). While we may not have memories as strong as the ancients did, writing has since augmented our thinking abilities, allowing us to easily record, recall, transmit, evaluate, analyse, and synthesise far more information than before.

 

This augmentative approach can also be applied to GenAI tools, like ChatGPT. 19.1% of my students (n=351) found ways to use ChatGPT as an augmentative tool rather than as a substitutive tool:

  • As an idea generator or a sounding board to help develop ideas before working on an assignment
  • As a learning resource to teach/explain concepts or clarify confusions
  • As a tool to improve their expression

 

Admittedly, it can be difficult for non-savvy users to think of augmentative uses. Students are commonly exposed to substitutive applications of ChatGPT in learning, and 65.5% of students did not think skills were required to use it well.

 

How can educators encourage effective augmentative uses of GenAI tools? I believe there are three learning objectives we should focus on:

 

(1) Cultivate a collaborative mindset working with GenAI. Knowing how to talk is not the same as knowing how to work well in a team. Learners must feel comfortable and empowered working with GenAI as a collaborative partner if they are to use it as an augmentative tool. One approach is to incorporate activities that involve collaborating with GenAI. In my course, students are to work alongside ChatGPT to develop an evaluation criterion for ride-sharing services, seeking feedback from it while also evaluating its feedback.

 

(2) Develop critical questioning skills. Learners need to learn how to scrutinise GenAI output as the content may be inaccurate or shallow. Throughout the same tutorial, students were challenged to find flaws with ChatGPT’s suggestions, and to find areas where they can improve the quality of ChatGPT’s output. The exercise helped them to recognise that an AI’s answer is far from perfect, and that they cannot take a seemingly well-written work as the final answer. Human intervention and scrutiny is still necessary as the AI’s work is, at best, a draft suggestion.

 

(3) Master the art of prompting. The quality of AI output is dependent on the quality and clarity of instructions given to it. Learners need to hone their ability to articulate their requirements well. Later in the same tutorial, students were given a prompt for ChatGPT to generate a pitch. They were then tasked with identifying shortcomings to the output and to produce better prompts to overcome those issues.

 

After the tutorial, many students reported newfound confidence and competency in utilising ChatGPT (n=351):

Table 1
Students’ perception of ChatGPT competency before and after tutorial

I considered myself very competent in using ChatGPT
Before Tutorial
(Average 2.76)
After Tutorial
(Average 3.71)
5 – Strongly Agree  5.41% 15.67%
4 22.79% 47.29%
3 27.07% 29.91%
2 31.91% 6.84%
1–Strongly Disagree 12.82% 0.28%

 

Table 1
Students’ perception of ChatGPT competency before and after tutorial

The tutorial taught me how to effectively collaborate and work with an AI for work.
(Average 4.19)
The tutorial taught me how to effectively critique and evaluate AI generated output so that I don’t take the answers for granted.
(Average 4.34)
The tutorial taught me how to design better prompts to get better results.
(Average 4.38)
I believe the skills taught in Tutorial 4 are useful for me when I go out to work.
(Average 4.28)
30.77% 40.46% 43.87% 38.46%
58.69% 53.56% 50.43% 52.71%
9.69% 5.70% 5.41% 7.41%
0.85% 0.28% 0.28% 1.42%
0% 0% 0% 0%

 

Overall, students had positive experiences learning this new approach to AI. They felt empowered and even an optimism about their future—knowledge of using AI in an augmentative way opens doors of opportunities that seemed too distant previously. In one case, a social science major shared how he felt so empowered by the tutorial that he took on a coding internship (despite being new to coding). He used ChatGPT to learn how to code which facilitated him to handle coding projects at work. This augmentative approach not only allowed him to produce solutions but also evaluate them much faster than if he did it on his own.

 

I firmly believe that teaching students how to augment their learning with GenAI tools holds immense potential in empowering our students for the future.

 

REFERENCES

Eors Szathmary et al. (2018). Artificial or augmented intelligence? the ethical and societal implications. In J. W. Vasbinder, B. Gulyas. & J. W. H Sim (Eds.), Grand Challenges for Science in the 21st Century. World Scientific.

Plato. (1952). Phaedrus. Trans. Reginald Hackforth. Cambridge University Press.

 

Harnessing the Potential of Generative AI in Medical Undergraduate Education Across Different Disciplines—Comparative Study on Performance of ChatGPT in Physiology and Biochemistry Modified Essay Questions

W. A. Nathasha Vihangi LUKE1*, LEE Seow Chong2, Kenneth BAN2, Amanda WONG1, CHEN Zhi Xiong1,3, LEE Shuh Shing3 , Reshma Taneja1,
Dujeepa SAMARASEKARA3, Celestial T. YAP1

1Department of Physiology, Yong Loo Lin School of Medicine (YLLSOM)
2Department of Biochemistry, YLLSOM
3Centre for Medical Education, YLLSOM

*nathasha@nus.edu.sg

 

Luke, W. A. N. V., Lee, S. C., Ban, K., Wong, A., Chen, Z. X., Lee, S. S., Taneja, R., Samarasekara, D., & Yap, C. T. (2023). Harnessing the potential of generative AI in medical undergraduate education across different disciplines—comparative study on performance of ChatGPT in physiology and biochemistry modified essay questions [Paper presentation]. In Higher Education Campus Conference (HECC) 2023, 7 December, National University of Singapore. https://blog.nus.edu.sg/hecc2023proceedings/harnessing-the-potential-of-generative-ai-in-medical-undergraduate-education-across-different-disciplines-comparative-study-on-performance-of-chatgpt-in-physiology-and-biochemistry-modified-es/ 
 

SUB-THEME

AI and Education

 

KEYWORDS

Generative AI, artificial intelligence, large language models, physiology, biochemistry

 

CATEGORY

Paper Presentations

 

INTRODUCTION & JUSTIFICATION

Revolutions in generative artificial intelligence (AI) have led to profound discussions on its potential implications across various disciplines in education. ChatGPT passing the United States medical school examinations (Kung et al., 2023) and excelling in other discipline-specific examinations (Subramani et al., 2023) displayed its potential to revolutionise medical education. Capabilities and limitations of this technology across disciplines should be identified to promote the optimum use of the models in medical education. This study evaluated the performance of ChatGPT, a large language model (LLM) by Open AI, powered by GPT 3.5, in modified essay questions (MEQs) in physiology and biochemistry for medical undergraduates.

 

METHODOLOGY

Modified essay questions (MEQs) extracted from physiology and biochemistry tutorials and case-based learning scenarios were encoded into ChatGPT. Answers were generated for 44 MEQs in physiology and 43 MEQs in biochemistry. Each response was graded by two examiners independently, guided by a marking scheme. In addition, the examiners rated the answers on concordance, accuracy, language, organisation, and information and provided qualitative comments. Descriptive statistics including mean, standard deviation, and variance were calculated in relation to the average scores and subgroups according to Bloom’s Taxonomy. Single factor ANOVA was calculated for the subgroups to assess for a statistically significant difference.

 

RESULTS

ChatGPT answers (n = 44) obtained a mean score of 74.7(SD 25.96) in physiology. 16/44(36.3%) of the ChatGPT answers scored 90/100 marks or above. 29.5%, numerically 13/44, obtained a score of 100%. There was a statistically significant difference in mean scores between the higher-order and lower-order questions on the Bloom’s taxonomy (p < 0.05). Qualitative comments commended ChatGPT’s strength in producing exemplary answers to most questions in physiology, mostly excelling in lower-order questions. Deficiencies were noted in applying physiological concepts in a clinical context.

 

The mean score for biochemistry was 59.3(SD 26.9). Only 2/43(4.6%) obtained 100% scores for the answers, while 7/43(16.27%) scored 90 or above marks. There was no statistically significant difference in the scores for higher and lower-order questions of the Bloom’s taxonomy. The examiner’s comments highlighted those answers lacked relevant information and had faulty explanations of concepts. Examiners commented that outputs demonstrated breadth, but not the depth expected.

nathasha luke et al, - Distribution of scores

Figure 1. Distribution of scores.

 

CONCLUSIONS AND RECOMMENDATIONS

Overall, our study demonstrates the differential performance of ChatGPT across the two subjects. ChatGPT performed with a high degree of accuracy in most physiology questions, particularly excelling in lower-order questions of the Bloom’s taxonomy. Generative AI answers in biochemistry scored relatively lower. Examiners commented that the answers demonstrated lower levels of precision and specificity, and lacked depth in explanations.

 

The performance of language models largely depends on the availability of training data; hence the efficacy may vary across subject areas. The differential performance highlights the need for future iterations of LLMs to receive subject and domain-specific training to enhance performance.

 

This study further demonstrates the potential of generative AI technology in medical education. Educators should be aware of the abilities and limitations of generative AI in different disciplines and revise learning tools accordingly to ensure integrity. Efforts should be made to integrate this technology into learning pedagogies when possible.

 

The performance of ChatGPT in MEQs highlights the ability of generative AI as educational tools for students. However, this study confirms that the current technology might not be in a state to be recommended as a sole resource, but rather be a supplementary tool along with other learning resources. In addition, the differential performance in subjects should be taken into consideration by students when determining the extent to which this technology should be incorporated into learning.

 

REFERENCES

Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198. https://doi.org/10.1371/journal.pdig.0000198

 Subramani, M., Jaleel, I., & Krishna Mohan, S. (2023). Evaluating the performance of ChatGPT in medical physiology university examination of phase I MBBS. Advances in Physiology Education, 47(2), 270–71. https://doi.org/10.1152/advan.00036.2023

 

Incorporating Generative AI in Project-based Learning: Case Study of How Students Utilise Generative AI in Tech-enabled Projects

Kate Sangwon LEE* and KHOO Eng Tat
Engineering Design & Innovation Centre
*katelee@nus.edu.sg

 

Lee, K. S., & Khoo, E. T. (2023). Incorporating generative AI in project-based learning: Case study of how students utilise generative AI in tech-enabled projects [Paper presentation]. In Higher Education Campus Conference (HECC) 2023, 7 December, National University of Singapore. https://blog.nus.edu.sg/hecc2023proceedings/incorporating-generative-ai-in-project-based-learning-case-study-of-how-students-utilise-generative-ai-in-tech-enabled-projects/ 

SUB-THEME

AI and Education 

 

KEYWORDS

Generative AI, technology-enabled project, project-based learning, interdisciplinary learning

 

CATEGORY

Paper Presentation 

 

ABSTRACT

As generative artificial intelligence (AI), such as Chat GPT and Midjourney, continues to permeate various industries, we have witnessed a recent surge in its adoption within project-based learning in education. (Gimpel et al., 2023; Su & Yang, 2023). However, as this technology is rapidly evolving and new services are introduced by various software platforms, understanding the appropriate software services and how they could be utilised in the students’ projects are challenging. This paper presents three case studies (under the module EG3301R “Ideas to Proof- of-Concept,” offered by the Innovation & Design Programme) that highlight how students identified design opportunities where utilisation of generative AI technology could enhance and improve the effectiveness of the learning process.

 

CONTEXT

Generative AI usually refers to AI systems that generate new content, including images, texts, music, and synthetic data (Cooper, 2023; Gimpel et al., 2023). One of the most representative services of generative AI is ChatGPT, a conversational service that uses large language models to interact with users (Gimpel et al., 2023).

 

Project-based learning is student-centred, context-specific, and inquiry-based learning where students can be actively involved in the learning process by interacting with other students and teachers within real-world practices (Kokotsaki et al., 2016). EG3301R is a project-based module that guides students to learn how to develop technology-enabled product ideas to address defined problems, and generate and evaluate concept designs by building prototypes and performing user testing.

 

CASE STUDIES

This paper introduces the three projects which utilised generative AI technology in their development process. The first project, “the Dentistry-geriatric patients’ communication training with VR service,” adopted Midjourney to generate geriatric patient characters and D-ID to create animation (see Figures 1 and 2).

Kate Sangwon Lee + Khoo Eng Tat Fig 1
Figure 1. Geriatric patient characters generated by using Midjourney.

 

Kate Sangwon Lee + Khoo Eng Tat Fig 2
Figure 2. Simulation video of geriatric patients by adopting D-ID.

 

The second project is an interprofessional education training service in healthcare and used Inworld to create various types of patient characters (Figure 3).

Kate Sangwon Lee + Khoo Eng Tat Fig 3
Figure 3. AI patient creation by using Inworld.

 

The third project involved developing a Korean language training AI chatbot that can help Korean learners practice diverse conjugation by adopting ChatGPT to generate various sentences (Figure 4).

Kate Sangwon Lee + Khoo Eng Tat Fig 4
Figure 4. Introduction about Edubot, an AI chatbot for Korean learners.

 

CHALLENGES AND IMPLICATIONS

The use of generative AI can be challenging due to its novelty and students’ lack of experience. Thus, supervisors should introduce available services and help them scrutinise possible opportunities to adopt the most appropriate generative AI technology from the market. To facilitate this process, it would be helpful to establish a database of previous cases and share it with students to spread knowledge. Generative AI services can simplify recurring tasks in students’ technology- enabled projects, such as creating various characters and scenarios, as shown in Table 1. Supervisors should closely observe their concept design and development process and advise on how to effectively incorporate generative AI technologies. Educators can further encourage the use of generative AI tools by sharing case studies and promoting their integration in students’ technology projects.

 

Table 1
Three projects, their objectives to use generative AI and used services

Project Objectives to use generative AI services Used generative AI services
Dentistry-geriatric patients’ communication training Various characters, emotions, and animations generation Midjourney, D-ID
Interprofessional education training service in healthcare Various patient characters and scenario generation Inworld
Edubot Sentences and questions generation, character generation ChatGPT, D-ID

 

 

REFERENCES

Cooper, G. (2023). Examining science education in ChatGPT: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32(3), 444-52. https://doi.org/10.1007/s10956-023-10039-y

Gimpel, H., Hall, K., Decker, S., Eymann, T., Lämmermann, L., Mädche, A., Röglinger, M., Ruiner, C., Schoch, M., & Schoop, M. (2023). Unlocking the power of generative AI models and systems such as GPT-4 and ChatGPT for higher education: A guide for students and lecturers. Hohenheim Discussion Papers in Business, Economics and Social Sciences No. 02-2023. http://hdl.handle.net/10419/270970

Kokotsaki, D., Menzies, V., & Wiggins, A. (2016). Improving Schools, 19(3), 267-77. https://doi.org/10.1177/1365480216659733

Su, J., & Yang, W. (2023). Unlocking the power of ChatGPT: A framework for applying generative AI in education. ECNU Review of Education, 20965311231168423. https://doi.org/10.1177/20965311231168423

 

Viewing Message: 1 of 1.
Warning

Blog.nus accounts will move to SSO login, tentatively before the start of AY24/25 Sem 2. Once implemented, only current NUS staff and students will be able to log in to Blog.nus. Public blogs remain readable to non-logged in users. (More information.)