Generative AI

Generative AI Use In The Classroom: Student Perception And Learning Outcomes

cdtlsd Lighting Talks , , , ,

Marissa K. L. E*  and Misty So-Sum WAI-COOK 

Centre for English Language Communication (CELC), NUS 

*elcmari@nus.edu.sg 

E, M. K., L., & Wai-Cook, M. S. S. (2024). Generative AI use in the classroom: Student perception and learning outcomes [Lightning talk]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-marissakle-wai-cook/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

Generative AI, academic literacies, higher education, academic writing 

CATEGORY

Lightning Talk

EXTENDED ABSTRACT

The emergence of Generative AI has required educators to re-think their approaches to teaching and learning. In the context of the academic literacies classroom, the use of Generative AI has caused mixed reactions among educators. This is because of how the technology can potentially provide useful writing support to students, yet also be easily used to produce texts on demand that can meet assignment requirements. As such, calls have been made for writing instructors to teach students how to appropriately harness Generative AI as a tool to enhance their writing and research skills (Johinke et al., 2023).  

 

In response to this call, we describe here, two exploratory case studies from two different writing courses where Generative and semi-Generative AI tools have been used in the academic literacies classroom to teach students how to develop initial topic ideas for a research paper, assist them with the writing process and provide them with the opportunity to critically assess different platforms to search for scholarly sources for the research paper. The main difference between Generative and semi-Generative AI tools is that the latter is not as freely generative as the former. 

 

The focus in the first case is to examine if, and how, students use Generative and semi-Generative AI tools to conceptualise ideas and assist them in writing research papers, and their perceptions of the effectiveness of using such tools during their writing process. Querying student perception regarding their experience with such tools is important as it can be associated with their motivations underlying the use of the tools, as well as useful insights about the utility of such tools for future student cohorts (Chan & Hu, 2023). A survey was used in this first case.  

 

In the second case, the use of Consensus.ai, a semi-Generative AI tool employed for searching for research sources, is described. We show here how students not only used the tool to search for research sources, exhibiting useful research skills, but also used critical thinking (Huang et al., 2024) to discern whether these sources are useful in the context of a proposed small-scale research study assignment.  

 

Figure 1 shows an extracted portion from a graphic organizer used in class where students had to critically compare three platforms—Consensus.ai, Google Scholar and NUS Libraries databases like JSTOR. The first platform is the semi-Generative AI platform while the remaining two are more traditional platforms used for searching for research sources. Students used the organiser to pen down how the three platforms helped them with their literature search. In Figure 1, we can see how the student has formed a critical opinion about two of the platforms. He first points out how Consensus.ai has its strengths, like its ability to facilitate quick identification of the thesis of an article, in comparison to Google Scholar. He is also able to identify how Google Scholar has an advantage over Consensus.ai with its ability to cover a wider range of articles in comparison.  

 

Figure 1. Graphic Organiser

 

The fact that Generative AI is increasingly capable of producing writing that is grammatically correct and relevant to a given prompt, with only the characteristic of style lacking at this point, means that academic literacies courses need to go beyond text production and focus as well on how thinking skills like ideation that go into the process of producing text can be facilitated (Dai et al., 2023).  

REFERENCES

Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education, International Journal of Educational Technology in Higher Education, 20(43), https://doi.org/10.1186/s41239-023-00411-8 

Dai, Y., Liu, A., & Lim, C. P. (2023). Reconceptualizing ChatGPT and generative AI as student-driven innovation in higher education. Procedia CIRP, 119, 84-90.  https://doi.org/10.1016/j.procir.2023.05.002

Huang, C. W., Coleman, M., Gachago, D., & Van Belle, J. P. (2024). Using ChatGPT to Encourage Critical AI Literacy Skills and for Assessment in Higher Education. In Van Rensburg, H.E., Snyman, D.P., Drevin, L., Drevin, G.R. (Eds.), ICT Education. SACLA 2023. Communications in Computer and Information Science, 1862. Springer, Cham. 

Johinke, R., Cummings, R., & Di Lauro, F. (2023). Reclaiming the technology of higher education for teaching digital writing in a post-pandemic world. Journal of University Teaching and Learning Practice, 20(2), https://doi.org/10.53761/1.20.02.01 

Opportunity, Implementation, Evaluation: Raising Awareness Of Generative AI Use In A Business Communication Course

cdtlsd Lighting Talks , , ,

Aileen Wanli LAM 

Centre for English Language Communication (CELC), NUS 

aileenlam@nus.edu.sg

Lam, A. W. (2024). Opportunity, implementation, evaluation: Raising awareness of generative AI use in a business communication course [Lightning talk]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-awlam/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

Generative AI, Awareness, Business Communication 

CATEGORY

Lightning Talk

EXTENDED ABSTRACT

Generative AI adoption has surged over the past two years, and AI is now being used in many business functions and by more individuals than in previous years (Mckinsey, 2024b). A growing number of global CEOs and investors believe that AI adoption is crucial for productivity and value creation, so they seek out employees who are able to harness the use of AI (PWC, 2024) while employing higher cognitive skills such as critical thinking and decision-making (Mckinsey, 2024a). Since AI advancements have led to “significant implications for business communication and…pedagogy” (Sharma & Pandey, 2024, p.2), there is a need to relook business communication curriculum to remain relevant and up-to-date.   

  

This paper outlines an action research which started from a gap observed in a business communication course, an opportunity to gather information to develop a plan in Semester 1, the implementation of the plan in Semester 2, and the observation and analysis of the implementation post-semester. The findings were reflected upon and used to refine the approach in the next academic year. Though the findings are unique to the context of the course, the approach and tools used can be applied to other courses and disciplines.  

 

As a course coordinator of an undergraduate business communication course, I had two questions on my mind: “What generative AI content should be included in a business communication course?” and “How effective would the introduction of generative AI content be?”   

  

In the first semester, an opportunity to collect information presented itself with the call from the academic community to acknowledge and cite the use of generative AI. This led to the creation and introduction of the AI Declaration forms (Lam, 2024b) for two assessments—the business proposal and the pitch. Students declared their use of generative AI and more specifically wrote about the tools they were using, and how they were using it (i.e. prompts and what they did to the outputs). The data was analysed and coded into themes (e.g. stages of writing or preparation) to inform the curriculum updates that focused on the principles and limitations of using AI tools in business communication. The key learning objective that was emphasised throughout was critical thinking through evaluation of generated outputs and through creation—their ability to combine parts to form a coherent whole or new product (Bloom’s 2001 revised taxonomy)  

 

The following topics (AlAfnan et al, 2024; Lam, 2024a; Riapina, 2023) were discussed over the semester 

  • Use of AI tools for research and idea generation with an emphasis on evaluation and creation (critical thinking)  
  • Issues of ownership of AI-generated outputs (e.g. terms of use, licenses, laws) and the implications of using AI in school   
  • Use of AI for the editing process (e.g. grammar, spelling, enhancing vocabulary, paraphrasing, style, tone) and implications of using AI at work   
  • Use of AI for presentations (i.e. creation of slides and visuals; writing of scripts)  
  • Use of AI to prepare for the Q&A (e.g. generating questions and practicing answers) with an understanding of the context  

  

In order to understand the effectiveness of the implementation, (1) a perception survey, and (2) an analysis of the AI declaration forms for semester 2 were carried out.  

 

In the perception survey, participants responded that their awareness of the principles and potential issues of using AI had increased from an average of 3.43 (pre-course) to 4.08 (post-course) on a scale of 5 (0= No Knowledge; 5= Very Knowledgeable) but they had varying responses to the usefulness of each topic. The analysis of responses in the AI declaration forms for Semester 2 found instances of critical thinking (i.e. student evaluation of generated outputs and the creation of coherent and/or original work). 

 

This presentation shares the findings of this action research and the significance of the approach. 

REFERENCES

AlAfnan, M. A., Dishari, S., & MohdZuki, S. F. (2024). Developing Soft Skills in the Artificial Intelligence Era: Communication, Business Writing, and Composition Skills. Journal of Artificial Intelligence and Technology. http://dx.doi.org/10.37965/jait.2024.0496

Anderson, L.W. (Ed.), Krathwohl, D.R. (Ed.), Airasian,P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R.,Raths, J., & Wittrock, M.C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of Educational Objectives. Longman

Lam, A. W. (2024, May 1). Embracing AI in education. Faculty Focus. https://www.facultyfocus.com/articles/teaching-with-technology-articles/embracing-ai-in-education 

Lam, A. W. (2024, May 28). Learning from learners: Student use of AI. The Teaching Professor.  https://www.teachingprofessor.com/topics/teaching-strategies/teaching-with-technology/learning-from-learners-student-use-of-ai/ 

Riapina, N. (2023). Teaching AI-enabled business communication in higher education: A practical framework. Business and Professional Communication Quarterly, 23294906231199249. https://doi.org/10.1177/23294906231199249

Sharma, D., & Pandey, H. (2024). Pedagogical Impact of Text-Generative AI and ChatGPT on Business Communication. Business and Professional Communication Quarterly, 23294906241249113. https://doi.org/10.1177/23294906241249113 

Mckinsey. (2024a, March 18). The human side of generative AI: Creating a path to productivity    https://www.mckinsey.com/capabilities/people-and-organizational-performance/our-insights/the-human-side-of-generative-ai-creating-a-path-to-productivity 

Mckinsey. (2024b, May 20). The state of AI in early 2024: Gen AI adoption spikes and starts to generate value. https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai 

PWC. (2024, May 21) AI Jobs Barometer. https://www.pwc.com/gx/en/issues/artificial-intelligence/ai-jobs-barometer.html 

Navigating Authentic Learning In The Age Of Generative AI

cdtlsd Lighting Talks , , , ,

Sharon LAU Pui Wan 

NUS-ISS 

sharon.lau@nus.edu.sg 

Lau, S. P. W. (2024). Navigating authentic learning in the age of generative AI [Lightning talk]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-spwlau/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

Generative AI, Authentic Learning, Critical Thinking, Educational Strategies 

CATEGORY

Lightning Talk

EXTENDED ABSTRACT

In an era where generative AI is rapidly transforming educational landscapes, educators face both unprecedented opportunities and formidable challenges. As an adult educator in NUS, I propose to address the most pressing issue: how can we foster authentic learning experiences while leveraging the capabilities of generative AI? This lightning talk aims to provide a roadmap for educators to navigate this complex terrain, addressing the following aspects: 

Embrace AI as a co-educator 

Generative AI can serve as a dynamic partner in the educational process, providing personalised feedback, facilitating adaptive learning, and enabling interactive simulations. By integrating AI-driven tools, educators can create more engaging and tailored learning experiences. An example from my own practice involves using AI tools to simulate real-world business scenarios. Students interact with AI-generated market data, making strategic decisions that mirror professional challenges. This method not only deepens their understanding but also prepares them for the complexities of the professional world. The AI acts as a co-educator, providing immediate feedback and alternative perspectives, thereby enriching the learning experience. 

Cultivate critical thinking  

One of the core components of authentic learning is the development of critical thinking skills. AI can play a crucial role in this area by presenting students with complex, open-ended problems that require creative solutions. Research has shown that AI can aid in the development of higher-order thinking skills by challenging students to analyse, interpret, and synthesise information. For example, in my classes, I utilise AI to design challenging case studies that require students to engage deeply with the material. These case studies present AI-generated data that students must critically evaluate and interpret. This approach fosters a deeper level of engagement and helps students develop the critical thinking skills necessary to navigate an increasingly complex world. The use of AI in presenting diverse scenarios and data sets encourages students to consider multiple perspectives and develop well-rounded solutions. 

 

During this talk, I will draw from the latest research and my practical experiences to highlight key strategies and insights. I will share examples of real-world practice to illustrate my key messages. 

 

Rethinking Assessment in the Age of GenAI

cdtlsd Lighting Talks , , ,

YANG Yi 

Department of Civil and Environmental Engineering,
College of Design & Engineering (CDE), NUS

yangyi@nus.edu.sg  

Yang, Y. (2024). Rethinking assessment in the age of GenAI [Lightning talk]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-yangyi/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

Generative AI, assessment methodology, academic integrity 

CATEGORY

Lightning Talk

INTRODUCTION

The advent of generative AI (GenAI) has profoundly transformed various sectors, with education being no exception. As GenAI technology advances, it presents both opportunities and challenges to traditional educational practices. While GenAI can automate many teaching tasks such as generating content (such as Figures 1, 2, 3 generated by AI for the subsequent paragraphs) and preparing lesson plans, one primary concern among educators is upholding academic integrity, given that GenAI can complete many assessments with minimal human effort. To address these challenges and leverage the potential of GenAI, there is a need to rethink and redesign our assessment models. This paper explores the strategies to adapt our assessment approaches in alignment with GenAI to enhance education outcomes. 

BE CLEAR OF THE OBJECTIVE 

First and foremost, educators should define clear objectives for their assessments. The format of assessments should align with specific learning goals in the context of GenAI. For example, if the aim is to gauge students’ retention of basic concepts, an in-class test that forbids GenAI use might be the most suitable. In contrast, if the goal is to create an authentic environment and enhance students’ ability to search for and organise information, a take-home assignment will be suitable. As the availability of GenAI has already become part of the authentic learning environment, the use of GenAI should be permitted or even encouraged in such assessments. It is critical that when AI tools are accessible for the assessments, all questions and tasks should be tested with the latest version of GenAI to ensure the assessments meet the intended learning objectives. 

Figure 1. Be clear of the objectives of the assessment (image generated by Microsoft Copilot) 

EMPHASISE SOFT SKILLS 

Secondly, assessments should place greater emphasis on soft skills. In the age of GenAI, competencies such as communication skills and teamwork become even more vital, as many tasks that traditionally require hard skills can now be automated by AI. These soft skills truly differentiate humans from machines and are becoming increasingly valuable in the professional sphere. To cultivate these soft skills, educators may consider designing project-based assessments with greater scoring weightage on oral presentations, Q&A sessions, and role-playing activities [1]. These assessment models not only enhance the development of soft skills, but also eliminate concerns regarding AI-related academic integrity issues by evaluating real-time interactions. 

Figure 2. Emphasise soft skills in assessment (image generated by Microsoft Copilot) 

INTEGRATE AI INTO ASSESSMENT MODELS 

Thirdly, integrating AI into assessments can foster innovative evaluation models. Instead of viewing AI merely as a challenge to academic integrity, educators and institutions can harness its capabilities to enrich and enhance learning experiences. For instance, an assessment could involve students reviewing and critiquing answers generated by GenAI. In an Urban Planning and Design course at the University of Hong Kong, students are tasked to: i) generate an essay using GenAI tools, ii) critically evaluate the AI-generated content, and iii) compose their own essay based on their critique [2]. This approach not only deepens students’ understanding of the subject but also helps them recognise the strengths and limitations of AI tools. By engaging with AI-generated content, students also develop critical thinking and analytical skills as they learn to evaluate and refine the information provided.  

Figure 3. Integrate genAI in assessment design (image generated by Microsoft Copilot)   

CONCLUDING REMARKS 

Educators should swiftly adapt to the evolving landscape of AI technology. By aligning assessments with clear learning objectives, emphasising the development of soft skills, and leveraging AI capabilities in assessment design, educators can create a more effective and relevant educational experience. This proactive approach ensures that students are not only knowledgeable in their subjects but also well-prepared to navigate and excel in a world increasingly shaped by AI.

REFERENCES​

Sharma, R. C., & Bozkurt, A. (2024). Transforming education with generative AI: Prompt engineering and synthetic content creation. IGI Global. 

Chan, C. K. Y., & Colloton, T. (2024). Generative AI in higher education: The ChatGPT effect (1st ed.). Taylor & Francis. 

Harnessing GPT to Develop a Personalised eLearning Web Application for Java Programming

cdtlsd Lighting Talks , , , ,

LIU Fan*, Esther TAN Meng Yoke, and CHIA Yuen Kwan 

NUS-ISS 

*isslf@nus.edu.sg  

Liu, F., Tan, E. M. Y., & Chia, Y. K. (2024). Harnessing GPT to develop a personalised elearning web application for Java programming [Lightning talk]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-lfan-et-al/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

Generative AI, Personalised eLearning, ChatGPT, Java Programming 

CATEGORY

Lightning Talk

BACKGROUND

E–learning systems are gaining popularity due to their extensive scalability, flexibility, self-paced capability as well as easy accessibility. Artificial Intelligence (AI) has been used in e-Learning systems (Murtaza et al., 2022; Sayed et. al., 2023) to make personalised learning recommendations to learners. Specifically, Generative Pre-trained Transformers (GPT) such as ChatGPT (OpenAI, 2023) has transformed education by offering intelligent assistance, customised content, personalised learning materials to cater to the individual learner’s needs (Rasul, 2023; Jin & Kim, 2023; Gharbi et al., 2024). AI-based eLearning can significantly enhance learning efficiency through personalised content delivery to learners (Bozkurt, 2021; Chen et al., 2023). However, these AI-based personalised eLearning techniques have not been effectively integrated to provide diverse and dynamic eLearning content for learners based on their personalities and learning preferences. 

PURPOSE

Propose an approach to provide dynamic and personalised content for learners through integrated Generative AI techniques. We developed an eLearning web application that connects to ChatGPT API, which will automatically generate dynamic course contents/exercises/quizzes based on the learners’ personality such as level of language proficiency and personal learning preferences for Java foundations. 

RESEARCH QUESTIONS

  1. How to automatically generate dynamic and personalised Java foundational content for each learner on the fly?
  2. How to automatically generate dynamic and personalised coding exercise questions for each learner on the fly?
  3. How to provide a more efficient learning experience for each learner based on the learner’s personality and learning preferences? 

METHOD

Design and develop an eLearning web application that seamlessly integrates the following technologies:

  1. Automatically assesses learner’s learning preferences using the Felder-Silverman learning style model (FSLSM) (Felder & Silverman, 2012). The FSLSM categorises learners into specific dimensions based on their preferred ways of acquiring and processing information. This model is used in our eLearning application to customise the course delivery format to suit the learners’ learning preferences.
  2. Harness the GPT model to generate dynamic course contents
  3. Provide diverse and personalised eLearning content based on learners’ personality and learning preferences.

 

The general architecture of the eLearning application consists of three modules: web browser client, eLearning server, and Microservices (ChatGPT server). The web browser client enables learners to interact with the eLearning application. The eLearning server delivers the core functionality of the system which includes the generation of dynamic topics, personalised exercises, and quizzes while the Microservice represents the external ChatGPT server.  

KEY FINDINGS 

This study explores how ChatGPT with FSLSM can be harnessed to create a customised Java programming course and provide learners with dynamic and personalised course materials. This method promotes a customised learning journey catering to the different learner’s needs in terms of their levels of Java proficiency and learning preferences.  

FUTURE WORK

The future work of the proposed eLearning application is summarised as follows: (1) using ChatGPT model to generate comprehensive course content including advanced topics of Java, (2) extend this research work to generate dynamic and personalised course content for other object-oriented programming languages as C++, C#, and Python, (3) introduce foundational topics, exercises and quizzes with support of different learning methods, such as hands-on workshops, auto-grading and learning feedback and report., (4) extend the application of the personalised eLearning application to include other academic disciplines. This future work should consider the challenges of the use of ChatGPT in education with regards to the accuracy and reliability of content.

REFERENCES​

Bozkurt, A., Karadeniz, A., Baneres, D., Guerrero-Roldán, A. E., & Rodríguez, M. E. (2021). Artificial intelligence and reflections from educational landscape: A review of AI Studies in half a century. Sustainability, 13(2), 800. https://doi.org/10.3390/su13020800

Chen, E., Huang, R., Chen, H. S., Tseng, Y. H., & Li, L. Y. (2023, June). GPTutor: a ChatGPT-powered programming tool for code explanation. In International Conference on Artificial Intelligence in Education (pp. 321-327). Springer Nature Switzerland. 

Felder, R., & Silverman, L. (2012). Index of learning styles questionnaire. North Carolina State University. Retrieved from http://www.engr.ncsu.edu/learn ingst yles/ilswe b.html. 

Gharbi, M., Taib Mohtadi, M., & Fal Merkazi, A. (2024). Revolutionizing Moocs with fine-tuned Chatgpt: Personalized Learning At Scale. International Journal of Computing and Digital Systems, 16(1), 1-11. 

Jin, J., & Kim, M. (2023). GPT-Empowered Personalized eLearning System for Programming Languages. Applied Sciences, 13(23), 12773. https://doi.org/10.3390/app132312773

Murtaza, M., Ahmed, Y., Shamsi, J. A., Sherwani, F., & Usman, M. (2022). AI-based personalized e-learning systems: Issues, challenges, and solutions. IEEE Access, 10, 81323–81342. https://dx.doi.org/10.1109/ACCESS.2022.3193938

OpenAI (2023). ChatGPT: Optimizing Language Models for Dialogue. https://openai.com/index/chatgpt/

Rasul, T., Nair, S., Kalendra, D., Robin, M., de Oliveira Santini, F., Ladeira, W. J., & Heathcote, L. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning and Teaching, 6(1), 41-56. http://dx.doi.org/10.37074/jalt.2023.6.1.29

Sayed, W. S., Noeman, A. M., Abdellatif, A., Abdelrazek, M., Badawy, M. G., Hamed, A., & El-Tantawy, S. (2023). AI-based adaptive personalized content presentation and exercises navigation for an effective and engaging E-learning platform. Multimedia Tools and Applications, 82(3), 3303-3333. https://doi.org/10.1007/s11042-022-13076-8

Leveraging Multi-Agent Generative AI for Next-Generation Education and Career Development

cdtlsd Lighting Talks , , , , ,

WANG Hongtai* and YEO Wee Kiang 

Department of Information Systems and Analytics,
School of Computing (SOC), NUS

*e1132287@nus.edu.sg

Wang, H., & Yeo, W. K. (2024). Leveraging multi-agent generative AI for next-generation education and career development [Lightning talk]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-hwang-wkyeo/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

Educational Support, Career Advisory, Generative AI, Multi-Agent System, Large Language Model 

CATEGORY

Lightning Talk

INTRODUCTION

In today’s complex education and career planning landscape, students face challenges in navigating vast resources and making informed decisions. Traditional career advisory services often lack the necessary personalisation. This project aims to leverage AI technologies, particularly large language models (LLMs) and multi-agent systems, to develop a novel solution that provides tailored guidance and support. 

The system’s functionality is divided into two core functionalities: Tutor and Career Advisory. Together, these features create a cohesive system that guides students through both academic and career development with personalised solutions. 

KEY CAPABILITIES 

Tutor Section 

  • Tutorial Teaching: This function introduces subject matter concepts progressively, from simple to complex, with clear examples for better understanding. 
  • Question-and-Answer: This function allows users to ask questions about the course materials and receive contextually relevant answers. 
  • Providing Review Questions: This function generates questions based on provided materials, testing students’ mastery of concepts through customised review questions. 

 

Career Advisory Section 

  • Generate Interview Questions. Generates tailored interview questions based on job descriptions by analysing required skills and knowledge. 
  • Analyse Shortcomings. Compare the student’s resume and the job description to highlight skill and knowledge gaps. 
  • Generate Cover Letters. Automatically examine the job description and user’s resume, extract the most relevant resume highlights, and generate a cover letter emphasizing the user’s advantages for the role. 
  • Assist Users in Gaining Essential Skills. Creates technology tutorials tailored to job descriptions, providing links to online courses and materials to enhance learning. 

METHODOLOGY 

Architecture 

The architecture of this project is designed as a dialogue system based on a generative AI model. The core component is a LLM agent that acts as a traffic controller and interacts with the user. Based on the user’s input prompt, this agent will discern whether the user’s queries are related to Tutor or Career Advisory and thereafter provide appropriate responses accordingly. 

If the user’s request is straightforward, the LLM agent handles it directly. For more complex requirements, the agent collects detailed information about the user’s needs and activates specialised agent groups (Cheng et al., 2024) designed to address specific issues. An agent group consists of multiple LLM agents created using Autogen (Wu et al., 2023) that is responsible for one specific task. These specialised agent groups include: 

  1. Education Agent Groups: Responsible for tasks such as coaching users with course materials, answering their questions, and providing review questions. 
  2. Career Agent Groups: Responsible for generating interview questions, analysing the user’s shortcomings, generating cover letters, and providing learning resources. 

Figure 1. Diagram of our system with LLM agent and role-based groups.

KEY FINDINGS 

Interface Design 

In the Tutor section, users engage directly with agent tutors through a system that allows seamless switching between instructional content and question-and-answer sessions. 

Figure 2. Our AI platform assists users in learning Operations Management through dialogue. 

 

The Career section offers unrestricted communication with AI-powered LLM agents, providing essential career advisory. This includes generating personalised tables that identify skill gaps relative to job requirements and assisting in crafting tailored cover letters to improve job applications. 

Figure 3. User interface for generating a comparison table between job requirements and applicant qualifications. 

SIGNIFICANCE OF THE PROJECT 

  • Enhanced Personalisation: Tailored learning experiences with progressive subject matter introduction, customized Q&A support, and personalised review questions. Career guidance includes tailored interview questions, skill gap analysis, and personalized cover letters. 
  • Improved Accessibility: 24/7 support with easy access to relevant online courses and materials. 
  • Efficiency: Streamlined assistance through specialized agent groups, providing quick and precise help. Identifies skill gaps and offers targeted learning recommendations. 
  • Career Preparation: Generates mock interview questions and skill gap analyses for better preparation. Automatically creates personalised cover letters and offers resume advice. 

REFERENCES​

Cheng, Y., Zhang, C., Zhang, Z., Meng, X., Hong, S., Li, W., Wang, Z., Wang, Z., Yin, F., Zhao, J., & He, X. (2024). Exploring large language model-based intelligent agents: Definitions, methods, and prospects.  ArXiv. abs/2401.03428. 

Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A.H., White, R.W., Burger, D., & Wang, C. (2023). AutoGen: Enabling next-gen LLM applications via multi-agent conversation. ArXiv. abs/2308.08155v2.

Harnessing Generative AI As A Personalised Tutor: Enhancing Interdisciplinary Learning Outcomes For Biotechnology Graduate Students

cdtlsd Lighting Talks , , , , ,

Xin Xiang LIM 

Department of Biological Sciences,
Faculty of Science, NUS 

xinxiang@nus.edu.sg 

Lim, X. X. (2024). Harnessing generative AI as a personalised tutor: Enhancing interdisciplinary learning outcomes for biotechnology graduate students [Lightning talk]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-xxlim/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

ChatGPT, Student’s Prompt Analysis, Interdisciplinary learning, Generative AI, Personalised Tutor

CATEGORY

Lightning Talk

EXTENDED ABSTRACT

The escalating complexity of societal issues necessitates that graduates from higher educational institutions engage in problem-solving endeavours that transcend singular disciplines (Mansilla & Duraising, 2007; Repko., 2007). When students grasp and interconnect a diverse array of knowledge and skills, their educational experiences become more fulfilling, and their employment prospects broaden (Ivanitskaya & Montgomery., 2002). This phenomenon is particularly pertinent in biotechnology, where innovation is paramount in transforming laboratory discoveries into marketable products that address societal challenges. The innovation process extends beyond biology, requiring an understanding of market needs, funding sources, business models, risk management, and competitor analysis—critical real-world considerations. Thus, fostering innovation demands collaborative, interdisciplinary approaches to facilitate the cross-pollination of ideas and the integration of multiple perspectives. Equipping students with the skills to identify problems, develop prototypes, and conduct market research can provide a robust framework for innovation (Boms et al., 2022). 

 

Pharmaceutical sciences, chemistry, and biotechnology, typically have limited exposure to essential business considerations necessary for evaluating product market viability. This gap underscores the need for substantial scaffolding in business and innovation concepts to enable students to integrate their biotechnological expertise with business innovation, ultimately facilitating product creation. 

 

Generative AI offers significant potential to enrich learning experiences, fostering creativity, critical thinking, and motivation among students. ChatGPT, for instance, has demonstrated efficacy in enhancing interactive learning and personalised tutoring (Baidoo-Anu & Ansah., 2023). Leveraging ChatGPT promotes inquiry-based learning and student-centric approaches, both of which are effective in enhancing learning outcomes. Previous studies indicate that generative AI tools increase intrinsic motivation, conversational engagement, and continuous idea expression among students (Ryan & Deci., 2020) This context presents two educational opportunities: 1) inquiry-based active learning through prompt generation (prompt engineering), and 2) learning from generative AI responses. Developing writing prompts and prompting strategies has become a critical skill in handling generative AI. Prompt datasets collected from generative AI, including log data, can capture students’ learning processes in non-invasive ways. Prompt analysis thus provides an opportunity for educators to gain insights into students’ perceptions, motivations, and behaviors concerning interdisciplinary learning. 

 

Evaluation of students’ interdisciplinary learning outcomes will be conducted through three primary methods: 1) pre- and post-course surveys to capture students’ self-perceptions of their interdisciplinary skills and knowledge, providing insights into their metacognition and epistemology (Lattuca et al., 2012); 2) Assessment of student assignments and presentations using published, peer-reviewed rubrics for interdisciplinarity; and 3) Analysis of student prompts generated with ChatGPT. The student prompts will be analysed using natural language processing techniques, as outlined in recent studies, to gain insights into students’ cognitive processes and modes of thinking (Lee et al., 2023). Through this triangulated approach, the overall aim of the study is to comprehensively and rigorously evaluate the cognitive processes underpinning interdisciplinary learning. 

 

This lightning talk will specifically describe the insights gleaned from the analysis of students’ prompts so as to identify and evaluate possible learning barriers faced by students in the process of interdisciplinary learning. 

 

Figure 1. Interdisciplinary approach to biotechnology innovation. 

REFERENCES

Baidoo-Anu, D., & Ansah, L. O. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52-62. psychology, 61, 101860. https://doi.org/10.1038/s41587-022-01253-x

Boms, O., Shi, Z., Mallipeddi, N., Chung, J. J., Marks, W. H., Whitehead, D. C., & Succi, M. D. (2022). Integrating innovation as a core objective in medical training. Nature Biotechnology, 40(3), 434-437. 

Boix Mansilla, V., & Dawes Duraising, E. (2007). Targeted assessment of students’ interdisciplinary work: An empirically grounded framework proposed. Journal of Higher Education, 78(2), 215-237. https://doi.org/10.1080/00221546.2007.11780874

Ivanitskaya, L., Clark, D., Montgomery, G. et al. (2002). Interdisciplinary Learning: Process and Outcomes. Innovative Higher Education 27, 95–111. https://doi.org/10.1023/A:1021105309984 

Lattuca, L. R., & Knight, D. B., & Bergom, I. M. (2012, June), Developing a Measure of Interdisciplinary Competence for Engineers. Paper presented at 2012 ASEE Annual Conference & Exposition, San Antonio, Texas. https://dx.doi.org/10.18260/1-2—21173 

Lee, U., Han, A., Lee, J., Lee, E., Kim, J., Kim, H., & Lim, C. (2023). Prompt Aloud!: Incorporating image-generative AI into STEAM class with learning analytics using prompt data. Education and Information Technologies, 1-31. https://doi.org/10.1007/s10639-023-12150-4

Repko, A. F. (2007). Integrating interdisciplinarity: How the theories of common ground and cognitive interdisciplinarity are informing the debate on interdisciplinary integration. Issues in Integrative Studies, 25, 1-31. http://hdl.handle.net/10323/4501

Content Analysis Of Student AI Use In A First-Year Writing Course

cdtlsd Paper Presentations , , , ,

Jonathan FROME  

NUS College 

frome@nus.edu.sg

Frome, J. (2024). Content analysis of student AI use in a first-year writing course [Paper presentation]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-jfrome/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

Generative AI, undergraduate, AI-assisted writing, content analysis 

CATEGORY

Paper Presentation 

 

EXTENDED ABSTRACT

The take-home essay has traditionally served as a reliable proxy for evaluating student writing skills. The rise of Generative AI (GenAI), however, has led to concerns that the take-home essay may no longer be a valid assessment tool. If instructors cannot determine whether a student or GenAI completed an assignment, such assignments may fail to demonstrate whether students have achieved the intended course learning outcomes. This concern is widespread among educators who rely on essays for assessment. For instance, Cardon et al.’s (2023) survey of over 300 communication instructors confirms the widespread concern that GenAI will increase plagiarism, reduce critical thinking, diminish writing skills, and make student assessment difficult. These fears often stem from intuitions about student behavior, such as the belief that “students just want the tool’s output without engaging in the actual [writing] process” (Chang et al. 2023). The speed with which GenAI can produce relatively high-quality essays has led some to suggest that university writing might shift to a model where “young writers will [try] to craft something meaningful and precise from the rough block of generic text that AI has provided them” (Moore 2023). 

 

Yet we cannot determine whether these concerns are justified because of a critical gap in the literature: the lack of research on how students actually use GenAI tools. Although instructors have strong intuitions about the effects of allowing students to use GenAI for writing assignments, few of these intuitions are evidence-based. We simply know very little about how students use GenAI in their coursework. While some instructors are beginning to incorporate GenAI into classroom activities, the primary concerns revolve around its use outside the classroom, which could undermine the effectiveness of essay writing for skill-building and assessment. 

 

This study aims to address this knowledge gap by exploring the following questions: How do students actually use GenAI tools for writing assignments when allowed to do so? How does their use relate to the primary concerns expressed by instructors? And what implications does this relationship have for designing college writing courses? 

 

In this study, students in a first-year writing class were allowed to use ChatGPT freely for their coursework, provided they shared links to their chat transcripts. The chats were downloaded, formatted into a spreadsheet, and analysed as pairs of user prompts and ChatGPT outputs. Over 600 pairs of prompts and outputs were collected and coded to understand how students used ChatGPT to complete their assignments. The coding categories were based on academic writing as a process involving discrete activities: reading and analysing sources, generating ideas, drafting, revising content, and revising form. Additional categories were added inductively during the coding process. 

 

The most serious concerns among instructors included fears that students would “offload” important writing activities (Watkins, 2024) to GenAI, such as active reading, thesis generation, and initial drafting. Such use could undermine the pedagogical value of assignments. Our findings suggest these concerns are supported only to a limited extent. Students were more likely to use GenAI as a reading aid (e.g., clarifying specific sentences) than as a substitute for active reading (e.g., summarising entire texts). Additionally, students used GenAI more often for revising their drafts than for generating initial drafts. 

 

These preliminary results suggest that in the context of take-home essays, the most salient instructor concerns about GenAI use are not entirely borne out. The stereotype that students will use GenAI to write essays for them was not supported, at least for the observed participants (though different students and assignments might yield different results). The findings also underscore the importance of considering specific course learning outcomes when evaluating the disruptive potential of GenAI. 

 

More fundamentally, this study provides an evidence-based account of how students use GenAI for writing assignments, which is crucial for developing more effective teaching strategies. Understanding student use of GenAI allows educators to design assignments that enhance learning and integrate GenAI into courses in ways that support, rather than undermine, critical thinking and writing skills. 

 

REFERENCES

Cardon, P., Fleischmann, C., Aritz, J., Logemann, M., & Heidewald, J. (2023). The challenges and opportunities of AI-assisted writing: Developing AI literacy for the AI age. Business and Professional Communication Quarterly, 86(3), 257–295. https://doi.org/10.1177/23294906231176517 

Chang, D. H., Lin, M. P.-C., Hajian, S., & Wang, Q. Q. (2023). Educational design principles of using AI chatbot that supports self-regulated learning in education: Goal setting, feedback, and personalization. Sustainability, 15(17), 12921. https://doi.org/10.3390/su151712921  

Moore, A. (2023, June 25). Is there any point still teaching academic writing in the AI age? Times Higher Education. https://www.timeshighereducation.com/blog/there-any-point-still-teaching-academic-writing-ai-age 

Watkins, M. (2024). Automated Aid or Offloading Close Reading? Student Perspectives on AI Reading Assistants. https://uen.pressbooks.pub/teachingandgenerativeai/chapter/automated-aid-or-offloading-close-reading-student-perspectives-on-ai-reading-assistants/ 

Leveraging Chatgpt For Analysing Student Reflections In A Design Thinking Course

cdtlsd Paper Presentations , , , , ,

Qian HUANG1,*, Ameek Kaur2, Thijs WILLEMS1

1Lee Kuan Yew Centre for Innovative Cities, Singapore University of Technology and Design (SUTD)
2NUS Business School

*qian_huang@sutd.edu.sg

Huang, Q., Kaur, A., & Willems, T. (2024). Leveraging ChatGPT for analysing student reflections in a design thinking course [Paper presentation]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/qhuang-et-al/

SUB-THEME

Opportunities from Generative AI 

KEYWORDS

Generative AI, ChatGPT, large-scale reflection, qualitative analysis, design education 

CATEGORY

Paper Presentation 

 

EXTENDED ABSTRACT

Generative Artificial Intelligence (Gen-AI) is increasingly being integrated into teaching and research methodologies, particularly since the advent of ChatGPT (Albdrani & Al-shargabi, 2023; Hwang & Chen, 2023). As educators navigate this evolving landscape, it becomes crucial to understand how to effectively and critically utilise Gen-AI tools in academic settings. This study explores the application of ChatGPT in analysing student reflections in a design thinking course at a university in Singapore. The course involved 550 first-year students across 11 cohorts, each student required to write four reflections over a semester. The significant volume of reflections presented a unique opportunity to deploy ChatGPT-4.0 for large-scale qualitative analysis. 

 

Initially, researchers manually analysed the reflections of 50 students from one class to establish a benchmark. These manual analyses were then compared to ChatGPT’s results to verify the reliability of the AI-driven approach. Upon confirming ChatGPT’s reliability, the tool was employed to analyse reflections from the entire cohort through the semester (550 students X four phases). The analysis focused on two primary objectives: first, to assess the impact of pedagogical interventions on students’ Affect, Behavior, and Cognition (ABC); and second, to understand how students applied these interventions and the frequency of their application. 

 

The study aimed to uncover how specific pedagogical interventions influenced students’ emotional responses, behavioural changes, and cognitive developments by using ChatGPT. For instance, it was observed that interventions such as confirmation bias were frequently applied by students during site visits to explore problems from multiple perspectives. This detailed analysis provided insights into the effectiveness of various teaching strategies and highlighted areas for potential improvement. 

 

Key findings from the study revealed several noteworthy trends. Firstly, some interventions, including case studies and activities, did not significantly impact students’ affective responses to the ideas emphasised in these interventions. This suggests that educators may need to refine these interventions to better support students emotionally. Secondly, the analysis highlighted variations in the delivery and emphasis of interventions across different cohorts, attributable to individual teaching styles of different instructors. ChatGPT’s analysis provided a nuanced understanding of how these differences influenced student outcomes. 

 

By leveraging ChatGPT, the research team was able to conduct a comprehensive analysis of a large dataset, providing valuable insights that might not have been feasible through manual analysis alone. The findings underscore the potential of Gen-AI tools in educational research, particularly in scaling qualitative analyses and uncovering patterns that inform pedagogical practices. 

 

In summary, this study demonstrates the utility of ChatGPT in analysing student reflections to gauge the impact of pedagogical interventions on students’ action, emotion, and cognition. The application of Gen-AI in this context not only facilitated the processing of a large volume of qualitative data but also offered educators deeper insights into how classroom interventions can be optimised to achieve desired educational outcomes. This method represents a significant advancement in educational research, providing a scalable and reliable approach to understanding and enhancing student learning experiences. 

 

This study contributes to the growing body of literature on the use of AI in education and offers practical implications for educators seeking to integrate Gen-AI tools into their teaching practices. Future research could expand on these findings by exploring the application of ChatGPT in different educational contexts and with diverse student populations to further validate and refine this approach. 

 

REFERENCES

Albdrani, R., & Al-shargabi, A. (2023). Investigating the effectiveness of ChatGPT for providing personalized learning experience: A case atudy. International Journal of Advanced Computer Science and Applications. https://doi.org/10.14569/ijacsa.2023.01411122.  

Hwang, G. J. & Chen., N. S. (2023). Exploring the potential of generative artificial intelligence in education: Applications, challenges, and future research directions. Educational Technology & Society, 26(2). https://doi.org/10.30191/ETS.202304_26(2).0014 

Using Generative AI in Design Thinking Courses: Towards Educators’ Guidelines

cdtlsd Paper Presentations , , , ,

Kate Sangwon LEE1,* and Jung-Joo LEE2

1Engineering Design and Innovation Centre, College of Design and Engineering (CDE)
2Division of Industrial Design, CDE

*katelee@nus.edu.sg

 

Lee, K. S. W., & Lee, J. J. (2024). Using generative AI in design thinking courses: Towards educators’ guidelines [Paper presentation]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-kswlee-jjlee/

SUB-THEME

Opportunities from Generative AI

KEYWORDS

Generative AI, Design-thinking, UIUX course, students assignment

CATEGORY

Paper Presentation 

 

INTRODUCTION

Generative AI (GenAI) applications have been extensively used in students’ assignments in design thinking courses (Hsiao & Tang, 2024) to express ideas and complete tasks (Chung et al., 2024), as shown in Figure 1. However, there are few clear guidelines about their usage due to GenAI’s novelty (Sun et al., 2024; Tholander & Jonsson, 2023; Wadinambiarachchi et al., 2024). This lack of guidance may confuse students and instructors regarding assignment guidelines and evaluation (Chung et al., 2024; Wadinambiarachchi et al., 2024). Therefore, this paper shares findings from observations on the current usage of GenAI by students in three design thinking courses at the College of Design and Engineering (CDE) at NUS during 2023-2024, aiming to identify challenges and opportunities. Finally, this paper proposes guidelines outlining students’ usage of GenAI at each stage of the design thinking process.

HECS2024-a39-Fig1Figure 1. Observed GenAI usages on the design thinking process.

METHOD

The three courses of CDE, including CDE3301R “Ideas to Proof-of-Concept”, CDE4301A “Ideas to Start-up”, and CDE5311 “Essential Skills in UI/UX Design”, were observed by instructors in 2023-2024, and students’ assignments served as supplementary material. All three courses were design thinking-based, and students conducted relevant methods in each phase as their projects progressed. They used GenAI in this process, and the lecturer and teaching assistant analysed the submitted assignments in terms of the GenAI usage’s relevance and effectiveness.

FINDINGS

Inspired and restricted at the same time by conceptual images (Discover phase)

  • In CDE3301R, students used the GenAI, Midjourney, to create conceptual images (Figure 2) by inputting project keywords as prompts to get inspiration for prototypes. Though the images helped specify initial ideas, they sometimes restricted students’ imagination boundaries.

HECS2024-a39-Fig2

Figure 2. An example of a GenAI-generated conceptual image.

Ambiguity in AI-generated images for storyboard (Define phase)

  • Students used GenAI to create storyboards in CDE5311 (Figure 3). However, many of the storyboards were abstract and ambiguous, which is not aligned with the method’s purpose. A storyboard as a UX method should convey an accurate environment or facial expressions, which are desired to be shown to describe the exact situations and users’ pain points, such as in a student’s drawing storyboard (Figure 4). Current GenAI applications are not capable enough of generating accurate images. Thus, using GenAI to create a storyboard may not be recommendable.

HECS2024-a39-Fig3

Figure 3. An example of a GenAI-generated storyboard.

 

HECS2024-a39-Fig4

Figure 3. An example of a storyboard drawn by a student.

 

Quality gap in ideation and prototyping (Develop phase)

  • In CDE4301A, students used GenAI for UI ideation in the early stage of UI prototyping as an inspiration tool (Figure 5). When compared to the UI created by the student (Figure 6), he reflected, “It is hard to generate specific UI details with acceptable quality…It is more suitable for ideation or early prototyping.”

HECS2024-a39-Fig5-6

Figure 5. An example of UI created by GenAI (Left),
Figure 6. An example of UI created by a student (Right).

 

Generic draft required refinement (Deliver phase)

  • In CDE5311, students used ChatGPT to generate a user test protocol. GenAI effectively created a draft, but it lacked specificity about the project context; thus, the experienced instructor needed help refining it.

DISCUSSION AND CONCLUSION

In the divergent phases (Discover and Develop), GenAI can be used as a supporting tool to get inspiration (Tholander & Jonsson, 2023). However, in the convergent phases (Define and Deliver), GenAI’s ambiguity did not effectively convey the exact ideas needed due to its lack of specificity and accuracy, which are attributes required in those phases (Tholander & Jonsson, 2023). Furthermore, depending on the students’ ability in each phase, the effectiveness of GenAI can differ (Cai et al., 2023). For example, if a student is a design novice, they would not have enough ability to discern the most effective and relevant outcomes from GenAI. Educators must selectively recommend using GenAI regarding students’ expertise and experience in each phase. Our proposal includes a few suggestions for the Usage of GenAI in the design thinking courses as below:

  • In the divergent phases (Discover and Develop), instructors should advise students to use various prompts to generate more diverse outcomes that can support ideation processes.
  • In the convergent phases (Define and Deliver), students can use GenAI to create initial drafts, but experienced instructors should assist in refining them to increase specificity reflecting project context.

REFERENCES

Cai, A., Rick, S. R., Heyman, J. L., Zhang, Y., Filipowicz, A., Hong, M., Klenk, M., & Malone, T. (2023). DesignAID: Using Generative AI and Semantic Diversity for Design Inspiration Proceedings of The ACM Collective Intelligence Conference. https://doi.org/10.1145/3582269.3615596

Chung, A., He, Y. C., Lin, L. F., & Liang, Y. W. (2024). Importance of Different AI-Generated Journey Map Modules from Industrial Design Students’ Perspectives. 2024 IEEE 7th Eurasian Conference on Educational Innovation (ECEI).

Hsiao, H. L., & Tang, H. H. (2024). A Study on the Application of Generative AI Tools in Assisting the User Experience Design Process. In International Conference on Human- Computer Interaction (pp. 175-189). Springer Nature Switzerland.

Sun, Y., Jang, E., Ma, F., & Wang, T. (2024). Generative AI in the Wild: Prospects, Challenges, and Strategies. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-16).

Tholander, J., & Jonsson, M. (2023). Design Ideation with AI – Sketching, Thinking and Talking with Generative Machine Learning Models. Proceedings of the 2023 ACM Designing Interactive Systems Conference. https://doi.org/10.1145/3563657.3596014

Wadinambiarachchi, S., Kelly, R. M., Pareek, S., Zhou, Q., & Velloso, E. (2024). The Effects of Generative AI on Design Fixation and Divergent Thinking. Proceedings of the CHI Conference on Human Factors in Computing Systems.

Viewing Message: 1 of 1.
 Success

Blog.nus login is now via SSO. Don't see or cannot edit your blogs after logging in? Please get in touch with us, and we will fix that. (More information.)