Authentic assessments

The Other Benefits of Making AI-resistant Assessments

cdtlsd Paper Presentations , , ,

Olivier LEFEBVRE1,2
1Department of Civil and Environmental Engineering
2NUS Teaching Academy

ceelop@nus.edu.sg

 

Lefebvre, O. (2023). The other benefits of making AI-resistant assessments [Paper presentation]. In Higher Education Campus Conference (HECC) 2023, 7 December, National University of Singapore. https://blog.nus.edu.sg/hecc2023proceedings/the-other-benefits-of-making-ai-resistant-assessments/

SUB-THEME

AI and Education 

 

KEYWORDS

AI chatbots, AI-resistant assessments, authentic assessments

 

CATEGORY

Paper Presentation 

 

ABSTRACT

Over the past one year, the world has witnessed growing concerns in relation with the rise in performance of artificial intelligence (AI) chatbots at a rate much faster than our own ability to comprehend all the implications, leading to questions on whether we should slow down or even halt the “race to god-like AI”1. In the academic world, concerns are mostly related about the way to assess students in this new day and age, where ChatGPT has been found to pass entry exams in fields as varied as medicine, law or business schools (Wilde, 2023). Such legitimate concerns have resulted in diverse responses from many universities over the world, from banning AI chatbots altogether, such as in French University Sciences Po (Sciences Po, 2023), to providing guidelines and recommendations for staff and for students, such as the choice made by NUS in our interim policy guidelines (NUS Libraries, n. d.).

 

The plagiarism issues and risks of other acts of academic dishonesty are real, but this is not the first time that we have to face such issues. At the peak of the COVID-19 pandemic, students were asked to take their exams from home, and in many cases, the simple conversion of a pen-and-paper exam into a digital one, without rethinking the entire assessment, had led to a rising number of plagiarism and other cheating cases. Similarly, as with the COVID-19 pandemic, can we once again rethink our assessments and not only proof them against abuse from AI but also take this opportunity to deliver more meaningful assessments, in better adequation with the skills that our students need in this day and age (Mimirinis, 2019)? Instead of banning ChatGPT and the like, should we just acknowledge that AI is here to stay, and design assessments that question higher-order thinking skills, allowing at the same to distinguish between students who engage in surface learning and those who have achieved a real deep understanding of the topic? Such exams would constitute a form of authentic assessment, by recreating the conditions that students will apply their knowledge in their professional environment (Shand, 2020).

 

In this talk, I will present some general guidelines on what kind of exam can at the same time test students for higher-order thinking skills and resist AI chatbots. Real examples will be provided, where students are asked to:

  • Deliver a critical analysis of a scientific paper
  • Interpret graphs or images
  • Solve ill-defined and complex problems

 

I will show how well (or not) these exams resist ChatGPT and compare the AI output to that of real (anonymised) students over a range of performance (excellent, average, marginal). I will conclude with the limitations, e.g., the risk to increase the difficulty of the exam by a too large margin, making it difficult for the weaker students to perform reasonably well.

 

ENDNOTE

  1. Refer to https://www.ft.com/content/03895dc4-a3b7-481e-95cc-336a524f2ac2 for details.

 

REFERENCES

Mimirinis, M. (2019). Qualitative differences in academics’ conceptions of e-assessment. Assessment & Evaluation in Higher Education, 44(2), 233-48. http://dx.doi.org/10.1080/02602938.2018.1493087

NUS Libraries (n. d.). Academic integrity essentials. https://libguides.nus.edu.sg/new2nus/acadintegrity

Sciences Po (2023, January 27). Sciences Po bans the use of ChatGPT without transparent referencing. https://newsroom.sciencespo.fr/sciences-po-bans-the-use-of-chatgpt/

Shand, G. (2020). Bringing OSCEs into the 21st century: Why internet access is a requirement for assessment validity. Medical Teacher, 42(4), 469-71. http://dx.doi.org/10.1080/0142159X.2019.1693527

Wilde, J. (2023, January 27). ChatGPT passes medical, law, and business exams. Morning Brew. https://www.morningbrew.com/daily/stories/2023/01/26/chatgpt-passes-medical-law-business-exams

 

Making Learning “Real” in Communication Courses: Authentic Assessments for Communities

cdtlsd Lighting Talks , , ,

Suwichit (Sean) CHAIDAROON
Department of Communications and New Media, Faculty of Arts and Social Sciences (FASS)

sean.chaidaroon@nus.edu.sg

 

Chaidaroon, S. (2023). Making learning “real” in communication courses: Authentic assessments for communities [Lightning talk]. In Higher Education Campus Conference (HECC) 2023, 7 December, National University of Singapore. https://blog.nus.edu.sg/hecc2023proceedings/making-learning-real-in-communication-courses-authentic-assessments-for-communities/

SUB-THEME

Communities and Education

 

KEYWORDS

Authentic assessments, significant learning experience, co-creational pedagogy

 

CATEGORY

Lightning Talks 

 

ABSTRACT

One of the common challenges we are facing in university teaching is designing assessments that facilitate students’ learning. Traditional assessments may measure students’ cognitive knowledge well but they often fail to motivate students to engage in higher-order learning. Well-designed assessments not only motivate students to excel themselves in their studies, but also serve as a catalyst to foster significant learning experiences. This refers to is education that makes a difference in how students live their lives meaningfully by enhancing individual lives, social interactions with others, inculcating a civic mindset as well as a readiness for the world of work (Brackenbury, 2012; Fink, 2013).

 

In this Lightning Talk, the author shares his practices and reflection on authentic assessments he designed for the communication courses he has taught. Defined as learning tasks that allow students to apply competencies or combination of knowledge, skills and attitudes that they need in their professional lives, the assessments in three communication courses in particular will be discussed to illustrate the authentic assessment framework (Gulikers et al., 2004) as outlined below.

 

  • In NMC5323 “Crisis Communications and Leadership”, one assessment was a press conference simulation where professional journalists were invited to participate and ask questions to students who role-play as spokespersons in crisis situations.
  • In NMC5306 “Communications and Leadership”, students were asked to write thought leadership blog posts on their personal LinkedIn profiles. Please search #NMC5306 on LinkedIn for examples of students’ work.
  • In NM3215 “Advertising Strategies” (later converted to NM4257 “Multiplatform Advertising Strategies”), students developed actual video and printed advertisements for local clients.

 

The author’s reflection highlights two key factors for the successful implementation of authentic assessments in his clases. First, the co-creational pedagogy, or student-centred teaching as some scholars may call this approach, should be adopted where students’ inputs are consistently taken through the facilitation and coaching process that the instructors will conduct (Hou & Chaidaroon, 2022). This co-creational pedagogy highlights the importance of the learning process as much as the product, allowing students to excel themselves in their studies. Second, there must be authentic audiences for students’ work as most traditional assessment deliverables, such as tests or essays, were solely read and assessed by the instructors. This is an opportunity for the local communities to be invited to serve as authentic audiences who may also benefit from students’ work.

 

Ultimately, the authentic assessments implemented by the author were proven to be meaningful teaching and learning experiences for the instructor, students, and the communities involved in all projects. They have proven to be good indicators for significant learning experience that students gain from attending university, which reflects students’ achievement not only in their studies but also in their development of a citizenry mindset.

 

REFERENCES

Brackenbury, T. (2012). A qualitative examination of connections between learner-centered teaching and past significant learning experiences. Journal of the Scholarship of Teaching and Learning, 12(4), 12-28. https://scholarworks.iu.edu/journals/index.php/josotl/article/view/3139

Fink, L. D. (2013). Creating significant learning experiences, revised and updated: An integrated approach to designing college courses. John Wiley & Sons.

Gulikers, J. T. M., Bastiaens, T. J. & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology and Development, 52, 67–86. https://doi.org/10.1007/BF02504676

Hou, J. Z., & Chaidaroon, S. S. (2022). A co-creational turn to online strategic communication education: Prospects and challenges. Asia Pacific Public Relations Journal, 24. https://www.pria.com.au/pria-newsroom/asia-pacific-public-relations-journal/apprj-volumes/volume-24/?fbclid=IwAR1qjBSnYHm7vn1Qqlp8KN1uuif2ZbqErNeW5yKO1gzoXBMysLlV8JsyIBw

 

Viewing Message: 1 of 1.
 Warning

Blog.nus accounts will move to SSO login, tentatively before the start of AY24/25 Sem 2. Once implemented, only current NUS staff and students will be able to log in to Blog.nus. Public blogs remain readable to non-logged in users. (More information.)