(Note from the blog owner: The following is the account by Dr. Lee Wang Yen, a colleague from the Philosophy Department, concerning how he created a computer programme for assessing logic. Since 2016, Dr. Lee has been teaching GET1028 Logic and GET1026 Effective Reasoning for the Department of Philosophy. As both modules are relatively large (200-300 students each semester), we are always looking for ways to streamline or automate processes. This is not just about saving human work; more importantly, it is also about cutting down on human errors and ensuring that our teaching is scalable across larger groups of students without compromising on quality. Knowing a little about what he went through to make his logic assessment program happen, I encouraged him to do a write up for sharing and offered him a platform for hosting it. You can find out more about Dr Lee’s research at http://nus.academia.edu/WangYenLee and contact him at wangyen.lee@nus.edu.sg)
* * * * *
1. Problem
When I first taught GET1028 Logic at NUS, I set a mixture of Multiple Choice Questions (MCQs) and short-answer questions (SAQs). My detailed marking method (deducting 0.1, 0.2, 0.25, etc. marks depending on the seriousness of the mistakes) entailed a huge workload not only for myself, but also for the admin staff in my department, who had to help check the marks calculation. Given the size of my class and the small admin team in my department, everyone was under huge pressure to meet the deadline without compromising on accuracy. I had to work 10-11 hours a day for more than a week. The admin staff had to work over weekends. It occurred to me, given the prompting of the admin staff and department leadership, that I had to change my assessment method.
One option would be to set only MCQs. However, after many long and detailed discussions with Prof. Loy, my deputy head of department (a keen user of MCQs in the large exposure module) in 2017, we came to the conclusion that not all logic skills could be tested by MCQs. While MCQs can test a student’s ability to evaluate a proof, it cannot test a student’s ability to conduct the proof from beginning to the end. As a result, I took up my then teaching assistant’s suggestion and began to write a computer programme that eventually evolved into LogiProof. Continue reading
