Jingcheng FU
Residential College 4 (RC4), NUS
Fu, J. (2024). Bridging the gap: Comparing student beliefs and existing empirical data on ChatGPT’s job market impact [Paper presentation]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-jingchengfu/
SUB-THEME
Opportunities from Generative AI
KEYWORDS
ChatGPT, student beliefs, job characteristics, labour market trends
CATEGORY
Paper Presentation
INTRODUCTION
The integration of AI technologies, particularly language models like ChatGPT, is poised to transform various job sectors and higher education (Dempere et al., 2023). This study explores student beliefs about ChatGPT’s impact on the labour market. Specifically, students are asked to predict about the association between different skill requirements and other job characteristics and the exposure to ChatGPT. The beliefs are elicited in a class survey after which the students are informed of the latest empirical findings. Understanding these beliefs is crucial for educators and career advisors to guide students effectively in a rapidly evolving job market.
LITERATURE REVIEW
Eloundou et al. (2023) developed an “exposure index” to measure the extent to which different jobs are affected by ChatGPT. Specifically, based on a detailed description of the work activities and tasks for each job, the researchers use some rubrics to determine the proportion of tasks that ChatGPT is expected to make at least 50% faster. This index is computed for 1,016 occupations in the US labour market. The occupation dataset also provides quantitative measures of job characteristics, including the importance of different skills. Many of those characteristics are significantly correlated with the exposure index.
RESEARCH QUESTIONS
- What are students’ perceptions of the relationship between a job’s requirements of different types of skills and the exposure to ChatGPT?
- How do these student beliefs compare with the existing empirical findings?
METHODOLOGY
To investigate these questions, a survey was administered to 34 first-year NUS students who took the course UTC1702G “Thinking in Systems – Markets and Inequality” in April 2023. The class was 55% female, and the faculty composition was 32% NUS Business School, 16% Faculty of Arts and Social Sciences (FASS), 14% Faculty of Science (FOS), 27% School of Computing (SOC), 9% College of Design and Engineering (CDE), and 2% from the Multi-disciplinary Programme. Students were given a brief explanation of the study design of Eloundou et al. (2023), together with the definitions of the skills, before they answered the multiple-choice questions (MCQs) to make guesses on the findings. I first asked students which skills are positively associated with substitutability, followed by which skills are negatively associated; the answers were coded as “Negative”, “Positive”, and “Neutral” for each skill (2% of the answers for a particular skill were inconsistent and not included in the analysis).
KEY FINDINGS
Figure 1 summarises the findings of Eloundou et al. (2023) and the distribution of student beliefs. Of the 11 skills tested, all are significantly associated with exposure except for speaking. For the three skills that are most strongly positively associated with exposure—reading, writing, and programming—the vast majority of student beliefs were accurate. For the other skills, however, their guesses departed from the paper’s findings. For the two skills that are most strongly negatively associated with exposure, only around 50% of students correctly predicted the association, and around 30% of the students believed the opposite. For the other two process-related skills, learning strategies and monitoring, which have a small negative association with exposure, the guesses were split, with more than 40% being incorrect positive guesses. Only 20-30% of students correctly saw the positive association for active listening, and the negative association for mathematics and science.
Figure 1. Beliefs about associations between ChatGPT exposure and different skill requirements compared to empirical findings.
SIGNIFICANCE OF THE STUDY
The findings indicate a significant disparity between student beliefs and the empirical data from the exposure index study. Many students hold incorrect assumptions about which skills and job characteristics are most vulnerable to AI substitution. This gap underscores the need for educational interventions to align student perceptions with actual labour market trends. By identifying these misconceptions, educators can develop targeted strategies to enhance career guidance and support, ensuring students have accurate information about AI’s effects on job characteristics and skills. This is essential for preparing them to navigate future career paths effectively. Addressing these misconceptions is crucial for students’ future success, providing a foundation for further research and practical applications in higher education and career planning..
REFERENCES
Dempere, J., Modugu, K., Hesham, A., & Ramasamy, L. K. (2023). The impact of ChatGPT on higher education. In Frontiers in Educatio, 8, 1206936. https://doi.org/10.3389/feduc.2023.1206936
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023). GPTs are GPTs: An early look at the labor market impact potential of large language models. https://doi.org/10.48550/arXiv.2303.10130