Technology-enhanced Education

In 2008, Professor Andrew Ng of Stanford University started the Stanford Engineering Everywhere programme to distribute free online education. Last year, he taught a Machine Learning class with an enrolment of more than 100,000 students from all over the world. Subsequently, Coursera was born with the intention of bringing materials designed and taught by world-class professors to every part of the world.

MITx was launched at the end of 2011 to deliver MIT courses online and free-of-charge. Later, Harvard and MIT each contributed US$30 million to form a new venture EdX, with the same aim. EdX runs on an open source platform, which provides videos and interactive software to engage online learners. Certificates of completion will be awarded for a modest fee, but at present, there are no plans to award college credits as yet.

Technology has enabled distant and online learning, even of degree courses.

Online courses have been around for some time. This trend started at the turn of the century. Many ventures had sprouted, but most of them lost momentum and direction after some time. NUS was previously involved in an online venture with the Universitas 21 group, but the venture folded due to financial non-viability. What is different this time with Coursera and EdX is the level of technology, and the high caliber of universities joining this bandwagon. 

The world was a different place just a mere decade ago. There was no YouTube or Facebook then. Technology has advanced to a stage where interactivity with students can be so dynamic and this can make learning much more effective. Take a look at Khan Academy’s videos and assessments of pre-college and college mathematics (the academic content is similar to what our students have to cover for the A levels in Mathematics) and you can see the power of technological advancement. 

Some colleagues (especially those of us who did not grow up with these gadgets) may be quite skeptical about technology, and view it as a fashion trend that might become passé in time to come. Some are perhaps wary that technology may displace teaching jobs in the future. Let me try to quell this misperception. In the words of John Hennessy, President of Stanford University, “the biggest way IT-enhanced education could go wrong is if we assume that we can fully automate education, if we think that we can throw students in front of terminals, that we don’t need any live instructors anywhere, that students can be totally successful without ever talking to anybody.”

Even with the best textbooks and good quality online resources, the role of the teacher remains critical. In fact, the teacher’s role will become even more important. I think the teacher’s role will shift from solely delivering subject content, to actively facilitating, and directing learning and interaction within a classroom, so that students are probed and prodded at the right time, with the right questions, to stimulate their thinking and learning.

Technology has revolutionised every part of our lives and surely, it will also impact teaching and learning. The Singapore Prime Minister, in his recent National Day Rally speech, provided many examples of how technology has and will impact our lives. Today’s students are more technologically savvy, and they respond very quickly to new innovations. We know that most young people are adept at using technology, and are completely comfortable, for instance, with watching a video online. They like the fact that they can hit the pause button at will; they know how to speed it up a little bit, to watch it 20% faster, and to personalise the experience, making the process of learning more efficient for themselves. We know some students prefer collaborative learning and habitually take and share notes using Twitter. Many are also familiar with crowd sourcing to complete their assignments and projects. The list just goes on and the possibilities are endless.

Should we then continue doing ‘business as usual’ and labour on in a large lecture theatre, earnestly endeavouring to deliver the same subject content, to 300 students with varying abilities and powers of absorption, year in and out, without change? Our aspirations have not changed – we want to deliver a high quality educational experience. Just as we urge our students to question and to discover, we lecturers too, should continually seek to explore how we can leverage on advances in technology, for what it’s worth, to enhance the learning process and its outcomes. 

 

‘Flipped Classroom’

Some of you may have read about the ‘flipped classroom’ concept. It is a fairly new look at the learning process, facilitated by a new space configuration that has been shown to be successful. The classroom is ‘flipped’, because there are no in-class didactic lectures – students instead access online materials to learn the basic concepts.

The ‘flipped classroom’ rests on a very strong scaffold of well-constructed online materials and the use of interactive learning platforms that further augment and reinforce understanding. In-class tasks and discussions allow for collaborative co-construction of knowledge. Again, learning effectiveness is predicated on well-designed assessment tasks that engender higher order learning outcomes. Effective ‘flipped classroom’ learning environments must also be facilitated by an appropriate learning space design that promotes collaborative learning.

The Duke-NUS Graduate Medical School (Duke-NUS) successfully employed the ‘flipped classroom’ concept in its TeamLEAD (Learn, Engage, Apply and Develop) approach. Under this learning structure, there are no hour-long lectures, but students watch videoed lectures or access online materials that disseminate basic content knowledge. The in-class time is devoted mainly to discussions and problem-solving activities. At the start of each TeamLEAD session, students are individually assessed about the material they were to learn prior to class to assure that they have come to class prepared.  During the first in-class discussion session, students are put in teams and answer the same set of questions (as the ones they had earlier individually answered). The only difference is that they now have to provide a team response. This is where further and deeper learning take place; and the active class dynamics surfaces – a student in a team must convince the others of his/her answer, or be convinced by others of their answers.  The next phase is applying that knowledge to more realistic (typically clinical) problems.  The students are then given another set of problems to solve as a team.  This phase is open resource/open internet.  The teams now must apply what they learned in the earlier phase, and not simply regurgitate a set of basic facts.  Simple questions which test only memory do not work in this instructional method, as it is too easy to just ‘Google’ the answer.  Instead the faculty must work hard to create questions that make the students think and apply facts and key principles that they have learned. 

One might ask, how have the students performed in such a learning environment? Duke-NUS students take the same national tests in basic science (i.e. the US Medical Licensing Examination, or USMLE Step 1) as students in American medical schools. Since its inception, Duke-NUS has year after year, performed significantly better than the US national average. What is most striking however, is the fact that the Duke-NUS curriculum devotes only 1 year to basic science instruction, rather than 2 years which is the norm in the US. Duke University is suitably impressed with TeamLEAD that it ‘transported’ the learning strategy back to Duke University, to be implemented at the School of Medicine and several courses in the undergraduate campus; this learning strategy was touted as one that would revolutionise medical education.  

With the success of TeamLEAD, NUS will be piloting the ‘flipped classroom’ strategy with a number of modules. We hope to harness the active engagement enabled by the ‘flipped classroom’ concept and good use of technology to maximize and personalise learning for our students. We do not advocate using technology for technology’s sake; but we do need to align the way we deliver education with how our students experience today’s technology-driven world in general, and life at the university in particular. Let’s all keep an open mind and see what this new technology-enabled learning strategy has to offer.

 

Entrepreneurship at NUS

 

Last month, the Straits Times conducted a survey of about 500 people on the values that mattered most to them. Honesty, kindness and gratitude came up tops. Curiousity was ranked last; creativity and courage were not too far from the bottom. Perhaps the sample size is too small for us to have any conclusive sense. But a few were quick to jump in to say that this is why we do not have great inventions and Nobel laureates.

 

Do Singaporeans have what it takes to stay relevant, ahead and prosperous in the next 50 years? Singapore has done well in the past 47 years. We have first world infrastructure (some may not agree, with the recent MRT breakdowns), the rule of law, reliable regulatory frameworks, and a hardworking and resilient labour force. But, in this innovation-driven era, ideas, creativity and enterprise – these are what will shape and define our future. You may like to read this great article by Farhad Manjoo on the competition in the IT industry.

 

Last Friday, I was invited to be a judge for the inaugural NUSSU Test-Bed Programme, a joint initiative by NUSSU and NUS Enterprise. Our students and alumni submitted a total of 63 business proposals, of which 10 were shortlisted to pitch their business ideas to a panel of judges. The panel would select a few of the winning ideas which will be test-bedded in NUS. NUS, with its 45,000 staff and students, provides a ready ‘customer’ base to seed and spawn these ideas.

 

Entrepreneurship can be a daunting endeavour. It begins with curiosity, ideas and dreams, of a product, technology or service that could bring value to society. However, it does not stop there. The next step entails venturing into the unknown – attempting to translate this idea into fruition. Much work goes into sourcing for support and resources to develop and fine-tune the product or service. And finally, the greatest challenge beholds, to capture and harness the value created in the marketplace.

 

While we know that it is important to nurture entrepreneurial skills and mindsets, some would say that the Asian upbringing is not particularly conducive for this purpose. In our growing years, many of us try not to question or challenge our parents, elders or teachers too much, as we do not want to be misconstrued as being disrespectful. Many parents prefer their children to pursue tried and tested professional careers, rather than to venture into start-ups.

 

I am thus very heartened by Friday’s event. The ideas of the 10 shortlisted teams are testament that there are budding seeds of adventure and enterprise within our community. I applaud and commend each team for their efforts. In formulating the business proposals, these students have had to “think outside of the box”, and though the course, they would have developed a sense of opportunism and savvy. It is an experience that textbooks cannot impart, yet the wisdom and acumen gained will come in useful in their future endeavours.  

 

Eventually, the panel of judges selected 4 proposals for test-bedding at NUS: SnapSell, Intraix, YourKaki and Munshi Labs. SnapSell is an app that will make selling and buying of second-hand items such a breeze and a delight. Intraix is an energy management system which incorporates an interesting gaming/challenge component. YourKaki is a refreshing one-stop community and directory for sport enthusiasts. And finally, Munshi Labs will facilitate researchers, consultancy firms and the like, with an easy database of respondents for research and surveys. There was a proposal (i.e., Clault which ensures security in cloud-based storage and applications) for which the judges thought was highly marketable, but unsuitable for test-bedding in NUS. Congratulations to all the teams for your fine participation.  

 

I also wanted to also convey the message to our students that if you think you have an enterprising knack, or if you are curious about creativity and innovation, there are developmental avenues and opportunities in NUS you can explore. The NUS Entrepreneurship Centre has been actively supporting and encouraging entrepreneurship endeavours within the university community. The Centre provides physical incubation space to NUS startup companies, and mentoring, financial and marketing support, as well as business network access that is vital for small businesses to thrive and take off.

 

Or perhaps, you will relish a work-study stint in an entrepreneurial hub. Take a look at the NUS Overseas College Programme (NOC). NOC is a distinct flagship educational programme which gives students the opportunity to be immersed in leading entrepreneurial hubs, such as Silicon Valley, Philadelphia, Stockholm, China, India and Israel. NOC students spend a year in these hubs, working as full-time interns in high-tech start-ups or innovative companies; they learn directly from founders and entrepreneurs, and witness firsthand, the business and operating environment. At the same time, NOC students will read entrepreneurship-related or discipline-based courses at established NUS partner universities at these overseas locations.

 

Finally, may I share a quote from Samuel Ullman, an American poet. He once aptly described youth as a state of mind – it is a temper of the will, a quality of the imagination, a vigour of the emotions, a predominance of courage over timidity, of the appetite for adventure over love of ease. This sums up the spirit of entrepreneurship that we hope will flourish in our community. Stay youthful, always!

Exams and CAs

In an earlier blog post, I had mentioned that NUS has been progressively moving towards decreasing the weightage of final exams.

 

I thought it might be interesting to share this table which shows the distribution of CA-weightage across modules offered in AY2010/11. One-third of the modules have 100% CA, and more than half our modules have CA components of 60% or higher. 

 

Table on Number of Modules by Level in AY2010/11 and the CA Component

Level

Modules with CA at 100%

Modules with CA at 80% to 99%

Modules with CA at 60% to 79%

Modules with CA at 40% to 59%

Modules with CA at 20% to 39%

Modules with CA at 0% to 19%

Total

Level 1000

111

0

67

147

30

5

360

Level 2000

120

2

128

251

66

4

571

Level 3000

288

1

145

251

81

5

771

Level 4000

274

2

107

214

68

14

679

Total

793

5

447

863

245

28

2381

Percentage

33%

0%

19%

36%

10%

1%

100%

 

There are however, some variations between the Faculties/Schools, as there are differences in the content and nature between the disciplines. The final exam weightage tends to be higher for the science and technology courses, than for the arts or humanities. In the Faculty of Arts and Social Sciences and Business School, 70% or more of the modules have CA components comprising at least 60% of the final grade. In Faculty of Engineering, slightly more than a quarter of modules offered have CA components comprising at least 60% of the final grade (and for the Faculty of Science, the corresponding percentage is 33%). This is probably not surprising as intuitively, many are aware that presentations, group case studies and discussions have traditionally been part of the pedagogies employed at business schools, whereas exams are the traditional testing method for technical subjects like mathematics, hard sciences and engineering. 

 

Table on Number of Modules by Faculty in AY2010/11 and the CA Component

Faculty/School

Modules with CA at 100%

Modules with CA at 80% to 99%

Modules with CA at 60% to 79%

Modules with CA at 40% to 59%

Modules with CA at 20% to 39%

Modules with CA at 0% to 19%

Total

Arts & Social Sci

304

2

291

278

3

878

Engineering

88

1

27

162

133

12

423

Science

131

32

232

102

2

499

Business

143

2

28

60

1

234

Computing

41

48

68

1

158

Design and Environment

51

19

63

6

13

152

CELC

35

2

37

 

There is a place for exams, and in many level 1000 or 2000 classes, exams will remain as an important assessment tool. But for the level 3000 and 4000 classes, we are moving towards placing more emphasis on continual assessment.

 

 

 

Admissions and Financial Aid

The days following the release of the GCE ‘A’ Level Results mark the traditional peak season for university applications. I thought it is timely for me to convey some key attributes of the NUS’s policies on admission, and on the related issues of tuition fees and financial aid.

 

Admissions

 

Admission to the NUS is competitive. We receive over 30,000 applications annually, but we only have places for about 6,700 freshmen. Each application is assessed solely on merit. For certain courses, candidates are shortlisted for further assessments, and the Faculty considers each applicant’s achievements, aptitude and personal qualities in assessing the applicant’s suitability for the course and/or the profession.  An applicant’s family background, schooling history or financial circumstances will not have a bearing on the selection process. Essays are reviewed without reference to the applicant’s personal details; interviewers and selection panels do not have access to the candidate’s financial status.

 

The NUS also sets aside up to 10% of our places for Discretionary Admissions (DA). DA provides us with an avenue to consider applications from deserving students whose high school grades may not meet the entry requirements for admission into NUS, but have the potential to pursue an undergraduate education, and to contribute to the NUS community. For DA applicants, NUS will consider their contributions and achievements in other areas, beside academic grades.

 

DA students add diversity to our campus and this scheme has increased the opportunities for students with different talents, achievements and experiences to join the NUS community. Many DA students are faring well. Some years back, when I was Dean of the Faculty of Science, I admitted an applicant with weak A level results to the Faculty of Science. We noted something special about her. She had been giving tuition to support her family since she was in Secondary 3, as her father was then retrenched. Her tuition business expanded, and by the time she was in JC 2, she founded a thriving tuition centre. Her involvement in the tuition business was probably one of reasons why her A level grades had suffered. She was also actively involved in community work. In spite of her weak A level grades, she coped well academically at NUS, and was even admitted to the University Scholars Programme. Eventually, she graduated with a Second Upper Class Honours, and was selected as the valedictorian for her commencement ceremony. She is now pursuing a medical degree at the Duke-NUS Graduate Medical School. I am thus grateful for the DA scheme, which has allowed me to give deserving students like her a chance to benefit from an NUS education.

 

An NUS education will remain affordable and accessible to all Singaporeans who qualify for admission

 

Some students have expressed concern that students from lower income families may be discouraged from pursuing a university education due to financial difficulties. I would like to take this opportunity to assure all existing and potential students, that NUS has in place, a robust financial aid framework to ensure that an NUS education remains affordable and accessible for all Singaporeans who are admitted to the university.

 

NUS is cognizant that society regards university education as an economic leveller and a means of social mobility. For many decades, an NUS degree is an aspiration of many Singaporean families. NUS counts many first generation graduates in each cohort. This is also true for the professional courses; each year, we train many first generation accountants, architects, dentists, doctors, engineers, lawyers and pharmacists. Although the income profile of NUS students does not exactly mirror that of the general national income profile, there is a substantial number of NUS students whose per capita household incomes are at the lowest quintile.

 

We remain committed to enabling needy and deserving students to pursue their university studies with us. Every needy Singaporean admitted to the NUS will have access to sufficient funds to meet the costs of undergraduate education. Students from middle-lower income families (this applies to about half of Singaporean households) will be able to obtain sufficient funds to meet the full costs of tuition and living expenses through a combination of bursaries and loans. 

 

Enhanced Financial Aid Framework – Helping the Needy

 

For AY2012/13, NUS will be extending even more help, especially to students whose families are in the lowest 20th percentile of per capita household income. As explained in my earlier circular to all students, NUS has conducted a comprehensive review of our financial aid schemes and we have made considerable enhancements. The key features of the enhanced financial framework are first, the annual bursary quanta for the neediest Singaporean students will increase from $4,750 to over $8,000.  This will significantly reduce the loan burden for students from very low income families, and more than 1,100 needy NUS students will benefit from this bursary increase. Second, the bursary quanta awarded will be more finely means-tested according to the student’s financial circumstances. The neediest students will receive more financial aid. This acknowledges the different circumstances that the neediest students with very low household incomes face. Third, eligible local students may receive more than one bursary, up to the maximum bursary quantum prescribed for their per capita household income. Fourth, needy Singaporean students participating in academic programmes such as the University Scholars Programme, Student Exchange Programmes, etc., will be provided with proportionately higher financial support. Overall, NUS will be increasing student financial aid by $4.5 million, which will bring overall student financial aid for AY2012/13 to $9.5 million.

 

Many students have expressed support for and welcome the enhanced Financial Aid Framework. NUS is also working hard at fund-raising, as philanthropic donations will allow us to expand the pool of funds available for financial aid. In the meantime, if you do know of any students who are struggling financially, please drop a note to financialaid@nus.edu.sg, and we will look into every case closely. For some students, their family circumstances may change midway through their studies at NUS, and as a result, they may be encountering financial adversity; please encourage them to approach us.

The Bell Curve

I chuckled when I read this article:   Desperate undergrads pray to ‘bell curve god’

 

Superstitions aside, students correctly know that the bell curve does affect them in some way or other. However, I hope that no one is feeling haunted by the bell curve.

 

What is this ‘bell curve’ all about?

 

In probability theory, the normal distribution is a continuous probability distribution that has a bell-shaped probability density function, known as the Gaussian function, or informally, the bell curve. The normal distribution is the most prominent probability distribution, because many large sets of data are approximately normally distributed.

 

 

 

 

For example, the heights of all students in NUS are likely to be normally distributed. The weights of NUS students probably follow a normal distribution too.  Likewise, if I set exams targeted at the average competency of a group of students, and if the class is large enough, the exam scores are likely to follow a normal distribution curve. Setting such an exam is, by no means, easy. Pitch it tough, most students will fail. Set it too easy, and many will score very high grades, and the resulting scores are hardly differentiated.

 

 

Grading and Moderation

 

Differentiation is necessary for CAP purposes, and for Honours classification, and these are here to stay for the foreseeable future. Most if not all major universities have variants of degree classes or GPA scores.  And because of the need for differentiation, many institutions from North America to Asia, use the bell curve as a mechanism to moderate marks.

 

Module requirements may encompass different modes of assessment such as tutorial presentations, laboratory reports, projects, essays, as well as mid-term and final examinations.  Grading may be based on absolute performance, relative performance, or a combination of the two.  Higher-level modules with small enrolments typically grade a student based on his absolute performance; larger lower-level modules take into account a student’s performance vis-à-vis the other students in the same module. Where necessary, the final grade which a student receives for a module may be subject to moderation.

 

One important reason for grade moderation is that examiners come from diverse academic backgrounds and may be accustomed to different marking regimes.  While we do make every effort to make sure modules are designed with clear learning outcomes, and professors are responsible to ensure their exams are pitched at the right level, grade moderation will prevent grade inflation or deflation, and helps to achieve consistency in assessment grading across modules. 

 

 

 

How NUS Applies the Bell Curve

 

At the end of the semester, a student is awarded a grade (and not specific marks) for each course taken. As such, NUS adopts a ‘recommended grade distribution’ in the following manner. I shall illustrate with an example – bear in mind that this is meant for illustrative purposes and is not the actual distribution that we are currently practising.

 

Example of a Recommended Grade Distribution: 

Grade A+, A, A- B+,B, B- C+,C D, F
Proportion Not more than 25% Not more than 40% About 30% Not more than 5%

   

One could compute the average grade point:

(5 x 0.17 + 4.5 x 0.08 + 4 x 0.14 + 3.5 x 0.13 + 3 x 0.13 + 2.5 x 0.15 + 2 x 0.15 + 1 x 0.05)

which is approximately 3.34, or roughly B grade.

And here are 3 possible course grade profile scenarios after moderation. For the first module, the test/exam may have been too easy, resulting in high cut-off marks for each grade. The average mark was 83 for a class of 630. The department will also look at the paper before moderation.

 

 

 

In the next scenario, the resulting cut-off marks for each grade after moderation, are moderate.

 

 

In this final scenario, the resulting cut-off marks for each grade after moderation are rather low.

 

 

 

I must emphasize that the recommended grade distribution is not applied blindly, and there are ample opportunities, within reason, for discretion and flexibility.  

  • First, the class size must be large enough, preferably above 30. For smaller classes, Professors are given discretion on an appropriate grade distribution because small sets of data may not be normally distributed.
  • Second, we are not looking for a perfect fit, i.e., we usually ignore small deviations. 
  • Third, if Professors have strong reasons to deviate from the recommended grade distribution, we are usually amenable to acceding to their requests. For example,  some modules belonging to special programs in the various Faculties/Schools have their own distributions.
  • Fourth, we sometimes also look at the CAP profiles of a class, and tweak the grade distributions appropriately. For example, if many students with high CAP choose a particular course, it will not be fair to apply the recommended grade distribution to this class. Another example: for Honours classes comprising students with an average CAP of at least 3.5, the grade distribution will be skewed higher.

 

Additionally, much goes on post-exams, before the grades are finalized and released to students. First, the grade profiles for individual modules are examined and compared at the Department level, and then across Departments at the Faculty level. All grades are carefully scrutinized by Department and Faculty Boards of Examiners before they are submitted to the Board of Undergraduate Studies and the Board of Graduate Studies for approval. Further checks are conducted at the University level by the Board of Undergraduate Studies and Board of Graduate Studies to ensure that there is consistency of assessment across Faculties/Schools.

 

In conclusion, I hope that this post has given a better picture of how the bell curve works at the NUS, and hopefully this helps to alleviate some bell curve anxiety. The bell curve is used primarily as a tool to moderate grades, and as a guide to prevent grade inflation or deflation. We do not apply the bell curve mindlessly or excessively. Students are sometimes worried about falling on a ‘wrong side’ of the bell curve. Do not worry too much, more often than not, we err on helping students along.

 

Here’s also wishing everyone a joyous Lunar New Year!